Fix native SQLite healing on Node 26

claude/ruflo-reference-full.md

@@ -107,39 +107,42 @@ doubt — see diagnostic table below.
 
 ### Node version compatibility (historically the ROOT cause of the WASM bugs)
 
-> **Resolved upstream in ruflo v3.10.6** ([ruvnet/ruflo#2219](https://github.com/ruvnet/ruflo/issues/2219)).
-> ruflo now ships an npm `overrides` entry forcing `better-sqlite3 ≥12.8.0` (which has
-> Node 20–26 prebuilts) across the agentdb copies, with a CI guard. So on ruflo ≥3.10.6
-> the WASM fallback **no longer happens by default**, even on Node 24/26, and the override
-> lives in ruflo's own `package.json` — an `npm i -g ruflo` upgrade keeps it, it does not
-> wipe it. the kit's natives heal (part of `ak sync`) is now a **safety net**, not a requirement.
+> **Partly resolved upstream in ruflo v3.10.6**
+> ([ruvnet/ruflo#2219](https://github.com/ruvnet/ruflo/issues/2219)). Ruflo added an
+> npm override forcing `better-sqlite3 ≥12.8.0` across the agentdb copies, which fixes
+> Node 24. Node 26 support starts at better-sqlite3 12.10.0; an exact 12.8/12.9
+> override can still fall back to WASM. The kit's natives heal (part of `ak sync`)
+> raises a stale v12 pin to a Node-compatible range and installs the binding.
 
-Background (why the bug existed, and the two cases where patching still matters):
+Background (why the bug existed, and the cases where patching still matters):
 
 The sql.js (WASM) backend is a *fallback*. ruflo prefers native `better-sqlite3`. The
 deeper `agentdb` packages still *declare* `better-sqlite3@^11.8.1` (no prebuilt for Node
-24+, won't compile against Node 26's V8) — but ruflo's `overrides` resolve that to v12 at
-install time, so the declared pin no longer decides the backend.
+24+, won't compile against Node 26's V8). Ruflo's ancestor override should decide the
+effective version, but nested repair installs must discover that override explicitly.
+On Node 26 they must also avoid v12 releases older than 12.10.0.
 
-| Node | ABI | ruflo ≥3.10.6 (override → v12) | ruflo <3.10.6 (declared ^11.8.1) |
-|------|-----|-------------------------------|----------------------------------|
+| Node | ABI | Compatible better-sqlite3 | Stale/incompatible resolution |
+|------|-----|---------------------------|-------------------------------|
 | ≤ 22 (LTS) | ≤ 127 | **native** | native |
-| 24 | 137 | **native (v12 prebuilt)** | ❌ WASM (buggy) |
-| 26 | 147 | **native (v12 prebuilt)** | ❌ WASM (buggy) |
+| 24 | 137 | **native (v12 prebuilt)** | v11 → ❌ WASM |
+| 26 | 147 | **native (v12.10+ prebuilt)** | v11 or v12.8/12.9 → ❌ WASM |
 
 **When the kit's natives heal still earns its keep:**
 - **npm ≥ 11.17** — npm's new `allow-scripts` blocks `better-sqlite3`'s build/`prebuild-install`
   during a global upgrade, so the native `.node` is skipped (the override resolves v12 but the
   prebuilt is never fetched) → WASM. **Always run `ak sync` after a global upgrade on npm
   ≥ 11.17**; its natives-heal step installs the binary even under `allow-scripts`.
+- **Node 26 with a stale v12 pin** — better-sqlite3 12.8/12.9 does not declare Node 26
+  support. `ak sync` raises it to `^12.10.0`, whose prebuilt supports ABI 147.
 - **ruflo < 3.10.6** on Node ≥24 — no override yet, so the agentdb copies fall to WASM;
   patch (or upgrade ruflo) to fix.
 - **agentic-qe** — a *separate* package with its own native-SQLite init that the override
   doesn't cover; `ak setup` repairs it.
 - As an idempotent re-assert if anything ever resolves a stale v11 binary.
 
 To check current state without changing anything: `ak status` (it reports
-`already native` when the override has done its job). Alternative to all of this: run ruflo
+`already native` when resolution and the binding are healthy). Alternative to all of this: run ruflo
 on **Node 22 LTS** (e.g. `mise install node@22`), where native resolves cleanly regardless.
 
 ### Hooks (learning + routing + workers)
@@ -523,4 +526,3 @@ Need to ... ?
 ├─ Set up agentic-qe in a repo       → ak setup   (opt-in)
 └─ Background analysis (long task)   → ruflo hooks worker dispatch -t <type>
 ```
-

docs/TROUBLESHOOTING.md

@@ -18,7 +18,7 @@ ak sync             # apply it
 | Symptom | What's happening | Fix |
 | --- | --- | --- |
 | Just upgraded ruflo/agentic-qe (`npm i -g …`) and things feel off | Upgrades re-resolve dependencies: native SQLite bindings and the aidefence package get dropped, and ruflo's helper auto-refresh regenerates the statusline without the footer | `ak sync` (this is its main job) |
-| `status` shows `natives … WASM fallback` | agentdb resolved a non-native better-sqlite3 — on this path **memory writes can silently vanish**. Root cause is npm ≥11.17 blocking install scripts during upgrades | `ak sync` installs the native binding |
+| `status` shows `natives … WASM fallback` | agentdb resolved a non-native better-sqlite3 — on this path **memory writes can silently vanish**. Common causes are npm ≥11.17 blocking install scripts during upgrades, or a stale better-sqlite3 ≤12.9 pin on Node 26 | `ak sync` selects a Node-compatible release and installs the native binding |
 | `status` shows `aidefence missing` | ruflo ≥3.28 stopped shipping `@claude-flow/aidefence` but `ruflo security defend` still imports it — injection defense is silently non-functional ([ruvnet/ruflo#2670](https://github.com/ruvnet/ruflo/issues/2670)) | `ak sync` reinstalls it; `ak x verify security` proves defend works (exit 1=threat / 0=clean) |
 | `status` shows oversized RVF store(s) | A runaway append after a hard exit grew a `.rvf` past the 2 GB cap (seen at ~277 GB once) | `ak sync` quarantines the oversized store; agentic-qe rebuilds it |
 | Statusline footer (🧠/🛡/🎓 lines) disappeared | `@claude-flow/cli`'s version-stamped helper auto-refresh pristine-copies `statusline.cjs` on the **first ruflo command after an upgrade** — including the statusline render itself | `ak sync` — it now triggers that refresh *first*, then re-injects, so the footer survives; `ak status` flags an armed wipe before it fires |

src/lib/heal.mjs

@@ -60,12 +60,19 @@ export async function ensureNativeBsq3(dir, { runner = run } = {}) {
   if (!pkgRoot) {
     // Derive the spec from THIS tree's own overrides/deps: a hardcoded `@^12` is
     // EOVERRIDE-rejected in a tree that pins better-sqlite3 (ruflo root pins
-    // 12.9.0, @claude-flow/cli pins ^12.9.0) — verified live. The declared spec
-    // installs clean and resolves a prebuilt.
-    await npmInstallInto(dir, `better-sqlite3@${deriveBsq3Spec(dir)}`, runner);
+    // 12.9.0, @claude-flow/cli pins ^12.9.0) — verified live. The derived spec
+    // also raises stale 12.x pins to Node 26's first supported release.
+    const installed = await npmInstallInto(dir, `better-sqlite3@${deriveBsq3Spec(dir)}`, runner);
     if (bsq3IsNative(dir)) return { ok: true, how: 'native installed' };
     pkgRoot = bsq3Root(dir);
-    if (!pkgRoot) return { ok: false, how: 'FAILED (better-sqlite3 not resolvable)' };
+    if (!pkgRoot) {
+      return {
+        ok: false,
+        how: installed.code === 0
+          ? 'FAILED (better-sqlite3 not resolvable after npm reported success)'
+          : failTail(installed),
+      };
+    }
   }
   // node-gyp compiling sqlite3 from source is slow; 300s truncated it mid-build.
   await runner('npm', ['run', 'install'], { cwd: pkgRoot, timeout: 600_000 });

src/lib/natives.mjs

@@ -71,19 +71,39 @@ export function rufloMemoryContexts() {
 const isPlainSemver = (v) =>
   typeof v === 'string' && /\d/.test(v) && !v.includes(':') && !v.trimStart().startsWith('$');
 
-/** The install spec for better-sqlite3 in `dir`, derived from that tree's OWN
- *  declarations so an npm `overrides` pin isn't fought with EOVERRIDE (verified
- *  live: `install better-sqlite3@^12` under @claude-flow/cli, which pins ^12.9.0,
- *  is rejected; the declared spec succeeds). Precedence: overrides →
- *  optionalDependencies → dependencies → fallback; the first PLAIN-semver value
- *  wins, non-semver forms are skipped. */
-export function deriveBsq3Spec(dir, fallback = '^12') {
-  const pkg = readJson(path.join(dir, 'package.json'), {}) ?? {};
-  for (const field of ['overrides', 'optionalDependencies', 'dependencies']) {
-    const v = pkg[field]?.['better-sqlite3'];
-    if (isPlainSemver(v)) return v;
+// better-sqlite3 first declares/supports Node 26 in 12.10.0. npm treats it as
+// optional through agentdb and silently removes 12.9.0 on Node 26 while still
+// exiting zero, so a stale exact ruflo override cannot be followed literally.
+const nodeCompatibleBsq3Spec = (spec, nodeMajor) => {
+  if (nodeMajor < 26) return spec;
+  const match = String(spec).match(/(\d+)\.(\d+)(?:\.\d+)?/);
+  if (match && Number(match[1]) === 12 && Number(match[2]) < 10) return '^12.10.0';
+  return spec;
+};
+
+/** The install spec for better-sqlite3 in `dir`, derived from the containing
+ *  npm tree. Ancestor overrides win over the nested package's dependency:
+ *  ruflo pins 12.x while agentdb still declares optional ^11.8.1, which npm
+ *  cannot build on Node 26. Installing from agentdb with the local declaration
+ *  therefore installs, fails, and removes 11.x even though the effective ruflo
+ *  tree requires 12.x. After overrides, use the target package's own optional
+ *  or regular dependency, then fallback. Non-semver forms are skipped. */
+export function deriveBsq3Spec(dir, fallback = '^12', nodeMajor = Number(process.versions.node.split('.')[0])) {
+  const start = path.resolve(dir);
+  let current = start;
+  for (;;) {
+    const override = readJson(path.join(current, 'package.json'), {})?.overrides?.['better-sqlite3'];
+    if (isPlainSemver(override)) return nodeCompatibleBsq3Spec(override, nodeMajor);
+    const parent = path.dirname(current);
+    if (parent === current) break;
+    current = parent;
+  }
+  const pkg = readJson(path.join(start, 'package.json'), {}) ?? {};
+  for (const field of ['optionalDependencies', 'dependencies']) {
+    const declared = pkg[field]?.['better-sqlite3'];
+    if (isPlainSemver(declared)) return nodeCompatibleBsq3Spec(declared, nodeMajor);
   }
-  return fallback;
+  return nodeCompatibleBsq3Spec(fallback, nodeMajor);
 }
 
 // A truthful load-test of the binding as node resolution finds it FROM `dir`: an

tests/kit/heal-natives.test.mjs

@@ -131,6 +131,17 @@ test('heal reports failure when better-sqlite3 stays unresolvable', async () =>
   fs.rmSync(root, { recursive: true, force: true });
 });
 
+test('heal reports the npm install error when the package stays unresolvable', async () => {
+  const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ak-heal-install-fail-'));
+  const runner = async () => ({ code: 1, stdout: '', stderr: 'npm error native build failed\n' });
+
+  const r = await ensureNativeBsq3(root, { runner });
+
+  assert.equal(r.ok, false);
+  assert.match(r.how, /native build failed/);
+  fs.rmSync(root, { recursive: true, force: true });
+});
+
 // ── FR-2: the install rung derives its spec from the tree (EOVERRIDE fix) ─────
 
 /** A better-sqlite3 that node resolution CANNOT find, in a dir that pins an npm
@@ -166,13 +177,13 @@ function eoverrideNpm(declared) {
 }
 
 test('ensureNativeBsq3 installs the tree-derived override spec, never the hardcoded ^12', async () => {
-  const dir = overrideDir('^12.9.0');
-  const { runner, specs } = eoverrideNpm('^12.9.0');
+  const dir = overrideDir('^12.10.0');
+  const { runner, specs } = eoverrideNpm('^12.10.0');
 
   const r = await ensureNativeBsq3(dir, { runner });
 
   assert.equal(r.ok, true, 'heal succeeds with the derived spec');
-  assert.ok(specs.includes('better-sqlite3@^12.9.0'), `derived spec used (saw ${JSON.stringify(specs)})`);
+  assert.ok(specs.includes('better-sqlite3@^12.10.0'), `derived spec used (saw ${JSON.stringify(specs)})`);
   assert.ok(!specs.includes('better-sqlite3@^12'), 'never falls into the EOVERRIDE-rejected hardcoded ^12');
   assert.equal(bsq3IsNative(dir), true);
   fs.rmSync(dir, { recursive: true, force: true });
@@ -189,15 +200,15 @@ test('healNatives heals @claude-flow/memory + /cli with each tree\'s derived spe
     const dir = path.join(nm, '@claude-flow', c);
     fs.mkdirSync(dir, { recursive: true });
     fs.writeFileSync(path.join(dir, 'package.json'),
-      JSON.stringify({ name: `@claude-flow/${c}`, overrides: { 'better-sqlite3': '^12.9.0' } }));
+      JSON.stringify({ name: `@claude-flow/${c}`, overrides: { 'better-sqlite3': '^12.10.0' } }));
   }
   _setGlobalRootForTest(g);
-  const { runner, specs } = eoverrideNpm('^12.9.0');
+  const { runner, specs } = eoverrideNpm('^12.10.0');
 
   const r = await healNatives({ runner });
 
   assert.ok(specs.length >= 2, 'installed into both contexts');
-  assert.ok(specs.every((s) => s === 'better-sqlite3@^12.9.0'), `only the derived spec (saw ${JSON.stringify(specs)})`);
+  assert.ok(specs.every((s) => s === 'better-sqlite3@^12.10.0'), `only the derived spec (saw ${JSON.stringify(specs)})`);
   assert.match(r.detail, /@claude-flow\/memory/);
   assert.match(r.detail, /@claude-flow\/cli/);
   assert.equal(r.ok, true);

tests/kit/natives-runtime.test.mjs

@@ -37,7 +37,7 @@ function fakeGlobalTree({ contexts = ['memory', 'cli'], pkg = {} } = {}) {
 test('deriveBsq3Spec honors an override pin exactly (12.9.0-style)', () => {
   const dir = tmp('ak-derive-');
   writePkg(dir, { overrides: { 'better-sqlite3': '12.9.0' } });
-  assert.equal(deriveBsq3Spec(dir), '12.9.0');
+  assert.equal(deriveBsq3Spec(dir, '^12', 24), '12.9.0');
   rm(dir);
 });
 
@@ -47,17 +47,29 @@ test('deriveBsq3Spec prefers overrides over optionalDependencies (EOVERRIDE avoi
     overrides: { 'better-sqlite3': '^12.9.0' },
     optionalDependencies: { 'better-sqlite3': '^12.0.0' },
   });
-  assert.equal(deriveBsq3Spec(dir), '^12.9.0');
+  assert.equal(deriveBsq3Spec(dir, '^12', 24), '^12.9.0');
   rm(dir);
 });
 
 test('deriveBsq3Spec falls back through optionalDependencies when no override', () => {
   const dir = tmp('ak-derive-');
   writePkg(dir, { optionalDependencies: { 'better-sqlite3': '^12.9.0' } });
-  assert.equal(deriveBsq3Spec(dir), '^12.9.0');
+  assert.equal(deriveBsq3Spec(dir, '^12', 24), '^12.9.0');
   rm(dir);
 });
 
+test('deriveBsq3Spec upgrades a stale ancestor 12.9 override for Node 26 before agentdb optional ^11', () => {
+  const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ak-bsq3-ancestor-'));
+  const agentdb = path.join(root, 'node_modules', 'agentdb');
+  fs.mkdirSync(agentdb, { recursive: true });
+  writePkg(root, { overrides: { 'better-sqlite3': '12.9.0' } });
+  writePkg(agentdb, { optionalDependencies: { 'better-sqlite3': '^11.8.1' } });
+
+  assert.equal(deriveBsq3Spec(agentdb, '^12', 26), '^12.10.0');
+  assert.equal(deriveBsq3Spec(agentdb, '^12', 24), '12.9.0');
+  fs.rmSync(root, { recursive: true, force: true });
+});
+
 test('deriveBsq3Spec skips a $ref override and a workspace protocol, falling back (EC-3)', () => {
   const refDir = tmp('ak-derive-');
   writePkg(refDir, {