diff --git a/claude/ruflo-reference-full.md b/claude/ruflo-reference-full.md index 100747ca..7180a393 100644 --- a/claude/ruflo-reference-full.md +++ b/claude/ruflo-reference-full.md @@ -107,31 +107,34 @@ doubt — see diagnostic table below. ### Node version compatibility (historically the ROOT cause of the WASM bugs) -> **Resolved upstream in ruflo v3.10.6** ([ruvnet/ruflo#2219](https://github.com/ruvnet/ruflo/issues/2219)). -> ruflo now ships an npm `overrides` entry forcing `better-sqlite3 ≥12.8.0` (which has -> Node 20–26 prebuilts) across the agentdb copies, with a CI guard. So on ruflo ≥3.10.6 -> the WASM fallback **no longer happens by default**, even on Node 24/26, and the override -> lives in ruflo's own `package.json` — an `npm i -g ruflo` upgrade keeps it, it does not -> wipe it. the kit's natives heal (part of `ak sync`) is now a **safety net**, not a requirement. +> **Partly resolved upstream in ruflo v3.10.6** +> ([ruvnet/ruflo#2219](https://github.com/ruvnet/ruflo/issues/2219)). Ruflo added an +> npm override forcing `better-sqlite3 ≥12.8.0` across the agentdb copies, which fixes +> Node 24. Node 26 support starts at better-sqlite3 12.10.0; an exact 12.8/12.9 +> override can still fall back to WASM. The kit's natives heal (part of `ak sync`) +> raises a stale v12 pin to a Node-compatible range and installs the binding. -Background (why the bug existed, and the two cases where patching still matters): +Background (why the bug existed, and the cases where patching still matters): The sql.js (WASM) backend is a *fallback*. ruflo prefers native `better-sqlite3`. The deeper `agentdb` packages still *declare* `better-sqlite3@^11.8.1` (no prebuilt for Node -24+, won't compile against Node 26's V8) — but ruflo's `overrides` resolve that to v12 at -install time, so the declared pin no longer decides the backend. +24+, won't compile against Node 26's V8). Ruflo's ancestor override should decide the +effective version, but nested repair installs must discover that override explicitly. +On Node 26 they must also avoid v12 releases older than 12.10.0. -| Node | ABI | ruflo ≥3.10.6 (override → v12) | ruflo <3.10.6 (declared ^11.8.1) | -|------|-----|-------------------------------|----------------------------------| +| Node | ABI | Compatible better-sqlite3 | Stale/incompatible resolution | +|------|-----|---------------------------|-------------------------------| | ≤ 22 (LTS) | ≤ 127 | **native** | native | -| 24 | 137 | **native (v12 prebuilt)** | ❌ WASM (buggy) | -| 26 | 147 | **native (v12 prebuilt)** | ❌ WASM (buggy) | +| 24 | 137 | **native (v12 prebuilt)** | v11 → ❌ WASM | +| 26 | 147 | **native (v12.10+ prebuilt)** | v11 or v12.8/12.9 → ❌ WASM | **When the kit's natives heal still earns its keep:** - **npm ≥ 11.17** — npm's new `allow-scripts` blocks `better-sqlite3`'s build/`prebuild-install` during a global upgrade, so the native `.node` is skipped (the override resolves v12 but the prebuilt is never fetched) → WASM. **Always run `ak sync` after a global upgrade on npm ≥ 11.17**; its natives-heal step installs the binary even under `allow-scripts`. +- **Node 26 with a stale v12 pin** — better-sqlite3 12.8/12.9 does not declare Node 26 + support. `ak sync` raises it to `^12.10.0`, whose prebuilt supports ABI 147. - **ruflo < 3.10.6** on Node ≥24 — no override yet, so the agentdb copies fall to WASM; patch (or upgrade ruflo) to fix. - **agentic-qe** — a *separate* package with its own native-SQLite init that the override @@ -139,7 +142,7 @@ install time, so the declared pin no longer decides the backend. - As an idempotent re-assert if anything ever resolves a stale v11 binary. To check current state without changing anything: `ak status` (it reports -`already native` when the override has done its job). Alternative to all of this: run ruflo +`already native` when resolution and the binding are healthy). Alternative to all of this: run ruflo on **Node 22 LTS** (e.g. `mise install node@22`), where native resolves cleanly regardless. ### Hooks (learning + routing + workers) @@ -523,4 +526,3 @@ Need to ... ? ├─ Set up agentic-qe in a repo → ak setup (opt-in) └─ Background analysis (long task) → ruflo hooks worker dispatch -t ``` - diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index 474b3549..44a4d74d 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -18,7 +18,7 @@ ak sync # apply it | Symptom | What's happening | Fix | | --- | --- | --- | | Just upgraded ruflo/agentic-qe (`npm i -g …`) and things feel off | Upgrades re-resolve dependencies: native SQLite bindings and the aidefence package get dropped, and ruflo's helper auto-refresh regenerates the statusline without the footer | `ak sync` (this is its main job) | -| `status` shows `natives … WASM fallback` | agentdb resolved a non-native better-sqlite3 — on this path **memory writes can silently vanish**. Root cause is npm ≥11.17 blocking install scripts during upgrades | `ak sync` installs the native binding | +| `status` shows `natives … WASM fallback` | agentdb resolved a non-native better-sqlite3 — on this path **memory writes can silently vanish**. Common causes are npm ≥11.17 blocking install scripts during upgrades, or a stale better-sqlite3 ≤12.9 pin on Node 26 | `ak sync` selects a Node-compatible release and installs the native binding | | `status` shows `aidefence missing` | ruflo ≥3.28 stopped shipping `@claude-flow/aidefence` but `ruflo security defend` still imports it — injection defense is silently non-functional ([ruvnet/ruflo#2670](https://github.com/ruvnet/ruflo/issues/2670)) | `ak sync` reinstalls it; `ak x verify security` proves defend works (exit 1=threat / 0=clean) | | `status` shows oversized RVF store(s) | A runaway append after a hard exit grew a `.rvf` past the 2 GB cap (seen at ~277 GB once) | `ak sync` quarantines the oversized store; agentic-qe rebuilds it | | Statusline footer (🧠/🛡/🎓 lines) disappeared | `@claude-flow/cli`'s version-stamped helper auto-refresh pristine-copies `statusline.cjs` on the **first ruflo command after an upgrade** — including the statusline render itself | `ak sync` — it now triggers that refresh *first*, then re-injects, so the footer survives; `ak status` flags an armed wipe before it fires | diff --git a/src/lib/heal.mjs b/src/lib/heal.mjs index 41392e59..b632b289 100644 --- a/src/lib/heal.mjs +++ b/src/lib/heal.mjs @@ -60,12 +60,19 @@ export async function ensureNativeBsq3(dir, { runner = run } = {}) { if (!pkgRoot) { // Derive the spec from THIS tree's own overrides/deps: a hardcoded `@^12` is // EOVERRIDE-rejected in a tree that pins better-sqlite3 (ruflo root pins - // 12.9.0, @claude-flow/cli pins ^12.9.0) — verified live. The declared spec - // installs clean and resolves a prebuilt. - await npmInstallInto(dir, `better-sqlite3@${deriveBsq3Spec(dir)}`, runner); + // 12.9.0, @claude-flow/cli pins ^12.9.0) — verified live. The derived spec + // also raises stale 12.x pins to Node 26's first supported release. + const installed = await npmInstallInto(dir, `better-sqlite3@${deriveBsq3Spec(dir)}`, runner); if (bsq3IsNative(dir)) return { ok: true, how: 'native installed' }; pkgRoot = bsq3Root(dir); - if (!pkgRoot) return { ok: false, how: 'FAILED (better-sqlite3 not resolvable)' }; + if (!pkgRoot) { + return { + ok: false, + how: installed.code === 0 + ? 'FAILED (better-sqlite3 not resolvable after npm reported success)' + : failTail(installed), + }; + } } // node-gyp compiling sqlite3 from source is slow; 300s truncated it mid-build. await runner('npm', ['run', 'install'], { cwd: pkgRoot, timeout: 600_000 }); diff --git a/src/lib/natives.mjs b/src/lib/natives.mjs index 8e58e3f3..4ae0cf01 100644 --- a/src/lib/natives.mjs +++ b/src/lib/natives.mjs @@ -71,19 +71,39 @@ export function rufloMemoryContexts() { const isPlainSemver = (v) => typeof v === 'string' && /\d/.test(v) && !v.includes(':') && !v.trimStart().startsWith('$'); -/** The install spec for better-sqlite3 in `dir`, derived from that tree's OWN - * declarations so an npm `overrides` pin isn't fought with EOVERRIDE (verified - * live: `install better-sqlite3@^12` under @claude-flow/cli, which pins ^12.9.0, - * is rejected; the declared spec succeeds). Precedence: overrides → - * optionalDependencies → dependencies → fallback; the first PLAIN-semver value - * wins, non-semver forms are skipped. */ -export function deriveBsq3Spec(dir, fallback = '^12') { - const pkg = readJson(path.join(dir, 'package.json'), {}) ?? {}; - for (const field of ['overrides', 'optionalDependencies', 'dependencies']) { - const v = pkg[field]?.['better-sqlite3']; - if (isPlainSemver(v)) return v; +// better-sqlite3 first declares/supports Node 26 in 12.10.0. npm treats it as +// optional through agentdb and silently removes 12.9.0 on Node 26 while still +// exiting zero, so a stale exact ruflo override cannot be followed literally. +const nodeCompatibleBsq3Spec = (spec, nodeMajor) => { + if (nodeMajor < 26) return spec; + const match = String(spec).match(/(\d+)\.(\d+)(?:\.\d+)?/); + if (match && Number(match[1]) === 12 && Number(match[2]) < 10) return '^12.10.0'; + return spec; +}; + +/** The install spec for better-sqlite3 in `dir`, derived from the containing + * npm tree. Ancestor overrides win over the nested package's dependency: + * ruflo pins 12.x while agentdb still declares optional ^11.8.1, which npm + * cannot build on Node 26. Installing from agentdb with the local declaration + * therefore installs, fails, and removes 11.x even though the effective ruflo + * tree requires 12.x. After overrides, use the target package's own optional + * or regular dependency, then fallback. Non-semver forms are skipped. */ +export function deriveBsq3Spec(dir, fallback = '^12', nodeMajor = Number(process.versions.node.split('.')[0])) { + const start = path.resolve(dir); + let current = start; + for (;;) { + const override = readJson(path.join(current, 'package.json'), {})?.overrides?.['better-sqlite3']; + if (isPlainSemver(override)) return nodeCompatibleBsq3Spec(override, nodeMajor); + const parent = path.dirname(current); + if (parent === current) break; + current = parent; + } + const pkg = readJson(path.join(start, 'package.json'), {}) ?? {}; + for (const field of ['optionalDependencies', 'dependencies']) { + const declared = pkg[field]?.['better-sqlite3']; + if (isPlainSemver(declared)) return nodeCompatibleBsq3Spec(declared, nodeMajor); } - return fallback; + return nodeCompatibleBsq3Spec(fallback, nodeMajor); } // A truthful load-test of the binding as node resolution finds it FROM `dir`: an diff --git a/tests/kit/heal-natives.test.mjs b/tests/kit/heal-natives.test.mjs index fdd0bba0..d6f8cb3f 100644 --- a/tests/kit/heal-natives.test.mjs +++ b/tests/kit/heal-natives.test.mjs @@ -131,6 +131,17 @@ test('heal reports failure when better-sqlite3 stays unresolvable', async () => fs.rmSync(root, { recursive: true, force: true }); }); +test('heal reports the npm install error when the package stays unresolvable', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ak-heal-install-fail-')); + const runner = async () => ({ code: 1, stdout: '', stderr: 'npm error native build failed\n' }); + + const r = await ensureNativeBsq3(root, { runner }); + + assert.equal(r.ok, false); + assert.match(r.how, /native build failed/); + fs.rmSync(root, { recursive: true, force: true }); +}); + // ── FR-2: the install rung derives its spec from the tree (EOVERRIDE fix) ───── /** A better-sqlite3 that node resolution CANNOT find, in a dir that pins an npm @@ -166,13 +177,13 @@ function eoverrideNpm(declared) { } test('ensureNativeBsq3 installs the tree-derived override spec, never the hardcoded ^12', async () => { - const dir = overrideDir('^12.9.0'); - const { runner, specs } = eoverrideNpm('^12.9.0'); + const dir = overrideDir('^12.10.0'); + const { runner, specs } = eoverrideNpm('^12.10.0'); const r = await ensureNativeBsq3(dir, { runner }); assert.equal(r.ok, true, 'heal succeeds with the derived spec'); - assert.ok(specs.includes('better-sqlite3@^12.9.0'), `derived spec used (saw ${JSON.stringify(specs)})`); + assert.ok(specs.includes('better-sqlite3@^12.10.0'), `derived spec used (saw ${JSON.stringify(specs)})`); assert.ok(!specs.includes('better-sqlite3@^12'), 'never falls into the EOVERRIDE-rejected hardcoded ^12'); assert.equal(bsq3IsNative(dir), true); fs.rmSync(dir, { recursive: true, force: true }); @@ -189,15 +200,15 @@ test('healNatives heals @claude-flow/memory + /cli with each tree\'s derived spe const dir = path.join(nm, '@claude-flow', c); fs.mkdirSync(dir, { recursive: true }); fs.writeFileSync(path.join(dir, 'package.json'), - JSON.stringify({ name: `@claude-flow/${c}`, overrides: { 'better-sqlite3': '^12.9.0' } })); + JSON.stringify({ name: `@claude-flow/${c}`, overrides: { 'better-sqlite3': '^12.10.0' } })); } _setGlobalRootForTest(g); - const { runner, specs } = eoverrideNpm('^12.9.0'); + const { runner, specs } = eoverrideNpm('^12.10.0'); const r = await healNatives({ runner }); assert.ok(specs.length >= 2, 'installed into both contexts'); - assert.ok(specs.every((s) => s === 'better-sqlite3@^12.9.0'), `only the derived spec (saw ${JSON.stringify(specs)})`); + assert.ok(specs.every((s) => s === 'better-sqlite3@^12.10.0'), `only the derived spec (saw ${JSON.stringify(specs)})`); assert.match(r.detail, /@claude-flow\/memory/); assert.match(r.detail, /@claude-flow\/cli/); assert.equal(r.ok, true); diff --git a/tests/kit/natives-runtime.test.mjs b/tests/kit/natives-runtime.test.mjs index 5ed36eda..85d0fd3c 100644 --- a/tests/kit/natives-runtime.test.mjs +++ b/tests/kit/natives-runtime.test.mjs @@ -37,7 +37,7 @@ function fakeGlobalTree({ contexts = ['memory', 'cli'], pkg = {} } = {}) { test('deriveBsq3Spec honors an override pin exactly (12.9.0-style)', () => { const dir = tmp('ak-derive-'); writePkg(dir, { overrides: { 'better-sqlite3': '12.9.0' } }); - assert.equal(deriveBsq3Spec(dir), '12.9.0'); + assert.equal(deriveBsq3Spec(dir, '^12', 24), '12.9.0'); rm(dir); }); @@ -47,17 +47,29 @@ test('deriveBsq3Spec prefers overrides over optionalDependencies (EOVERRIDE avoi overrides: { 'better-sqlite3': '^12.9.0' }, optionalDependencies: { 'better-sqlite3': '^12.0.0' }, }); - assert.equal(deriveBsq3Spec(dir), '^12.9.0'); + assert.equal(deriveBsq3Spec(dir, '^12', 24), '^12.9.0'); rm(dir); }); test('deriveBsq3Spec falls back through optionalDependencies when no override', () => { const dir = tmp('ak-derive-'); writePkg(dir, { optionalDependencies: { 'better-sqlite3': '^12.9.0' } }); - assert.equal(deriveBsq3Spec(dir), '^12.9.0'); + assert.equal(deriveBsq3Spec(dir, '^12', 24), '^12.9.0'); rm(dir); }); +test('deriveBsq3Spec upgrades a stale ancestor 12.9 override for Node 26 before agentdb optional ^11', () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ak-bsq3-ancestor-')); + const agentdb = path.join(root, 'node_modules', 'agentdb'); + fs.mkdirSync(agentdb, { recursive: true }); + writePkg(root, { overrides: { 'better-sqlite3': '12.9.0' } }); + writePkg(agentdb, { optionalDependencies: { 'better-sqlite3': '^11.8.1' } }); + + assert.equal(deriveBsq3Spec(agentdb, '^12', 26), '^12.10.0'); + assert.equal(deriveBsq3Spec(agentdb, '^12', 24), '12.9.0'); + fs.rmSync(root, { recursive: true, force: true }); +}); + test('deriveBsq3Spec skips a $ref override and a workspace protocol, falling back (EC-3)', () => { const refDir = tmp('ak-derive-'); writePkg(refDir, {