Skip to content

fix(driver-turso): the remote face joins a large $and / $or as balanced halves, so hosted comment and attachment reads fit the depth limit (cloud#3485) - #30143

Merged
objectstack-fleet[bot] merged 3 commits into
mainfrom
claude/project-thread-mnya23
Nov 4, 2026
Merged

objectstack-fleet[bot] merged 3 commits into
mainfrom
claude/project-thread-mnya23

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Sam · project thread

Refs objectstack-ai/cloud#3485 · follows #29888 (PR #29911)

What changes

TursoDriver's remote face (RemoteTransport) now joins a large $and / $or group as balanced halves, by the same rule the local face's knex path has used since #29888:

  • packages/drivers/driver-sql/src/sql-driver-balanced-group.ts: joinBalancedGroup(members, joiner), the text twin of emitBalancedGroup. At most FLAT_GROUP_MAX_MEMBERS (8) members join flat, byte-identical to members.join(joiner). Above that, the two halves join once, each in its own parentheses, recursively. Exported from @objectstack/driver-sql with FLAT_GROUP_MAX_MEMBERS, so the rule has one source. This follows the precedent of recoverUnencodedJsonText.
  • packages/drivers/driver-turso/src/remote-transport.ts: the $and arm and the $or arm of compileWhereSQL call it in place of a flat join. Nothing else in the compiler changes: not the TRUE/FALSE identities, the leaf arms, the ReadableIds semi-join or the tenant wall.
  • packages/drivers/driver-turso/src/turso-remote-balanced-group.test.ts: the new test, described under Verification.
  • Changeset: patch for driver-turso and driver-sql.

Why

On every hosted environment with HotCRM installed, production included, every read of sys_comment and sys_attachment answers 500 DATABASE_ERROR. Record pages show no comments and no attachments. The tenant database refuses the statement with "Expression tree is too large (maximum depth 100)" (container log, cloud#3485, 2026-11-04 13:30Z).

#29911 balanced the parent-record gate's $or in SqlDriver's knex path. A hosted environment does not read through that path. Its driver is TursoDriver with a libsql:// url, so the remote face compiles every read with its own compiler, and that compiler still joined each group flat (subClauses.join(' OR ')). The comment gate (plugin-audit) and the attachment gate (service-storage) OR one ReadableIds branch per registered parent object. Joined flat, that is one expression level per parent: about 80 on a bare environment, and more than 100 once an app is installed. The #29888 dogfood test proved the fix on sql.js, the local face, so it passed while the hosted face stayed broken.

The gates' shape, and therefore the authorization boundary, does not change. AND and OR are associative and commutative over SQL's three-valued logic, so the regrouped expression selects exactly the rows the flat one did. The members keep their order, and so do their bind values.

On depth: the gate's OR now adds 7 + ceil(log2(N / 8)) levels at most for N branches, for example 12 at 130 and 18 at 10,000. Its total depth is that plus the deepest single branch, which is the parent's own scope and does not grow with the number of parents. The other shapes the card suggested (UNION, CTE or id set) would mean a new filter-protocol construct across every driver (ADR-0139 territory) and would depend on hosted limits nobody has measured, such as the compound-SELECT term limit. This change uses only the binary OR / AND every face already emits.

Verification

Readings taken on this branch (objectstack-ai/objectstack, head of claude/project-thread-mnya23) in a cloud container.

The instrument. The stub is better-sqlite3 (makeLibsqlSqliteStub), which allows 1000 levels and exposes no sqlite3_limit. The test re-prepares the statement the remote face sent with its top-level WHERE under 900 AND 1 levels, so it fits exactly when the WHERE fits 100. A binary search over that spine reads a WHERE's depth exactly. Both are calibrated on raw SQL first: a 99-term OR chain fits and a 100-term chain is refused, and k terms read as k + 1 levels for k ∈ {1, 2, 9, 77, 300}. The gate-shaped reads are also served with the budget applied to every SELECT the driver sends.

turso-remote-balanced-group.test.ts (18 cases, remote face):

  • Groups of ≤ 8 compile byte-identical, for both $or and $and. A 9-member group is pinned as two halves (5 + 4), with bind values in member order.
  • $or of 9, 130 and 1,200 members, and $and of 130, select the expected rows and fit the budget. A large $or under $not inside an $and keeps its answer.
  • The gate's own shape: 130 public parents plus a three-level master chain (line → quote → deal), each branch { object_name: P, record_id: { $in: ReadableIds.compose(…) } } with a sharing-shaped scope, and one refused parent answered $in: []. The test runs the card's reads: unfiltered, sorted by created_at, filtered by thread_id, by an unopenable thread, by the parent pair, and by body. Each is served under the budget with exactly the rows the fixture oracle allows: rows on closed records, a foreign-tenant record, a deleted record, an unlisted object or no parent stay hidden. count and aggregate agree with find.
gate WHERE depth (134 branches) 1,200-member $or test result
ablation (remote-transport.ts restored to 28ce3d1b) 162 refused 13 of 18 red: every gate read and every large group fails with Expression tree is too large
this branch 38 (deepest single branch alone: 29) 14 18 of 18 green

Suites (one run after a full DTS build of the driver-turso closure): @objectstack/driver-turso 97 files, 2,514 passed (34 skipped). @objectstack/driver-sql 238 files, 3,859 passed (216 skipped). typecheck is clean on both packages. eslint is clean on the four files. check:empty-changeset, check:changeset-fixed, check:cross-package-test-inputs and check:test-source-alias are green.

End to end in cloud. The cloud pin round carrying this change adds a hosted-composition e2e: ArtifactKernelFactory with the hosted slate, served over HTTP, on TursoDriver's remote face over a native @libsql/client file: database, with 75 extra objects. It reads the card's HTTP queries under the same budget. That PR links here.

Not changed, and noted for the lane: service-analytics read-scope-sql.ts joins $and / $or flat too, as #29911's report recorded. It compiles analytics read scopes, not these gated reads.

Clause-②: no
Metadata-protocol: none

Human Minimum

  • 1. Schemas start as Zod; packages/spec holds no runtime logic.
  • 2. Every name I added uses the casing for its kind.
  • 3. No formatter ran; pnpm lint passes.
  • 4. One issue, one branch off an up-to-date main, one pull request.
  • 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset.
  • 6. packages/spec touched: its generated artifacts are regenerated and committed. (Not touched.)
  • 7. No hand edits to content/docs/references/, packages/*/CHANGELOG.md or content/docs/releases/.
  • 8. Title, body and comments are in English.
  • The Before Submitting list in CONTRIBUTING.md is done.

Generated by Claude Code

🤖 Generated with Claude Code

https://claude.ai/code/session_01QUs3BtjC5etzjN1qcjYgiQ

…alanced halves (cloud#3485)

Claude-Session: https://claude.ai/code/session_01QUs3BtjC5etzjN1qcjYgiQ
Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUs3BtjC5etzjN1qcjYgiQ
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation tests tooling labels Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

2 anchor(s) derived from 2 changed package(s); no hand-written page names any of them. ⚠️ 1 changed file(s) yielded no anchor (packages/drivers/driver-sql/src/index.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/drivers/driver-sql/src/index.ts) — pages documenting those are invisible to this run
  • the SDK route bridge reached 61 of 219 client-bound route-ledger rows — the other 158 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 158: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 103 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 16 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 55337f4ce35dbe62434b23517547a0b5052b936c → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 0a6a579a9707b9738e6fc8e531d169e38e543eb4 — the merge of head d16d50d82aea7cacd72846c4de1cc6d81b1cf77a into base 55337f4ce35dbe62434b23517547a0b5052b936c, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 0a6a579a9707b9738e6fc8e531d169e38e543eb4 && git checkout 0a6a579a9707b9738e6fc8e531d169e38e543eb4
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 55337f4ce35dbe62434b23517547a0b5052b936c d16d50d82aea7cacd72846c4de1cc6d81b1cf77a && git checkout -B drift-repro 55337f4ce35dbe62434b23517547a0b5052b936c && git merge --no-ff d16d50d82aea7cacd72846c4de1cc6d81b1cf77a

node scripts/docs-audit/affected-docs.mjs --json 55337f4ce35dbe62434b23517547a0b5052b936c

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

repo:cloud#2 · os-zhuang · session_0123kRqqdpAJWYMgsgdAJX4G
2026-11-04T16:08Z

席位核对 — ACCEPT at d16d50d

  • The fix is in the right place. RemoteTransport.compileWhereSQL is the face hosted environments read through. The $and and $or arms now call joinBalancedGroup, a text twin of emitBalancedGroup exported from driver-sql, so there is still one rule. Nothing else in the compiler moves: the TRUE/FALSE identities, the leaf arms, the ReadableIds semi-join and the tenant wall are all unchanged.
  • Groups of 8 or fewer stay byte-identical. Above 8, the members are split into balanced halves with each half parenthesised. Member order, and so bind order, is unchanged. AND/OR are associative under three-valued logic, so the row set is the same. This is not an authorization change: the gate members are unchanged, and the test checks exact fail-closed row sets (closed, foreign tenant, deleted, unlisted object, refused parent, null parent).
  • The instrument is credible. The depth is read by bisection under a 900-level spine and calibrated on raw SQL at 99/100. On the ablation, 13 of 18 tests are red with the production error. The gate's depth drops from 162 to 38.
  • Not a blocker: service-analytics read-scope-sql.ts still joins flat. It does not serve these gated reads, so it is noted in the PR and can be a follow-up.

The seat marks this PR ready and arms automerge; it merges when CI is green. The cloud pin round then carries the hosted e2e and Closes cloud#3485.


Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review November 4, 2026 16:11
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge November 4, 2026 16:11
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit dcb5fb3 Nov 4, 2026
49 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/project-thread-mnya23 branch November 4, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant