Skip to content

chore(schema): re-vendor manifest schema for cross-app storage imports (+ fork-manifest test) - #868

Merged
ZacxDev merged 5 commits into
mainfrom
chore/revendor-schema-imports
Nov 3, 2026
Merged

ZacxDev merged 5 commits into
mainfrom
chore/revendor-schema-imports

Conversation

@ZacxDev

@ZacxDev ZacxDev commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

What

Re-vendors schema/app-block.manifest.schema.json from the live canonical schema (https://civitai.com/schemas/app-block/v1.json, civitai web v5.1.425) using the repo's own mechanism, scripts/revendor-canonical-schema.sh. It also adds a test that uses a real fork manifest as its fixture.

Why

civitai web v5.1.425 added cross-app storage imports. The embedded schema predates them, so civitai app validate on a real fork manifest fails with two findings the server does not raise:

scopes[6]: value must be one of …            (apps:storage:import is not in the enum)
storage: additional properties 'imports' not allowed

The only way to submit a fork today is --skip-validate.

Schema changes (canonical vs. current main)

  • storage.imports: a new array, 1 to 8 items. Each item is {from, collections, access}, all three required.
    • from is a blockId: ^[a-z][a-z0-9-]{1,38}[a-z0-9]$, length 3 to 40.
    • collections is 1 to 16 unique collection names.
    • access is an enum: read | readwrite.
  • storage.collections.<name>.exportable: a new boolean.
  • Scope enum: adds apps:storage:import.
  • storage previously required collections. It now uses anyOf: [required collections, required imports].
  • Description text: the storage description and the scopeJustifications list of sensitive scopes now name apps:storage:import.

Drift vs. the bot PR #867

#867 (head 90fbe193) is byte-identical to current main (cmp exit 0 on the schema file). Its 56 added lines were measured against its older base a9ba713, and #865 has since brought that content onto main. So #867 has nothing left to land, and every change listed above is beyond #867. This PR supersedes it.

Pattern-gloss ledger

The new imports[].from pattern has no gloss in patternRules. It is therefore recorded as owed in internal/validate/pattern_gloss_owed.json. This is the same sorted row that the revendor workflow's -update-gloss-ledger step writes. The file was written to that format by hand, because tests are not run locally here, and TestPatternRulesCoverTheVendoredSchema passing in CI is the check on it.

Test

internal/validate/storage_imports_test.go uses the fixture internal/validate/testdata/fork-manifest.json. That fixture is the real fork seed manifest minus iframe.src. iframe.src is server-owned and the CLI refusing it is intended; the seed carrying it is a separate platform fix. The test asserts:

  • (a) the fork manifest validates with zero errors;
  • (b) imports[0].access = "write" gives exactly one error, at field storage.imports[0].access, with message storage.imports[0].access: value must be one of 'read', 'readwrite';
  • (b) an import missing from gives exactly one error, at storage.imports[0], with message storage.imports[0]: missing property 'from';
  • (c) a minimal manifest that is still coherent validates clean. It declares apps:storage + apps:storage:import (with a justification), one collection with exportable: false, and one import.

Every expected message is a literal string. None is read from the schema.

Red/green matrix (CI)

commit content CI
eaaa477 test only, main's schema red: https://github.com/civitai/cli/actions/runs/41675032766. build-test fails on exactly the 4 new tests: TestForkManifestWithStorageImportsValidates, TestStorageImportRejectsUnknownAccess, TestStorageImportRejectsMissingFrom, TestStorageImportScopeValidates. The failures are the reported scopes[N] enum error and additional properties 'imports' (plus 'exportable' for (c)). Every other job is green.
ff6993a + re-vendor + ledger row green: https://github.com/civitai/cli/actions/runs/41675338616, and schema-drift is green: https://github.com/civitai/cli/actions/runs/41675338664

On attempt 1 of the green run, template-page-money failed: src/e2e.test.tsx "estimate -> consent -> submit -> poll -> succeeded" timed out at 6115 ms against a 5000 ms limit. That run was slow across the board (tests 22.8s vs 10.1s, setup 1.72s vs 0.71s compared with the red run). That job tests the JS scaffold template and does not touch the schema. Re-running the failed job passed (attempt 2).

Server-rule parity for imports (added after review)

The server's rules come from civitai main src/shared/constants/block-storage.constants.ts (appStorageScopeErrors and parseImports) and packages/civitai-shared/src/sensitive-block-scopes.ts. Messages are the server's text.

  • storagemanifest.go storageScopeRuleChecks: the existing "nothing to reach" rule fired on len(names)==0 && hasBase. It now also requires no imports, matching the server's names.length === 0 && sources.length === 0. An imports-only app (no collections of its own, apps:storage + apps:storage:import) now validates. Three import rules are added, in the server's order:
    • storage imports from <src> but scopes does not include apps:storage — add it, or remove storage (imports only, no collections);
    • storage imports from <src>, which requires the apps:storage:import scope — add it to scopes with a scopeJustifications entry;
    • scopes includes apps:storage:import but storage declares no imports — declare them under storage.imports, or remove the scope.
  • storagemanifest.go storageImportChecks covers the only two parseImports refusals the schema cannot express:
    • storage.imports[i].from names this app itself — declare its collections instead;
    • storage.imports[i] imports from "<src>" again — declare one import per app. Like the server, this compares only against imports it KEPT, so an entry the schema already refuses never makes a later one a "duplicate".
    • Every other parseImports refusal (array bounds, keys, from pattern, access enum, collection names/bounds/uniqueness) is already in the schema.
  • semantic.go: apps:storage:import is added to SENSITIVE_BLOCK_SCOPES. It is in the server set on both main and release, which now holds 11 scopes.
  • storagescope.go source advisory, rule 1:
    • A call with a literal from option, e.g. app.storage.collection('kv', { from: 'parent' }), is now checked against the import from that source:
      • if no import names that source, the advisory is located at storage.imports;
      • if the import does not list the collection, it is located at storage.imports[i].collections.
      • Both are refused at runtime with import_not_declared.
    • Before this change, such a call was wrongly warned "storage.collections does not declare 'kv'".
    • A call with no from keeps today's own-collection check.
    • A computed from ({ from: PARENT }, { from }) cannot be resolved statically, so it is not flagged.
    • An options variable (collection('kv', opts)) is still read as an own collection. This is a documented residual.
  • The field-coverage corpus and the field/pair ledgers gained fixtures and rows for each new finding.

Red/green matrix for the parity change (CI)

commit content CI
a1ab6d0 tests only red: https://github.com/civitai/cli/actions/runs/41678441179. build-test fails on TestStorageImportsOnlyValidates (with the OLD guard's own message: scopes includes apps:storage but storage declares no collections), TestStorageImportScopeRules (3 subtests), TestStorageImportParseRefusals (self-import, duplicate), TestStorageImportScopeIsSensitive, TestStorageImportedCollectionAdvisory (6 subtests), TestIsSensitiveBlockScope, TestEveryCheckEmitsAField, TestEveryFindingCarriesItsDocumentedField, TestPairRuleNamesTheMissingField. All other jobs are green.
51dc961 + the fix build-test green, lint red: literalArg was unused. https://github.com/civitai/cli/actions/runs/41678846649
c8724b0 + drop the unused wrapper all green: https://github.com/civitai/cli/actions/runs/41679154652

The accept twins pass on both sides by design, so they are invariant guards, not regression proof: collections plus imports with both scopes, no duplicate of a refused entry, and the silent from cases.

Until a CLI release and the seed fix

Users validating or submitting a fork manifest with a released CLI still need --skip-validate. A CLI release with this PR is not enough on its own for a real fork seed. The seed also carries the server-owned iframe.src, which the CLI refuses on purpose. A separate civitai PR that drops iframe.src from fork seeds is in flight. A fork seed validates clean only once both have shipped.

🤖 Generated with Claude Code

ZacxDev and others added 2 commits November 3, 2026 10:30
Adds the fork seed manifest (minus the server-owned iframe.src) as a
fixture and asserts: it validates clean; imports[0].access "write" and an
import missing "from" are each refused with their exact message and field;
a coherent manifest declaring apps:storage:import validates.

Expected RED on this commit: the embedded schema predates cross-app
storage imports (civitai web v5.1.425). The re-vendor follows.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Regenerated with scripts/revendor-canonical-schema.sh from the live
canonical schema (civitai web v5.1.425). Adds storage.imports
(from / collections / access read|readwrite), collection-level
`exportable`, the apps:storage:import scope, and relaxes storage's
required `collections` to anyOf collections|imports.

The new imports[].from pattern has no gloss in patternRules yet, so it is
recorded as owed in internal/validate/pattern_gloss_owed.json, the same
row the revendor workflow's -update-gloss-ledger step writes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ZacxDev and others added 3 commits November 3, 2026 10:51
…all advisory

Mirrors civitai appStorageScopeErrors / parseImports for imports:
- an imports-only manifest (no own collections, both scopes) validates;
- imports without apps:storage, imports without apps:storage:import, and
  apps:storage:import with no imports are each refused with the server's text;
- an import from the app itself and a second import from one source are refused;
- apps:storage:import is a sensitive scope (needs a justification);
- a source call with a literal `from` is checked against that import, not
  against storage.collections.

Corpus fixtures and field/pair ledger rows for the new findings.

Expected RED on this commit; the fix follows.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- storageScopeRuleChecks: the three import rules of appStorageScopeErrors
  (imports-only without apps:storage; imports without apps:storage:import;
  apps:storage:import with no imports), and "apps:storage with nothing to
  reach" now also requires no imports, so an imports-only app validates.
- storageImportChecks: parseImports' two refusals the schema admits, an
  import from the app itself and a second import from one source (compared,
  like the server, against the imports it kept).
- SENSITIVE_BLOCK_SCOPES gains apps:storage:import (server set, main and
  release).
- storageScopeChecks: a collection call with a literal `from` option is
  checked against the import from that source, not storage.collections.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lint (unused): every caller reads literalArgEnd now.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@ZacxDev
ZacxDev marked this pull request as ready for review November 3, 2026 16:58
@ZacxDev
ZacxDev merged commit 2d66c6a into main Nov 3, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant