Repository navigation
packages data layer: datascript delta pipeline (replay, conflicts, validation) #95
Description
Spec: the packages-devx design, §Datascripts, agreed 2026-08-10 (public proposal: #1; the full design doc lives in the private working origin). Builds on #90.
The data layer for packages: partial-column row deltas shipped in the package (data/.generated/), applied as the FINAL stage of the existing data pipeline, and re-applied automatically whenever a base import family reloads — the importer's wholesale-per-family reload semantics make one-shot row edits a time bomb, so patches must be a pipeline stage, not edits.
- A delta claims table + PK + only the columns it sets → survives base-data updates.
- Uninstall = replay the affected families without that package's deltas. No pre-image ledger.
- Conflicts: two packages claiming the same table+row+COLUMN → apply fails naming both (packer precedent). Same row, different columns → fine. NO priority numbers.
- New rows: full row + explicit insert marker; custom-content ID floor (documented, above vanilla ranges) + apply-time PK collision detection across installed packages.
- Referential validation at apply (IDs exist, faction rep-index sanity, effect kinds known) + a
--checkdry-run printing the delta without applying. - Per-package provenance stamps in
game_import_meta(existing per-family upsert pattern). - Shard routing inherits the importer's family→database routing; no new routing concept.
- Row DELETION is deferred — override-to-inert covers the real cases; revisit if "remove content" becomes first-class.
Done when: with two test packages (one same-row-different-column overlap, one same-cell conflict): apply lands deltas on all targeted databases; a base-family re-import re-applies deltas automatically; disabling one package and replaying removes exactly its delta; the same-cell conflict fails loudly naming both packages; --check reports without writing.
Migrated 2026-08-11 from the private tracker (spacetime-core#507).
Activity
Mathih13 commentedon Aug 24, 2026
Human Brief
Category: enhancement
Summary: Coordinate the Package Delta artifact, replay pipeline, Shard orchestration, and table-family expansions.
Why this needs a human:
This parent owns the sequence and supported-table policy. Its children are bounded implementation tickets. Live multi-Shard verification and later table admission still need maintainer oversight.
Current behavior:
Base imports replace whole families. Packages have no typed data artifact, field-level ownership, replay stage, Package provenance, or all-Shard orchestration.
Desired behavior:
Start with the spell artifact and conflict tracer in #308. Add single-Shard spell replay in #309 and Realm orchestration in #310. Extend support through #311-#313 only after each table has an explicit Import Family, identifier policy, routing rule, and referential checks.
The settled contract uses versioned canonical JSON, typed primary keys, partial updates, complete inserts, and no deletes. Conflicts occur at the same table, row, and field. Identifier rules are table-specific. Realm application preflights all targets, applies Shards sequentially, and reports an exact resume plan after partial completion.
Acceptance criteria:
Out of scope:
Mathih13 commentedon Aug 28, 2026
Closing in favor of the Package Deltas breakdown: #308 (artifact and spell conflict tracer), #309 (spell replay), #310 (Shard orchestration), and the formal expansion tickets #311 (dump-backed catalogues), #312 (spatial creature and game-object families), #313 (non-spell DBC catalogues), #354 (EventAI claims). The core pipeline shipped; the expansion tickets carry the remaining table coverage this issue asked to gate through formal tickets.