Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion documentation/notes/wd.codebase-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Core owns portable Semantic Flow behavior: request/result types, RDF/Turtle help
- `src/core/weave/inventory_append_planner.ts` prepares current inventory bytes and parsed default-graph quads as one opaque value, plans append/no-op/conflict outcomes, and renders planner-approved append facts through one shared compact renderer. The prepared current bytes remain the exact output prefix; a self-contained suffix-only proof checks rendered facts against `plan.missing`, avoiding a second parse of carried blank nodes. `knop.create` MeshInventory growth, FoundingReferentData settlement, current-only ReferenceCatalog and PageDefinition weave, current-shape extract and integrate, import source-registry insertion, first `knop add-reference` catalog registration, extracted-term versioned MeshInventory growth, and shared current-only payload-like page claims use the planner.
- `src/core/knop/add_reference.ts` retains its unwoven/current-only/versioned shape validation and known support-family consistency checks, but treats the shape renderer only as the requested ReferenceCatalog fact set. The original KnopInventory is the append planner's prepared input and exact output prefix, so unknown facts, comments, prefixes, and carried blank nodes survive; single-valued conflicts refuse before the runtime stages writes.
- `src/core/weave/mesh_inventory_renderers.ts` builds only the term, Knop, page, membership, and optional history/state facts owned by each migrated path, then appends them against the original MeshInventory. Versioned first-Knop weave, singular/delegated-batch versioned first-payload weave, homogeneous extracted batches, versioned sequential extracted weave, and the shared current-only first-payload/extracted helper preserve carried bytes and graph. Versioned first-Knop/first-payload weave rejects legacy inventory-owned mutable progression before planning; first-payload also rejects conflicting values or locator kinds. Current-only semantic no-ops return exact input bytes and create no history progression.
- `src/core/weave/mesh_support_pages.ts` keeps initial support-history construction separate. Initial versioned support history writes only settled artifact/history/state/manifestation membership into MeshInventory and writes current/latest/next progression for every versioned mesh-owned support artifact into MeshMetadata. Its no-initial-history page-only arm prepares the carried MeshInventory and appends only named support-page claims/types. Runtime ResourcePage history/raw-source assembly combines inventory structure with metadata progression. Exact page-claim duplicates produce no inventory update; comments, repeated blocks, opaque facts, trailing bytes, and carried blank nodes remain untouched prefix data.
- `src/core/weave/mesh_support_pages.ts` keeps initial support-history construction separate. Both initial-history and page-only arms prepare the carried MeshInventory once and append named settled facts through `planInventoryAppend`; the initial arm adds artifact/history/state/manifestation membership while current/latest/next progression for every versioned mesh-owned support artifact stays in MeshMetadata. Runtime ResourcePage history/raw-source assembly combines inventory structure with metadata progression. Exact duplicates produce no inventory update, single-valued conflicts fail closed, and legacy inventory-owned progression is refused before arm selection. Comments, repeated blocks, opaque facts, trailing bytes, and carried blank nodes remain untouched prefix data; no new blank nodes are generated.
- `src/core/weave/knop_inventory_renderers.ts` uses the same append path for current-only ReferenceCatalog and ResourcePageDefinition page claims. PageDefinition working-file validation is RDF-semantic rather than substring/block based, so unrelated mentions cannot hide a conflicting locator; exact duplicates return the original KnopInventory bytes.
- `src/core/extract/extract.ts` keeps its legacy-carried-shape renderer separate, but normal current-shape MeshInventory registration prepares and preserves the original bytes before appending only `_mesh` membership, Knop type/working-inventory, and inventory-file facts. Carried source-locator blank nodes remain prefix data and are not part of the named-node-only append request.
- `src/core/weave/source_locator_renderers.ts` gives every persisted floating repository locator the deterministic `<D/_knop/_sources#payload-source-repository-locator>` identity. Payload inventories and source registries point to and describe that same named resource; current readers reject anonymous or differently named locator shapes, while unrelated carried blank nodes remain ordinary opaque inventory data.
Expand Down
13 changes: 7 additions & 6 deletions documentation/notes/wd.todo.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Groomed on 2026-05-27 from all `wa.task.*` notes in the Weave archive. Updated a
- [x] Closed [[wa.completed.2026.2026-07-03_1332-stagecraft-weave-planner-generalization]] after auditing `main`: later-payload advancement, deterministic multi-target and untargeted first-payload batching, Stagecraft replay coverage, condition-specific diagnostics, and the approved `malformed-knop-metadata` finding code are delivered. Three deferred first-payload fixture gates and current-mode extracted/progress work remain with [[wa.task.2026.2026-05-04-refactor-planFirstPayloadWeave]].
- [d] Fix later binary payload advancement: cut as [[wa.task.2026.2026-08-01_1411-binary-payload-advancement]] and PARKED 2026-08-01 — no consumer ever asked for binary payloads (audit: neither feedback round mentions them; the "press includes non-text artifacts" line was a Stagecraft PM-seat open issue whose consumption was deferred to their own lane). The underlying code defect is real and documented (later advancement decodes bytes; later renderers mistype binary payloads as `sflo:RdfDocument`), and v1 refuses binary at API admission, so live exposure is narrow — existing binary working payloads through file-backed CLI `version`. Revive on a real occurrence or when the Stagecraft press lane lands.
- [x] Closed [[wa.completed.2026.2026-08-02_1330-extracted-term-weave-batch-path]] and its generate carve [[wa.completed.2026.2026-08-21_1111-generate-streaming-memory]]. The landed weave batch/history-index work plus bounded render→write→discard generation now complete the faithful N=1,700 nested-source sequence at 802 MiB weave and 674 MiB standalone generate peak RSS; both N=500→1,700 curves are sublinear, and the affordable N=40/120 regression runs in per-PR CI.
- [ ] Execute [[wa.plan.2026.2026-05-17-append-onlyish-inventory]] through independently reviewable child tasks. Add-reference, extracted/current-only MeshInventory, current-only PageDefinition, current-shape extract, mesh-support page-only growth, named locators, progression correction, first-Knop, and [[wa.completed.2026.2026-08-31_1905-versioned-first-payload-mesh-inventory-append]] are delivered. [[wa.task.2026.2026-08-31_2101-initial-versioned-mesh-support-inventory-append]] runs next.
- [ ] Execute [[wa.plan.2026.2026-05-17-append-onlyish-inventory]] through independently reviewable child tasks. Add-reference, extracted/current-only MeshInventory, current-only PageDefinition, current-shape extract, mesh-support page-only growth, named locators, progression correction, first-Knop, versioned first-payload, and [[wa.task.2026.2026-08-31_2101-initial-versioned-mesh-support-inventory-append]] are delivered. Audit the residual legacy/raw MeshInventory and KnopInventory/progression writers next.
- [x] Closed [[wa.completed-plan.2026.2026-08-22_1550-stagecraft-iri-initialization]] and [[wa.completed-plan.2026.2026-08-23_0950-founding-capability-releases]]: SFLO v0.5.0 source/Pages and Weave/`weave-lib` v0.9.0 are published; packed and installed Node founding parity passed; no batch task is owed. See [[release-receipt.v0.9.0]].
- [x] `v0.7.0` RELEASED 2026-08-07 (npm `@semantic-flow/weave@0.7.0` and `weave-lib@0.7.0`; PRs #31/#33/#34/#35 plus ESM-only packaging #39; first release able to generate the published SFLO mesh). Details in [[release-notes.v0.7.0]].
- [ ] Build sites from mesh-held Markdown via the public API: [[wa.task.2026.2026-08-06_0854-markdown-site-pipeline]], queue item 2 behind the serial initial mesh-support append child. `unified`/`remark`/`rehype` was chosen by blind evaluation; packaging is proven in Deno and Node with byte-identical rendering for 1,467 documents at 242 MB, and four contract rulings leave slice 1 unblocked.
Expand All @@ -29,15 +29,16 @@ Groomed on 2026-05-27 from all `wa.task.*` notes in the Weave archive. Updated a
1. **Remote resolution** — [[wa.task.2026.2026-05-20_2152-workingAccessUrl]]: first concrete second-consumer signal (2026-08-21) — another of Dave's projects may need a mesh targeting an ontology held remotely. No fetch path exists in `src/` today; needs the `sfcfg` remote-policy runtime surface plus a bounded fetch helper. The in-flight digest work is its fail-closed substrate.
2. **Fingerprint verification** — [[wa.task.2026.2026-05-04-fingerprint-verification]]: still a stub note, but the digest substrate makes the verify surface a thin slice. Natural first bite of the remote/digest arc rather than an arc of its own.
3. **Markdown site pipeline** — [[wa.task.2026.2026-08-06_0854-markdown-site-pipeline]]: ten slices, slice 1 unblocked, packaging proven. Largest new-capability arc; currently queue 2.
4. **Append-only inventory migration** — [[wa.plan.2026.2026-05-17-append-onlyish-inventory]]: shared planner, independent writers, named locators, integrate append, progression correction, first-Knop, and versioned first-payload append have shipped; initial versioned mesh-support append is queue 1.
4. **Append-only inventory migration** — [[wa.plan.2026.2026-05-17-append-onlyish-inventory]]: shared planner, independent writers, named locators, integrate append, progression correction, first-Knop, versioned first-payload, and initial versioned mesh-support append have shipped; audit the residual legacy/raw MeshInventory and KnopInventory/progression writers next.

### P0: Current Config And Resolution Follow-Ups
- [ ] Keep path and URL trust policy aligned with [[wa.completed.2026.2026-04-11_1723-operational-config-for-runtime-resolution]], [[wa.task.2026.2026-05-20_2152-workingAccessUrl]], and the config-source resolver. Do not let portable mesh config silently grant broader host trust.
### P1: Publication, History, And Runtime Correctness

- [ ] Keep Stagecraft's persisted roleplaying-data use case in view when prioritizing runtime work: prefer slices that improve stable identifiers, exact state citation, append-onlyish histories, source provenance, and generated inspection pages for ordinary application data.- [ ] Remove the remaining first-payload planner blockers from [[wa.task.2026.2026-05-04-refactor-planFirstPayloadWeave]]. Multi-pending first-payload weave shipped in `v0.7.0` (PR #33), extracted-term sets joined the batch path post-regroom (PR #41), and condition-specific diagnostics landed (PR #42). Still open: current-mode extracted-term weave, and the SFLO/URPX docs pass to drop the old failure-mode warnings.
- [ ] Implement append-onlyish inventory writes through [[wa.plan.2026.2026-05-17-append-onlyish-inventory]]: normal inventory operations append new settled facts, no-op existing facts, and fail closed on conflicts; current/latest/next progression belongs in metadata or explicit repair/regeneration/retraction modes. `planInventoryAppend` now has consistency-by-construction prepared current input and one exactness-checked shared append renderer; current-only ReferenceCatalog and PageDefinition, first/later `knop.create`, FoundingReferentData, import source-registry insertion, first `knop add-reference` catalog registration, extracted/current-only/versioned first-Knop/versioned first-payload MeshInventory weave growth, current-shape extract/integrate, and mesh-support page-only growth use it, while ResourcePage policy deletion is fixed. Mesh-support progression producers and affected fixture tails are corrected. [[wa.task.2026.2026-08-31_2101-initial-versioned-mesh-support-inventory-append]] now owns the remaining current MeshInventory block mutation; legacy/raw disposition, Knop-local progression, other KnopInventory writers, and fixture/docs closure follow.
- [ ] In the next Weave release notes, call out the first-Knop and versioned first-payload MeshInventory output changes from [[wa.completed.2026.2026-08-31_1111-versioned-first-knop-mesh-inventory-append]] and [[wa.completed.2026.2026-08-31_1905-versioned-first-payload-mesh-inventory-append]]: carried bytes/facts now survive as the exact prefix, stale inventory-owned progression and conflicting first-payload locator kinds refuse pending explicit repair, and the affected Alice live fixture refs were regenerated.
- [ ] Keep Stagecraft's persisted roleplaying-data use case in view when prioritizing runtime work: prefer slices that improve stable identifiers, exact state citation, append-onlyish histories, source provenance, and generated inspection pages for ordinary application data.
- [ ] Remove the remaining first-payload planner blockers from [[wa.task.2026.2026-05-04-refactor-planFirstPayloadWeave]]. Multi-pending first-payload weave shipped in `v0.7.0` (PR #33), extracted-term sets joined the batch path post-regroom (PR #41), and condition-specific diagnostics landed (PR #42). Still open: current-mode extracted-term weave, and the SFLO/URPX docs pass to drop the old failure-mode warnings.
- [ ] Implement append-onlyish inventory writes through [[wa.plan.2026.2026-05-17-append-onlyish-inventory]]: normal inventory operations append new settled facts, no-op existing facts, and fail closed on conflicts; current/latest/next progression belongs in metadata or explicit repair/regeneration/retraction modes. `planInventoryAppend` now has consistency-by-construction prepared current input and one exactness-checked shared append renderer; current-only ReferenceCatalog and PageDefinition, first/later `knop.create`, FoundingReferentData, import source-registry insertion, first `knop add-reference` catalog registration, extracted/current-only/versioned first-Knop/versioned first-payload/initial mesh-support MeshInventory weave growth, current-shape extract/integrate, and mesh-support page-only growth use it, while ResourcePage policy deletion is fixed. Mesh-support progression producers and affected fixture tails are corrected. Legacy/raw disposition, Knop-local progression, other KnopInventory writers, and fixture/docs closure remain.
- [ ] In the next Weave release notes, call out the first-Knop, versioned first-payload, and initial versioned mesh-support MeshInventory output changes from [[wa.completed.2026.2026-08-31_1111-versioned-first-knop-mesh-inventory-append]], [[wa.completed.2026.2026-08-31_1905-versioned-first-payload-mesh-inventory-append]], and [[wa.task.2026.2026-08-31_2101-initial-versioned-mesh-support-inventory-append]]: carried bytes/facts now survive as the exact prefix, stale inventory-owned progression and single-valued/locator conflicts refuse pending explicit repair, and the affected Alice, sidecar, and branch-published live fixture refs were regenerated.
- [ ] Harden the shared append renderer against an internally mismatched prepared-inventory/plan pair, even though current production callers construct and pass the pair together and the final G1 review found no public reachability. Keep this separate from FoundingReferentData unless that work touches the renderer.
- [ ] Stop reusing one stateful N3 Parser across multiple planned Turtle files in import, integrate, version execution, and payload update validators; instantiate per document so `@base`/`@prefix` state cannot leak between files. Found by [[wa.review.2026-08-22_2303-stagecraft-phase1-g1-claude]], outside G1.
- [ ] Replace FoundingReferentData history progression's compact-`sflo:` regex rewrite with the shared subject-block/progression machinery or an equivalently representation-independent path, preserving fail-closed diagnostics for foreign-but-equivalent inventory serialization. Accepted as a non-landing G2 residual in [[wa.review.2026-08-23_0939-stagecraft-phase2-final-claude]].
Expand Down Expand Up @@ -85,7 +86,7 @@ Groomed on 2026-05-27 from all `wa.task.*` notes in the Weave archive. Updated a
### Active

- [[wa.task.2026.2026-05-04-refactor-planFirstPayloadWeave]]: narrowed further; multi-pending batching (`v0.7.0`), extracted-term batching (PR #41), and condition-specific diagnostics (PR #42) landed, leaving current-mode extracted-term weave and the SFLO/URPX docs pass.
- [[wa.plan.2026.2026-05-17-append-onlyish-inventory]]: active and ruled; named locators, progression correction, first-Knop, and [[wa.completed.2026.2026-08-31_1905-versioned-first-payload-mesh-inventory-append]] are delivered. [[wa.task.2026.2026-08-31_2101-initial-versioned-mesh-support-inventory-append]] is next.
- [[wa.plan.2026.2026-05-17-append-onlyish-inventory]]: active and ruled; named locators, progression correction, first-Knop, versioned first-payload, and [[wa.task.2026.2026-08-31_2101-initial-versioned-mesh-support-inventory-append]] are delivered. Audit the residual legacy/raw MeshInventory and KnopInventory/progression writers next.
- [[wa.task.2026.2026-05-05-optional-history-and-slim-support-artifacts-by-default]]: partially superseded by config synthesis, but the current-only `ResourcePageDefinition` and history-policy runtime behavior remain active.- [[wa.task.2026.2026-05-26_1321-relative-links]]: active but underspecified; needs expansion before implementation.
- [[wa.task.2026.2026-05-16_1625-manifest-completeness-check]]: active fixture-infrastructure work; scenario indexes exist, but manifest completeness checking remains.
- [[wa.task.2026.2026-05-19_1536-latest-state-conformance]]: active conformance fixture backlog.
Expand Down
34 changes: 34 additions & 0 deletions src/core/weave/mesh_inventory_progression_assertions.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import { SFLO_NAMESPACE } from "../rdf/namespaces.ts";
import { WeaveInputError } from "./errors.ts";
import type { PreparedCurrentInventory } from "./inventory_append_planner.ts";

const LEGACY_INVENTORY_PROGRESSION_PREDICATES = new Set([
`${SFLO_NAMESPACE}currentArtifactHistory`,
`${SFLO_NAMESPACE}nextHistoryOrdinal`,
`${SFLO_NAMESPACE}latestHistoricalState`,
`${SFLO_NAMESPACE}nextStateOrdinal`,
]);

export function assertNoLegacyMeshInventoryProgression(
preparedCurrentInventory: PreparedCurrentInventory,
operation: string,
): void {
const carriedPredicates = [
...new Set(
preparedCurrentInventory.quads
.map((quad) => quad.predicate.value)
.filter((predicate) =>
LEGACY_INVENTORY_PROGRESSION_PREDICATES.has(predicate)
),
),
].sort((left, right) => left.localeCompare(right));
if (carriedPredicates.length === 0) {
return;
}

throw new WeaveInputError(
`Could not ${operation} because the current MeshInventory contains legacy inventory-owned mutable progression predicates: ${
carriedPredicates.map((predicate) => `<${predicate}>`).join(", ")
}. Regenerate the fixture or use an explicit repair path before retrying weave.`,
);
}
Loading