Summary
ChangeSetWriterProvider currently emits a stamp's lifecycle in an order
that is correct on the wire but inverts the temporal narrative: when a
stamp commits, the committed chronology is written before the
uncommitted entities that were held back waiting for it. A reader of
the change set ZIP sees [commit, the-uncommitted-stuff-that-led-up-to-it]
rather than the natural [uncommitted-stuff-that-led-up-to-it, commit].
The on-disk format is unchanged, the loader is order-insensitive, and
no existing consumer breaks if we flip this. But the recent change-set
diagnostic tool surfaced the inversion as actively confusing during
manual review — the journal is now a thing humans read, not just a
machine transport.
This issue covers the writer-side reorder and the matching rollover
flush partition, plus a small ordering contract in javadoc.
Background
The relevant code is the service-thread loop in
ChangeSetWriterProvider.java#L230-L264.
Its current behavior:
- Uncommitted entities arrive at the queue → stashed in
uncommittedEntitiesByStamp[stampNid] keyed by the stamp nid each
uncommitted version references.
- A committed stamp arrives:
writeEntity(committed_stamp) — writes the now-2-version chronology.
- Then
uncommittedEntitiesByStamp.removeAll(stampNid).forEach(writeEntity) —
drains the held-back uncommitted entities (concept / semantic / pattern
versions, plus any prior uncommitted snapshot of the stamp itself
that was stashed against its own nid).
- Inactivity rollover / explicit save() / shutdown() triggers
checkpoint() → state transitions from RUNNING → ROTATING or
STOPPED → loop exits → uncommittedEntitiesByStamp.forEachValue(writeEntity)
flushes everything still held back, in undefined multimap-value order.
The per-stamp commit drain (step 2) writes commit first and the
uncommitted history second. The rollover flush (step 3) groups
nothing — stamp snapshots and their dependent entities are interleaved
arbitrarily across stamp nids.
Observed effect
A simple flow — create concept → reference some stamp uncommitted →
commit the stamp — produces:
StampChronology (2 versions: uncommitted, committed)
ConceptChronology (1 uncommitted version referencing the stamp)
SemanticChronology (1 uncommitted version referencing the stamp)
…
A natural reading of the journal would expect:
StampChronology (1 uncommitted version)
ConceptChronology (1 uncommitted version)
SemanticChronology (1 uncommitted version)
StampChronology (2 versions: uncommitted, committed)
The first matches "snapshot-of-state-at-flush-time". The second matches
"sequence of edits as they happened, terminated by the commit event."
The second is what humans expect when they read a change set as a
journal.
Proposed change
Two narrow edits in ChangeSetWriterProvider.startService():
1. Reorder the per-stamp commit drain
Drain held-back entities before writing the committed stamp.
Within the drain, write StampEntity snapshots first and other
entities (concepts / semantics / patterns) second, so each unit reads
as uncommitted-stamp → its-dependents → committed-stamp.
2. Partition the rollover/shutdown flush by stamp nid
Same partition strategy applied to the final flush, grouped per stamp
nid so each stamp's snapshot is followed by its own dependents instead
of all stamp snapshots clumped together followed by all dependent
entities. Mirrors the per-stamp commit-drain shape so the on-disk
narrative is consistent regardless of which path produced the entries.
3. Document the ordering contract
A short javadoc block on startService() (or a class-level note)
establishing the contract:
Within a change set ZIP, entries appear in a journal order where each
unit is uncommitted-stamp-snapshot → dependent-uncommitted-versions → committed-stamp, terminated either by the commit event itself or by
a rollover/shutdown flush. Importers should treat this ordering as
informational; correctness must not depend on it.
That clarifies that we have a contract for human readers without
trapping future contributors into thinking importers can rely on it.
Compatibility
- Loader (
LoadEntitiesFromProtobufFile): order-insensitive — merges
by PublicId. Existing change set ZIPs (with the old order) and new
change set ZIPs (with the new order) load identically.
- Schema: unchanged.
- Existing change set files on disk: unchanged. The reorder is
forward-only; old files keep their old ordering and load just fine.
- Downstream consumers: only consumer that cares about order is the
change-set summary tool, which displays whatever order it finds. No
changes needed there.
Acceptance
- The commit-drain path emits
[uncommitted-stamp(s)] [dependents] [committed-stamp]
in that order.
- The rollover-flush path emits one group per stamp nid:
[uncommitted-stamp(s)] [dependents] per group, groups in any order.
- Roundtrip test: write a transaction with held-back uncommitted
dependents, close the writer, read back the ZIP, assert the new
ordering message-by-message.
- Roundtrip-via-loader test: same fixture, run through
LoadEntitiesFromProtobufFile, assert the resulting in-store
chronologies are equivalent to a baseline (i.e., the loader produces
the same data regardless of writer order).
- Javadoc contract block added; no existing javadoc removed.
Effort
~30 lines source + ~50 lines test, no schema or compatibility changes.
Ships in the next tinkar-core release.
Summary
ChangeSetWriterProvidercurrently emits a stamp's lifecycle in an orderthat is correct on the wire but inverts the temporal narrative: when a
stamp commits, the committed chronology is written before the
uncommitted entities that were held back waiting for it. A reader of
the change set ZIP sees
[commit, the-uncommitted-stuff-that-led-up-to-it]rather than the natural
[uncommitted-stuff-that-led-up-to-it, commit].The on-disk format is unchanged, the loader is order-insensitive, and
no existing consumer breaks if we flip this. But the recent change-set
diagnostic tool surfaced the inversion as actively confusing during
manual review — the journal is now a thing humans read, not just a
machine transport.
This issue covers the writer-side reorder and the matching rollover
flush partition, plus a small ordering contract in javadoc.
Background
The relevant code is the service-thread loop in
ChangeSetWriterProvider.java#L230-L264.Its current behavior:
uncommittedEntitiesByStamp[stampNid]keyed by the stamp nid eachuncommitted version references.
writeEntity(committed_stamp)— writes the now-2-version chronology.uncommittedEntitiesByStamp.removeAll(stampNid).forEach(writeEntity)—drains the held-back uncommitted entities (concept / semantic / pattern
versions, plus any prior uncommitted snapshot of the stamp itself
that was stashed against its own nid).
checkpoint()→ state transitions fromRUNNING→ROTATINGorSTOPPED→ loop exits →uncommittedEntitiesByStamp.forEachValue(writeEntity)flushes everything still held back, in undefined multimap-value order.
The per-stamp commit drain (step 2) writes commit first and the
uncommitted history second. The rollover flush (step 3) groups
nothing — stamp snapshots and their dependent entities are interleaved
arbitrarily across stamp nids.
Observed effect
A simple flow — create concept → reference some stamp uncommitted →
commit the stamp — produces:
A natural reading of the journal would expect:
The first matches "snapshot-of-state-at-flush-time". The second matches
"sequence of edits as they happened, terminated by the commit event."
The second is what humans expect when they read a change set as a
journal.
Proposed change
Two narrow edits in
ChangeSetWriterProvider.startService():1. Reorder the per-stamp commit drain
Drain held-back entities before writing the committed stamp.
Within the drain, write
StampEntitysnapshots first and otherentities (concepts / semantics / patterns) second, so each unit reads
as
uncommitted-stamp → its-dependents → committed-stamp.2. Partition the rollover/shutdown flush by stamp nid
Same partition strategy applied to the final flush, grouped per stamp
nid so each stamp's snapshot is followed by its own dependents instead
of all stamp snapshots clumped together followed by all dependent
entities. Mirrors the per-stamp commit-drain shape so the on-disk
narrative is consistent regardless of which path produced the entries.
3. Document the ordering contract
A short javadoc block on
startService()(or a class-level note)establishing the contract:
That clarifies that we have a contract for human readers without
trapping future contributors into thinking importers can rely on it.
Compatibility
LoadEntitiesFromProtobufFile): order-insensitive — mergesby PublicId. Existing change set ZIPs (with the old order) and new
change set ZIPs (with the new order) load identically.
forward-only; old files keep their old ordering and load just fine.
change-set summary tool, which displays whatever order it finds. No
changes needed there.
Acceptance
[uncommitted-stamp(s)] [dependents] [committed-stamp]in that order.
[uncommitted-stamp(s)] [dependents]per group, groups in any order.dependents, close the writer, read back the ZIP, assert the new
ordering message-by-message.
LoadEntitiesFromProtobufFile, assert the resulting in-storechronologies are equivalent to a baseline (i.e., the loader produces
the same data regardless of writer order).
Effort
~30 lines source + ~50 lines test, no schema or compatibility changes.
Ships in the next
tinkar-corerelease.