Skip to content

Re-evaluate dependent queries on touches and streamline dependency bookkeeping - #42

Open
danReynolds wants to merge 51 commits into
mainfrom
feature/touchAsWrite
Open

danReynolds wants to merge 51 commits into
mainfrom
feature/touchAsWrite

Conversation

@danReynolds

@danReynolds danReynolds commented Sep 5, 2026 •

Copy link
Copy Markdown
Owner

Dependency changes now re-evaluate documents in observed queries, allowing a previously excluded document to enter the result set or an included document to leave it. Writes, deletions, and manual rebroadcasts schedule BroadcastEvents.touched for affected dependents; a rebroadcast does not persist data or rebuild dependencies. A touch clears the cached query results of the dependent's collection, once per collection, and keeps its document observers' cached values, since it doesn't change the dependent's data.

The dependency manager maintains both indexes eagerly, removes deleted subtree documents from their dependencies' reverse indexes before propagating deletion events, and snapshots builder-owned sets. Only documents with at least one dependency have an entry. Observers no longer maintain separate dependency trees. Plain and observable document handles compare by path; active observers are tracked by identity.

The implementation lives under lib/src, and lib/loon.dart exports the public API through an explicit list. It now also exports the persistors, DataStoreEncrypter and the ID generators, which needed deep imports before, and no longer exports internal classes such as the value stores and the managers. The value stores parse paths as they walk them and remember the last parent node they resolved, so runs of reads and writes within a collection skip walking its path, and propagation records each touch with a single putIfAbsent. A fuzz test of the stores found three bugs in graft, now fixed; one kept a data store from being deleted after its documents moved to another store. Observable queries read a broadcast's snapshots by collection and cache them by document ID. Inspection follows the existing serialization convention: entries implement toJson() using reference paths without serializing document data.

Performance

benchmark/ compares versions with a standalone runner (value_store/run_core.dart, JIT and native AOT) and an end-to-end harness over the public API (loon_benchmark.dart, flutter test). CI checks that both workloads still produce correct results without enforcing timings. The decisions behind this PR and their numbers are in benchmark/DECISIONS.md.

Against main (82b8a7b) on macOS arm64. Store rows are AOT store_core, where a second copy of this PR varied by about ±3%. The other rows run through the public API in JIT, median of three runs, where differences under 10–20% are noise.

main this PR
Store: get, 20k deep paths 9.7 ms 2.2 ms
Store: write, 20k new deep paths 12.1 ms 3.6 ms
Store: get of a missing path, 100k 33.9 ms 3.1 ms
Write 50k documents with events, per write 1.64 µs 0.84 µs
Read 200k documents, per read 0.33 µs 0.24 µs
Broadcast with 5k idle observers 3.35 ms 1.04 ms
Update feeding a live query, 10k dependents 16.4 ms 3.8 ms
Update touching 20k dependents in one collection 30.0 ms 5.5 ms
Delete a collection of 20k dependents 0.04–0.5 ms 5.7–8.9 ms
  • Every store operation is faster: reads by 49–79%, misses and existence checks by 61–91%, writes by 24–76%, deletes by 34–49% and extraction by 4–47%.
  • main tracked dependencies inside queries, so the dependency rows compare mechanisms rather than identical work.
  • Deleting a collection is slower because this PR removes its documents from their dependencies' reverse indexes. main left them there, so every later update of a dependency walked the deleted documents, and their memory was never released.

Validation

  • 150 core, 90 native and 42 web (Chrome) tests pass, and flutter analyze is clean.
  • CI's benchmark checks pass: the store_core workload in flutter test and a one-trial manager_core run.
  • Each commit from 75c9a73 onward passes analysis, the core and native tests, and the benchmark checks on its own.

Release considerations

  • This releases as 6.0.0. Public APIs are removed (PathRefStore, ObservableDocument.inspect(), and the exports of internal classes such as ValueStore, BroadcastManager and DependencyManager), imports of other package:loon/... paths move to package:loon/loon.dart, inspection shapes change, including ObservableQuery.inspect() keying docSnaps by document ID, observable documents compare equal by path, and Document.dependencies() returns null instead of an empty set when the dependencies builder returns one. The changelog marks each as breaking.
  • Failed-filter recovery remains explicitly deferred. A query can retain a stale snapshot after its filter throws; other observers continue receiving broadcasts.
  • The branch history adds and later removes about 12 MB of raw benchmark results. Squash merging, the repository's only merge option, keeps them out of main.

🤖 Generated with Claude Code

A rebroadcast previously re-emitted observers' cached values, so it could
never change a query: filters and sorts were not re-run, a touched
document outside a query's result set was ignored, and the query's
snapshot and dependency caches were never refreshed. Any query whose
filter or sort reads state outside of the store went stale.

Document.rebroadcast now writes the document's current value without
persisting it: observer caches are invalidated, queries re-evaluate the
document like a modified one (adding, refreshing or evicting it), and its
dependencies are rebuilt. A query touched through a deleted dependency
path re-evaluates its result set from the store, and an observer that
throws during a broadcast no longer prevents subsequent broadcasts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 5, 2026 19:48
The lockfile was re-resolved by the local SDK and is unrelated to this change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The updated broadcast error-handling can drop pending events for observers that haven’t yet run in the current broadcast when another observer throws, causing missed updates.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR changes Document.rebroadcast() semantics so a rebroadcast behaves like a non-persisted write of the document’s current value, ensuring observer caches, query membership/order, and dependency caches are refreshed when state outside the store affects filters/sorts.

Changes:

  • Make Document.rebroadcast() go through Loon.writeDocument(..., event: touched, persist: false) when the document exists, so queries re-evaluate and dependencies rebuild.
  • Treat touched like modified for query evaluation, and force full query recomputation when the query itself is touched (e.g., dependency path deleted).
  • Improve broadcast robustness, expand docs/changelog, and add comprehensive rebroadcast tests.
File summaries
File Description
lib/document.dart Re-implements rebroadcast() as a non-persisted write of the current stored value (or re-notify if missing).
lib/broadcast_manager.dart Makes touched invalidate observer caches; adds finally cleanup for broadcast scheduling state.
lib/observable_query.dart Treats touched like modified for membership/sort refresh; recomputes from store when the query is touched.
test/core/rebroadcast_test.dart Adds coverage for rebroadcast effects on docs/queries, dependency rebuild, membership/sort changes, and broadcast resilience.
README.md Documents the rebroadcast contract and intended usage pattern for filters/sorts reading external state.
CHANGELOG.md Adds 5.7.0 entry describing rebroadcast and dependency-path deletion behavior changes.
pubspec.yaml Bumps version to 5.7.0.
Review details
  • Files reviewed: 7/7 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lib/broadcast_manager.dart Outdated
Comment on lines +61 to +65
try {
for (final observer in _observers.toList()) {
observer._onBroadcast();

// Recalculate the set of observers with dependencies after they process the broadcast
// and update their dependency stores.
if (!observer._deps.isEmpty) {
_depObservers.add(observer);
// Recalculate the set of observers with dependencies after they process the broadcast

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea should we continue for each observer if one throws?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes — for query filters. A query whose filter throws reports through FlutterError.reportError and does not abort the loop, so the remaining observers still process this broadcast.

Other throws (anything that escapes an observer) still abort the rest of that turn. _broadcast now always clears the event store and timer in a finally, so that cannot stick the scheduler. _depObservers is gone; dependents are indexed by path and touched per document.

The query also rebuilds its result from the store on the next broadcast, since the events from the failed turn are cleared. See c5356f6.

Comment thread README.md Outdated
Comment on lines +289 to +293
final transfers = transactions.where((snap) => snap.data.isTransfer);

// `isTransfer` reads in-memory rules. When the rules change, rebroadcast the affected
// transactions so that the query re-evaluates them.
transaction.rebroadcast();

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — the example is now a self-contained onRulesChanged that declares its collection and rebroadcasts by id.

Comment thread lib/document.dart Outdated
void rebroadcast() {
Loon._instance.broadcastManager
.writeDocument(this, BroadcastEvents.touched);
// The store is read directly rather than through [get], which observable documents override

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That early version made rebroadcast() a non-persisted write of the current value (which also rebuilt dependencies). It is now a no-op for a missing document, and otherwise just schedules BroadcastEvents.touched — observers re-read / queries re-evaluate, nothing is persisted, dependencies are not rebuilt.

Comment thread lib/observable_query.dart Outdated
if (Loon._instance.broadcastManager.eventStore.get(_observerId) ==
BroadcastEvents.touched) {
shouldRebroadcast = true;
_value = _resetCache(super.get());

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What scenario is this covering?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And do we see it being hard on performance if every dep change now and touch now causes a full refilter/sort?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That early version recomputed the whole query when it was touched (e.g. a dependency path was deleted). That is gone.

A dependency write/delete now schedules touched on each affected document. The query re-runs its filter/sort only for those documents — same path as modified, complexity scales with the size of the change, not the result set. A full recompute is not involved.

Comment thread lib/observable_query.dart Outdated
// same way as a modified one, since a touch is a write of the document's current value that
// signals state its filter or sort depends on may have changed outside of the store.
case BroadcastEvents.modified:
case BroadcastEvents.touched:

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this just consoldating basically duplicate spots under one case handler blocK?

danReynolds and others added 22 commits September 5, 2026 16:05
An observer that throws while processing a broadcast no longer aborts the
loop: the remaining observers still process the events (which are cleared
once the broadcast completes), the first error is rethrown afterwards and
any others are reported to the zone.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Dependents are now indexed by the path of the document they depend on, so
deleting a document or collection can find the dependents of everything under
the deleted path and touch each of them, exactly as a document write already
does through _broadcastDependents. Queries then re-evaluate only the affected
documents instead of recomputing their whole result set.

That makes the observer-level dependency tracking redundant, so it is
removed: the per-observer PathRefStore, _updateDeps, the broadcast manager's
_depObservers, ObservableDocument's dependency cache and inspect(), and
ObservableQuery's _docDepCache, along with the observer-id touch that deleted
paths used to write.

Alongside it, Document.rebroadcast is a plain touched event again (queries
re-evaluate touched documents like modified ones; dependencies are not
rebuilt), the query keeps its cached value across a broadcast that follows a
read, and an observer that throws during a broadcast is reported through
FlutterError.reportError without interrupting the remaining observers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Deletes now broadcast dependents in one place, _deletePath, through the
renamed _broadcastPathDependents / getPathDependents. Loon.deleteCollection
deletes from the store before broadcasting, matching deleteDocument, so
documents deleted along with a dependency are not touched and their
observers see a removed event rather than a touched one.

PathRefStore and its tests are removed now that nothing uses it, and the
rebroadcast and query doc comments are trimmed to the final semantics.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…cture doc

Rebroadcasting a document that does not exist has no value for observers to
re-evaluate, and scheduling a touched event for it invalidated the cached
values of every query on the collection for nothing. It now returns early.

docs/architecture.md described the removed observer-level dependency
tracking (PathRefStore, per-observer dependency trees); it now documents the
path-indexed dependents store and the per-document touched fan-out.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e cleared

updateDependencies removed a document's forward entry with a recursive delete,
which also wiped the dependency entries of the documents in its subcollections
while their reverse-index memberships remained, so their old dependencies kept
touching them. The delete is now non-recursive; deleting a document or
collection still removes the subtree.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
An ObservableDocument was never equal to the plain document it observes, an
exception that existed only so that two observables of one document stay
distinct in the broadcast manager's observer set. That exception leaked into
every structure keyed by documents: a document written through an observable
handle could not be matched by plain references in the query snapshot cache
(duplicate, un-evictable entries) or removed from its old dependency's
dependents set (spurious touches).

Documents now compare by path unconditionally and the observer set is keyed
by identity instead.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A disposed observer now drops its stream controllers and last emitted value,
so that anything still referencing it (such as a snapshot written through an
observable handle) retains nothing of its observer state. Its streams are
empty after disposal, disposing twice is a no-op, and observe() on a disposed
observable returns a fresh observer instead of the dead one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The dependents store is now a ValueStore of ValueStores: each dependency path
holds its dependents keyed by their own paths. Deleting a path drops the
dependents under it, which are deleted along with it, with a single delete of
that path in each affected dependents store, so they are neither touched nor
left behind in the index, and a single document's membership is removed by
path without relying on document equality. Loon.inspect()['dependentsStore']
now maps each dependency path to the path tree of its dependents.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Deleting a document or collection now removes every deleted document from
the dependents of its dependencies, using the dependency index before it is
cleared: the union of the deleted documents' dependencies is collected, and
each of those dependents sets drops its members under the deleted path. A
document that is deleted and re-created with different dependencies no
longer keeps its old memberships, deleting all of a document's dependents
leaves the index empty, and dependents deleted along with their dependency
are neither touched nor left behind.

With the index exact on delete, the dependents store is a flat
ValueStore<Set<Document>> again, the dependency document cache is removed
(document equality is by path), the two dependents-broadcast loops are one
_broadcastDependents(ref, recursive:), and Document.isDescendant is added.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An observer that throws while processing a broadcast now recovers itself
rather than being left half-applied: the broadcast observer mixin catches
the error, has the observer discard its partially updated state, and
delivers the error on the observer's own stream so its listener learns
which observer failed. A query rebuilds its result set from the store on
its next broadcast; a document observer drops its cached value. The
broadcast manager resets its events and timer in a finally so that nothing
thrown while broadcasting can prevent subsequent broadcasts.

Tests: the broadcast tests assert the error on the failing query's stream
and a correct result on the next broadcast; a test covers re-creating a
document with different dependencies; the rebuilt-dependencies test asserts
between the two dependency writes; the fakeAsync reset helper is shared
from test/utils.dart. README documents the touched change event and the
delete behaviour of dependencies; CHANGELOG marks the removals as breaking.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Restores the broadcast observer, observable document, observable query and
broadcast manager to the query-level try/catch: a query whose filter throws
reports the error through FlutterError.reportError and resets its cached
result, and the broadcast loop is unchanged. The broadcast test asserts that
contract again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deleting a path removes each deleted document from the dependents of its
dependencies by walking the dependency index, rather than scanning every
dependency's dependents set for descendants of the deleted path. The cost
of a delete no longer grows with how many documents share a dependency:
measured per deleted document, 199us -> 12us with a dependency shared by
1,000 documents and 901us -> 6us at 5,000.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The observer controllers are late final again: dispose() closes them and
releases the last value, and listening to a disposed observer's streams
again throws rather than yielding an empty stream. The dispose test asserts
that contract and the null-aware handling the nullable controllers required
is removed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The _addDependent/_removeDependent parameters were renamed to
(doc, dep) with doc as the dependency, and _deleteRef was updated to match,
but the three call sites in updateDependencies still passed the dependent
first, so writes and deletes keyed the reverse index by different sides.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The rebroadcast tests join the Document > rebroadcast group, the dependency
tests join the Dependencies group, observable equality and dispose join
ObservableDocument, and the observable-write and error-recovery tests join
ObservableQuery. The broadcast timing and query equivalence tests from main
stay as their own files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The equality, dispose, observable-write and error tests are loose tests in
the ObservableDocument and ObservableQuery groups, alongside the existing
cache tests, and are named as sentences about the observer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The broadcast batching tests join the Document and ObservableQuery groups in
loon_test (the unchanged-update case already existed there), the coalesced
delete-and-recreate test and the query equivalence fuzzer join
ObservableQuery (trimmed to one sorted and one unsorted walk), the store
property tests join value_store_test and value_ref_store_test as property
groups (50 seeds each), and the generated-ID test joins Document.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The path generation helpers duplicated at the top of value_store_test and
value_ref_store_test move to test/store_paths.dart.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@danReynolds danReynolds changed the title Treat rebroadcast as a write of the document's current value Re-evaluate dependent queries on touches and streamline dependency bookkeeping Sep 16, 2026
danReynolds and others added 9 commits September 21, 2026 10:15
Scan paths incrementally and reuse dependency maps and sets during propagation and deletion. Preserve extraction and unusual path-boundary semantics.

Add Dart JIT/AOT and retention workloads with measured comparisons and rejected prototypes. Run native Flutter profiling hosts without windows or app activation.

Validation: 253 native/core/benchmark tests and 44 headless Chrome tests passed; static analysis and diff checks are clean.
The store remembers the last path it resolved, where that path's final
segment begins and the node that holds it. A read or write under the
same parent, such as the next document of a collection, reuses the node
instead of walking the path again. Only resolved nodes are remembered,
and writes never remove or replace nodes. clear, delete and graft (on
the source store) forget it, so it cannot go stale.

Add ValueStore.putIfAbsent, which writes a value unless the path
already has one, in a single resolution.

store_core AOT against the previous commit, per 20k operations: deep
get 4.7 -> 2.3 ms, write_new 6.3 -> 3.7 ms, child_values 19.3 -> 5.1
ms. Reads that never share a parent are about 12% slower, and early
misses and deletes about 5%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Propagation writes a touched event to each dependent with
eventStore.putIfAbsent. A dependent with a pending event was already
invalidated and has already touched its own dependents, so it is
skipped. Otherwise its cached observer values are invalidated and its
dependents are touched in turn. The value store's parent reuse replaces
the collection-map cache that propagation kept for itself.

Against the propagation it replaces, in manager_core AOT with 20k
dependents: faster across 5,000 collections (11.9 -> 10.5 ms) and on
chains that alternate collections, slower when every dependent shares
one collection (2.7 -> 5.2 ms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deleting a path unlinks each deleted document from its dependencies'
reverse indexes while visiting the dependency entries under the path
with the new ValueStore.forEachValue, then deletes the entries, instead
of extracting them into a set first. _removeDependent looks up each
dependency's set again, which the value store's parent reuse keeps
cheap, so the operation-local reuse of the last set is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A document whose dependencies builder returns null or an empty set has
no dependency entry, so Document.dependencies() returns null for it.
Previously an initial empty result was stored and returned as an empty
set, while clearing a non-empty set removed the entry. Documents with
conditional dependencies no longer keep an entry, a set and their
document handle while they have none. The builder's set is copied only
when it is stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On broadcast, an observable query reads its collection's snapshots with
one getChildValues call and caches its result set by document ID,
instead of building a Document and walking the store for every event.
Snapshots that are not parsed yet, such as hydrated data, are read
through Document.get so the query's serializer applies; a regression
test covers typed queries receiving hydrated documents.
ObservableQuery.inspect() keys docSnaps by document ID.

Updating 20k documents in a 20k-document query: 8.6 -> 4.1 ms per
broadcast (JIT).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…shapes

manager_core adds 20k writes without dependents, and chains whose
writes alternate between two or three collections at shallow and deep
paths. store_core adds reads where every path has a different parent,
the worst case for parent reuse. The end-to-end harness adds writes
with broadcasting on, dependency fan-out into a queried collection,
dependency shapes (registering, propagating to and deleting 20k
dependents) and sparse updates in a large collection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep run_core.dart with the store_core and manager_core workloads, and
the end-to-end harness. Remove the Flutter host runner and its source
rewriting variants, which broke as the library changed, along with
prototype drafts, experiment-only workloads and tests, retention
runners and 12 MB of raw results. A decision log replaces the result
writeups, and a shorter README covers both tools.

Exclude benchmark/ from the published package, and check the
manager_core workload in CI with one trial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
danReynolds and others added 9 commits September 23, 2026 14:51
_getParent walks the path itself and takes the create flag, so reads
and writes share one function and _resolveParent is removed. As a
prototype it measured neutral in store_core and manager_core, with
empty-store gets about 7% faster.

_removeDependent no longer returns the remaining set, which nothing
used, and _deleteRef's documentation says why the entries are read
before they are deleted.

manager_core adds deleting 20k records that share one dependency or
each have their own. Visiting their entries with forEachValue takes
32-37% less time than extracting them into a set first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ValueStore, ValueRefStore and their path helpers move to an internal
library, lib/src/store/store.dart, that loon.dart imports but does not
export. Loon's persistors, tests and benchmarks import it directly. No
public persistor API takes a store, so nothing outside the package
needs them.

Document and Collection use the store's splitReferencePath, and
ValueStore.fromJson takes a plain map so the store library needs
nothing from Loon. run_core.dart reads the stores from either layout,
so versions from before the move still compare.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Check reads, grafts and clears across two stores against reference
models, reading under each operation's target first so a stale cached
parent would be read back. The fuzz found three bugs in graft:

- Merging into existing values replaced them with an untyped map, which
  typed reads such as extractValues then rejected.
- Emptied branches of the other store were kept when an ancestor had
  other children, so a data store whose documents all moved out was
  never empty and never deleted.
- Grafting a path the other store doesn't have left empty nodes behind.

Also check that reads through the cached parent match reads from the
root for paths built from runs of underscores, and that the other store
of a graft writes next to moved data into its own tree. Each of the
four places that forget the cached parent now fails a test without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The propagation fixture wrote observer values at the dependents' and
collections' own paths, so the first collection clear emptied the store
and later clears returned at once. It now keys them under the observed
paths, as observers do, and checks that each touched collection's
cached query results are cleared.

Each manager sample starts after settleHeap() promotes what its setup
allocated. Without it, collections during the timed operation copied
the setup's objects, and builds with the same algorithm differed by up
to 24%.

Also remove what the Flutter host runner left behind (unused settings,
ProfileResults.measure, the profile mode and the Flutter report
branches), and replace commit hashes that won't exist after the squash
merge in the benchmark docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A touch doesn't change a dependent's data, so its document observers
keep their cached values and only its collection's cached query results
are cleared, once per collection. With observer values keyed as
observers key them, propagation to 20k dependents in one collection
takes 5.2 instead of 16.5 ms in AOT manager_core, and 11.0 instead of
22.3 ms across 5,000 collections. 5.6.1 didn't clear anything for a
touch, so document observers behave as they did then.

writeDocument writes an event directly, or with putIfAbsent for a
touch, instead of reading the pending event first, which takes 31-35%
less time for writes without dependents. extractValues collects through
forEachValue instead of its own walk, with no measurable difference.

Also tidy the PR for merge:

- Collect recursive dependents with forEachValue and drop the public
  flatten extension, which was only used there.
- Rename the top-level _lastSegmentStart to _finalSegmentStart, since a
  field of the same name shadowed it inside the stores.
- Restore the comment on why deletes prune dependencies before touching
  dependents, and fix comments that said observers re-read a touched
  document or that touches only come from outside the store.
- Fold the dependency traversal tests into loon_test, restore the query
  fuzzer's thresholds of 1, 4 and 8, and drop an unused test helper.
- Rewrite the changelog against 5.6.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The attributes only marked benchmark results that are no longer committed, and the lockfile changes came from resolving the example with a newer SDK.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Breaking changes include removed exports and APIs, changed inspection output, path equality for observable documents, and Document.dependencies() returning null instead of an empty set. The changelog marks each one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The last resolved path, the index at which its final segment starts and
the parent node now live in a small immutable _PathCache
(lib/src/store/path_cache.dart), with the prefix check as its isMatch
method. A walk that resolves replaces it, and anything that removes
nodes sets it to null, so it can't be half-set. The path helpers move
to lib/src/store/utils/paths.dart, and the benchmark runner reads both
new files.

Against the three fields in AOT, every store_core operation stayed
within its A/A noise and empty-store gets took 1% longer, unlike the
helper class that made them 25% slower. manager_core propagation and
writes moved by -8% to +4%, also within noise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread lib/src/store/value_store.dart Outdated
static const root = _BaseValueStore.root;

static ValueStore fromJson(Json json) {
static ValueStore fromJson(Map<String, dynamic> json) {

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use this instead of the Json type?

final (parent, key) = _getParent(path, create: true)!;
final Map<String, T> values =
parent[_BaseValueStore._values] ??= <String, T>{};
if (values[key] != null) {

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is null not a valid value store value for a key?

///
/// Segments are parsed as they are visited rather than split up front, so a lookup allocates
/// only the segments it reaches and stops at the first missing node.
@pragma('vm:prefer-inline')

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do these pragmas do and why did we add them?

Comment thread README.md
Additionally, whenever a document is updated, it will rebuild its set of dependencies, allowing documents to support dynamic dependencies
that can change in response to updated document data.

### Rebroadcasting

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewrite this following the best practices/theme of examples elsewhere in the readme.

danReynolds and others added 5 commits September 25, 2026 10:51
Shorten the comments on touching dependents, drop comments that restate the code, and document forEachValue and the unparsed snapshot fallback in queries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Move the Json typedef into lib/src/json.dart, which the store and the
  loon library both import and loon.dart re-exports, so
  ValueStore.fromJson takes a Json again.
- Document that putIfAbsent treats a null value as no value, like get
  and hasValue, and test it.
- Say why the store's hot helpers are marked vm:prefer-inline. Without
  the pragmas, empty-store gets took 46% longer, misses 5-8% longer and
  propagation and writes 5-14% longer in AOT, so they stay, and
  DECISIONS.md records the numbers.
- Rewrite the README's rebroadcasting example around UserModel, like the
  README's other examples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…citly

Only public entry points live directly under lib/, following Dart's
package layout conventions. lib/loon.dart is now the package's public
API, with an explicit export list over the implementation in lib/src:

- The core library moves to lib/src/loon.dart, with the broadcast and
  dependency managers in lib/src/broadcast and lib/src/dependencies,
  and the persistors, widgets, utilities and extensions under lib/src.
- loon.dart now also exports FilePersistor, SqlitePersistor,
  IndexedDBPersistor, DataStoreEncrypter, generateSecureId and
  generateFastId, which needed deep imports before.
- Internal classes are no longer exported: BroadcastManager,
  BroadcastObserver, DependencyManager, PersistManager, the persistor
  operations and the iterable extensions.
- The platform stubs take the same constructor parameters as the real
  persistors (encrypter, and useFfi for SQLite), since the analyzer
  checks code against the stubs that loon.dart exports by default.

Tests import internals from lib/src, the example and the end-to-end
benchmark import only loon.dart, the benchmark runner reads the managers
from their new paths with the old ones as fallbacks, and the unused
distinct extension is removed. The architecture doc describes the
layout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reasons are recorded in benchmark/DECISIONS.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Add the changelog entry for queries reporting filter and sort errors
  instead of stopping later broadcasts, which 5.6.1 did.
- Drop the changelog entry on snapshotting builder-owned dependency sets,
  which 5.6.1 already did.
- Say in the changelog that the persistors' implementation details, such
  as SqlitePersistor.initDB, are internal, and say in the stubs why only
  constructors are mirrored.
- Note that the end-to-end benchmark needs the old ID generator import
  before 6.0.0, and fix a stale path and two stale comments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants