AniList coverage and the whole-operation derivation

.github/workflows/ci.yml

@@ -127,7 +127,7 @@ jobs:
       # The device tests and the ingest benchmark need a device and do not
       # run here; the benchmark's APKs build.
       - run: >
-          gradle --no-daemon :baton-gradle:build :baton:build :baton:jvmRelayTest :baton:jvmFateTest
+          gradle --no-daemon :baton-gradle:build :baton:build :baton:jvmRelayTest :baton:jvmFateTest :baton:jvmAniListTest
           :baton-testing:build :baton-inspector:build :baton-okhttp:build
           :goldens:compileKotlinJvm :samples:shared:build :samples:desktop:build
           :samples:exchange:build :samples:github:build :samples:android:assembleDebug

BENCHMARKS.md

@@ -21,6 +21,83 @@ Best ingest and best commit of the fixture at each release below.
   <img alt="Ingest and commit, best, from 0.1.0 through 0.6.0" src="benchmarks/charts/read-path.svg">
 </picture>
 
+## Unreleased, the AniList list — 2026-10-18
+
+Revision: `c4b9078`. Machine: Apple M1 Pro (MacBook Pro), macOS 26.5.2,
+Xcode 26.6, Swift 6.3.3, release build; the JVM is OpenJDK 21.0.12.1
+(Homebrew) on the same machine. `swift run -c release BatonBenchmarks`,
+the whole suite, three runs, then `gradle :baton:runAniListBench`, the
+Kotlin twin of the suite's AniList section, two runs; best of the runs,
+the median of the medians in parentheses. Each JVM measurement takes as
+many warm-up rounds as samples before it samples. Not a quiet machine:
+the load average stood between 8 and 14 and other sessions were building.
+
+The fixture is `spec/anilist/`: two chunks of AniList's
+`MediaListCollection`, 500 entries each, 457,499 and 458,874 bytes,
+recorded from a public list and anonymized (`spec/anilist/README.md`);
+6,014 records after both. *Every field* is the row's fragment and its
+media's card, 44 slots an entry, 44,006 over the list; *five values* is
+the reading of the derived-state recipe's model, seven slots an entry.
+The save is entry 991's `progress` changed by a payload of the mutation's
+shape, committed with a body observing every row and a detail's body
+observing entry 991, which the lookup `MediaList(id:)` answers from the
+list's records.
+
+### The list into the store, the image and a launch
+
+| Measurement | Swift | Kotlin, the JVM |
+|---|---|---|
+| Ingest of a chunk of 500 entries, bytes → change set, per chunk | 1.81 ms (1.95) | 1.66 ms (1.79) |
+| Commit of both chunks into an empty store, 6,014 records | 5.26 ms (5.45) | 1.58 ms (2.01) |
+| Write-behind of both chunks, off the main actor or the store's thread | 5.92 ms (10.0) | 12.5 ms (16.3) |
+| The image after both chunks, closed | 811,008 bytes | 811,008 bytes |
+| A launch in a new process: open and hydrate at once, both handles ready | 21.6 ms (23.0) | 297 ms (307) |
+| The same, once the image has opened off the main actor | 18.1 ms (19.4) | 97.5 ms (101) |
+| The save's commit, 1,000 row bodies and a detail observing | 27.4 µs (34.0) | 108 µs (114) |
+| Notifications of that commit | 2 | 2 |
+
+The ingest of a 500-entry chunk is 3.6 µs an entry in Swift and 3.3 µs on
+the JVM. The commit of the two chunks is 0.87 µs a record in Swift,
+against 0.51 µs a record for the 899-record fixture in the same runs
+(462 µs). A launch from the
+image reaches both handles ready, 6,014 records hydrated, in 21.6 ms in
+Swift, 18.1 ms of it the hydration once the file is open: 3.0 µs a
+record. The JVM's launch is a new JVM, interpreting and loading classes
+as it goes, and says little of a launch on Android, which a device would
+measure. The save notifies the two bodies that read `progress` of
+`MediaList:991`, the row's and the detail's, and none of the other 999
+rows; the count is in `benchmarks/counts.txt`.
+
+### A reading of the whole list, tracked and untracked
+
+| Measurement | Swift | Kotlin, the JVM |
+|---|---|---|
+| Every field, untracked, the whole list | 891 µs (923) | 646 µs (849) |
+| The same, per field | 20.2 ns (21.0) | 13.6 ns (15.0) |
+| The first reading of a fresh store, untracked, its channels made | | 1.13 ms (1.33) |
+| Every field inside one tracking scope or read-observing snapshot, the whole list | 28.5 ms (29.9) | 2.56 ms (3.13) |
+| The same, per field | 647 ns (682) | 57.3 ns (62.5) |
+| One chunk's every field in its own scope, 500 entries | 14.2 ms (14.9) | 1.20 ms (1.27) |
+| One chunk's five values in its own scope | 2.57 ms (2.68) | 264 µs (293) |
+| Every field through `Observations` or `snapshotFlow`, made to its first value | 28.3 ms (29.8) | 2.16 ms (2.56) |
+| A one-field save in a chunk, the commit to that chunk's reading delivered, every field | 15.6 ms (16.1) | 1.16 ms (1.40) |
+| The same, the chunk read for five values | 2.70 ms (2.80) | 305 µs (377) |
+
+In the same Swift runs, the suite's untracked lens read was 20.3 ns a
+field and its tracked read of a row body of eight fields 585 ns (608).
+A tracked field in one scope over the whole list costs 647 ns, and 643 ns
+over one chunk: linear in the scope's size, and what a row body's field
+costs, about thirty untracked reads. That is Observation's registrar,
+an access registered with each record and an observer installed on it
+when the scope ends; through `Observations` the first value costs the same
+as the scope. On the JVM a tracked field is 57.3 ns, four untracked reads:
+Compose's read observer adds the slot's state to a set, and the channel,
+made at a slot's first read, is made by the untracked reading too, which
+costs 1.13 ms on a fresh store where a later one costs 646 µs. The
+composition's re-read after a save is the chunk's tracked reading and the
+delivery: 15.6 ms in Swift for every field, 2.70 ms for the model's five
+values.
+
 ## Unreleased, a streamed list — 2026-10-13
 
 Revision: the working tree on top of `d027f18`; the runtime change was

CHANGELOG.md

@@ -27,6 +27,51 @@ are expected and listed without apology.
   a mutation's response moves the row with no updater, and carried by the
   image to the next launch. The README, the docs index and the site link
   them.
+- `docs/recipes/derived-state.md`, *A derivation over a whole operation*:
+  how a model that derives one value from every entry of a large
+  operation keeps the tracked reads off the launch and each change to one
+  chunk. One `Observations` or `snapshotFlow` per operation handle, the
+  first reading untracked and synchronous before the streams start, a
+  reading applied only when it differs from the last, and a closure that
+  reads only what the value needs. With the arithmetic to budget a pass of
+  N entries by M fields: a field read costs 20 ns untracked and 647 ns
+  tracked in Swift, and 14 and 57 ns on the JVM. The model is compiled on
+  both runtimes beside the AniList tests and proven by `ListModelTests`. No
+  coarser observation channel is added: per-slot registration is what the
+  store's principle rests on, and the record
+  [derived state is observed, not signaled](docs/decisions/derived-state-is-observed-not-signaled.md)
+  now names the operation-level and per-record channels as not chosen and
+  reopens on a whole-operation derivation the composition cannot keep in
+  budget at a measured size ([#53](https://github.com/shergin/baton/issues/53)).
+- `spec/anilist/`: a user's list recorded from AniList's public API, two
+  chunks of `MediaListCollection` of 500 entries each, with the SDL the API
+  answers to the standard introspection query. The user's id, the entries'
+  ids and the notes are anonymized; the rest is the server's bytes. Frozen
+  by one dump of 6,014 records on both runtimes, by the Swift target
+  `BatonAniListTests` and the Kotlin harness's `jvmAniListTest`, with an
+  entry answered by the lookup `MediaList(id:)` from the list's records and
+  a save that notifies the two bodies that read the field it changed
+  ([#51](https://github.com/shergin/baton/issues/51)).
+- Bench rows for that list on both runtimes, the AniList section of
+  `swift run -c release BatonBenchmarks` and its Kotlin twin
+  `gradle :baton:runAniListBench`: the ingest of a 500-entry chunk, 1.81 ms
+  in Swift; the image after both chunks, 811,008 bytes; a launch from it
+  to both handles ready, 21.6 ms; a one-field save's notifications, 2; and
+  a tracked reading of the whole list, 28.5 ms in Swift and 2.6 ms on the
+  JVM, against 0.89 and 0.65 ms untracked. Recorded in `BENCHMARKS.md`,
+  18 October.
+- The test schema learns AniList's shapes: capitalized root fields, one
+  spelled like its type, an enum that grows behind a `version` argument,
+  and a union of twenty members with no interface. The goldens of both
+  emitters hold documents over them, and three cases,
+  `capitalized-roots-1`, `episode-status-1` and `notifications-1`, freeze
+  them at run time on both runtimes. A root field's accessor keeps its
+  spelling, `MediaListCollection`, and its lens takes a number,
+  `MediaListCollection2`.
+- AniList's schema is a second set of the compiler's goldens, Swift,
+  Kotlin and plans, compiled with its own configuration from the AniList
+  documents and built by a Swift target and the Kotlin goldens module of
+  their own.
 
 ## 0.17.0 (Kulich) — 2026-10-18
 

Package.swift

@@ -92,7 +92,7 @@ let package = Package(
             exclude: ["README.md", "runtime.md"],
             sources: ["Spec.swift", "Manifest.swift"],
             resources: [
-                .copy("documents"), .copy("fate"), .copy("manifest.json"), .copy("relay"), .copy("rickandmorty"),
+                .copy("anilist"), .copy("documents"), .copy("fate"), .copy("manifest.json"), .copy("relay"), .copy("rickandmorty"),
                 .copy("scripts"), .copy("sources"), .copy("tests"), .copy("tokenizer"),
             ],
             swiftSettings: [.swiftLanguageMode(.v6)]
@@ -122,6 +122,24 @@ let package = Package(
             swiftSettings: [.swiftLanguageMode(.v6)],
             plugins: ["BatonPlugin"]
         ),
+        // The documents of the list recorded from AniList under
+        // `spec/anilist/`, compiled against the schema recorded beside it,
+        // for the tests that freeze the recording and the bench that
+        // measures it.
+        .target(
+            name: "AniListDocuments",
+            dependencies: ["Baton"],
+            path: "swift/Tests/AniListDocuments",
+            exclude: ["baton.json"],
+            swiftSettings: [.swiftLanguageMode(.v6)],
+            plugins: ["BatonPlugin"]
+        ),
+        .testTarget(
+            name: "BatonAniListTests",
+            dependencies: ["AniListDocuments", "Baton", "BatonInspector", "BatonSpec", "BatonTesting"],
+            path: "swift/Tests/BatonAniListTests",
+            swiftSettings: [.swiftLanguageMode(.v6)]
+        ),
         // The emitter's goldens, compiled as an app compiles them with the
         // Xcode template's default of main-actor isolation: generated code
         // states its isolation, so it builds under either default.
@@ -131,6 +149,15 @@ let package = Package(
             path: "compiler/src/tests/goldens",
             swiftSettings: [.swiftLanguageMode(.v6), .defaultIsolation(MainActor.self)]
         ),
+        // The goldens of the second schema, AniList's, compiled the same way
+        // in a module of their own, since each schema's shared file declares
+        // its own types and slots.
+        .testTarget(
+            name: "BatonAniListGoldenTests",
+            dependencies: ["Baton"],
+            path: "compiler/src/tests/goldens-anilist",
+            swiftSettings: [.swiftLanguageMode(.v6), .defaultIsolation(MainActor.self)]
+        ),
         .testTarget(
             name: "BatonMacrosTests",
             dependencies: [
@@ -187,7 +214,7 @@ let package = Package(
         ),
         .executableTarget(
             name: "BatonBenchmarks",
-            dependencies: ["Baton", "BatonTesting", "BatonSpec"],
+            dependencies: ["AniListDocuments", "Baton", "BatonTesting", "BatonSpec"],
             path: "swift/Benchmarks",
             swiftSettings: [.swiftLanguageMode(.v6)],
             plugins: ["BatonPlugin"]

benchmarks/counts.txt

@@ -15,3 +15,5 @@ count notifications-refetch-first-page 1
 count log-events-during-edge-commits 0
 count eviction-kept-the-last-launch 1
 count eviction-dropped-the-older-launch 1
+count records-anilist-two-chunks 6014
+count notifications-anilist-save-one-field 2

compiler/src/tests/emit_tests.rs

@@ -6,7 +6,10 @@
 //! goldens. Those
 //! documents exist to exercise every directive the runtime's tests prove, so
 //! they cover every shape the lowering and the emitter write, and a change to
-//! either shows up here as a diff to review.
+//! either shows up here as a diff to review. A second schema, AniList's, and
+//! the documents of the list recorded from it have goldens of their own,
+//! `goldens-anilist/`, `goldens-anilist-kotlin/` and `plans-anilist/`: a
+//! production schema's shapes, compiled with a configuration of their own.
 //!
 //! After an intended change, `BATON_BLESS=1 cargo test` rewrites the goldens.
 
@@ -92,13 +95,19 @@ fn compile_kotlin_tests() -> (Plan, BTreeMap<String, Option<String>>) {
 /// named from the repository's root, with the package each Kotlin host
 /// declares, by its source.
 fn compile_sources(hosts: &[PathBuf]) -> (Plan, BTreeMap<String, Option<String>>) {
-    let target = repository().join("swift/Tests/BatonTests");
+    compile_target(&repository().join("swift/Tests/BatonTests"), hosts)
+}
+
+/// The plan of a Swift target's sources, compiled with the `baton.json`
+/// beside them, and of `hosts`, named from the repository's root, with the
+/// package each Kotlin host declares, by its source.
+fn compile_target(target: &Path, hosts: &[PathBuf]) -> (Plan, BTreeMap<String, Option<String>>) {
     let config_path = target.join("baton.json");
-    let config = swift_tests_config();
+    let config = Config::load(&config_path).expect("the target has a baton.json");
     let schema_path = config.schema_path(&config_path);
     let schema = std::fs::read_to_string(&schema_path).expect("the test schema is readable");
-    let mut sources: Vec<PathBuf> = std::fs::read_dir(&target)
-        .expect("the test target is readable")
+    let mut sources: Vec<PathBuf> = std::fs::read_dir(target)
+        .expect("the target is readable")
         .filter_map(|entry| entry.ok().map(|entry| entry.path()))
         .filter(|path| {
             path.extension()
@@ -199,6 +208,12 @@ fn plan_swift_tests() -> BTreeMap<String, String> {
         .retain(|fragment| fragment.source != hostile_names::CORPUS);
     plan.operations
         .retain(|operation| operation.source != hostile_names::CORPUS);
+    plan_by_source(&plan)
+}
+
+/// A plan as `batonc plan` prints it, one source's documents to a golden,
+/// each with the enums and inputs its own documents read or pass.
+fn plan_by_source(plan: &Plan) -> BTreeMap<String, String> {
     let mut by_source: BTreeMap<String, Plan> = BTreeMap::new();
     for fragment in &plan.fragments {
         by_source
@@ -460,6 +475,119 @@ fn the_report_reproduces_its_golden_byte_for_byte() {
     check_goldens(reports(), &report_swift_tests(), ".report.json");
 }
 
+/// The Swift target of the documents of the list recorded from AniList under
+/// `spec/anilist/`, compiled against the schema AniList answered by
+/// introspection with the `baton.json` beside them.
+const ANILIST_TARGET: &str = "swift/Tests/AniListDocuments";
+
+/// The AniList documents' configuration.
+fn anilist_config() -> Config {
+    Config::load(&repository().join(ANILIST_TARGET).join("baton.json"))
+        .expect("the AniList target has a baton.json")
+}
+
+/// The plan of the AniList documents, named from the repository's root.
+fn compile_anilist() -> Plan {
+    compile_target(&repository().join(ANILIST_TARGET), &[]).0
+}
+
+/// The AniList documents' Swift goldens, which the Swift test target
+/// `BatonAniListGoldenTests` compiles.
+fn anilist_goldens() -> PathBuf {
+    Path::new(env!("CARGO_MANIFEST_DIR")).join("src/tests/goldens-anilist")
+}
+
+/// The AniList documents' Kotlin goldens, which the Gradle module
+/// `kotlin/goldens` compiles beside the first schema's, in a package of
+/// their own.
+fn anilist_kotlin_goldens() -> PathBuf {
+    Path::new(env!("CARGO_MANIFEST_DIR")).join("src/tests/goldens-anilist-kotlin")
+}
+
+/// The AniList documents' plan goldens.
+fn anilist_plans() -> PathBuf {
+    Path::new(env!("CARGO_MANIFEST_DIR")).join("src/tests/plans-anilist")
+}
+
+/// The Swift the AniList documents generate, by output name, the shared
+/// file among them.
+fn emit_anilist_swift() -> BTreeMap<String, String> {
+    let output = swift(&compile_anilist(), &anilist_config()).unwrap_or_else(|duplicates| {
+        let messages: Vec<String> = duplicates.iter().map(ToString::to_string).collect();
+        panic!(
+            "the AniList documents emit names twice:\n{}",
+            messages.join("\n")
+        )
+    });
+    let mut files: BTreeMap<String, String> = output
+        .files
+        .into_iter()
+        .map(|(source, text)| (golden_name(&source, ".baton.swift"), text))
+        .collect();
+    files.insert("Baton.baton.swift".to_string(), output.shared);
+    files
+}
+
+/// The Kotlin the AniList documents generate under the package
+/// `baton.goldens.anilist`, by output name, the shared file among them.
+fn emit_anilist_kotlin() -> BTreeMap<String, String> {
+    let mut config = anilist_config();
+    config.language = crate::config::Language::Kotlin;
+    config.kotlin = Some(crate::config::KotlinConfig {
+        package: Some("baton.goldens.anilist".to_string()),
+    });
+    let output = super::kotlin::kotlin(&compile_anilist(), &config, BTreeMap::new())
+        .unwrap_or_else(|duplicates| {
+            let messages: Vec<String> = duplicates.iter().map(ToString::to_string).collect();
+            panic!(
+                "the AniList documents emit names twice in Kotlin:\n{}",
+                messages.join("\n")
+            )
+        });
+    let mut files: BTreeMap<String, String> = output
+        .files
+        .into_iter()
+        .map(|(source, text)| (golden_name(&source, ".baton.kt"), text))
+        .collect();
+    files.insert("Baton.baton.kt".to_string(), output.shared);
+    files
+}
+
+#[test]
+fn the_swift_emitter_reproduces_the_anilist_goldens_byte_for_byte() {
+    check_goldens(anilist_goldens(), &emit_anilist_swift(), ".baton.swift");
+}
+
+#[test]
+fn the_kotlin_emitter_reproduces_the_anilist_goldens_byte_for_byte() {
+    check_goldens(
+        anilist_kotlin_goldens(),
+        &emit_anilist_kotlin(),
+        ".baton.kt",
+    );
+}
+
+#[test]
+fn the_plan_reproduces_the_anilist_goldens_byte_for_byte() {
+    check_goldens(
+        anilist_plans(),
+        &plan_by_source(&compile_anilist()),
+        ".plan.json",
+    );
+}
+
+#[test]
+fn compiling_the_anilist_documents_twice_emits_the_same_bytes() {
+    assert!(
+        emit_anilist_swift() == emit_anilist_swift(),
+        "two compilations of the AniList documents emitted different Swift"
+    );
+    assert!(
+        emit_anilist_kotlin() == emit_anilist_kotlin(),
+        "two compilations of the AniList documents emitted different Kotlin"
+    );
+}
+
 #[test]
 fn compiling_the_same_sources_twice_emits_the_same_bytes() {
     assert!(