journal: record what we ran, what came out, and what we concluded

.claude/settings.json

@@ -0,0 +1,38 @@
+{
+  "hooks": {
+    "PostToolUse": [
+      {
+        "matcher": "Bash",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "python3 \"$CLAUDE_PROJECT_DIR/scripts/journal.py\" --mode post-tool",
+            "timeout": 20
+          }
+        ]
+      }
+    ],
+    "SessionStart": [
+      {
+        "hooks": [
+          {
+            "type": "command",
+            "command": "python3 \"$CLAUDE_PROJECT_DIR/scripts/journal.py\" --mode session-start",
+            "timeout": 20
+          }
+        ]
+      }
+    ],
+    "Stop": [
+      {
+        "hooks": [
+          {
+            "type": "command",
+            "command": "python3 \"$CLAUDE_PROJECT_DIR/scripts/journal.py\" --mode stop",
+            "timeout": 20
+          }
+        ]
+      }
+    ]
+  }
+}

.claude/skills/ddrs-journal/SKILL.md

@@ -0,0 +1,78 @@
+---
+name: ddrs-journal
+description: Use when a ddrs run or experiment has finished, when a journal entry in docs/journal/ has unfilled TODO fields, or when recording what a training run or experiment concluded.
+---
+
+# ddrs research journal
+
+`docs/journal/YYYY-MM.md` is the running record of what we tried and what came of
+it. Facts are written for you by `scripts/journal.py` (hooked on `ddrs run` and
+`ddrs experiment`). Judgement is yours.
+
+## What fires, and when
+
+| Event | What happens |
+|---|---|
+| A Bash call matching `ddrs run` / `ddrs experiment` returns | Any finished, unjournaled run gets a ledger row; non-smoke runs also get an entry stub. You are told which. |
+| Your turn ends with a stub **this session** opened still unfilled | The Stop hook blocks and names it. |
+| A session starts | Runs finished outside a session are reported. Nothing is written. |
+
+Smoke tests (`--max-mini-batches`, fewer than 5 epochs) and experiment *shards*
+get a ledger row only. There is no question for them to answer.
+
+Manual: `python3 scripts/journal.py --mode status` (what is pending),
+`--mode backfill` (ledger rows for history), `--mode backfill --with-stubs`
+(also open entries — only for runs recent enough that you remember why).
+`DDRS_JOURNAL_OFF=1` disables the hooks for one command.
+
+## Filling an entry
+
+Four fields. Keep the whole entry under ~15 lines; if the conclusion needs more,
+it has earned a `docs/YYYY-MM-DD-<topic>-findings.md` and the entry links to it.
+
+**Question** — what we wanted to find out, phrased so "no" is a possible answer.
+
+- Good: "Does `nse-batch` + AdaDelta beat L1 on the 2,365-gauge CONUS set?"
+- Bad: "Improve NSE." That cannot come out false, so it cannot be answered.
+
+**What we did** — 2 to 4 bullets: the one thing under test, and what it was held
+against. Name the arm it is being compared to. If nothing changed but the seed,
+say that; seed spread is a result.
+
+**Result** — the number *next to the number it must beat*. A bare "NSE 0.72"
+is not a result. The comparison is the summed-Q' baseline for the **same gauge
+population** (`references/research-status.md` in `ddrs-dev` has the right bars;
+the CONUS one is 0.6781 NSE / 0.7172 KGE on 2,365 gauges). Record the metric the
+experiment was designed to move, not the one that happens to look best.
+
+**Conclusion** — supported, refuted, or inconclusive, and what changes next.
+"Inconclusive" is a real and frequent answer; write it rather than rounding a
+null result up into a weak positive.
+
+## Rules
+
+1. **Never invent a Question or Conclusion.** If you did not run the experiment
+   and cannot tell why it was run, write `unknown — run predates this entry` and
+   leave it. A journal that contains one confident guess is no longer evidence.
+2. **A NO-GO is a result and gets a full entry.** The most valuable thing in this
+   repo is the leakance non-identifiability finding, which closed a line of work.
+   Failed and aborted runs get entries too: record the failure mode.
+3. **Flag a dirty tree.** The facts block marks `**dirty**` when the run's git SHA
+   did not describe the working tree. That run is not reproducible; say so in the
+   conclusion rather than quietly citing its number later.
+4. **Watch for the stale-binary trap.** A run whose SHA looks current can still
+   have executed an old `~/.cargo/bin/ddrs`. Before trusting a surprising result,
+   check the checkpoint layout (see `ddrs-dev` trap T1) and note what you checked.
+5. **Link, don't duplicate.** If a findings doc, spec, or figure directory exists,
+   point at it. The journal is the index of the research, not a second copy of it.
+6. **Append only.** Never rewrite a past entry's conclusion. If it turns out to be
+   wrong, write a new entry saying so and link back. The record of having been
+   wrong is part of the research.
+
+## What this is not
+
+Not a changelog (that is git), not a run log (that is `.ddrs/runs/<id>/run.log`),
+and not a substitute for updating `ddrs-dev` when a run establishes a durable
+fact. CLAUDE.md's rule still holds: knowledge from a completed run belongs in the
+skill library in the same session that produced it. The journal records *that we
+learned it and when*; the skill records *what to do about it*.

CLAUDE.md

@@ -574,6 +574,37 @@ Subsequent plans on the same input set are cache hits and instant for both.
 Implementation: `src/baseline/`. Mirrors
 `~/projects/ddr/scripts/summed_q_prime.py`.
 
+## Research journal (`docs/journal/`, added 2026-09-10)
+
+One file per month, append-only, recording what we tried and what came of it.
+Two parts per month: a **ledger** with one row per completed run or experiment
+(including smoke tests and shards, so nothing is lost), and **entries** carrying
+four judgement fields — Question, What we did, Result, Conclusion.
+
+`scripts/journal.py` writes the facts, scraped from `.ddrs/runs/<id>/manifest.json`
+and `.ddrs/experiments/<name>/<ts>/manifest.json`. **It never writes judgement**,
+and neither should you invent it: an entry for a run whose purpose is unrecoverable
+says so. Hooks in `.claude/settings.json` run it after `ddrs run` / `ddrs experiment`
+(ledger row always; a `_TODO_` stub for anything that is not a smoke test), at
+session start (reports runs finished outside a session, writes nothing), and at
+turn end (blocks only on stubs the current session opened). `DDRS_JOURNAL_OFF=1`
+disables it for one command.
+
+```bash
+python3 scripts/journal.py --mode status     # what is pending
+python3 scripts/journal.py --mode backfill   # ledger rows for anything missed
+python3 scripts/test_journal.py              # stdlib only; no tier gate needed
+```
+
+A hook only sees commands Claude runs. A run started in your own terminal is
+caught at the next session start, not immediately. Entries before 2026-09-10 are
+ledger-only by construction.
+
+The journal is the index of the research, not a replacement for it: long-form
+results still earn a `docs/YYYY-MM-DD-<topic>-findings.md`, and a durable fact
+still belongs in the skill library in the same session that produced it. Rules for
+writing an entry: `.claude/skills/ddrs-journal/SKILL.md`.
+
 ## Conventions specific to this repo
 
 - **Read DDR first when porting.** Cite line numbers in comments
@@ -590,11 +621,12 @@ Implementation: `src/baseline/`. Mirrors
 
 ## When in doubt
 
-- **Two skills cover this repo.** `ddrs-dev` — building, coding, configuring,
+- **Three skills cover this repo.** `ddrs-dev` — building, coding, configuring,
   testing, running, debugging; its `references/` carry the full config reference,
   the change→gate matrix, the trap catalog, and the authoritative research-status
-  numbers. `ddrs-eval-plots` — evaluating and visualizing run output. The other 16
-  skills were consolidated into these two on 2026-07-30; see
+  numbers. `ddrs-eval-plots` — evaluating and visualizing run output.
+  `ddrs-journal` — recording what an experiment concluded (see §Research journal).
+  The other 16 skills were consolidated on 2026-07-30; see
   `docs/2026-07-30-docs-and-skills-audit.md`.
 - Sparse / autograd questions → `docs/reference/burn-autograd.md`
 - Algorithm questions → `.claude/ARCHITECTURE.md` and `~/projects/ddr/CLAUDE.md`

docs/journal/2026-06.md

@@ -0,0 +1,38 @@
+# Research journal — 2026-06
+
+Facts are written automatically from run manifests by `scripts/journal.py`.
+The Question / What we did / Result / Conclusion fields are judgement and are
+written by hand. See `.claude/skills/ddrs-journal/SKILL.md` for what makes a
+good entry. Append-only: on a merge conflict, keep both sides.
+
+## Ledger
+
+Every completed run, including smoke tests, so nothing is lost.
+
+<!-- ledger:start -->
+| finished (UTC) | id | kind | result | git |
+|---|---|---|---|---|
+| 2026-06-02 02:36 | `2026-06-02T02-10-30Z-train-and-test` | run | NSE 0.6937 | `de7e80b`+dirty |
+| 2026-06-02 13:14 | `2026-06-02T12-48-52Z-train-and-test` | run | NSE 0.6937 / KGE 0.7206 | `6ad4d41`+dirty |
+| 2026-06-03 10:13 | `2026-06-03T09-45-09Z-train-and-test` | run | NSE 0.6937 / KGE 0.7206 | `61ae38d`+dirty |
+| 2026-06-03 13:39 | `2026-06-03T13-13-26Z-train-and-test` | run | NSE 0.6978 / KGE 0.7233 | `856930a`+dirty |
+| 2026-06-04 00:05 | `2026-06-03T23-38-42Z-train-and-test` | run | NSE 0.6980 / KGE 0.7234 | `30501af`+dirty |
+| 2026-06-04 01:01 | `2026-06-04T00-35-13Z-train-and-test` | run | NSE 0.6980 / KGE 0.7234 | `4728dd8`+dirty |
+| 2026-06-04 01:29 | `2026-06-04T01-03-23Z-train-and-test` | run | NSE 0.7001 / KGE 0.7238 | `4728dd8`+dirty |
+| 2026-06-04 02:23 | `2026-06-04T01-57-45Z-train-and-test` | run | NSE 0.7001 / KGE 0.7239 | `15f7d00`+dirty |
+| 2026-06-04 17:13 | `2026-06-04T16-46-58Z-train-and-test` | run | NSE 0.6996 / KGE 0.7238 | `3b907f1`+dirty |
+| 2026-06-04 22:31 | `2026-06-04T22-31-49Z-train-and-test` | run | status `failed` | `27b36e0`+dirty |
+| 2026-06-04 23:25 | `2026-06-04T22-58-43Z-train-and-test` | run | NSE 0.7004 / KGE 0.7243 | `9422006`+dirty |
+| 2026-06-05 01:40 | `2026-06-05T01-40-41Z-train-and-test` | run | status `failed` | `9422006`+dirty |
+| 2026-06-05 02:07 | `2026-06-05T01-41-16Z-train-and-test` | run | NSE 0.7004 / KGE 0.7244 | `6588b9f`+dirty |
+| 2026-06-23 02:29 | `2026-06-23T01-43-07Z-conus-hourly-train-and-test` | run | trained, no eval metrics | `ba6b3c1`+dirty |
+| 2026-06-23 02:59 | `2026-06-23T02-46-56Z-conus-hourly-train` | run | trained, no eval metrics | `a5972d9`+dirty |
+| 2026-06-23 03:59 | `2026-06-23T02-49-12Z-conus-hourly-train-and-test` | run | NSE 0.7153 / KGE 0.7104 | `a5972d9`+dirty |
+| 2026-06-23 13:56 | `2026-06-23T13-03-35Z-train-and-test` | run | NSE 0.6957 / KGE 0.6926 | `3c33994`+dirty |
+| 2026-06-24 01:13 | `2026-06-24T00-03-01Z-conus-hourly-train-and-test` | run | NSE 0.7101 / KGE 0.7099 | `19955d1`+dirty |
+| 2026-06-24 03:39 | `2026-06-24T02-10-49Z-conus-hourly-train-and-test` | run | NSE 0.7151 / KGE 0.7086 | `8908f0b`+dirty |
+<!-- ledger:end -->
+
+## Entries
+
+<!-- entries -->

docs/journal/2026-07.md

@@ -0,0 +1,44 @@
+# Research journal — 2026-07
+
+Facts are written automatically from run manifests by `scripts/journal.py`.
+The Question / What we did / Result / Conclusion fields are judgement and are
+written by hand. See `.claude/skills/ddrs-journal/SKILL.md` for what makes a
+good entry. Append-only: on a merge conflict, keep both sides.
+
+## Ledger
+
+Every completed run, including smoke tests, so nothing is lost.
+
+<!-- ledger:start -->
+| finished (UTC) | id | kind | result | git |
+|---|---|---|---|---|
+| 2026-07-01 01:56 | `2026-07-01T01-07-17Z-train-and-test` | run | NSE 0.6990 / KGE 0.7234 | `2cdd341` |
+| 2026-07-01 02:47 | `2026-07-01T01-58-03Z-train-and-test` | run | NSE 0.6990 / KGE 0.7234 | `2cdd341` |
+| 2026-07-01 15:00 | `2026-07-01T13-43-32Z-train-and-test` | run | NSE 0.7145 / KGE 0.7150 | `b5bd087`+dirty |
+| 2026-07-01 22:19 | `2026-07-01T21-20-27Z-train-and-test` | run | NSE 0.6963 / KGE 0.7250 | `b5bd087`+dirty |
+| 2026-07-02 11:28 | `2026-07-02T11-27-48Z-daily-lstm-train` | run | trained, no eval metrics | `ed6653c` |
+| 2026-07-02 11:32 | `2026-07-02T11-30-33Z-hourly-lstm-train` | run | trained, no eval metrics | `ed6653c` |
+| 2026-07-02 11:32 | `2026-07-02T11-32-36Z-hourly-lstm-train` | run | status `failed` | `ed6653c` |
+| 2026-07-02 11:40 | `2026-07-02T11-40-16Z-daily-lstm-train` | run | trained, no eval metrics | `ed6653c`+dirty |
+| 2026-07-02 11:40 | `2026-07-02T11-40-47Z-hourly-lstm-train` | run | status `failed` | `ed6653c`+dirty |
+| 2026-07-07 04:49 | `2026-07-07T03-55-53Z-train-and-test` | run | NSE 0.5894 / KGE 0.6219 | `b261e1d` |
+| 2026-07-07 06:50 | `2026-07-07T04-49-19Z-train-and-test` | run | NSE 0.6201 / KGE 0.5969 | `b261e1d` |
+| 2026-07-07 09:51 | `2026-07-07T06-50-28Z-train-and-test` | run | NSE 0.5543 / KGE 0.4852 | `b261e1d` |
+| 2026-07-10 17:22 | `2026-07-10T14-33-16Z-train-and-test` | run | NSE 0.7294 / KGE 0.7028 | `7cc4081`+dirty |
+| 2026-07-11 05:37 | `2026-07-11T02-45-27Z-train-and-test` | run | NSE 0.7274 / KGE 0.7051 | `98f464c` |
+| 2026-07-11 17:08 | `2026-07-11T14-20-47Z-train-and-test` | run | NSE 0.7281 / KGE 0.7112 | `98f464c`+dirty |
+| 2026-07-12 04:25 | `2026-07-12T02-25-05Z-train-and-test` | run | NSE 0.6777 / KGE 0.6785 | `98f464c`+dirty |
+| 2026-07-13 00:12 | `2026-07-12T18-25-24Z-train-and-test` | run | NSE 0.6766 / KGE 0.7223 | `98f464c`+dirty |
+| 2026-07-13 07:08 | `2026-07-13T00-15-47Z-train-and-test` | run | NSE 0.7330 / KGE 0.7188 | `9178934` |
+| 2026-07-16 10:49 | `2026-07-16T02-23-20Z-train-and-test` | run | NSE 0.5259 / KGE 0.5175 | `ec84987`+dirty |
+| 2026-07-16 14:47 | `2026-07-16T11-31-52Z-train-and-test` | run | NSE 0.5543 / KGE 0.4852 | `ec84987`+dirty |
+| 2026-07-29 07:11 | `2026-07-29T01-39-25Z-train-and-test` | run | NSE 0.6734 / KGE 0.6845 | `98ac814`+dirty |
+| 2026-07-30 00:55 | `2026-07-30T00-54-17Z-train` | run | trained, no eval metrics | `1242216`+dirty |
+| 2026-07-30 08:22 | `2026-07-30T00-24-24Z-train-and-test` | run | NSE 0.6799 / KGE 0.7194 | `b12c3ff`+dirty |
+| 2026-07-30 20:46 | `2026-07-30T01-58-07Z-train-and-test` | run | NSE 0.6209 / KGE 0.6903 | `b12c3ff`+dirty |
+| 2026-07-31 14:04 | `2026-07-31T00-54-47Z-train-and-test` | run | NSE 0.6143 / KGE 0.6853 | `24cb4d0`+dirty |
+<!-- ledger:end -->
+
+## Entries
+
+<!-- entries -->

docs/journal/2026-08.md

@@ -0,0 +1,32 @@
+# Research journal — 2026-08
+
+Facts are written automatically from run manifests by `scripts/journal.py`.
+The Question / What we did / Result / Conclusion fields are judgement and are
+written by hand. See `.claude/skills/ddrs-journal/SKILL.md` for what makes a
+good entry. Append-only: on a merge conflict, keep both sides.
+
+## Ledger
+
+Every completed run, including smoke tests, so nothing is lost.
+
+<!-- ledger:start -->
+| finished (UTC) | id | kind | result | git |
+|---|---|---|---|---|
+| 2026-08-01 17:04 | `2026-08-01T03-01-18Z-train-and-test` | run | NSE 0.6855 / KGE 0.7207 | `6fd31c0`+dirty |
+| 2026-08-02 12:37 | `2026-08-02T00-13-35Z-train-and-test` | run | NSE 0.6758 / KGE 0.7179 | `a7fe36e`+dirty |
+| 2026-08-03 17:00 | `2026-08-03T13-11-00Z-train-and-test` | run | NSE 0.6738 / KGE 0.6960 | `e91541a`+dirty |
+| 2026-08-04 17:06 | `2026-08-04T13-53-07Z-train-and-test` | run | NSE 0.6782 / KGE 0.7186 | `5858d15`+dirty |
+| 2026-08-04 21:03 | `2026-08-04T17-53-55Z-train-and-test` | run | NSE 0.7043 / KGE 0.7433 | `feeba13`+dirty |
+| 2026-08-05 13:22 | `2026-08-05T04-58-58Z-conus-experimental-train-and-test` | run | NSE 0.6201 / KGE 0.6986 | `a2a1c65`+dirty |
+| 2026-08-09 09:30 | `2026-08-09T03-05-54Z-train-and-test` | run | NSE 0.7058 / KGE 0.7296 | `54cd386`+dirty |
+| 2026-08-09 12:04 | `2026-08-09T09-30-39Z-train-and-test` | run | NSE 0.7071 / KGE 0.7379 | `54cd386`+dirty |
+| 2026-08-09 14:38 | `2026-08-09T12-05-08Z-train-and-test` | run | NSE 0.5775 / KGE 0.6160 | `54cd386`+dirty |
+| 2026-08-09 22:13 | `2026-08-09T22-13-26Z-train-and-test` | run | status `failed` | `54cd386`+dirty |
+| 2026-08-10 04:59 | `2026-08-09T22-13-43Z-train-and-test` | run | NSE 0.4833 / KGE 0.5039 | `8140f0d`+dirty |
+| 2026-08-17 04:07 | `2026-08-16T21-23-48Z-train-and-test` | run | NSE 0.5694 / KGE 0.5342 | `e44cd53`+dirty |
+| 2026-08-17 12:40 | `2026-08-17T02-03-02Z-train-and-test` | run | NSE 0.7344 / KGE 0.7567 | `e44cd53`+dirty |
+<!-- ledger:end -->
+
+## Entries
+
+<!-- entries -->