M6: stateful correlation across a time window

[dylanpatriarchi]

Milestone 6 — stateful correlation

Stack: 5 of 6 · base feat/m5-network-collector (PR #5) · next: M7 CLI and rule pack

The rules that justify the whole pipeline. Each step alone is ordinary; it's the sequence, from the same actor, inside a time window, that means something. Correlation rules have parsed and validated since M2 — this makes them fire.

One CorrelationState per rule, holding partial matches in an insertion-ordered seq rather than a hash table — iteration order is alert order, so determinism is structural instead of incidental. advance() does prune → advance → seed; seeding last is what stops one event walking a brand-new partial through two steps at once.

The design decisions, each pinned by a test

• The window anchors on the FIRST event of the chain, not the previous one. window: 60s should mean the whole chain happened inside a minute, which is what the rule author is saying. A per-hop anchor lets a ten-step chain paced at 59s gaps span ten minutes while calling itself a one-minute rule — and lets a partial match live forever, breaking the memory bound too. The cost (a chain of individually fast steps whose total span exceeds the window is a false negative) is documented and asserted.
• An event advances every partial match it can. Picking one is quieter but silently discards a chain that genuinely completed, which contradicts this project's own position that silent loss is unacceptable. Honest cost: near-duplicate alerts. Dedup belongs downstream.
• A completed chain is removed and does not re-fire, or a beaconing implant would re-alert on every callback.
• An empty join means all events share one bucket. Distinct from a join field being absent on an event, which makes it unjoinable entirely — counted separately.

Eviction is bounded and counted

Window expiry is not reported as loss — a window closing is the rule working. Hitting the per-rule cap is. The cap evicts the oldest partial, the opposite of the bus dropping the newest, and the comment says why: the bus must never stall a collector, whereas here every candidate is in hand and the oldest is nearest its window closing, so it has the least chance left of completing.

reset() clears partials and counters — a half-finished chain surviving into the next trace would be a fabricated detection.

One existing test changed, and why

test_engine.nim's "a correlation rule raises nothing through the single-event path" now fails — correctly. It was written when correlation was inert, and its rule (process → network, no join) genuinely completes twice on the sample trace once correlation works. There's no design that keeps it silent.

Narrowing its second step to net_accept (absent from the trace) keeps the real intent — an incomplete chain stays silent — and the reasoning is in a comment so the next reader doesn't think it was weakened to go green. Worth a second opinion on that call.

Verification

45 new tests, 639 total at this point, all green. Verified in an isolated worktree.

Known cost: prune and advance are O(partials) per event per rule. Bounded by the cap so it's a fixed ceiling, but at real event rates the partials want indexing by join key. Documented on the type rather than hidden.