Repository navigation
fix(feed): Kalshi changed its wire format and the books went quietly empty - #69
Merged
Merged
Conversation
…empty Kalshi's orderbook_delta channel now sends decimal dollar strings under new field names: snapshots carry yes_dollars_fp / no_dollars_fp holding ["0.5100", "1200.50"] pairs, and deltas carry price_dollars / delta_fp with a signed, fractional size. The parser asked for yes / no and get_int64, so nothing matched. The failure mode is the reason this is a fix and not a feature. A missing side key was treated as "a legal empty book", so every snapshot parsed successfully into nothing at all: books empty, malformed counter zero, integrity checks green. A live capture reported 0 malformed and 6 deltas where it should have had 240. Throughput and integrity counters cannot see this - only reading prices out of the book can. Both wire forms are accepted so the captures under docs/bench/ still replay byte for byte. Prices reuse parse_cents, whose OffGrid rejection refuses to round a sub-cent price (Kalshi now lists some markets on a 0.0001 grid) into a neighbouring cent it can represent. Sizes get to_rounded_contracts: signed, because a resting level shrinks as well as grows, and rounded to whole contracts rather than kSizeScale, because a Kalshi contract is a countable thing and not a fractional base unit. An unrecognised snapshot is now Malformed rather than empty. An empty Kalshi book still sends its side keys holding empty arrays, so keys absent under every known name means a schema this parser does not know, and saying so loudly is the whole point.
…ands
.gitignore covered secrets/, which protects a path and nothing else. A key
pasted into a scratch file at the root, or into a markdown note, committed
cleanly - and that is exactly how keys reach public history.
scripts/check_no_secrets.py scans content instead of paths: private key
blocks under any PEM label, and the exact value of any credential sitting
in secrets/, since a bare UUID is too generic to blocklist by shape. It
runs over the staged set from scripts/git-hooks/pre-commit and over every
tracked file standalone, so CI catches what a clone without
core.hooksPath would miss.
Both cases are fault-injected rather than assumed: a key pasted into
docs/notes_scratch.md and a key id pasted into src/cli/usage.cpp were each
rejected with HEAD unmoved. The .md case is the one that matters, because
the extension rules below would have waved it straight through.
Enable in a fresh clone with:
git config core.hooksPath scripts/git-hooks
This was referenced Sep 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Kalshi's
orderbook_deltachannel now sends decimal dollar strings under newfield names. Snapshots carry
yes_dollars_fp/no_dollars_fpholding["0.5100", "1200.50"]pairs; deltas carryprice_dollars/delta_fpwith asigned, fractional size. The parser asked for
yes/noandget_int64.Why this is a fix and not a feature
A missing side key was treated as "a legal empty book", so every snapshot
parsed successfully into nothing:
Books empty, malformed zero, integrity green. Throughput and integrity counters
structurally cannot see this - only reading prices out of the book can. It was
found by pointing the engine at production, not by any test in the suite.
What changed
Both wire forms are accepted, so the captures under
docs/bench/still replaybyte for byte and all 250 existing tests pass unchanged.
parse_cents, whoseOffGridrejection refuses to round asub-cent price (Kalshi now lists some markets on a 0.0001 grid) into a
neighbouring cent it can represent.
to_rounded_contracts: signed, because a resting level shrinks aswell as grows, and rounded to whole contracts rather than
kSizeScale,because a Kalshi contract is countable, not a fractional base unit.
Malformed. An empty Kalshi book still sendsits side keys holding empty arrays, so keys absent under every known name
means a schema this parser does not know.
Validation
Five new tests cover the new shape, the old one, empty-but-legal books, and
sub-cent rejection. The regression guard was fault-injected: removing
!saw_sidefailsSnapshotWithNoRecognisedSideKeyIsMalformedspecifically,so the test can actually fail.
Verified live against the September 2026 FOMC decision quoted on both venues
(both closing 2026-09-16), which now produces a real cross-venue basis:
Also here
chore(security)adds content-based secret scanning..gitignorecoveredsecrets/, which protects a path; a key pasted into a markdown note committedcleanly. Both the key-in-
.mdand key-id-in-.cppcases are fault-injected.