diff --git a/README.md b/README.md index 0b83bf5..62271b7 100644 --- a/README.md +++ b/README.md @@ -1,86 +1,151 @@ # basis -A real-time, cross-venue market-data engine in C++20. It reads live order -books from several venues over TLS WebSocket, normalizes them into one -schema, and serves them to consumers through a BLPAPI-style interface - -subscription and request, both entitlement-checked. - -That is four pieces, and the middle two are the ones a market-data team -spends its time on: - - **feed handlers.** Four venue parsers behind one adapter seam, each - turning a venue's own dialect into one canonical delta. Sequence gaps - are detected and answered by dropping the book and re-requesting a - snapshot, because a book known to be stale must never be served. - **normalization.** Venue-native market ids disappear at the registry; - everything downstream is keyed by a venue-neutral event id. - **distribution.** Conflated fan-out, so a slow consumer receives the - current price rather than a backlog, with memory bounded by subscribers - times topics rather than by publish rate. - **entitlements.** Default-deny, granted per subscriber and per topic, - enforced identically on push and pull, with denials counted rather than - merely prevented. - -On top of that it measures which venue's price moves first. - -**The measured result: a repricing on Binance is answered by Coinbase far -more often than the reverse, and it now holds on two instruments at once.** -A 29 minute capture carrying BTC and ETH quoted by both venues confirms -the ordering separately on each, under identical market conditions: BTC at -z = 5.19 and ETH at z = 9.43, with Binance's moves answered roughly four -times as often as Coinbase's on both. The second instrument is a control -on the first rather than a separate study, which is the strongest form -this claim has taken. - -It also replicates across sessions. Across two earlier 45 minute captures -of Binance and Coinbase quoting BTC, Binance was answered 0.577 of the -time against Coinbase's 0.264 in the first session -(z = 11.76, 204,864 messages) and 0.903 against 0.539 in a second session -eight days later that was three times as busy (z = 39.33, 642,919 -messages). The ordering holds at every threshold from $0.25 to $2.00 in -both, and the measurement is biased *against* that conclusion by roughly -36 to 86 ms of network asymmetry. - -The follow rates themselves do **not** replicate - both roughly doubled in +A cross-venue market-data engine in C++20. It reads live order books from +four venues over TLS WebSocket, normalizes them into one schema, and serves +them to consumers through a BLPAPI-style interface: subscription and +request, both entitlement-checked. + +Then it measures which venue's price moves first. + +## Numbers + +Every row traces to one command on a capture committed to this repo. No +figure here is aspirational; if something is not yet measured, it says so. + +| Measured | Result | Where | +|---|---|---| +| Cross-venue price discovery | Binance repricings answered 57.7% of the time, Coinbase 26.4% (z = 11.76, 204,864 messages) | [`cross_venue_lead.md`](docs/bench/cross_venue_lead.md) | +| Ingest throughput | 2.15M messages/sec median of five runs (2.00 to 2.19M), 8.2M book deltas/sec | [`ingest.md`](docs/bench/ingest.md) | +| Ingest-to-signal latency | p50 0.8 us, p99 78 us at 458k records/sec | [`latency.md`](docs/bench/latency.md) | +| Coordinated omission | slowest 1% of records wait 13.3x their own processing time (116 us against 1,534 us) | [`latency.md`](docs/bench/latency.md) | +| Conflated fan-out | publisher runs 40 to 46x faster than inline fan-out at 16 subscribers | [`fanout.md`](docs/bench/fanout.md) | +| Hot path allocations | 1 to 2 heap allocations per message, BDE arenas at parity | [`allocator.md`](docs/bench/allocator.md) | +| Unattended soak | 4 hours live, 59,891 messages, 3 venue disconnects recovered, zero gaps | [`soak.md`](docs/bench/soak.md) | +| Cross-venue arbitrage | books cross on 59.4% of updates, 0 of 1,383 profitable after fees | [`cross_venue_fomc.md`](docs/bench/cross_venue_fomc.md) | +| Book reconstruction | 40 of 40 levels match the venue's own REST snapshot, zero mismatches | [`book_reconstruction.md`](docs/bench/book_reconstruction.md) | +| Matching engine | 42.6M operations/sec at 23.5 ns/op, 2.1x a `std::map` book | [`matching_engine.md`](docs/bench/matching_engine.md) | + +`scripts/bench.sh` regenerates these from the committed captures through +`replay --json`. `scripts/perf_gate.sh` runs in CI on every commit and +fails the build if lead recovery, integrity counters, the allocation +budget, or the throughput floor regress. + +## Try it in sixty seconds + +No network, no credentials. Generate a synthetic session with a known +cross-venue lead injected, then replay it through the real parsers and +watch the engine report that lead back. + +``` +cmake -B build -G Ninja +cmake --build build -j +./build/src/basis synth captures/demo.feedlog --steps 5000 --lead-ms 400 +./build/src/basis replay captures/demo.feedlog +``` + +Replay prints message accounting (nothing is ever silently dropped), +per-event basis, the recovered lead with a bootstrap confidence interval +and an independent event-study cross-check, and latency percentiles. The +same closed loop runs in CI: if the engine cannot recover an injected lead +through the real parsers, by both methods, the build is red. + +![Synthetic session with the injected 400 ms cross-venue lead visible](docs/img/synth-lead.png) + +The injected lead is visible in the session itself: one random walk quoted +by two synthetic venues, one of them 400 ms behind. Replay reports 0.400 s +at correlation 1.00. + +## How it fits together + +``` + Kalshi ----+ + Polymarket + + Binance ---+---> feed handlers ---> normalize ---> unified book + Coinbase --+ (4 parsers, (venue ids (per event, + one adapter become one both venues) + seam) event id) | + v + analytics + (basis, lead-lag, microprice, + crossed-book economics net of + fees and of reaction delay) + | + v + BLPAPI-style API: subscription (push, + conflated) + request (pull), both + entitlement-checked, default-deny + + exec/ ---> price-time-priority matching engine, Gtc/Ioc/Fok + re-executes the analytics' sweeps as order flow so the + two must agree +``` + +Four things happen here, and the middle two are where a market-data team +spends its time. + +**Feed handlers.** Four venue parsers behind one adapter seam, each turning +a venue's dialect into one canonical delta. Sequence gaps are detected and +answered by dropping the book and re-requesting a snapshot, because a book +known to be stale must never be served. + +**Normalization.** Venue-native market ids disappear at the registry; +everything downstream is keyed by a venue-neutral event id. + +**Distribution.** Conflated fan-out, so a slow consumer receives the current +price rather than a backlog, with memory bounded by subscribers times topics +rather than by publish rate. + +**Entitlements.** Default-deny, granted per subscriber and per topic, +enforced identically on push and pull, with denials counted rather than +merely prevented. + +## The cross-venue result + +**A repricing on Binance is answered by Coinbase far more often than the +reverse, and it holds on two instruments at once.** A 29 minute capture +carrying BTC and ETH quoted by both venues confirms the ordering separately +on each, under identical market conditions: BTC at z = 5.19 and ETH at +z = 9.43, with Binance's moves answered roughly four times as often as +Coinbase's on both. The second instrument is a control on the first rather +than a separate study. + +It replicates across sessions. Across two earlier 45 minute captures, +Binance was answered 0.577 of the time against Coinbase's 0.264 +(z = 11.76, 204,864 messages) and 0.903 against 0.539 in a session eight +days later that was three times as busy (z = 39.33, 642,919 messages). The +ordering holds at every threshold from $0.25 to $2.00 in both, and the +measurement is biased *against* that conclusion by roughly 36 to 86 ms of +network asymmetry. + +The follow rates themselves do **not** replicate. Both roughly doubled in the busier session, because a two second window catches more coincidence when the market is moving constantly. So the claim is the ordering and its -significance, not a percentage: those describe one afternoon. +significance, not a percentage. **A third estimator says why the other two could only ever report an -ordering.** Hayashi-Yoshida is the standard covariance estimator for -prices observed at different, irregular times: it pairs returns over -overlapping observation intervals instead of resampling both venues onto -a common grid. Run on the same two captures, it finds a co-movement of -0.71 and 1.18 where the grid estimator found 0.25 and 0.31 - the grid was -recovering roughly a quarter to a third of what is in these books, and -throwing the rest away as synchronization bias before it measured -anything. That gap replicates across both captures. The millisecond-scale +ordering.** Hayashi-Yoshida is the standard covariance estimator for prices +observed at different, irregular times: it pairs returns over overlapping +observation intervals instead of resampling both venues onto a common grid. +On the same two captures it finds co-movement of 0.71 and 1.18 where the +grid estimator found 0.25 and 0.31, so the grid was recovering roughly a +quarter to a third of what is in these books and discarding the rest as +synchronization bias. That gap replicates. The millisecond-scale *direction* does not survive the same scrutiny, and the writeup says so rather than picking the capture that agreed. ![Hayashi-Yoshida lag scan across two BTC/USD captures, against the peak the grid estimator reported on the same data](docs/img/hy-lead-scan.png) -Full methodology, all three estimators, the replication, and the bias -budget: [`docs/bench/cross_venue_lead.md`](docs/bench/cross_venue_lead.md). - -Both sockets are read by one process so the two streams share a clock, -which is the detail that makes the measurement possible at all - two -processes, or two clocks, would put an unknown offset directly into the -quantity being estimated: +Both sockets are read by one process so the streams share a clock, which is +what makes the measurement possible at all. Two processes, or two clocks, +would put an unknown offset directly into the quantity being estimated. ``` basis record out.feedlog --binance btcusdt --coinbase BTC-USD --seconds 2700 basis xvenue-lead out.feedlog ``` -**Contents:** [What it was built for](#what-it-was-built-for-and-where-that-stands) -- [Status](#status) - [Build and run](#build-and-run) -- [Live capture](#live-capture) - [Design](#design-at-a-glance) -- [Layout](#layout) - [Numbers](#numbers) - -Start with [Build and run](#build-and-run) if you want to see it work: it -generates a synthetic session with a known cross-venue lead injected and -replays it through the real parsers, with no network and no credentials. +Full methodology, all three estimators, the replication, and the bias +budget: [`docs/bench/cross_venue_lead.md`](docs/bench/cross_venue_lead.md). ## What it was built for, and where that stands @@ -103,15 +168,22 @@ have all since settled; that file is kept because the committed captures carry its token ids. `configs/contracts-live.toml` is the current one and `scripts/contracts.py check` says whether it still is. -**It has not been measured, and the reason is credentials, not code.** -Polymarket's market channel is public and is captured live here -(`docs/bench/latency.md` is a committed 30 minute session). Kalshi requires -an authenticated session even for market data, and this repo has never had -an account: the Kalshi adapter - signed session, gap-triggered -re-snapshot - is verified offline down to the RSA-PSS signature and waits -on a key. Binance and Coinbase are the pairing that is public on both -sides, which is why the real cross-venue result above is crypto rather -than prediction markets. +**Both venues are now captured, and the first time Kalshi connected it +found a bug.** Polymarket's market channel is public. Kalshi requires an +authenticated session even for market data, and for most of this repo's +life it had no account, so the adapter was verified offline down to the +RSA-PSS signature and never run. When a key finally arrived the handshake +worked on the first attempt and the books came back empty: Kalshi had +migrated its wire format to decimal dollar strings under new field names, +and the parser treated the missing fields as a legal empty book. Every +snapshot parsed, every book was empty, and the malformed counter read +zero. `docs/bench/cross_venue_fomc.md` is the capture that came out of +fixing it, and the failure is written up in +[`docs/postmortems.md`](docs/postmortems.md). + +The crypto pairing carries the headline lead-lag result anyway, because +Binance and Coinbase quote continuously while these prediction markets go +minutes between updates. That distinction is kept sharp everywhere in this repo: a figure is either measured on a committed capture or it is labelled as not yet measured. @@ -132,60 +204,29 @@ probability strings turned into one canonical cents-per-contract frame. Figures regenerate from the committed capture with `scripts/plot_bench.py`. -## Status +## Where to read more -The engine runs end to end, offline and live: real venue wire formats are -parsed, normalized into per-event unified books, and measured for basis, -lead-lag, and per-record ingest-to-signal latency, driven either by -deterministic replay or by `basis live` streaming the venues in real -time. Four venue parsers (Kalshi, Polymarket, Binance, Coinbase) sit -behind one `FeedAdapter` seam. The hot path is zero-copy and -allocator-instrumented, benchmarked against Bloomberg's BDE arenas -(`docs/bench/allocator.md`). A price-time-priority matching engine -(`docs/bench/matching_engine.md`) runs both as a throughput benchmark and -as an independent check on the analytics. Things that broke along the way, -and how they were found, are written up in -[`docs/postmortems.md`](docs/postmortems.md). See `PLAN.md` for the full -spec and `docs/design.md` for how the code is put together. +- [`docs/postmortems.md`](docs/postmortems.md) - five defects that survived + their own test suites, and why nothing caught them +- [`docs/analytics_notes.md`](docs/analytics_notes.md) - what the analytics + compute, and the failures that shaped them +- [`docs/design.md`](docs/design.md) - how the code is put together +- [`docs/bench/`](docs/bench/) - one writeup per measurement, each with the + command that regenerates it +- `PLAN.md` - the full spec -## Build and run +## Build options ``` -cmake -B build -G Ninja -cmake --build build -j ctest --test-dir build --output-on-failure ``` -Try it without any credentials or network: generate a synthetic session with -a known injected cross-venue lead, then replay it through the full pipeline -and watch the engine report that lead back. - -``` -./build/src/basis synth captures/demo.feedlog --steps 5000 --lead-ms 400 -./build/src/basis replay captures/demo.feedlog -``` - -The replay prints message accounting (nothing is ever silently dropped), -per-event basis statistics, the recovered lead with a bootstrap confidence -interval and an independent event-study cross-check, and ingest-to-signal -latency percentiles. The same closed loop runs in the test suite and in -CI's performance gate: if the engine cannot recover an injected lead -through the real parsers, by both methods, the build is red. - -![Synthetic session with the injected 400 ms cross-venue lead visible](docs/img/synth-lead.png) - -That injected lead is visible to the eye in the synthetic session itself: -the same random walk quoted by both synthetic venues, one of them 400 ms -behind. Replay reports 0.400 s with correlation 1.00. - The configure pulls GoogleTest and simdjson. With `-DBASIS_ENABLE_BDE=ON` (`brew install bde` on macOS), `replay --alloc bde` runs the hot path on Bloomberg `bdlma` arenas and `--alloc count` reports heap traffic per -message; `docs/bench/allocator.md` records what those measured. -`replay --breakdown` splits the ingest-to-signal time into parse versus -everything downstream (normalize, book apply, analytics, publish); on the -synthetic session parse is about 60% (simdjson plus canonicalization), -the rest of the pipeline the other 40%. +message. `replay --breakdown` splits ingest-to-signal time into parse +versus everything downstream: on the synthetic session parse is about 60% +(simdjson plus canonicalization), the rest of the pipeline the other 40%. ## Live capture @@ -245,24 +286,9 @@ zero message loss across the thread boundary is measured, not assumed: ./build-net/src/basis live --seconds 60 ``` -## Design at a glance +## Design details -``` -feed (Kalshi, Polymarket) -> normalize + match -> unified order book - | - analytics - (divergence, lead-lag, microprice, - crossed-book economics net of fees - and of reaction delay) - | - BLPAPI-style subscription + request API - -matching engine (exec/) -> price-time priority book, Gtc/Ioc/Fok - cross-checks the analytics by executing - the same sweeps as order flow -``` - -The consuming path above is one direction; `exec/` is the other. It is a +`exec/` is the other direction. It is a price-time-priority matching engine whose ladder is a flat 99-slot array because prediction-market prices are integer cents, and it exists for more than completeness: the arbitrage numbers the analytics report are @@ -349,73 +375,13 @@ scripts/ CI performance gate, levelization check, benchmark runner, README figure generator ``` -## Numbers +## Detail worth reading + +The table at the top links each measurement to its writeup. What follows is +the material those rows cannot carry: the failures the engine is built +against, and the checks that exist because a plausible number is more +dangerous than an error. -Filled in only from committed benchmarks, never aspirational (the same rule the -companion voxel-engine project follows). `scripts/bench.sh` regenerates the -summary below from the committed capture artifacts through `replay --json`, so -every figure traces to one command. Recorded so far: - -- `docs/bench/book_reconstruction.md`: consuming a venue API is not the - hard part; knowing that the book you assembled is the book the venue has - is. `feed::BookSequencer` implements snapshot-and-diff reconciliation - (buffer before the snapshot, discard what it already contains, require - the first event to straddle it, then demand unbroken `U == prev u + 1`), - and `basis book-verify` proves it on a committed live capture: 230 - events applied in sequence, and the reconstructed book matches the - venue's own independently taken REST snapshot on all 40 compared levels - with zero mismatches, at exactly the same sequence id. CI runs it on - every commit. -- `docs/bench/ingest.md`: the prediction markets are slow, about 19 - messages/sec on the committed capture, which makes a latency number hard - to interpret. So the engine also ingests Binance, which is there because - it produces load: 90 seconds of live public market data across 94 - symbols, 24,048 messages with zero malformed. The venue sustains 269 - messages/sec; the same pipeline parses and applies them at 2,189,981/sec, - 8.2M deltas/sec. Four orders of magnitude of headroom, *on the average - rate* - which is not the rate anything arrives at, see the latency entry - below. (This was 932,471/sec until a book-correctness fix; see the - cross-venue entry below.) -- `docs/bench/cross_venue_lead.md`: the first lead-lag measurement in this - repo taken on a real market rather than a synthetic session with a lead - injected on purpose. Binance BTCUSDT against Coinbase BTC-USD, both - sockets read by one process (`basis record --binance btcusdt --coinbase - BTC-USD`) so the two streams share a clock, 45 minutes - and 204,864 messages with zero malformed. A repricing on Binance is - answered by Coinbase 57.7% of the time; one on Coinbase is answered by - Binance 26.4% of the time (two-proportion z = 11.76), and the gap holds - at every threshold from $0.25 to $2.00. The measurement is biased - *against* that result: this host sits about 86 ms farther from Binance - than from Coinbase, which both suppresses the forward follow rate and - inflates the reverse one. Building it also exposed a real bug in shipped - code - `bookTicker` replaces the touch, but the parser emitted it as - plain level updates, so the book accumulated stale levels until the - spread went to minus $54.87. Throughput benchmarks cannot see a wrong - book; a measurement that reads prices out of one can. -- `docs/bench/latency.md`: on a committed 30-minute live capture (34,731 - messages, 266,597 deltas, zero loss), ingest-to-signal latency is - p50 0.8 us / p99 78 us at ~458k records/sec, stable across runs. Those - are **service** times from a back-to-back replay, and replaying the - capture at its own arrival schedule instead shows what that costs: - the p99 a consumer of the feed actually waits is **13x higher** - (116 us service against 1,534 us response, same run), because a loop - that never waits also never samples the stall it is in - coordinated - omission. The same experiment found the engine genuinely behind on - **one record in seven** at the venue's real rate, despite averaging 19 - messages/sec against a 458k/sec pipeline: 41% of this feed's - inter-arrival gaps are under 100 microseconds, so the mean rate - describes nothing that happens. -- `docs/bench/soak.md`: four unattended hours live (59,891 messages, 49 MB), - 3 natural venue disconnects survived, zero malformed, zero rejected, - zero gaps; same latency percentiles as the shorter capture, so the - numbers are properties of the engine, not of one lucky recording. -- `docs/bench/allocator.md`: hot path at 1-2 heap allocations per message, - 3.7M records/sec max-rate synthetic replay on an Apple M4, bdlma arenas - at parity with the global heap. -- `scripts/perf_gate.sh` runs in CI on every commit: lead recovery exact, - integrity counters zero, allocation budget held, throughput floor with - 10x headroom. It reads `replay --json`, so a change to the human report - format cannot silently break the gate. The failure this exists for is not a socket that errors. It is one that does not: a peer or a middlebox that vanishes without a FIN or an RST leaves a blocking read parked forever, and the reconnect path is driven by @@ -473,65 +439,6 @@ breaks the read the same way `stop()` does. study over the same flow separates fills from never-fills by queue position at placement: 3,600 contracts ahead versus 126,234, a 35x split, with 80.4% of resting orders eventually trading. -- Crossable dislocations are priced as a ladder, each rung answering the - question the previous one raises: how often the books cross, how long - the episodes persist, how deep they run, what one taker order at the - touch captures, what sweeping the full crossed depth captures, and what - survives Kalshi's taker fee (ceil(0.07 * C * P * (1-P)), assumptions in - `src/model/fees.h`). On the synthetic session the ladder inverts the - headline: the gross sweep averages $1.13 per crossed update, but only - 109 of 537 crossed updates survive fees (net at the touch: mean -$0.36), - while a taker free to decline losing fills keeps mean +$0.08, max - +$1.45 - and the survival ladder times the decay: 100 ms of reaction - delay leaves an expected $0.03 (59/210 episodes still crossed), 250 ms - leaves $0.01 (8/210). Crossable is not profitable; selective and fast - is. The fee and sweep arithmetic saturates on corrupt sizes the same - way the book does (UBSan-verified), so a bad feed cannot poison the - economics. Methodology and regeneration commands: - `docs/bench/economics.md`. -- The mid is not fair value on a wide book, and the engine says so with - numbers: it tracks queue imbalance and the size-weighted microprice at - every touch. On the committed 30-minute capture, 11 of 13 two-sided - events were bid-heavy and the microprice sat 7.99 cents above the mid - on average, scaling with the spread (+9.9c on books quoted wider than - 20 cents, +1.7c on tighter ones). That qualifies every mid-based - divergence statistic and says which events they can be trusted on; the - crossable-dislocation ladder is unaffected because it never used mids, - only executable bid and ask prices (`docs/bench/economics.md`). -- The engine also has the venue side: `src/exec/limit_order_book.h` is a - price-time-priority matching engine (Gtc/Ioc/Fok, O(1) submit, cancel - and best-price via a flat 99-slot ladder and two-word occupancy bit - scan, because prediction-market prices are integer cents 1..99). It - sustains 42.6M operations/sec at 23.5 ns/op on an M4, 2.1x a textbook - `std::map` book replaying the identical order flow. Two correctness - checks run in CI: the two books must produce identical fill streams over - 200k random operations, and executing a sweep as order flow through the - engine must reproduce `crossed_sweep_cents` exactly, so the executable - edge numbers are cross-checked rather than trusted - (`docs/bench/matching_engine.md`). -- The complementary YES/NO bound is monitored, an invariant that exists - only because these contracts settle at $0 or $1: the two sides are - worth exactly $1.00 together, so any gap is riskless. On the committed - 30-minute capture it effectively never broke. Getting that answer took - three attempts, and the failures are the interesting part: comparing - prices as doubles manufactured 267 phantom violations (0.53 + 0.47 - exceeds 1.0 in floating point), and sampling per delta rather than per - wire message manufactured more from states that exist only midway - through applying one message (`docs/bench/economics.md`). -- Mutually exclusive outcome groups are watched as baskets on live data: - best-bid sums are checked against the hard $1 no-arbitrage bound (valid - even for partial baskets), mid-price sums read the venue's probability - mass. On the committed 30-minute capture the Fed basket's mid-sum broke - coherence for a moment ($1.40) while the tradable bid-sum never came - within ten cents of the bound - quote noise and executable opportunity - are different things, measured on live data - (`docs/bench/economics.md`). -- Venue integrity hashes are recomputed, not trusted: the parser rebuilds - Polymarket's canonical book summary and checks its SHA-1 on every - snapshot that carries the hashed fields, 13/13 verified with 0 - mismatches on the committed capture (docs/api_integration.md has the - recipe). - -The cross-venue lead measurement waits on a simultaneous both-venue -recording (Kalshi credentials); until then that figure stays a bracketed -placeholder in `PLAN.md`. + +More of this kind of detail, on what the analytics compute and the failures +that shaped them: [`docs/analytics_notes.md`](docs/analytics_notes.md). diff --git a/docs/analytics_notes.md b/docs/analytics_notes.md new file mode 100644 index 0000000..d72e9da --- /dev/null +++ b/docs/analytics_notes.md @@ -0,0 +1,70 @@ +# What the analytics actually compute + +Pulled out of the README, which had grown a tail longer than its head. +Each item below is summarized from a `docs/bench/` writeup and links back +to it; nothing here is a separate measurement. + +- Crossable dislocations are priced as a ladder, each rung answering the + question the previous one raises: how often the books cross, how long + the episodes persist, how deep they run, what one taker order at the + touch captures, what sweeping the full crossed depth captures, and what + survives Kalshi's taker fee (ceil(0.07 * C * P * (1-P)), assumptions in + `src/model/fees.h`). On the synthetic session the ladder inverts the + headline: the gross sweep averages $1.13 per crossed update, but only + 109 of 537 crossed updates survive fees (net at the touch: mean -$0.36), + while a taker free to decline losing fills keeps mean +$0.08, max + +$1.45 - and the survival ladder times the decay: 100 ms of reaction + delay leaves an expected $0.03 (59/210 episodes still crossed), 250 ms + leaves $0.01 (8/210). Crossable is not profitable; selective and fast + is. The fee and sweep arithmetic saturates on corrupt sizes the same + way the book does (UBSan-verified), so a bad feed cannot poison the + economics. Methodology and regeneration commands: + `docs/bench/economics.md`. +- The mid is not fair value on a wide book, and the engine says so with + numbers: it tracks queue imbalance and the size-weighted microprice at + every touch. On the committed 30-minute capture, 11 of 13 two-sided + events were bid-heavy and the microprice sat 7.99 cents above the mid + on average, scaling with the spread (+9.9c on books quoted wider than + 20 cents, +1.7c on tighter ones). That qualifies every mid-based + divergence statistic and says which events they can be trusted on; the + crossable-dislocation ladder is unaffected because it never used mids, + only executable bid and ask prices (`docs/bench/economics.md`). +- The engine also has the venue side: `src/exec/limit_order_book.h` is a + price-time-priority matching engine (Gtc/Ioc/Fok, O(1) submit, cancel + and best-price via a flat 99-slot ladder and two-word occupancy bit + scan, because prediction-market prices are integer cents 1..99). It + sustains 42.6M operations/sec at 23.5 ns/op on an M4, 2.1x a textbook + `std::map` book replaying the identical order flow. Two correctness + checks run in CI: the two books must produce identical fill streams over + 200k random operations, and executing a sweep as order flow through the + engine must reproduce `crossed_sweep_cents` exactly, so the executable + edge numbers are cross-checked rather than trusted + (`docs/bench/matching_engine.md`). +- The complementary YES/NO bound is monitored, an invariant that exists + only because these contracts settle at $0 or $1: the two sides are + worth exactly $1.00 together, so any gap is riskless. On the committed + 30-minute capture it effectively never broke. Getting that answer took + three attempts, and the failures are the interesting part: comparing + prices as doubles manufactured 267 phantom violations (0.53 + 0.47 + exceeds 1.0 in floating point), and sampling per delta rather than per + wire message manufactured more from states that exist only midway + through applying one message (`docs/bench/economics.md`). +- Mutually exclusive outcome groups are watched as baskets on live data: + best-bid sums are checked against the hard $1 no-arbitrage bound (valid + even for partial baskets), mid-price sums read the venue's probability + mass. On the committed 30-minute capture the Fed basket's mid-sum broke + coherence for a moment ($1.40) while the tradable bid-sum never came + within ten cents of the bound - quote noise and executable opportunity + are different things, measured on live data + (`docs/bench/economics.md`). +- Venue integrity hashes are recomputed, not trusted: the parser rebuilds + Polymarket's canonical book summary and checks its SHA-1 on every + snapshot that carries the hashed fields, 13/13 verified with 0 + mismatches on the committed capture (docs/api_integration.md has the + recipe). + +A simultaneous both-venue recording now exists +(`docs/bench/cross_venue_fomc.md`), but it does not yet support a +prediction-market lead-lag figure: 218 Kalshi records in 23.6 minutes left +59.7% of samples priced against a quote more than five seconds old, so the +estimator reports no signal for want of data rather than want of effect. diff --git a/docs/postmortems.md b/docs/postmortems.md index c339b92..c19ef2c 100644 --- a/docs/postmortems.md +++ b/docs/postmortems.md @@ -1,6 +1,6 @@ # Things that broke, and how they were found -Four defects that survived their own test suites. They are collected here +Five defects that survived their own test suites. They are collected here because the interesting part of each is not the fix, which is usually a few lines, but the reason nothing caught it: in every case the code did exactly what it was written to do, reported success, and produced a plausible @@ -130,11 +130,51 @@ rather than a bug. followed in this repo: prices are integers, and any comparison that decides whether money is on the table runs on the integer representation. +## 5. A venue changed its wire format and every counter stayed green + +The Kalshi adapter was written against the documented `orderbook_delta` +schema and verified offline, down to the RSA-PSS signature. It had never +connected, because this repo had no account. When a key finally arrived the +handshake worked on the first attempt, the feed reported 0 malformed and 0 +gaps, and the books were empty. + +Kalshi had migrated the channel. Snapshots now carry `yes_dollars_fp` and +`no_dollars_fp` holding decimal strings (`["0.5100", "1200.50"]`) where the +parser expected `yes` and `no` holding integer pairs. Deltas carry +`price_dollars` and a signed, fractional `delta_fp` where the parser +expected `price` and `delta` as integers. + +The rename alone would have been a loud failure. What made it silent was +one line of defensive handling: + +```cpp +if (field.error() == NO_SUCH_FIELD) { + return true; // side absent entirely is a legal empty book +} +``` + +Written for a market with no resting orders, it also covers a market whose +field was renamed. Every snapshot parsed successfully into nothing. A live +capture reported `kalshi 10 msgs 6 deltas 0 malformed` where the correct +figure was 240 deltas. + +The fix accepts both wire forms, so committed captures still replay, and +makes the absence loud: an empty Kalshi book still sends its side keys +holding empty arrays, so keys missing under *every* known name is an +unrecognised schema rather than an empty market, and is now `Malformed`. + +The lesson is the one this file keeps repeating in a new place. Integrity +counters answer "did anything I recognised look wrong", not "did I +recognise anything". No throughput number, no gap counter, and no +malformed count could have caught this; only reading prices out of the +book against a venue that was quoting could, and that meant pointing it at +production. + ## The pattern None of these produced an error. Each one produced a plausible number, and -in three of the four cases the plausible number was *more* interesting than -the truth - a 13:1 move ratio, a quiet market, 267 arbitrages. That is the +in four of the five cases the plausible number was *more* interesting than +the truth - a 13:1 move ratio, a quiet market, 267 arbitrages, a clean feed. That is the direction the errors ran, and it is why this repo leans on cross-checks that share no code with the thing being checked: an independent Python replay, a matching engine that recomputes the analytics by executing the