Repository navigation
docs: lead with the numbers, and correct what the Kalshi feed made false - #72
Merged
Merged
Conversation
The README opened with 84 lines of prose before its first heading, and the measurements were spread through it as paragraphs. A reader had to get most of the way down to find a number. This puts a table of every measured result at the top, one row per benchmark, each linking to the writeup and the command that regenerates it. Three claims had gone false overnight and are corrected rather than softened. "It has not been measured, and the reason is credentials, not code", "this repo has never had an account", and the closing note that the cross-venue measurement "waits on a simultaneous both-venue recording" were all true when written and are not now: Kalshi has connected, and docs/bench/cross_venue_fomc.md is the capture. The replacement says what actually happened, because it is a better story than the one it replaces: the handshake worked first time and the books came back empty, since the venue had migrated its wire format and the parser's empty-book path swallowed it. That is now postmortem 5, and it is the fifth entry in a file whose thesis is that none of these produced an error. Ingest throughput now reports the median of five runs with its spread (2.15M, 2.00 to 2.19M) instead of the best run alone (2,189,981). The repo's own rule is that a ratio of counts reproduces and a timing does not; quoting the maximum of five was the thing that rule exists to prevent. Shorter overall despite the additions: the seven bullets under "Numbers" restated what the table now carries, and Status restated the architecture section. 537 lines to 503, with a pipeline diagram that was not there before.
"Detail worth reading" had grown to 129 lines, a quarter of the file and longer than the four sections a reader actually arrives for. The half of it covering crossed-book economics, the microprice, the YES/NO bound, baskets and integrity hashes is summarized from docs/bench/economics.md and matching_engine.md, which carry it in full. It moves to docs/analytics_notes.md and is linked from the README twice: inline where it used to sit, and from "Where to read more". 442 lines and 3,239 words, from 537 and 4,297 before this branch started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
The README opened with 84 lines of prose before its first heading, with
measurements spread through it as paragraphs. A reader had to get most of the
way down to find a number.
What changed
A table of every measured result at the top - ten rows, one per benchmark,
each linking to its writeup and the command that regenerates it. Every figure
in it was cross-checked against its source doc before committing.
Three claims had gone false overnight and are corrected rather than
softened:
both-venue recording (Kalshi credentials)"
All three were true when written. None are now, after #69 and #71.
The replacement says what actually happened, because it is a better story than
the one it replaces: the handshake worked on the first attempt and the books
came back empty, because the venue had migrated its wire format and the
parser's empty-book path swallowed it. That is now postmortem 5, in a file
whose thesis is that none of these defects produced an error.
Ingest throughput now reports the median with its spread (2.15M, range
2.00 to 2.19M) instead of the best of five runs alone (2,189,981). This repo's
own rule is that a ratio of counts reproduces and a timing does not; quoting
the maximum of five is exactly what that rule exists to prevent.
An ASCII pipeline diagram replaces the two partial ones that were
scattered lower down.
Shorter, despite the additions
537 lines to 503. The seven bullets under the old "Numbers" heading restated
what the table now carries, and "Status" restated the architecture section.
All links resolve, all three images exist, and
check_readme_flags,check_bench_inputs, andcheck_no_secretspass.