fix(cli): --speed banana silently replayed unpaced, which is the wrong measurement - #66
Merged
Merged
Conversation
…g measurement
Found by taking a defect out of the sibling repo and asking whether this
one has it. Voxel's argument parser used std::atoi on five count flags, so
a typo parsed to zero and silently disabled the feature. This repo's
parser is better - parse_int uses from_chars and requires the whole string
consumed, flag_value returns nullopt on malformed input, and callers check
it - so the answer was mostly no.
flag_double was the exception. It returned the FALLBACK on unparseable
input and on a flag given with no value at all, so:
basis replay capture.feedlog --speed banana
basis replay capture.feedlog --speed
both fell back to 0.0, which means "replay flat out". No error, no
warning, and a normal-looking report.
That is worse here than the equivalent was in voxel. --speed 1 exists to
replay at the venue's real arrival schedule and measure response time
instead of service time - docs/bench/latency.md's whole finding is that
the service p99 understates what a consumer waits by 13.3x. A typo
therefore returned exactly the service-time numbers that mode was built to
correct, in the benchmark built to correct them, and the only tell was the
absence of one line in the output.
flag_double now returns std::optional<double> like flag_value, signalling
rather than substituting, and both call sites reject with a message naming
the flag and the expected shape. The missing-value case is handled
explicitly rather than falling out of the loop bound, since that is how it
went unnoticed.
--speed banana -> exit 1
--speed 1x -> exit 1 (trailing junk)
--speed -> exit 1 (no value)
--speed -1 -> exit 1 (negative)
--pace-spin-ms xyz -> exit 1
--speed 0 and --speed 1 unchanged; 228 tests, the perf gate and the
bench-input check all pass.
Measured first. Test depth is this repo's strength and it is not close: 5,357 test lines over 10,828 source, a ratio of 0.49 against the sibling repo's 0.21, with every core library covered. The components with no tests are cli/ command wiring, which is mostly orchestration, and the one place that gap had teeth has just been closed. Ranked the three gaps that are real. Kalshi has never run live and the blocker is credentials rather than code, which makes it the only item that cannot be done by writing any, and the one that unlocks the most - with it the arbitrage backtester runs on a real both-venue capture instead of a synthetic session. Trades are absent and the engine carries only price levels, which is bigger than it looks because the live feed subscribes for depth, so it needs a channel change and a fresh capture rather than a parser change. And there is no storage layer at all. Also records what is explicitly not on the list and why, so the same proposals do not have to be re-litigated: Kafka (a milliseconds tool against a microsecond headline, and 1,170 repos), a fourth lead-lag estimator (three already agree), and chasing throughput (already four orders of magnitude above the venue).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by taking a defect out of the sibling repo and asking whether this one has it.
Voxel's parser used
std::atoion five count flags, so a typo parsed to zero and silently disabled the feature. This repo's parser is better —parse_intusesfrom_charsand requires the whole string consumed,flag_valuereturnsnullopton malformed input, and callers check it. So the answer was mostly no.flag_doublewas the exceptionIt returned the fallback on unparseable input and on a flag given with no value at all:
Both fell back to
0.0— which means "replay flat out". No error, no warning, a normal-looking report.Why it's worse here than in voxel
--speed 1exists to replay at the venue's real arrival schedule and measure response time instead of service time.docs/bench/latency.md's entire finding is that the service p99 understates what a consumer waits by 13.3x.So a typo returned exactly the service-time numbers that mode was built to correct — in the benchmark built to correct them. The only tell was the absence of one line in the output.
Fix
flag_doublenow returnsstd::optional<double>likeflag_value, signalling rather than substituting. Both call sites reject with a message naming the flag and the expected shape. The missing-value case is handled explicitly rather than falling out of the loop bound, since that's how it went unnoticed.--speed 0and--speed 1unchanged. 228 tests, perf gate, and bench-input check all pass.