Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# What is left here

An audit on 2026-09-03, run the same way as the sibling repo's: measure
first, then decide.

## Where this repo is strong

Test depth, and it is not close:

basis 5,357 test lines over 10,828 source ratio 0.49
voxel 1,993 test lines over 9,313 source ratio 0.21

Every core library - feed, model, analytics, api, exec, normalize, core -
has tests. The components with none are `cli/` command wiring and
`logger.cpp`, and CLI commands are mostly orchestration: open a file, call
a library, print. There is real value in testing them, and it is a long
way below the value of the gaps named next.

Argument parsing was the one place the CLI gap had teeth, and it has been
closed: `flag_double` used to substitute a fallback for unparseable input,
so `--speed banana` silently replayed unpaced.

## The three real gaps, ranked

### 1. Kalshi has never run live

The blocker is credentials, not code. The adapter exists and is verified
offline down to the RSA-PSS signature. `configs/contracts.toml` maps 14
cross-venue contracts. None of it has ever been exercised against the
venue, so every cross-venue result in this repo is Binance/Coinbase - the
substitute pairing, chosen because it is public on both sides.

The cost is a free account and an RSA key. It is the only item on this
list that cannot be done by writing code, and it is the one that unlocks
the most: with it, the fee-aware arbitrage backtester runs on a real
both-venue capture instead of a synthetic session, and the repo's stated
thesis becomes a measured result rather than a described one.

### 2. This is a book engine, not a market-data engine

`BookDelta` carries price levels. `Action` is Set, Add, Clear. There are
no trade prints, no last-trade, no volume - and every venue here publishes
them on the same socket.

The Coinbase parser can already read a `ticker` frame, which carries
trades. The live feed subscribes `level2_batch` instead, for depth, so no
committed capture contains a single trade. That is what makes this bigger
than it looks: it needs a channel change and a fresh capture, not a parser
change.

Worth it because it is half of what a market-data feed carries, and
because it would give `ConflatingSession` its counter-example. That doc
already argues fills and prints want a queue rather than conflation, and
there is nothing in the repo to point at.

### 3. There is no storage layer

`.feedlog` is one record per line, tab-separated, raw JSON, gzipped. That
is a capture format. There is no index, no columnar layout, no time-range
query; reading a thirty-minute window means scanning the file.

This is the largest piece of work on the list and the one furthest from
what exists. It is also the only one that would add a capability the repo
does not gesture at anywhere else.

## What is explicitly not on the list

**Kafka, for replay or event streaming.** Considered and rejected on
numbers. Kafka is a milliseconds tool and this engine's headline is a
microsecond service time; putting it in the path deletes the number the
repo is built on, and putting it outside the path leaves it doing nothing.
`kafka market data` is also 1,170 GitHub repositories, which is the
default tutorial architecture rather than a differentiator. The gap it
would fill - cross-process fan-out - is real, and the domain-correct
answer to it is a shared-memory ring buffer, which is what CME and
Chronicle actually use and which preserves the latency story instead of
destroying it.

**More estimators.** Three already agree on the cross-venue ordering:
cross-correlation, an event study, and Hayashi-Yoshida. A fourth would not
make the finding more true.

**Chasing throughput.** ~2M messages/sec against a venue producing 269.
The ratio is the point and it is already four orders of magnitude.
14 changes: 9 additions & 5 deletions src/cli/args.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,22 @@ std::string flag_string(const std::vector<std::string_view>& args,
return fallback;
}

double flag_double(const std::vector<std::string_view>& args,
std::string_view name, double fallback) {
for (std::size_t i = 0; i + 1 < args.size(); ++i) {
std::optional<double> flag_double(const std::vector<std::string_view>& args,
std::string_view name, double fallback) {
for (std::size_t i = 0; i < args.size(); ++i) {
if (args[i] != name) continue;
// Present as the final argument, so there is no value to read. Silently
// falling back here is what let `--speed` with no number look like a
// deliberate unpaced run.
if (i + 1 >= args.size()) return std::nullopt;
const std::string text(args[i + 1]);
try {
std::size_t consumed = 0;
const double v = std::stod(text, &consumed);
if (consumed != text.size()) return fallback; // trailing junk
if (consumed != text.size()) return std::nullopt; // trailing junk
return v;
} catch (...) {
return fallback;
return std::nullopt;
}
}
return fallback;
Expand Down
7 changes: 6 additions & 1 deletion src/cli/args.h
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,12 @@ std::string flag_string(const std::vector<std::string_view>& args,
// a rate, and every rate they accept is validated against its own bounds
// right after, so a malformed value is caught there with a message that
// names the flag.
double flag_double(const std::vector<std::string_view>& args,
// Returns nullopt when the flag is present but its value is not a
// complete number, or when it is present with no value at all - the same
// contract flag_value has, and for the same reason. It used to return the
// fallback in both cases, so `--speed banana` silently produced an
// unpaced replay: the one measurement that mode exists to correct.
std::optional<double> flag_double(const std::vector<std::string_view>& args,
std::string_view name, double fallback);

// Comma-separated flag values (--binance btcusdt,ethusdt).
Expand Down
14 changes: 12 additions & 2 deletions src/cli/replay_commands.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -121,11 +121,21 @@ int run_replay(const std::vector<std::string_view>& args) {
// factor. 0 keeps the historical flat-out replay, which can only ever
// report service time.
const auto speed = flag_double(args, "--speed", 0.0);
if (!speed || *speed < 0.0 || *speed > 100000.0) {
basis::log::error("replay: --speed expects a non-negative number "
"(0 replays flat out, 1 replays at the venue's rate)");
return usage();
}
// How much of each pacing wait is spun instead of slept. Past the
// capture's largest gap this spins every wait, which is what it takes
// for the harness's own timer error to stop dominating a
// microsecond-scale response time.
const auto spin_ms = flag_double(args, "--pace-spin-ms", 2.0);
if (!spin_ms || *spin_ms < 0.0 || *spin_ms > 60000.0) {
basis::log::error("replay: --pace-spin-ms expects a non-negative number "
"of milliseconds");
return usage();
}
const auto episodes_csv_path = flag_string(args, "--episodes-csv", "");

std::string error;
Expand Down Expand Up @@ -196,8 +206,8 @@ int run_replay(const std::vector<std::string_view>& args) {
basis::bench::ReplayHarness harness(*registry, &session, {}, book_mr);
harness.set_parse_arena(parse_arena);
harness.set_breakdown(breakdown);
harness.set_replay_speed(speed);
harness.set_pace_spin_ns(static_cast<std::int64_t>(spin_ms * 1e6));
harness.set_replay_speed(*speed);
harness.set_pace_spin_ns(static_cast<std::int64_t>(*spin_ms * 1e6));
const auto stats = harness.run(in_path, &error);
if (!stats) {
basis::log::error(error);
Expand Down
Loading