Repository navigation
integration: live diagnostics for end markers on dump-then-delta streams - #963
Merged
Merged
Conversation
Two #[ignore]d async tests: - end_markers_after_initial_dump: subscribes to account_summary, account_updates(_multi), positions(_multi); optional ES round trip (END_MARKER_FILL=1) forces pushes; prints per-frame timeline + summary - account_summary_end_markers_by_tag_set: regular tags vs $LEDGER:ALL vs both, one connection per case Findings (one paper account, group "All", IB Gateway, 2026-10-05): End only after initial dump on all five streams. account_summary sent two Ends in most runs (not the first smoke run), repeating the $LEDGER block; recorder shows both arrived from TWS. Other account types, named groups and TWS untested. Context: #957.
wboayue
force-pushed
the
end-marker-diagnostics
branch
from
October 6, 2026 05:03
51fd187 to
0daeb77
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Live diagnostics for when TWS sends end markers on the streams that send an initial dump and then pushes changes:
account_summary,account_updates,account_updates_multi,positions,positions_multi. Written while reviewing #957. Both tests are#[ignore]d. They only print findings, apart from one sanity check that each stream ends its initial dump.end_markers_after_initial_dump: merges the five streams into one timeline and prints each frame's arrival time and type (row, end marker, notice, error, closed), followed by a summary per stream.END_MARKER_FILL=1buys 1 front-month ES at 20s and sells it at 60s, so the run sees pushes that follow fills. This needs Globex open.account_summary_end_markers_by_tag_set: requests the regular tags,$LEDGER:ALL, and both, each on its own connection.Findings
All runs used one paper account with group
Allthrough IB Gateway, on 2026-10-05. Accounts with sub-accounts, named groups and TWS are untested.account_summarysent two end markers in 5 of 6 runs ofend_markers_after_initial_dump, and in all 4 runs ofaccount_summary_end_markers_by_tag_set. The first run, a 30s smoke test, had one. The frame recorder (IBAPI_RECORDING_DIR) logged 1 request out and 2AccountSummaryEndframes in, so our client did not duplicate a frame. The recorder logs each frame after it is read from the socket. A byte-level capture (IBAPI_RAW_CAPTURE_DIR) was not run.48 rows, End, End$LEDGER:ALL:75 rows, End, the same 75 rows again, Endaccount_summarypush arrives within one second, and pushes are at least 40s apart.cancel().awaiton two summary requests, a third request on the same connection gets 322 even 10s later. A fresh connection works. That's why the second test opens a connection per case.Test plan
cargo clippy -p ibapi-integration-async --tests -- -D warningscargo fmt --check