Skip to content

accounts: add account_summary_snapshots and document that account_summary stays open after End - #957

Open
faysou wants to merge 3 commits into
wboayue:mainfrom
faysou:account-summary-snapshots-pr
Open

faysou wants to merge 3 commits into
wboayue:mainfrom
faysou:account-summary-snapshots-pr

Conversation

@faysou

@faysou faysou commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Summary

account_summary yields one row at a time. TWS sends End once, after the initial snapshot, and then pushes only the rows whose values changed, with no further End. A consumer that wants the account as a whole has to keep the latest value of every row itself and decide when a pushed batch is finished. The End doc ("end marker for a batch of account summaries") reads as if it followed every push.

This PR adds two commits:

  1. Docs: AccountSummaryResult::End and both account_summary methods now say End marks the initial snapshot only and the subscription stays open.
  2. Client::account_summary_snapshots(&group, tags, quiet) on both clients. It wraps account_summary and yields AccountSummarySnapshot, the latest value of every row by account, tag and currency. A snapshot completes at an End, or once no row has arrived for quiet. Rows received before the subscription ends are returned as a final snapshot. Dropping the wrapper cancels the subscription.

Design notes

  • The rows carry no batch delimiter, so the quiet period is a heuristic. It is explicit and chosen by the caller, and account_summary itself is unchanged. Rows of one push arrive within milliseconds of each other, so one second has been enough in practice.
  • Row folding and pending-row tracking live in accounts::common::snapshots and are shared by the sync and async wrappers.
  • The async type is accounts::AccountSummarySnapshots with an async next. The blocking type is an Iterator; with both features enabled it is at accounts::blocking, as the other blocking types are exposed.
  • AccountSummary now derives Clone and PartialEq. This is additive.

Evidence

I found the behaviour while fixing an Interactive Brokers adapter that refreshed its account state only on End. Against a paper TWS, a probe logged 74 summary rows and 2 End markers, all End markers before the later rows, and none after them. After an ES fill, changed rows arrived on the open subscription within two seconds. I do not know why that run showed two End markers and each row twice, so the docs claim only that rows keep arriving after End and that no End followed them.

Tests: the shared row folding; async snapshots at End, after the quiet period over a channel that stays open, grouping of one push, and a final snapshot when the subscription ends; and the blocking equivalents. cargo test --lib passes with default features (1654), sync only (1671), and both (2140). cargo clippy --all-targets -- -D warnings is clean in all three, and cargo doc builds with warnings denied.

Not included

A changelog entry is added under Unreleased. I did not add an example.

faysou added 2 commits October 5, 2026 08:02
The docs called End the end marker for a batch of summaries. TWS sends End
once the initial snapshot is complete, then keeps pushing changed values on
the same subscription as Summary rows without another End. A consumer that
treats End as a per-update boundary stops refreshing after the first
snapshot, or drops the subscription and cancels the request.

Document this on AccountSummaryResult and on both account_summary methods.
account_summary yields one row at a time. TWS sends End once, after the
initial snapshot, and then pushes only the rows whose values changed, with
no further End. A consumer that wants the account as a whole has to keep
the latest value of every row itself and decide when a pushed batch is
finished.

account_summary_snapshots wraps account_summary on both clients and yields
AccountSummarySnapshot, the latest value of every row by account, tag and
currency. A snapshot completes at an End, or once no row has arrived for a
caller-chosen quiet period, because the rows of one push arrive within
milliseconds of each other. Rows received before the subscription ends are
returned as a final snapshot. Dropping the wrapper cancels the subscription.

The row folding and the pending-row tracking are shared by both clients.
When both features are enabled the blocking type is at accounts::blocking.
AccountSummary now implements Clone and PartialEq.
@tradatious

tradatious commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Some feedback in case it's helpful to hopefully smooth this being accepted, since this should probably land before my PRs.

A few notes from the repo's rules

  • src/accounts/common/snapshots.rs has an inline #[cfg(test)] mod tests { ... }. docs/rules/testing/sibling-test-files.md asks for a sibling snapshots_tests.rs wired in with #[path], so maybe move the tests.
  • You've exposed the blocking type through a new accounts::blocking module. docs/rules/parity/dual-feature-types.md (step 3) puts the sync version of a dual-feature type under client::blocking, and no other domain has its own blocking module, so it might be better to re-export it there.
  • The tests build AccountSummarySnapshots with a struct literal, so nothing calls Client::account_summary_snapshots itself on either client. docs/rules/testing/coverage-floor.md asks for a unit test per new pub function; maybe add one per client that goes through the method with a MessageBusStub.
  • AccountSummarySnapshot::get and iter might want an # Examples block (docs/rules/docs/public-api-examples.md exempts only field getters and is_* predicates).

Other feedback:

  • You note that you don't know why the probe showed two End markers and each row twice. Two small things on top of that: the rustdoc says the tags are "followed by End" (one), and IB documents a three-minute cadence ("every three minutes those values which have changed will be returned") where you saw changes within two seconds. A second capture, perhaps with IBAPI_RECORDING_DIR set, might explain both.
  • A snapshot is emitted at every End, even when no row arrived since the previous one, and pending is set by any row, even one whose value did not change. So the run with two End markers and each row twice would probably yield two identical snapshots. I see one test checks the emit on an End without new rows, so this may be deliberate? If not, maybe emit only when the snapshot changed, or skip an End with nothing pending once a snapshot has gone out (which still keeps an empty first snapshot)?
  • Both wrappers drop SubscriptionItem::Notice items silently. That might be the right choice here, but perhaps say so in the docs.
  • The tests wait in real time (50 ms quiet periods, a 300 ms timeout), which can be flaky on a loaded CI runner. Also the blocking side has two tests where the async side has three (the "one push is grouped" case is async only).
  • AccountSummarySnapshot does not derive the serde and utoipa traits that AccountSummary and its neighbours do; with the tuple-keyed map that would need a custom form (a list of rows, say), since JSON map keys must be strings. get allocates three Strings per lookup. The async type has its own next() where the async Subscription implements Stream. All three might be worth a look.

If you update the commit, probably also change the CHANGELOG bullets to end with the PR number now that you have it (docs/rules/docs/changelog-entry.md).

Emit a snapshot only when a row changed a value since the previous one.
An End with nothing changed is skipped once a snapshot has gone out, and
the first End still emits, possibly empty, so a consumer always gets an
initial snapshot. A repeated row with an unchanged value no longer counts
as a change.

Move the blocking type to client::blocking, as the dual-feature type rule
asks, and drop the accounts::blocking module. Move the snapshot builder
tests to a sibling test file. Add tests that go through
Client::account_summary_snapshots on both clients with the message bus
stub, and the grouping test on the blocking side.

Document that notices are dropped and what IB documents about the push
cadence, add examples to AccountSummarySnapshot::get and iter, and end the
changelog bullets with the PR number.
@faysou

faysou commented Oct 5, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, this was useful. I pushed 017fb89 with most of it.

Done:

  • The snapshot builder tests are in snapshots_tests.rs, wired with #[path].
  • The blocking type is at client::blocking::AccountSummarySnapshots, and the accounts::blocking module is gone.
  • Both clients have a test that goes through Client::account_summary_snapshots with MessageBusStub and checks the request and the cancel on drop. The blocking side also has the "one push is grouped" test now.
  • AccountSummarySnapshot::get and iter have # Examples.
  • The wrappers' docs say that notices are dropped, and that IB documents the pushes as every three minutes.
  • The CHANGELOG bullets end with (accounts: add account_summary_snapshots and document that account_summary stays open after End #957), and the blocking path in the first one is corrected.
  • A snapshot is now emitted only when a row changed a value. A row with an unchanged value no longer counts, and an End with nothing changed is skipped once a snapshot has gone out. The first End still emits, even when empty, so a consumer always gets an initial snapshot. The earlier test that expected a snapshot at an End without new rows was deliberate at the time, and I changed it to match this.

Not done:

  • The second capture with IBAPI_RECORDING_DIR that would explain the two End markers and the repeated rows. I only have the one probe run, so the docs still claim only what it showed. The new change check means a repeated batch no longer produces a second snapshot.
  • The tests still wait in real time. The quiet period starts only after a row has been received and the rows are queued before next is called, so a slow runner makes them slower but should not make them fail. I did not enable tokio's test-util just for this. Say so if you want paused time instead.
  • serde and utoipa derives on AccountSummarySnapshot. Nothing needs them yet, and the tuple-keyed map would need a custom form. I'd add that when a caller needs it.
  • Allocation in get, and a Stream implementation on the async type. Both are fair, but I left them to keep this change small. A Stream impl would also need the quiet-period timer inside poll_next.

cargo clippy -D warnings, the lib tests, and cargo doc pass with default features (1657 tests), sync only (1675), and both (2145).

wboayue added a commit that referenced this pull request Oct 6, 2026
Two #[ignore]d async tests:
- end_markers_after_initial_dump: subscribes to account_summary,
  account_updates(_multi), positions(_multi); optional ES round trip
  (END_MARKER_FILL=1) forces pushes; prints per-frame timeline + summary
- account_summary_end_markers_by_tag_set: regular tags vs $LEDGER:ALL vs
  both, one connection per case

Findings (one paper account, group "All", IB Gateway, 2026-10-05):
End only after initial dump on all five streams. account_summary sent
two Ends in most runs (not the first smoke run), repeating the $LEDGER
block; recorder shows both arrived from TWS. Other account types, named
groups and TWS untested. Context: #957.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants