Publish one Pub/Sub request per flush - #48
Open
BinaryFiddler wants to merge 1 commit into
Open
BinaryFiddler wants to merge 1 commit into
BinaryFiddler wants to merge 1 commit into
Conversation
AsyncPubSubPublisher already batches messages in an asyncio queue, so it configured the google.cloud.pubsub_v1 wrapper's own batching layer out of the way with BatchSettings(max_messages=1). That setting makes the wrapper spawn a commit thread and issue a publish RPC for every single message, no matter how many the flush loop collected. Publish each flush as one request through the GAPIC client instead, which is the layer the wrapper was calling anyway. Measured against google-cloud-pubsub 2.15.2 with the RPC stubbed at the same boundary in both versions: 74us -> 22us of client CPU per message for a flush of one message, 71us -> 3.4us for a flush of 13, and thread creations per message drop from 1.01 to 0. Batch size stays a product of arrival rate rather than of an added delay, so publish latency does not regress. Also fixed while rewriting the flush path: - stop() could drop the whole queue. The drain lived inside the flush task's CancelledError handler, so cancelling a task that had not taken its first step skipped it entirely: 500 queued messages, 0 delivered, reproduced against a Pub/Sub emulator. The drain now runs in stop(), and re-checks the queue instead of sampling qsize() once, which had abandoned anything _requeue put back mid-drain. - stop() could also hang forever: on a cancel landing after queue.get() completed, asyncio.wait_for returns the message and consumes the CancelledError, leaving the `while True` loop running with stop() awaiting it. A _stopping flag now ends the loop. - The idle path caught the builtin TimeoutError, which on <=3.10 is not asyncio.TimeoutError. - Cancelled and Unknown were missing from the transient set the library itself retries publish on. - A batch stops accumulating at 9MB, keeping requests inside the 10MB server limit, and a message oversized on its own is published alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
BinaryFiddler
force-pushed
the
codex/osprey-pubsub-request-per-flush
branch
from
August 20, 2026 19:28
2fb5b33 to
4783cef
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AsyncPubSubPublisheralready batches in an asyncio queue. WithBatchSettings(max_messages=1), the high-level client still created one commit thread and one publish RPC per message.This change calls the generated GAPIC client once per asyncio flush. The public API and flush timing are unchanged: messages already queued together share a request, without adding a batching delay.
Results
Microbenchmark: real
_sync_flush, RPC stubbed at the same boundary, 4,000 × 1.2 KB messages, three runs.Production averages about 15.2k attempts/s across these publishers. Most flushes still contain one message, so the steady-state gain is removing per-message wrapper and thread work; bursts batch naturally.
Other fixes
stop()drains the queue even if the flush task never starts, retries messages requeued during the drain, and records residual messages asshutdown_dropped._stoppingprevents the Python 3.11asyncio.wait_forcancellation race from hanging shutdown.CancelledandUnknownnow match the generated client's transient retry set.queue_depthandmessages_per_request; publish counters remain per message.Risk
One RPC now shares an outcome across its batch. Transient failures requeue the batch; permanent failures drop it. Seven-day production data shows ~0.17 RetryError/s, ~0.12 TimeoutError/s, and no permanent error tag.
messages_per_request.countmeasures logical GAPIC publish calls, not extra wire attempts made by internal retries.Review follow-ups before merge
Verification and rollout
CI is green for async unit tests, Python quality, integration tests, docs, UI, and Rust. The author also exercised burst, paced, immediate-stop, and missing-topic cases against a Pub/Sub emulator.
After rollout, verify attempt rate is stable, failure rate does not rise, queue depth stays near zero, and
shutdown_droppedstays zero. Client pooling remains out of scope.