Make the integration test deterministic#381
Merged
Merged
Conversation
The integration test previously waited for Docuum to finish processing events by sampling the process state in /proc, which couldn't distinguish "idle, waiting for the next event" from "blocked on I/O mid-vacuum" and also raced against Docker's asynchronous event delivery, causing flaky failures. The test now synchronizes on Docuum's own logs: after each step, it emits a uniquely named Docker network event as a sentinel and waits for Docuum to log its arrival at trace level. Since Docker delivers events in order and Docuum processes them sequentially, receipt of the sentinel proves all earlier events were fully processed. Network events are irrelevant to Docuum, so the sentinels don't perturb the state under test. Containers are removed explicitly rather than with --rm, whose asynchronous removal could emit events after the sentinel. Startup similarly probes with network events until Docuum reports receiving one, closing the race between spawning `docker system events` and its subscription being established. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
toast validate_releasewas flaky because the integration test waited for Docuum to finish processing events by sampling the process state in/proc, which can't distinguish "idle, waiting for the next event" from "blocked on I/O mid-vacuum", and which also raced against Docker's asynchronous event delivery.The test now synchronizes on Docuum's own logs instead:
LOG_LEVEL=trace, so it logs every incoming event before processing it.--rm, whose asynchronous removal could emit events after the sentinel.docker system eventsand its subscription being established.The two trace/info log lines the test greps for are now tagref-tagged (
[tag:incoming_event],[tag:listening_for_docker_events]), so changing them without updating the test failstoast lint. Thesrc/run.rschanges are comments only.Verified by running
toast validate_releaselocally end to end.Status: Ready
Fixes: N/A
🤖 Generated with Claude Code