Repository navigation
land: honest-walk-partials - #493
Open
aft-alfonso[bot] wants to merge 11 commits into
Open
aft-alfonso[bot] wants to merge 11 commits into
aft-alfonso[bot] wants to merge 11 commits into
Conversation
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
Oct 17, 2026
…ly for a stalled call Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
…lease note Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
Outline's breadth-first discovery named a directory once, then read all of its entries and checked each one's type with no further progress. A large directory whose entries each answer promptly but together take longer than the 5 s stall window was reported as a stall, and with nothing collected yet became a read_blocked error instead of a partial. - WalkControl::entry names the entry (or the directory being read), records progress and checks the walk deadline; outline's breadth-first walk calls it before every entry read and every per-entry check, so only one call that does not return is a stall, and it is named. - bounded_io::walk_entry records progress from walker entry filters (outline depth-first, the grep/glob fallback builder and the scope probes built on it). Entries those filters drop never reach the walk loop's advance(). Past the deadline it skips entries and marks the walk cut short; outline's depth-first walk now reports cut_short as walk_truncated. - Test hooks slow_walk_entries / stall_walk_entry act on per-entry calls only, so existing per-directory hook counts are unchanged. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
Past the walk deadline (or once the request stops waiting), the grep/glob fallback, scope-probe and depth-first outline walkers' entry filters used to return false. That only hides entries: the ignore walker kept reading every remaining entry of each open directory and opening each remaining sibling directory, holding the walk's helper slot after the reply. bounded_io::walk_entry is now walk_entry_must_stop and returns true once the walk must stop; the filters then keep that entry, and each walk loop checks WalkControl::advance before counting or examining any entry, so the walk ends there. advance no longer counts the entry it refuses. The walk helper frees its slot before sending its reply. Tests walk a 1,250-entry tree with a counting per-entry hook and a short deadline: at most one entry check after expiry, the helper slot free when the reply arrives, and only files checked in time kept. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
…k tests The previous commit's edit dropped the probe helper's own permit binding in run_with_limiter (it was meant only for the walk helper), so a probe's slot was freed when the request returned instead of when the helper finished. Restore it. The fallback-walk test now bounds entries_visited by the entries checked in time plus any checked after expiry, so counting the refused entry goes red. Register the outline expiry test helper with the list-envelope guard. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
A 300 ms window turned a scheduler stall on a loaded runner into an abandoned walk. The entry count after the deadline is what proves the walk stopped, so the window only needs to catch a hang. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
The expired-walk tests slowed each entry check by 1 ms and set a 400 ms walk deadline from before the helper thread started. On a loaded runner the deadline could pass before breadth-first outline finished checking the root directory's 50 entries, so it kept no file and failed "the walk kept no file it found" (seen on macOS CI for the breadth-first outline test; the depth-first outline, fallback-walk and scope-probe tests had the same timing dependence). The per-entry walk hook can now expire the walk right after its Kth hooked entry check by setting the walk's stop flag, the same flag the request sets when it stops waiting, so the production expiry path (WalkControl::expired reads stop or deadline) is what stops the walk. The four tests expire after 200 checks (breadth-first makes 101 in the root before keeping its files; the tree has 1,250 entries) with a 60 s hang-guard deadline, and no longer slow the walk. All assertions are kept: at most one entry check after expiry, slot free when the reply arrives, kept files only ones checked in time, truncated flag, and the fallback walk's entries_visited bound. Test-only change; no production code. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
ualtinok
force-pushed
the
land/honest-walk-partials
branch
from
October 17, 2026 23:42
41e20b4 to
2887421
Compare
Collect each match's mtime and the resolved filter root inside the bounded walk, publishing paths before metadata so stalled stats retain honest partials. Sort partials only from in-memory keys, preserve complete-walk ordering, and reuse discovery metadata and presence through glob merges. Outline already sorts only collected strings after discovery and needs no equivalent change. Existing assertions remain intact. Add stalled-metadata, post-walk I/O, complete-order, and missing-mtime regressions; independently restoring grep and glob post-walk lookups reddens the respective stall regression. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
The engine-unwired quality predicate requires the pinned tool-call parity fixture. Add that fixture after verifying all 16 parity tests and replaying all 103 real-query rows byte-identically against the reference. Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
…st walk partials branch Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
github-merge-queue
Bot
removed this pull request from the merge queue due to a conflict with the base branch
Oct 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Accepted agent delivery; branch-keyed search descriptor included.
Summary by cubic
Changes
glob,grep, andoutlineon large directories outside the project (such as~/.cargo/registry/src) so a slow walk returns the files found so far, marked as a partial walk, instead of failing with aread_blockederror.read_blockedis now reserved for one filesystem call that stops making progress; the reply names that path, and a walk that stalls keeps what it had already found.complete: falseand lists the stalled path in itsgaps, with a trailingread_blockedline. A stall before any results is still theread_blockederror.Written for commit 406908e. Summary will update on new commits.


Confidence Score: 5/5
The latest changes appear safe to merge; no new actionable issue was found.Summary
Directory walks now keep partial results when time runs out or a filesystem call stalls.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart TD A[Bounded directory walk] --> B[Publish file and unknown modification time] B --> C[Read modification time inside helper] C --> D{Walk ends} D -->|Finished| E[Take snapshot] D -->|Stopped| E E --> F[Sort snapshot without filesystem calls] F --> G[Return files and completeness]Reviews (6) · Last reviewed commit: "search-quality descriptor: one engine_un..." · Reviewed by Greptile