Repository navigation
Conversation
FillDocsTask filled fields that are both stored and doc-valued twice per hit: fetchSlice read the value from doc values, then getStoredFieldFetchContext also selected the field for the stored document read, which decompressed a stored fields block and overwrote the doc-values value. Only collect stored fields without doc values, so these fields are read from doc values once, as FillFieldsTask already does. Multi-valued fields that are stored and doc-valued are now returned in doc-values order for the default and parallel-by-document fetch, the same as with parallel fetch by field. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vim345
marked this pull request as ready for review
October 9, 2026 17:08
aprudhomme
reviewed
Oct 9, 2026
Comment on lines
+796
to
+797
| && indexableFieldDef.isStored() | ||
| && !indexableFieldDef.hasDocValues()) { |
Contributor
There was a problem hiding this comment.
One thing that is useful about stored fields is that it preserves ordering, where doc values get sorted. It may be better to give stored fields priority when both are enabled, to allow retrieval of the unsorted values.
Contributor
Author
There was a problem hiding this comment.
Good point, thanks. I changed it so stored fields take priority whenever both are enabled, although not a single field enables them both right now.
Stored values keep the original order and duplicates, while doc values are sorted (and de-duplicated for sorted set), so a field that is both stored and doc-valued should return its stored values. FillDocsTask already did, but also read the doc values first and discarded them, and FillFieldsTask (parallel fetch by field) returned doc values instead. Keep the stored read in FillDocsTask, skip the redundant doc values read in fetchSlice, and check the stored case before doc values in FillFieldsTask, so every fetch path returns stored values. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a retrieved field is both stored and has doc values, fetch reads it twice or from the wrong place, depending on the fetch path:
FillDocsTask(default fetch and parallel fetch by document) reads it twice per hit.fetchSlicefills it from doc values, then the stored-document read fills it again and overwrites the doc-values result.FillFieldsTask(parallel fetch by field) checks doc values first, so it returns doc values while the other paths return stored values.This PR makes stored fields take priority everywhere, as discussed in review. Stored values keep the original order and duplicates, while doc values come back sorted (and de-duplicated for
SORTED_SET). The changes:fetchSliceskips the doc-values read for stored fields. They are filled once, from the stored document.FillFieldsTask.getFieldForHitchecks the stored case before the doc-values case.Fields that are only stored or only have doc values are unaffected.
Behavior change
Default fetch and parallel fetch by document return exactly what they did before, minus the wasted doc-values read. Parallel fetch by field (
parallelFetchByField: true) now returns stored values for these fields. For multi-valued fields that means original order instead of doc-values order, matching the other paths.Tests
FillDocsTaskTest:fetchSlicereturns the stored values (b, a, b) for a stored + doc-values field and never callsgetDocValues.StoredDocValuesFetchTest: a multi-valued ATOM field that is stored + doc-valued returns stored values (b, a, b), and a stored-only field returns its values. It runs three ways: the default fetch, parallel fetch by document, and parallel fetch by field. Each parallel variant first asserts that its fetch config took effect. Againstmain, only the by-field run fails (it returneda, b)../gradlew :testpasses, including spotless.Side note:
MultiSegmentParallel*FetchTestset parallel fetch options as YAML keys that nothing reads, soMultiSegmentParallelFieldsFetchTestnever runsFillFieldsTask. Worth a follow-up.🤖 Generated with Claude Code