Repository navigation
Persist use_colpali on the document record so requeue keeps it - #450
Merged
Merged
Conversation
/ingest/requeue resolves use_colpali from the request, then from the document's system_metadata, and otherwise falls back to False. The flag was only written to system_metadata when ingestion completed, so a document that failed before ever completing was silently requeued without ColPali when the requeue request omitted the flag. Store the requested flag when the document record is created and when its content is replaced, so requeue keeps the original mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adityav369
added a commit
that referenced
this pull request
Oct 14, 2026
/ingest/requeue resolves use_colpali from the request, then from the document's system_metadata, and otherwise falls back to False. The flag was only written to system_metadata when ingestion completed, so a document that failed before ever completing was silently requeued without ColPali when the requeue request omitted the flag. Store the requested flag when the document record is created and when its content is replaced, so requeue keeps the original mode. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
/ingest/requeuedecidesuse_colpaliper document in this order:use_colpalivalue in the request job, if given;use_colpaliin the document'ssystem_metadata;False.The worker only writes
use_colpalitosystem_metadatawhen ingestion completes. A document that was ingested withuse_colpali=truebut failed before ever completing has no stored flag. If a client later requeues it without passing the flag, the document is silently reprocessed with text embeddings only. Nothing reports that the mode changed.Fix
Store the requested
use_colpaliinsystem_metadatawhen the document record is first created (_ingest_file_content_locked) and when its content is replaced (queue_document_update). Requeue then keeps the original mode. An explicituse_colpaliin the requeue request still takes precedence.Tests
use_colpali, and a queued content update persists it. Both tests fail without the fix.pytest core/tests/unit/test_ingestion_service_metadata_update.py core/tests/unit/test_week1_ingest_fixes.pyand the other ingest unit tests: 58 passed.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.