Read text in images with a vision model instead of local OCR

[ArnavAgrawal03]

Images uploaded with use_colpali=false go through Docling, which runs RapidOCR on the worker CPU and downloads its model files from modelscope.cn the first time it runs. A CPU-only on-prem install had no way to hand that work to a hosted model. iQor asked for exactly that: OCR for Azure DevOps screenshots through OpenAI, without a GPU server.

[parser] image_ocr_model names a key in [registered_models]. When it is set:

• PNG, JPEG, GIF, WebP, BMP and TIFF files are converted to PNG, scaled to at most 2048 px, and sent to that model through LiteLLM. The prompt asks for a Markdown transcription, or a one or two sentence description when the image has little text.
• The deep-parse fallback returns empty for these images instead of rerunning Docling.
• Rate limits, timeouts, connection errors and 5xx responses are raised as ConnectionError, so the ingestion worker retries the job. An invalid key fails the document.
• Startup fails if the key is not in [registered_models].

When it is unset, nothing changes. PDFs, Office files and ColPali ingests are not affected. DOCKER.md and both example tomls document the setting.

Testing

• core/tests/unit/test_image_ocr_model.py (9 tests): routing, PNG conversion of BMP/TIFF, downscaling, Docling still used when unset and for PDFs, deep fallback, retryable vs non-retryable errors, unknown key.
• Ran the API and worker locally with image_ocr_model pointing at a fake OpenAI-compatible server. A PNG and a BMP were each sent once as a high-detail PNG data URI, and their text became chunks. A PDF still went through Docling. With the fake server stopped, the image job logged a transient retry and stayed processing. After the server came back, it completed.
• Not run: a call to the real OpenAI API, because no key was available locally.
• Unit files that fail or hang without local services (test_docker_lifecycle backup test, test_colpali_embedding, test_document_download, test_reranker, test_multivector) behave the same on main.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}