Bump dependencies, drop unused ones, OCR images embedded in PDFs

Dockerfile

@@ -9,8 +9,6 @@ RUN apt-get update \
     && apt-get install -y --no-install-recommends \
     pandoc \
     netcat-openbsd \
-    libgl1 \
-    libglib2.0-0 \
     && rm -rf /var/lib/apt/lists/*
 
 COPY requirements.txt .

README.md

@@ -214,8 +214,14 @@ The following environment variables are required to run the application:
 - `MISTRAL_SERVER_URL`: (Optional) Base URL of the Mistral API. Leave unset for the SDK default (`https://api.mistral.ai`); set `https://api.eu.mistral.ai` to keep OCR traffic on Mistral's [EU regional endpoint](https://docs.mistral.ai/inference/regional-inference). Regional endpoints don't offer the Files API, which is fine here: documents and images are sent inline as base64. Confirm with Mistral that your OCR model is served on the regional endpoint before relying on it.
     - Note: LibreChat's own `mistral_ocr` strategy uploads through the Files API, so on an EU-only deployment route PDFs and images to this service's `/text` endpoint instead.
     - For EU embeddings as well, point `RAG_OPENAI_BASEURL` at an EU OpenAI-compatible endpoint (e.g. `https://eu.openrouter.ai/api/v1`) with a non-OpenAI model such as `mistralai/mistral-embed-2312` (1024 dimensions), and set `RAG_CHECK_EMBEDDING_CTX_LENGTH=False`: with the check on, inputs are sent as OpenAI tiktoken token IDs, which non-OpenAI models don't accept. Switching embedding models requires re-indexing existing files (see `EMBEDDINGS_DIMENSIONS`).
-- Standalone image uploads (`.png`, `.jpg/.jpeg`, `.gif`, `.bmp`, `.tif/.tiff`, `.webp`) — including multi-page TIFF — are run through **Mistral OCR**, yielding one result per page. Requires `MISTRAL_API_KEY` (same key used for PDF OCR). This is separate from PDF handling; images embedded inside PDFs are not extracted.
+- Standalone image uploads (`.png`, `.jpg/.jpeg`, `.gif`, `.bmp`, `.tif/.tiff`, `.webp`) — including multi-page TIFF — are run through **Mistral OCR**, yielding one result per page. Requires `MISTRAL_API_KEY` (same key used for PDF OCR). This is separate from PDF handling; images embedded inside PDFs are covered by `MISTRAL_OCR_EMBEDDED_IMAGES` below.
 - `IMAGE_OCR_MAX_PAGES`: (Optional) Max number of frames/pages of a single image upload (e.g. animated GIF or large multi-page TIFF) sent to OCR; frames beyond the cap are skipped with a warning. Default value is `100`.
+- `MISTRAL_OCR_EMBEDDED_IMAGES`: (Optional) Boolean. Mistral OCR returns only a placeholder (`![img-0.jpeg](img-0.jpeg)`) for a raster image embedded in a PDF page, such as a photo or scan of a page, and does not read the text inside it. When "True" (default), those images are extracted, OCR'd in a second Mistral call each, and the recognized text replaces the placeholder. **Each embedded image is billed as an extra OCR page**; set "False" to OCR only the PDF itself. If an image's OCR call fails, its placeholder is kept and the document still loads.
+- `MISTRAL_OCR_IMAGE_MIN_SIZE`: (Optional) Minimum image size in pixels for an embedded image to be extracted, so logos and icons are skipped. Default value is `100`.
+- `MISTRAL_OCR_MAX_EXTRACTED_IMAGES`: (Optional) Max number of embedded images Mistral extracts per document (minimum `1`), before any filtering; this bounds the size of the OCR response. A warning is logged when the document has more, and the later images are not extracted. Default value is `200`.
+- `MISTRAL_OCR_MAX_EMBEDDED_IMAGES`: (Optional) Max number of distinct embedded images OCR'd per document (minimum `1`), i.e. the number of extra billed OCR calls. It is applied after the page-fraction filter and the removal of identical images, in page order; a warning is logged when the document has more, and the extra images are left as placeholders. Default value is `50`.
+- `MISTRAL_OCR_IMAGE_MIN_PAGE_FRACTION`: (Optional) Embedded images covering less than this fraction of their page (`0` to `1`) are not OCR'd and keep their placeholder, so letterheads and logos repeated on every page cost nothing. Set to `0` to OCR every extracted image. Skipped images are logged at debug level. Identical images are always OCR'd only once per document. Default value is `0.1`.
+- `MISTRAL_OCR_IMAGE_CONCURRENCY`: (Optional) Max number of embedded-image OCR calls in flight at once across all documents and requests (minimum `1`); it is also the parallelism within a single document. Default value is `4`.
 - `DEBUG_RAG_API`: (Optional) Set to "True" to show more verbose logging output in the server console, and to enable postgresql database routes
 - `DEBUG_PGVECTOR_QUERIES`: (Optional) Set to "True" to enable detailed PostgreSQL query logging for pgvector operations. Useful for debugging performance issues with vector database queries.
 - `CONSOLE_JSON`: (Optional) Set to "True" to log as json for Cloud Logging aggregations

app/config.py

@@ -224,6 +224,40 @@ async def dispatch(self, request, call_next):
 # Override the Mistral API base URL, e.g. https://api.eu.mistral.ai for EU-only
 # deployments. Empty keeps the SDK default (https://api.mistral.ai).
 MISTRAL_SERVER_URL = get_env_variable("MISTRAL_SERVER_URL", "").rstrip("/")
+# Mistral OCR returns only a placeholder (![img-0.jpeg](img-0.jpeg)) for a raster
+# image embedded in a PDF page, without transcribing the text inside it. When
+# enabled, the extracted images are requested and OCR'd separately, and the
+# recognized text replaces the placeholder. Every image costs one extra OCR page.
+MISTRAL_OCR_EMBEDDED_IMAGES = (
+    get_env_variable("MISTRAL_OCR_EMBEDDED_IMAGES", "True").lower() == "true"
+)
+# Smallest image side (pixels) Mistral extracts, so logos/icons aren't OCR'd.
+MISTRAL_OCR_IMAGE_MIN_SIZE = max(
+    0, int(get_env_variable("MISTRAL_OCR_IMAGE_MIN_SIZE", "100"))
+)
+# Max embedded images Mistral extracts per document (minimum 1), before any
+# filtering. This bounds the size of the OCR response; images beyond it are not
+# extracted at all.
+MISTRAL_OCR_MAX_EXTRACTED_IMAGES = max(
+    1, int(get_env_variable("MISTRAL_OCR_MAX_EXTRACTED_IMAGES", "200"))
+)
+# Max distinct embedded images OCR'd per document (minimum 1), counted after the
+# page-fraction filter and the dedupe. Each one is a billed OCR call.
+MISTRAL_OCR_MAX_EMBEDDED_IMAGES = max(
+    1, int(get_env_variable("MISTRAL_OCR_MAX_EMBEDDED_IMAGES", "50"))
+)
+# Embedded images covering less than this fraction of their page (0-1) are not
+# OCR'd, so letterheads and logos repeated on every page cost nothing. 0 disables.
+# Applied before MISTRAL_OCR_MAX_EMBEDDED_IMAGES, so skipped images don't count.
+MISTRAL_OCR_IMAGE_MIN_PAGE_FRACTION = min(
+    1.0,
+    max(0.0, float(get_env_variable("MISTRAL_OCR_IMAGE_MIN_PAGE_FRACTION", "0.1"))),
+)
+# Max embedded-image OCR calls in flight at once across all documents and
+# requests (minimum 1); also the parallelism within a single document.
+MISTRAL_OCR_IMAGE_CONCURRENCY = max(
+    1, int(get_env_variable("MISTRAL_OCR_IMAGE_CONCURRENCY", "4"))
+)
 
 # DOCX handling for the /text endpoint: when enabled, .docx files are converted
 # via pandoc so tracked changes and comments are preserved (the embedding path