Repository navigation
Convert legacy Office files before Docling and quiet server log noise - #449
Merged
Merged
Conversation
Docling only reads OOXML, so a .doc attachment failed with "File format
not allowed" and only came back through the slow PDF-and-OCR fallback.
.xls and .rtf had no fallback at all. LibreOffice now converts .doc, .rtf
and .odt to .docx, .ppt and .odp to .pptx, and .xls and .ods to .xlsx
before parsing, so .xls also takes the openpyxl path. Each conversion
gets its own LibreOffice profile, since concurrent runs that share one
fail. If conversion fails, the original file goes to Docling as before.
litellm's aiohttp transport caches a ClientSession per client for an
hour, then drops it without closing it. That logged "Unclosed client
session" every hour from the API and the worker, in bursts when the
cache evicted many at once. Core now uses litellm's httpx transport.
GET /documents/{id} logged every 404 at ERROR. The Documents page polls
each processing document every 2 s and never stopped for one that was
deleted, so one deleted document filled the log. The route now logs a
miss at DEBUG, and the page refreshes the list on a 404, which drops the
document and stops the polling.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ArnavAgrawal03
force-pushed
the
arnav/d0o16
branch
from
October 13, 2026 20:25
898ce36 to
6951f59
Compare
ArnavAgrawal03
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes three problems found in a customer's on-prem server log. The install is CPU-only and doesn't use ColPali.
.docattachments failed in Docling. The log showedDocling parsing failed for <name>.doc: File format not allowed. Docling only reads OOXML. Before this change,.docand.pptwere recovered only by the slow deep fallback (PDF, then OCR), and.xlsand.rtfwere not recovered at all. LibreOffice now converts legacy formats before parsing:.doc,.rtf,.odt.docx.ppt,.odp.pptx.xls,.ods.xlsx(then the openpyxl fast path)Each call gets a private LibreOffice profile, because concurrent conversions that share the default profile fail. If conversion fails, the original file goes to Docling as before, and the deep fallback now also covers
.xls,.rtfand the OpenDocument formats.Unclosed client sessionevery hour from the API and the worker. litellm 1.74.7 caches one aiohttp-backed client per model/key for an hour (_DEFAULT_TTL_FOR_HTTPX_CLIENTS). When it evicts a client,InMemoryCache._remove_keydrops it without closing it, and GC then logs the warning. Evictions run in batches, which explains the bursts of about 16. Upstream reverted closing on eviction because it broke clients that were still in use, so a litellm upgrade won't help. Core now setslitellm.disable_aiohttp_transport = True, so litellm uses httpx.About 150
Error getting document: 404lines, one every 3 s. The Documents page pollsGET /documents/{id}every 2 s for each document it last saw as processing, and it ignored non-OK responses. A document deleted mid-processing was polled until the tab closed. The page now refreshes the list on a 404, which removes the document and ends the polling. The route logs a miss at DEBUG instead of ERROR.Not changed: the
Relative URI reference without a base URIline comes from WeasyPrint. Since #447, WeasyPrint only runs for ColPali, so it won't appear on installs without ColPali once they upgrade.Testing
core/tests/unit/test_legacy_office_parsing.py: conversion targets,.xlsreaching the openpyxl path, fallback when conversion fails, and modern files left alone. The real-LibreOffice test skips whensofficedoesn't run.python:3.11-slimwith Debian's LibreOffice. Two conversions in parallel with private profiles (.doc→.docx,.xls→.xlsx) both succeeded.MorphikParserwith Docling then read the converted files correctly: the.doctext, and the.xlsas a Markdown table.pyteston the touched modules and the vision OCR tests from Read text in images with a vision model instead of local OCR #448: 74 passed, 1 skipped.tsc --noEmit, eslint and prettier pass onDocumentsSection.tsx.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.