Make the per-record text cap settable via LLM_MAX_TEXT_CHARS - #124
Open
Airwhale wants to merge 4 commits into
Open
Make the per-record text cap settable via LLM_MAX_TEXT_CHARS#124Airwhale wants to merge 4 commits into
Airwhale wants to merge 4 commits into
Conversation
MAX_TEXT_CHARS was a constant, so comparing two values meant editing code between runs. It now reads LLM_MAX_TEXT_CHARS, the way MAX_TOKENS already read LLM_MAX_TOKENS, and defaults to the same 8000. Measured on the 2-week 15-community corpus: 9.8% of aggregated patients exceed 8000 chars and 20.2% of all corpus text was being discarded to the cap. Raising it to 60000 recovered 21.1% more field fills from those patients for $0.04 on a $1.71 run -- the bill is dominated by output tokens, which scale with fields found rather than input length. Wall time is the real cost, +47%. The comment's warning still holds and is why the default did not move: at 30000 a reply overran 8192 output tokens mid-JSON. Raise this together with LLM_MAX_TOKENS, not alone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Airwhale
marked this pull request as ready for review
August 3, 2026 19:04
Without this the branch adds LLM_MAX_TEXT_CHARS and tests nothing: the suite sits at 318, the same as main, and the only surviving mention of the constant reads it rather than checking it is settable. MAX_TEXT_CHARS is bound at import, so reload is the only way to exercise the env var in-process. The reload back runs in a finally rather than after the assert -- the module is shared with the rest of the suite, and a failing assert would otherwise leave every later test on a 1234-character cap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One fix, one line of behaviour. Second of the Path B commits needed to reconstruct the
2026-07-31 full-corpus run (
s3://patientpunk/full_corpus_2026-07-31/).Why
MAX_TEXT_CHARSwas a module constant:Changing this to a variable allows us to read larger input (which happened to me), and record it in an easy way.