Keep doses linked to their treatments - #142
Draft
Airwhale wants to merge 9 commits into
Draft
Conversation
added 2 commits
August 25, 2026 10:51
Replace flat dosage values with validated treatment: dose pairs so posts that mention multiple interventions cannot be matched by list position or textual proximity. Add a Pydantic v2 treatment-value boundary model, reject unlinked values in new extraction records, and decompose the structured CSV field into aligned dosage_treatment and dosage_value columns. Preserve bare legacy dosage values with a blank treatment rather than inventing an attribution. Cover prompt behavior, multi-treatment extraction through CSV export, malformed values, legacy normalization, and clustering exclusions. Update the schema and methods documentation.
Keep the Pydantic treatment-value boundary, but remove the callback-based generic normalizer and decomposer. Normalize dosage pairs directly and place dosage CSV decomposition beside the existing treatment-outcome normalization code. The JSON and CSV contracts are unchanged: new records still require treatment: dose pairs, and legacy bare doses still produce a blank treatment column. The full suite remains green.
added 2 commits
August 25, 2026 15:16
Split the dosage prompt guidance into explicit evidence, treatment linkage, formatting, and omission decisions. Exclude prescribed-but-unstarted, planned, declined, third-party, and general-information doses so structured output represents treatment exposure rather than nearby dosage text. Add prompt contract coverage for the negative examples and align the shipped schema description with the author-exposure requirement.
Delete the explanatory comment above dosage pair normalization and shorten the dosage decomposition docstring by removing the legacy bare-dose sentence. Runtime behavior and tests remain unchanged.
…eatment-dose-pairs
…eatment-dose-pairs
…eatment-dose-pairs
…eatment-dose-pairs
…eatment-dose-pairs
Airwhale
pushed a commit
that referenced
this pull request
Aug 27, 2026
Add a validated Pipeline B record boundary and shared study paths so analyses run reliably from the repository root or against versioned outputs. Summarize explicit treatment-linked dosages and administration routes by compound, centralize dose attribution helpers, and remove the audit script's source exec. Update the workbook and runbook for the #142/#141 stack, fresh caches, group attribution guarding, and versioned extraction artifacts. Add focused contract tests for stale CSV rejection, pair alignment, derivative precedence, and single-letter compound attribution.
Airwhale
pushed a commit
that referenced
this pull request
Sep 3, 2026
Add a validated Pipeline B record boundary and shared study paths so analyses run reliably from the repository root or against versioned outputs. Summarize explicit treatment-linked dosages and administration routes by compound, centralize dose attribution helpers, and remove the audit script's source exec. Update the workbook and runbook for the #142/#141 stack, fresh caches, group attribution guarding, and versioned extraction artifacts. Add focused contract tests for stale CSV rejection, pair alignment, derivative precedence, and single-letter compound attribution.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#120 -> #142 -> #141
Why
The variable-extraction pipeline currently returns dosage as a flat list. When a post names several treatments, downstream code cannot tell whether 4.5 mg belongs to LDN or B12. Matching separate treatment and dose lists by position would silently create false records.
Approach taken
User-facing changes
Closes #128