Draft/exploration: 7,8-DHF comparator study and side-effect severity - #140
Draft
Airwhale wants to merge 58 commits into
Draft
Draft/exploration: 7,8-DHF comparator study and side-effect severity#140Airwhale wants to merge 58 commits into
Airwhale wants to merge 58 commits into
Conversation
Airwhale
changed the title
Airwhale
force-pushed
the
shaun/study-78dhf-nootropics
branch
from
August 25, 2026 23:55
e967756 to
c4f5f99
Compare
Airwhale
changed the base branch from
shaun/fix-canonicalize-batch-split
to
codex/match-dose-administration-route
August 25, 2026 23:55
The study existed only as untracked files in one working tree. Its scripts, alias list, extraction schema, findings (NOTES.md) and a step-by-step RUNBOOK.md are here; no corpus data is, and none should be added -- source/, source_B/, outputs_A/ and the databases are patient text. Sits on top of the pipeline stack because it needs it: on main the classify stage asks for 10 tokens per prefilter item and dies on the first batch. RUNBOOK.md gives exact commands from S3 pull through corpus build, both pipelines, and each analysis, with the expected counts at each stage so a wrong turn is visible early. It also records what this branch cannot do: the published 71.1% was measured with reasoning enabled, this stack suppresses reasoning, and the constants needed to re-enable it are not here. The LLM cache does not key on that flag, so the two regimes silently share cache entries -- documented rather than fixed. The audit_*.py scripts are included deliberately. Each found a real defect in an analysis that looked finished: proximity dose-matching pulls other compounds' doses out of stack posts, and the side-effect-by-indication diagonal is a tagging artifact that collapses from 13% to 3% once outcome sentences stop assigning the indication. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add a validated Pipeline B record boundary and shared study paths so analyses run reliably from the repository root or against versioned outputs. Summarize explicit treatment-linked dosages and administration routes by compound, centralize dose attribution helpers, and remove the audit script's source exec. Update the workbook and runbook for the #142/#141 stack, fresh caches, group attribution guarding, and versioned extraction artifacts. Add focused contract tests for stale CSV rejection, pair alignment, derivative precedence, and single-letter compound attribution.
Analyze the complete 752-author nootropics extraction using treatment-linked dose and route fields. Normalize comparable mass doses, audit excluded values, and report both entry-level and author-level summaries. Replace the invalid overlapping full-sample Fisher comparison with matched McNemar and mutually exclusive sensitivity analyses. Update the study workbook, notes, and runbook with reproducible paths, repair settings, and the completed run results. Declare the workbook dependency and add regression coverage for mass-dose parsing and exclusion behavior.
Build an atomic SQLite artifact that preserves the completed Pipeline A database and imports all 752 normalized Pipeline B records. Add queryable linked dose, route, treatment-outcome, canonical side-effect, and manifest tables. Create one author-compound exposure table that combines dose, route, desired-result efficacy, and conservative outcome summaries without inventing event-level pairings. Bucket quantitative doses, route families, efficacy targets, and safety signals for proposed-study planning. Generate a reproducible aggregate Markdown analysis, document exact rebuild and validation commands, add Typer and Rich CLI support, and cover the database contract and bucket boundaries with tests.
Define a versioned, tiered comparator cohort and build one privacy-preserving Reddit thread corpus for all compounds. Run each compound through the same targeted sentiment pipeline, exclude derivative-only spans from the parent target, link side effects to treatment IDs, and add aggregate independent and matched-author comparisons. Separate PEM from general fatigue, include dose and route sections from linked Pipeline B records, key LLM caches by reasoning mode, and record that mode in extraction provenance. Add end-to-end corpus and report contract tests, alias exclusion coverage, runbook commands, and methodology notes. Full suite: 391 passed.
Add a bounded three-attempt retry around prefilter and sentiment calls when the provider returns an empty streamed response. Fall back from failed batch calls to item-level classification while preserving the existing global transport and truncation retry policy. Add regression tests for recovery, non-empty errors, and the retry bound. Full suite: 394 passed.
Treat OpenRouter upstream stream failures as bounded transient errors even when they are raised through the OpenAI SDK without an HTTP status code. Preserve fail-fast behavior for deterministic 4xx responses and cover the DigitalOcean stream-failure signature with a regression test.
Add the privacy-safe author-level sentiment, matched sensitivity, treatment-linked side-effect, dose, route, and symptom-outcome report for the completed comparator cohort. Document executed cohort sizes and interpretation boundaries, make zero PEM coverage explicit, and keep all source text, author identifiers, caches, manifests, and databases outside the repository.
Join the private comparator and linked-variable artifacts by hashed author and treatment to report any-side-effect percentages for every dose and route bucket. Include classifier coverage, Wilson intervals, leading mapped-effect percentages, and explicit cross-report and non-causal interpretation boundaries. Regenerate only the aggregate study report and add contract coverage for denominators and percentages.
MAX_TEXT_CHARS was a constant, so comparing two values meant editing code between runs. It now reads LLM_MAX_TEXT_CHARS, the way MAX_TOKENS already read LLM_MAX_TOKENS, and defaults to the same 8000. Measured on the 2-week 15-community corpus: 9.8% of aggregated patients exceed 8000 chars and 20.2% of all corpus text was being discarded to the cap. Raising it to 60000 recovered 21.1% more field fills from those patients for $0.04 on a $1.71 run -- the bill is dominated by output tokens, which scale with fields found rather than input length. Wall time is the real cost, +47%. The comment's warning still holds and is why the default did not move: at 30000 a reply overran 8192 output tokens mid-JSON. Raise this together with LLM_MAX_TOKENS, not alone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HEALTH_SUBREDDITS decides which text is sent to the model first, and so which text survives the MAX_TEXT_CHARS cut. Every member is a long-COVID or ME-CFS community, so on the r/Nootropics corpus every single text fell to the "other" bucket: the ordering ran, changed nothing, and truncation kept whatever came first in scrape order. That is a silent recall loss. Measured on the 89 tropoflavin authors who provably discuss 7,8-DHF but produced no compound exposure row, 6 had the compound truncated away entirely. A schema now sets priority_subreddits and gets its own communities ordered first. Schemas that name none keep HEALTH_SUBREDDITS, so existing long-COVID runs are byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The base medications description reads "Prescription drugs and daily supplements (LDN, Paxlovid, gabapentin, magnesium, probiotics)". Every example is a long-COVID drug. Against an r/Nootropics population that wording dropped research chemicals the model had already read: of the 42 authors who discuss 7,8-DHF and produced no record of it at all, 36 had the compound sitting inside the input window. Truncation explains only 6. Overriding medications in extension_fields keeps this study-local -- the base description and every other schema are untouched. The same file now names its priority_subreddits, so its own communities survive truncation. Re-running the 89 affected authors with these two changes plus the raised cap takes "compound named in medications" from 43 to 70 and "yields a compound exposure row" from 0 to 48, against 22 for an unchanged re-run. The recall is not free: auditing every extracted dose and route against source text, 3 of 30 numeric doses were attributed to the wrong compound and 4 of 20 routes were unsupported. The schema notes field says so; corroborate before trusting a dose. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
llm_provenance.json named the model and temperature but not the code, so an output could not be traced to what produced it. That mattered here: re-running the unchanged 2026-08-27 extraction today yields 22 compound exposure rows on 89 authors where the original produced 0. Nothing in the artifacts distinguishes a hosted-model change from a code change, because neither was pinned. Provenance now also carries commit, extracted_at, and max_text_chars. The commit is read out of .git rather than shelled out for, because the pipeline is deliberately subprocess-free (TestPipelineNoSubprocess). This makes drift detectable, not impossible. Pinning the model itself is a policy call and is left alone here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Coverage was only ever reported as absolute counts -- "752 author histories", "202 compound exposures" -- with no denominator, so a recall collapse looked identical to a small cohort. This supplies the denominator by scanning the same corpus the extractor was given. Run against the 2026-08-27 extraction it reports: of 748 authors whose text mentions a 7,8-DHF alias, 273 have it named anywhere in their record and 173 in a field that can reach a compound exposure row. Read that as a trend signal, not a defect count. The denominator is "mentions the compound", and the prompt deliberately excludes treatments a patient only asked about or was offered, so the miss rate is an upper bound. The script says so in its output rather than leaving the number to be quoted bare. For a true recall figure, intersect with an independent judgement that the author used the compound; treatment_reports shares the author_hash namespace and serves. It also reports attribution, because exhaustiveness and correctness trade off: asking a model for complete stack lists makes it pair a compound with whatever dose is nearby. Every numeric dose and route is corroborated against source text and flagged when it sits far from any mention of the compound it was attached to. On the patched re-run that catches 3 of 29 doses, all three verified by hand as another compound's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both new files pointed at C:/Users/scgee/OneDrive/... , which works on exactly one machine and names a person in a repo that may be shared. They now resolve PatientPunk_data from the repo root and honour PATIENTPUNK_DATA, matching build_corpus.py and the rule now written down in AGENTS.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous read of this question rested on an extraction that had silently missed a large share of the compound mentions in its own corpus (#143). This rebuilds it on the merged set: 202 original compound exposures plus 25 recovered by the 2026-09-01 repair run, gated on the corroboration check that drops a dose sitting too far from any mention of the compound it was attributed to. The conclusion is unchanged, which is the point worth recording. Every dose band's 95% interval still contains the cohort baseline of 35.8%; the trend across the six milligram bands is z = -0.04, p = 0.97; sublingual against every other known route is p = 0.78. The added data moved the mean interval width from 59 to 55 percentage points and moved nothing off the baseline. This is a better-supported null, not a new result. The notebook carries its own limits: the outcome is a reporting rate rather than incidence, dose and route are still rarely co-reported so no interaction is estimable, and the recovered route values are weaker than the recovered doses -- 4 of 20 were unsupported by any route language near the compound. Also gitignores the canonicalisation outputs and executed-notebook artifacts that were accumulating untracked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous build charted dose bands and routes with both tropoflavin compounds combined. Neither pools. Milligrams are not comparable between substances. An earlier panel-wide version of this made the point vividly: its >=100 mg band was 7/10 gram-scale lion's mane, which manufactured a downward trend that vanished the moment the compounds were separated. The same objection applies to 7,8-DHF and 4'-DMA, which differ about tenfold in potency and barely overlap in range -- 4'-DMA has nothing above 25 mg while 7,8-DHF concentrates between 25 and 100. A shared "<5 mg" bucket was stacking doses an order of magnitude apart. Route does not pool either. The label "sublingual" is compound-independent but the exposure is not: how much bypassing first-pass metabolism is worth depends on the compound. Pooling averages heterogeneous effects into a number that estimates nothing. Everything is now within-compound. That leaves 7,8-DHF as the only testable stratum (49 dosed, 45 routed); 4'-DMA is reported descriptively and explicitly not tested at 15 dosed and a 14-vs-3 route split. The conclusion is unchanged: trend across bands z = +0.70, p = 0.49; sublingual vs other routes p = 0.53; every band's interval contains the compound's own baseline. Adds a section verifying the sample size against raw text rather than the pipeline, with the two scan scripts. Extraction found 59 people stating a dose; scanning the source directly brackets the truth at 60-75. The loose 146 is inflated by rodent mg/kg figures quoted from papers and by stack lists where the milligrams belong to a neighbour. 747 of 752 authors mention this compound and about a twelfth ever write down a number -- that, not extraction quality, is what caps this analysis. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scope is now one compound. 4'-DMA is dropped rather than shown alongside: 15 dosed users and a 14-vs-3 route split cannot support an inference, and its presence invited exactly the cross-compound comparison the previous commit removed. Dose and route still predict nothing. Trend across six bands z = +0.70, p = 0.49; <25 mg vs >=25 mg p = 0.31; sublingual vs other routes p = 0.53. The route table now prints the no-route group beside the rest, because the apparent sublingual effect was a small-denominator illusion -- swallowed oral and nasal each contain a single user who reported anything. Two new sections carry the actual result. What users report: sleep/wakefulness and activation/anxiety at 9.5% of the cohort each, headache 8.6%, everything else below 6%. Raw strings are uncanonicalised in comparators.db, so this needs the new SE_CATEGORIES map; the 33 terms it cannot place are printed rather than swept into an "other" bucket. No mucosal language appears anywhere, so there is no route-specific local effect to find. What predicts reporting: not the drug. Every one of the 16 users who said 7,8-DHF worsened something also has a side effect recorded -- complete separation, so the term is not estimable. It is not a risk factor; Pipeline B's "worsened: insomnia" and Pipeline A's side_effects ["insomnia"] are one sentence read twice, and 20 of 31 symptom strings match literally. With it excluded the model reaches pseudo R2 0.06, and the only surviving term is how much the user wrote about the compound -- an increase smaller than independent accumulation across reports would predict. A side-effect rate built this way measures reporting intensity, not risk, which limits how any drug in this panel can be compared. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every earlier estimate joined a dose to an outcome because the same author wrote both, somewhere across their history. That pairing is frequently not the author's: only 45 of 173 side-effect reports sit in a post that also states a dose, the person-level band disagrees with what the post itself says 21% of the time, and 33 of 59 dosed users have no dose in any post that recorded a side effect. The exposure column was often the wrong dose, which is a measurement error no sample size fixes. A row is now one report whose own post states a dose, so the pairing is the author's. That costs independence -- a person can contribute several posts -- so the model is fitted twice, naive and clustered by author. At 1.09 posts per author the correction is nearly a no-op (design effect 1.02 at ICC 0.2), which is the honest answer to the pseudo-replication objection rather than a reason to wave it away. The conclusion holds and gets weaker, as it should: 36 posts from 33 authors, trend z = -0.94, p = 0.35, odds ratio per e-fold dose 0.75 naive and clustered alike. The post-level estimate is smaller and less precise than the person-level one because it discards every user whose dose and side effect were never stated together. That is the trade -- the larger number meant less. The direction also flips negative, consistent with self-titration: someone who reacts badly at 25 mg does not go on to try 100 mg, so high-dose rows are selected for having tolerated the compound. That biases against finding harm at dose and no re-analysis of this data removes it. Attribution is now compound-specific. "4'-DMA-7,8-DHF" contains the string "7,8-DHF", so the previous pattern would hand a 4'-DMA dose to 7,8-DHF on any post naming both; compound_mentions excludes an occurrence that is the tail of the other name, and the notebook carries the unit test showing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add explicit subreddit and source-hash provenance to comparator corpus and sentiment runs. Build author-level variable extraction corpora for all configured comparators, enforce OpenRouter-only execution, and record provider-reported token usage. Correct comparison orientation and use mutually exclusive authors for independent Fisher tests while retaining matched-author sensitivity tests. Generalize Pipeline B treatment mapping across the full comparator cohort and add focused contract tests.
Require same-segment compound evidence for Pipeline B dose and route rows, and add a stricter post-level dose and outcome summary. Add explicit sparse-sample warnings, coverage and recall-proxy reporting, privacy scanning, a cross-subreddit overlap matrix, and an unpooled summary. Document the independent-cohort architecture and test attribution, overlap, and artifact privacy contracts.
Delete executed notebook artifacts that contained record-level derived text and local paths. Keep regenerated analyses in privacy-scanned aggregate Markdown reports. Also decouple Pipeline B author records from Pipeline A's user foreign key so independently versioned but hash-compatible runs can be joined without rejecting valid external records.
Replace a base-field registry row when the study schema intentionally overrides that field. This prevents duplicate field names from consuming coverage statistics twice and allows Pipeline B runs to finish their codebook phase. Add a focused regression test for the medications override used by the nootropics schema.
Allow author-level Pipeline B corpora to intersect deterministic comparator mentioners with authors who have a retained treatment-specific sentiment report. This removes false-positive and non-self-report authors before costly variable extraction while preserving the same subreddit boundary. Record the eligibility basis and add an integrity-checked regression test.
Normalize Pipeline B CSV output into aligned treatment, dosage, route, and outcome columns before database import. Preserve provider usage totals across resume-only finalization, add side-effect percentages to cohort reports, and test the linked export contract.
Follow Git's commondir pointer when a worktree HEAD references a branch stored in the shared repository metadata. Variable extraction manifests now record the actual 40-character commit instead of null when the pipeline runs from a Codex worktree.
added 22 commits
September 3, 2026 13:03
Expose the approved aggregate side-effect vocabulary and extend privacy scans to fail if a report table contains an unapproved label. This provides an automated guard against accidentally committing raw side-effect phrases.
Annotate accumulator structures and separate optional-path, SQLite-row, and author-vote variables so the complete multisubreddit study module set passes focused mypy checking.
Declare the optional API base, make the stream context manager's non-suppressing exit contract explicit, and reject missing target drugs before alias resolution. This keeps the token-accounting adapter type-safe without changing runtime behavior.
Compare Pipeline B output with the retained-author corpus, persist incomplete attempt provenance and token usage, and fail with a resumable action when records are missing. Aggregate reports now reject incomplete variable manifests.
Use explicit connection closure and commits in database fixtures so the 84 percent focused coverage run completes without resource warnings or relying on implicit context-manager lifetime.
Extract and cover the exact Pipeline B coverage calculation, including missing-record detection and rejection of impossible record inflation.
Capture the study commit at process start and carry provider-reported usage forward when a completed database is resumed. Add a contract test for additive token accounting so clean provenance normalization cannot erase prior model usage.
Keep local pytest coverage databases and reports out of version control so study provenance checks and privacy reviews see only source and aggregate deliverables.
Publish privacy-safe, unpooled reports for the original r/Nootropics cohort and eight additional subreddit cohorts, together with an author-overlap matrix and cross-cohort sentiment summary. Preserve treatment-to-value alignment when empty model-output placeholders occur, add a regression test, document multi-account overlap limits, and resolve focused type-checking collisions without changing report calculations.
Add a dedicated OpenRouter extraction for explicit reasons for 7,8-DHF use so outcome-derived symptoms are not reused as predictors. Analyze dosage, route, and multi-label reasons against sentiment and mapped side effects both per subreddit and after global author deduplication. Normalize sentiment against an equal-weight leave-target-out compound mean, retain raw rates, add subreddit-adjusted Mantel-Haenszel contrasts, document the external-artifact workflow, and cover normalization, deduplication, validation, and response parsing with tests.
Add a generated findings summary with coverage percentages and explicitly describe cross-compound normalization. Reassemble recursively split reason-extraction caches without repeated provider calls, validate cache identity, and cover the resume path with a regression test.
Add the privacy-safe, globally deduplicated and per-subreddit report for dosage, administration route, explicit reason for use, sentiment, and mapped side-effect reporting. Include between-compound normalization, uncertainty intervals, multiple-testing correction, provenance hashes, and explicit interpretation boundaries.
Extract quantitative dose, route, and explicit reason from each individual 7,8-DHF report without sending identifiers to the model. Add author-clustered ordinal-sentiment and side-effect dose-trend models, secondary descriptive tables, strict provenance validation, private-cache resumption, documentation, and regression tests.
Load and validate completed private episode records from an incomplete manifest, preserve their original item identifiers, and submit only missing source episodes on retry. Add regression coverage for incomplete-run resumption.
Switch incomplete-run retries to single-episode provider requests while preserving completed records and recording every batch size used in provenance. This bounds the retry tree for schema-resistant residual episodes.
Persist only exception-class counts for incomplete episode runs so persistent provider, parsing, and schema failures can be diagnosed without exposing source text or identifiers.
Capture only Pydantic error codes and schema locations for incomplete extraction runs, omitting response inputs, source text, and identifiers.
Add counts of local schema-validator messages to incomplete manifests while continuing to omit response inputs, identifiers, and source text.
Deduplicate exact structured dose, route, and reason repeats before validation and derive single versus multiple status from distinct values. Preserve every reported numeric dose and add regression coverage for repeated identical doses.
Report positive-versus-other sentiment as a prespecified sensitivity, expose primary outcome counts, and document why sparse same-post reason and route overlap cannot support multivariable confounder adjustment.
Add the privacy-safe aggregate report for 1,828 globally unique author-post episodes. Report author-clustered ordinal-sentiment and side-effect dose trends, binary sentiment sensitivity, per-subreddit estimates, route and reason descriptives, uncertainty intervals, and full external-artifact provenance.
Airwhale
force-pushed
the
shaun/study-78dhf-nootropics
branch
from
September 3, 2026 20:04
34f31bc to
ec7b8c4
Compare
Airwhale
changed the base branch from
codex/match-dose-administration-route
to
codex/side-effect-severity
September 3, 2026 20:06
added 5 commits
September 11, 2026 13:06
Teach the combined Pipeline A and B builder to read both legacy string arrays and the new structured side-effect records. Persist the validated severity alongside raw and canonical side-effect labels so downstream study reports can distinguish ungraded effects from explicitly mild, moderate, severe, or life-threatening reports. Extend the combined-database contract test to verify structured input and severity persistence. No generated data is included.
Add a read-only SQLite corpus builder that scans configured patient communities for the comparator cohort, hashes Reddit authors before writing artifacts, and retains only matched threads plus ancestor context. This supports the ten existing ME/CFS and Long COVID community datasets without placing corpus text in version control. Add an aggregate explicit-severity analysis that keeps independent community summaries, produces globally deduplicated nootropic, patient, and combined scopes, and reports dose-route linkage feasibility without exporting author rows. Include focused tests for hashing, context retention, cross-report severity deduplication, and linked exposure summaries.
…udit Publish the reviewed six-episode correction with Wilson intervals while explicitly preserving historical models as pre-audit. Guard the correction with cohort, source hash, and outcome-count contracts and refuse mismatched input rather than infer private exclusions. Cover idempotence, invalid counts, provenance mismatches, and generator integration with synthetic tests.
…nd provenance Add typed read-only author/effect loading and six outcome families across dose, route, joint and interaction designs. Suppress unsupported or failed fits, retain missing grades, use author-robust uncertainty and compound-adjusted pooled models, and write only aggregate external artifacts. Document where current eligibility and estimates differ from the saved September 3 analyses. Add synthetic end-to-end, privacy, source-immutability, and inference support tests.
…le study documentation Make the saved September 3 aggregates reviewable through typed schemas, a deterministic offline publisher, readable confidence-interval figures, exposure and model appendices, and privacy-checked provenance. Clearly separate frozen historical results from reconstructed models and flag unstable or degenerate historical estimates. Add a study README, current summary, sample availability and version history, updated runbook, synthetic publication tests, and scoped lint/type/coverage gates. Keep raw data, databases, and document exports external.
Airwhale
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR should likely never be merged: it's a disorginized workbook.
The maintainer's exploration-only note above is retained. This update organizes the study record; it does not change the draft status or request a merge.
Stack from
main: #121 -> #119 -> #122 -> #118 -> #120 -> #142 -> #141 -> #144 -> #140.This PR is the study layer and continues to target #144 (
codex/side-effect-severity). No other PR or branch in the stack is changed by this update.Why
The 7,8-DHF work has expanded from r/Nootropics sentiment exploration into ten-compound comparisons, independent community analyses, overlap/deduplication checks, same-post exposure models, and explicit side-effect severity. Those pieces need an understandable study record and reproducible review commands, without publishing raw posts or databases.
The latest update also addresses a stale >=100 mg descriptive row and distinguishes saved September 3 results from a newly reconstructed model implementation. The historical fine-grained modeling scripts were not recovered. Republishing their aggregate outputs is not the same as reproducing their calculations.
Approach taken
User-facing changes
Start at
studies/tropoflavin_nootropics/README.md. It describes work completed, the ten compounds, 19 communities, counting units, sample availability, report versions, limitations, and remaining questions.The study summary tells the overall story. Separate reports expose the frozen severity checkpoint, readable exposure/model tables and figures, and the new reconstruction with its differences. Positive sentiment is a perceived-benefit proxy, not clinical efficacy. Reported side effects and conditional severity proportions are not clinical incidence or safety rankings.
Only code, synthetic tests, small configuration/provenance artifacts, aggregate Markdown, and aggregate figures are published. Raw corpora, account-level records, SQLite databases, CSV exports, workbooks, and Word documents stay under external
PatientPunk_data/, honoringPATIENTPUNK_DATA. This update uses existing local data and makes no paid extraction calls.Detailed test plan
Run from this PR's checkout. The test suite needs no private datasets or provider credentials.
Expected: tests pass, scoped coverage is at least 75%, mypy and pre-commit pass, and compilation/whitespace checks exit successfully. CI enforces the same scoped lint/type/coverage rules.
Local verification on September 11: 486 tests passed, 94.84% coverage for the seven new modules, all three pre-commit gates passed, compilation and privacy scans passed. The seven published artifacts match a second independent publication byte-for-byte. Reconstructed eligibility, status, and coefficient CSVs also match a repeated offline run. Four SQLite resource warnings remain in the full suite; there were no test failures.
Check the entire stacked diff for prohibited artifacts:
Expected: no matches. Review JSON changes as configuration/provenance, not participant records.
With the preserved external datasets available, set the data root and use new output directories:
Verify end to end:
reports/severity_reconstruction.mdinstead of expecting identical historical coefficients.git status --shortcontains no generated data, participant records, credentials, or document exports.See
studies/tropoflavin_nootropics/RUNBOOK.mdsection 10 for portable commands and detailed interpretation limits. New provider extraction is a separate, paid operation and is not required to verify this update.