CI's integration-test job runs on every push to main and on a weekly cron, but it uses the committed public BABEL_RELEASES_URL, which does not publish the DuckDB Parquet files. Every Parquet-dependent test therefore skips, and CI reports green without having exercised any of the code those tests cover.
That is working as designed — a public contributor should not need a private URL to get a green build — but it means the integration suite is only ever run by hand, on a developer machine, by someone who remembers to. The bugs it would have caught are real and have happened twice already:
Both were invisible for the same reason: the code path only executes against a release that publishes the files.
What would need deciding
The blocker is not the workflow, it is where the URL lives. Options, roughly in increasing order of commitment:
- A repository secret plus a scheduled job.
BABEL_RELEASES_URL as an Actions secret, used only by a workflow_dispatch + cron job, never by PR builds. Keeps the URL out of the tree and out of fork PRs, but puts it somewhere GitHub admins can read, and this repository has already had one incident of that URL leaking (see the history-rewrite note in CLAUDE.md).
- A self-hosted runner with the URL in its environment. No secret in GitHub at all, but somebody has to own the runner.
- Leave CI as is and run it manually before releases, with a documented checklist. Cheapest; relies on discipline, which is what we have now.
Also worth sizing before committing to 1 or 2: Concord.parquet is 4.6 GB in 2026jul22 and its tests are not marked slow, so even -m "integration and not slow" is a multi-gigabyte download per run. See #18, which tracks that sizing question — this issue is about whether CI ever gets the URL at all, and the two should probably be decided together.
Affected
.github/workflows/ci.yml — the integration-test jobCLAUDE.md — the note that a couple of dozen skips is the expected result
CI's
integration-testjob runs on every push tomainand on a weekly cron, but it uses the committed publicBABEL_RELEASES_URL, which does not publish the DuckDB Parquet files. Every Parquet-dependent test therefore skips, and CI reports green without having exercised any of the code those tests cover.That is working as designed — a public contributor should not need a private URL to get a green build — but it means the integration suite is only ever run by hand, on a developer machine, by someone who remembers to. The bugs it would have caught are real and have happened twice already:
get_curie_xref.cache_clear()calls that raisedAttributeError, left over from anlru_cachethat had been removed. Found by reading, in Copilot review on Add babel-explorer: a CLI for querying Babel cross-references via DuckDB and NodeNorm #20, not by running.downloaded_identifiersfixture because a release publishedConcord.parquetwithoutIdentifiers.parquet(Skip absent test dependencies, and stop unit runs deleting the integration cache #27).Both were invisible for the same reason: the code path only executes against a release that publishes the files.
What would need deciding
The blocker is not the workflow, it is where the URL lives. Options, roughly in increasing order of commitment:
BABEL_RELEASES_URLas an Actions secret, used only by aworkflow_dispatch+ cron job, never by PR builds. Keeps the URL out of the tree and out of fork PRs, but puts it somewhere GitHub admins can read, and this repository has already had one incident of that URL leaking (see the history-rewrite note in CLAUDE.md).Also worth sizing before committing to 1 or 2:
Concord.parquetis 4.6 GB in2026jul22and its tests are not markedslow, so even-m "integration and not slow"is a multi-gigabyte download per run. See #18, which tracks that sizing question — this issue is about whether CI ever gets the URL at all, and the two should probably be decided together.Affected
.github/workflows/ci.yml— theintegration-testjobCLAUDE.md— the note that a couple of dozen skips is the expected result