fix(cache): reject the pre-#90 cache format instead of trusting it (#87 follow-up)

[kuanlinhuang]

#90 stopped writing region-restricted transcript sets to the sidecar path. It could do nothing about the ones already on disk, and nothing rejects them.

Every <gff3>.fastvep.cache written by a build between 2026-06-10 (6bc6bd5, which added the tabix read path) and 2026-08-18 (#90) may hold only the transcripts overlapping one input VCF's variants. Reading one back is #87 exactly: 47,196 of 47,196 chr17 variants reported as intergenic_variant at exit code 0, all 2,761 P/LP BRCA1 variants among them.

There is no check to add

Nothing in the file distinguishes a poisoned cache from a whole-file one:

• the magic is FSTVEP02 either way — fix(cache): three ways an incomplete transcript set became a silent wrong answer (#87, #88, #89) #90 did not bump it
• the bytes after the header are the same bincode
• the transcript count is plausible either way, and is not recorded anywhere to compare against
• the path is keyed on the GFF3 alone
• cache_is_fresh compares mtimes only, so a cache written after its GFF3 was downloaded is fresh — which is every poisoned cache, since it was written by a run that had just read that GFF3

Both released versions are inside the affected window: v0.2.0 was tagged the same day the tabix path landed, v0.3.0 on 2026-07-28.

So the only way to tell a trustworthy cache from an ambiguous one is to stamp the guarantee into the format. FSTVEP03 means "written by a build that cannot persist a region-restricted set."

What happens now

• save_cache writes FSTVEP03.
• FSTVEP02 and FSTVEP01 are recognised and refused, with an error naming the format, what goes wrong if it were read, and how to recover. Recognising them specifically is the point — a user with an old cache should be told to rebuild, not told their file is corrupt.
• The two recovery paths fix(cache): three ways an incomplete transcript set became a silent wrong answer (#87, #88, #89) #90 already built then apply unchanged and correctly: a sidecar cache is rebuilt from its GFF3 and republished as V3; an explicit --transcript-cache stops the run, because there is nothing to substitute and annotating without it reports every variant as intergenic.

Rejecting costs one rebuild, which the sidecar path does by itself. Reading costs a silently wrong annotation nothing downstream can detect. Same trade as #88's decision to error rather than annotate against an empty transcript set.

V1 is refused for the same reason rather than kept as read-only legacy support: it predates the zstd format and therefore predates the tabix path too, so it carries the identical ambiguity. test_legacy_gzip_cache_loads is replaced by the test asserting it does not load.

Tests

Six, all mutation-checked — with V2 accepted again, each one fails:

a_pre_90_cache_is_rejected_rather_than_read
a_legacy_gzip_cache_is_rejected_with_the_same_reason
a_v3_cache_written_now_round_trips
the_published_magic_is_v3
a_pre_90_sidecar_cache_is_rebuilt_rather_than_trusted
a_pre_90_explicit_transcript_cache_stops_the_run

The two integration tests build a real FSTVEP02 cache with the real codecs and set its mtime an hour into the future, so the run genuinely attempts the load rather than skipping it as stale. They assert the annotation survives (missense_variant present, intergenic_variant absent) and that the rebuild republishes as V3.

931 tests pass, up from 926. clippy --all-targets -D warnings clean, rustfmt clean.

Release note worth writing

Anyone holding a reference GFF3 with a sidecar cache beside it gets one rebuild on the next run. On a 1.2 GB Ensembl GFF3 that is minutes, and someone will wonder why.

Concretely relevant to Huang-lab/germline-plp-carrier-nf, whose setup_fastvep.sh clones fastVEP master unpinned and builds it, so its Minerva reference directory has a sidecar cache of unknown provenance. That pipeline is not exposed to #87 today — setup_fastvep.sh downloads the plain .gff3 and gunzips it, so there is no .tbi and the whole-file path is taken — but it fans out one bsub per chunk against a single shared --gff3, so it becomes exposed the moment anyone bgzips and tabixes that GFF3 to share it with VEP. Filed as Huang-lab/germline-plp-carrier-nf#3.

Separately

The anchor_candidates first-match ambiguity from #93 is filed as its own issue rather than bundled here — different subject, and it is narrow.

🤖 Generated with Claude Code

[kuanlinhuang]

Verified before merge

Suite: 931 pass, clippy --all-targets -D warnings clean, rustfmt clean.

Mutation check. With V2 accepted again (writer back to V2, gzip reader restored), 5 of the 6 new tests fail:

a_pre_90_cache_is_rejected_rather_than_read              FAILED
a_legacy_gzip_cache_is_rejected_with_the_same_reason     FAILED
the_published_magic_is_v3                               FAILED
a_pre_90_sidecar_cache_is_rebuilt_rather_than_trusted   FAILED
a_pre_90_explicit_transcript_cache_stops_the_run        FAILED
a_v3_cache_written_now_round_trips                      still passes

The round-trip one can't fail under that mutation - writer and reader move together - so the_published_magic_is_v3 is the test carrying it. Worth knowing which of the six is load-bearing.

The failure this prevents, measured end to end. A poisoned pre-#90 sidecar (a whole-file cache built from a different GFF3, magic flipped to FSTVEP02, mtime fresh against its source) placed next to tests/test.gff3, then the same 8-variant VCF annotated with each binary:

binary	cache	intergenic	missense	exit
master (3a741b8)	trusted, 3 transcripts loaded	8 / 8	0	0
this PR	rejected, rebuilt from GFF3	0	4	0

Master reproduces #87 in miniature: a complete, well-formed, entirely wrong VCF at exit code 0. This branch rejects, rebuilds, republishes FSTVEP03, and the second run loads from cache with byte-identical output.

Also ran it against the genuine article - the FSTVEP02 sidecar sitting in tests/ on this machine, written 2026-06-10, inside the affected window and fresh against a GFF3 from April. Rejected, rebuilt, republished as V3, correct annotation.

Forward compatibility, since a shared reference directory will see both binaries (relevant to the germline-plp-carrier-nf note in the description): an old binary reading a V3 sidecar warns wrong magic header, expected FSTVEP02, rebuilds from the GFF3, gets the right answer, and downgrades the sidecar to V2. The new binary then rejects it and rebuilds again. Alternating versions against one reference directory therefore pays a rebuild every run, but is never wrong in either direction. An explicit --transcript-cache V3 file on an old binary is a hard error, which is also right. Worth a line in the release note alongside the one-rebuild warning.

One thing fixed on the branch: db0fef9

The explicit-cache rejection read as two diagnoses at once, because the format explanation was interpolated into the generic load-failure wording:

Error: Transcript cache old_v2.cache could not be loaded: Transcript cache is in the
FSTVEP02 format, which predates the #90 fix [...] Rebuild with `fastvep cache`, or
delete it and let the sidecar rebuild itself.. It is most likely truncated or corrupt -
delete it and rebuild with `fastvep cache`. Refusing to continue [...]

fastvep cache twice, a .., and a user sent looking for a truncated write or a full disk when the file is intact. A PR whose whole argument is tell the user what is actually wrong shouldn't ship that. StaleCacheFormat makes it a type instead of a string, so the explicit path selects the right sentence via downcast_ref while the sidecar path keeps quoting it verbatim. Now:

Error: Transcript cache old_v2.cache cannot be used: it is in the FSTVEP02 format, which
predates the #90 fix and cannot be trusted. [...] Rebuild it with `fastvep cache`, or
delete it and let the sidecar rebuild itself. Refusing to continue, because annotating
without it would report every variant as intergenic.

and a truncated cache still gets its own diagnosis, unchanged in substance:

Error: Transcript cache trunc.cache cannot be used: it could not be read (Reading cache
header), and is most likely truncated or corrupt - delete it and rebuild with
`fastvep cache`. Refusing to continue [...]

Pinned by an assertion that the pre-#90 error does not also blame corruption, which fails against the old wording.

Not blocking

• No version bump. The on-disk format changed and every crate is still 0.3.0, so two binaries that disagree about which caches are readable report the same version. The test's msg.contains("pre-0.3.1") fallback suggests this was considered. It belongs to the release, not here.
• AnnotationContext::new swallows the rejection (load_cache(..).ok() in crates/fastvep-annotate/src/lib.rs). The outcome is right - it falls through to a whole-file parse_gff3 and re-saves as V3 - but the web path rebuilds a 1.2 GB GFF3 with no log line saying why. A tracing::warn there would pay for itself the first time someone asks why startup got slow.