🤖 Written by Claude
Symptom
Running an analysis on a VCF (250328BCRABLv2_3_HG38_reanalysis.vardict.vcf.gz) failed with:
VCF '250328BCRABLv2_3_HG38_reanalysis.vardict.vcf.gz' contains variants that have not finished annotation
(in variant annotation version=v91. (06 July 2026) VEP: 116 / RefSeq/GRCh38)
uv.max_variant_id=295967374 > lowest_unannotated_variant=284363487
The block came from AnnotationRun on range-lock 34628 (min_variant=284363487, max_variant=288093038): its standard run is FINISHED but its structural-variant (AnnotSV) run is not.
Root cause
The annotation-completeness check collapses the independent annotation pipelines (VariantAnnotationPipelineType.STANDARD / STRUCTURAL_VARIANT, plus future types) onto a single variant-pk axis:
get_lowest_unannotated_variant_id() (annotation/annotation_versions.py:204) excludes non-FINISHED AnnotationRuns with no pipeline_type filter, so a stuck/slow SV run drags the threshold down.UploadedVCF.max_variant (upload/models/models.py:453) is the single highest variant pk in the VCF, regardless of type.
Net effect: a short-variant-caller VCF (vardict) whose short variants are fully annotated is blocked by an unfinished SV run — even when it contains no SVs at all. The counting query get_variants_qs_for_annotation() already supports a pipeline_type arg; only the lowest-unannotated helper and the per-VCF max are type-blind.
Proposed fix
Track max-variant per pipeline type in a normalized child model (UploadedVCFPipelineMaxVariant, keyed by the same VariantAnnotationPipelineType enum that AnnotationRun uses), and make get_lowest_unannotated_variant_id() pipeline-type aware.
A VCF is fully annotated when every per-type row is below that type's lowest-unannotated variant; the absence of a row means "no variants subject to that pipeline" → that pipeline is ignored. The child-model shape means adding a future pipeline type (e.g. SpliceAI, an alternate long-SV pipeline) is an enum value + a membership predicate, with no schema change, and correctly handles types that overlap rather than partition the variants.
Includes a backfill (via a ManualOperation migration) so the next annotation upgrade evaluates completeness correctly for existing VCFs.
Full plan is committed to the repo: claude/plans/split_uploaded_vcf_max_variant_short_sv_plan.md (permalink in a comment below).
🤖 Written by Claude
Symptom
Running an analysis on a VCF (
250328BCRABLv2_3_HG38_reanalysis.vardict.vcf.gz) failed with:The block came from
AnnotationRunon range-lock 34628 (min_variant=284363487,max_variant=288093038): its standard run is FINISHED but its structural-variant (AnnotSV) run is not.Root cause
The annotation-completeness check collapses the independent annotation pipelines (
VariantAnnotationPipelineType.STANDARD/STRUCTURAL_VARIANT, plus future types) onto a single variant-pk axis:get_lowest_unannotated_variant_id()(annotation/annotation_versions.py:204) excludes non-FINISHEDAnnotationRuns with nopipeline_typefilter, so a stuck/slow SV run drags the threshold down.UploadedVCF.max_variant(upload/models/models.py:453) is the single highest variant pk in the VCF, regardless of type.Net effect: a short-variant-caller VCF (vardict) whose short variants are fully annotated is blocked by an unfinished SV run — even when it contains no SVs at all. The counting query
get_variants_qs_for_annotation()already supports apipeline_typearg; only the lowest-unannotated helper and the per-VCF max are type-blind.Proposed fix
Track max-variant per pipeline type in a normalized child model (
UploadedVCFPipelineMaxVariant, keyed by the sameVariantAnnotationPipelineTypeenum thatAnnotationRunuses), and makeget_lowest_unannotated_variant_id()pipeline-type aware.A VCF is fully annotated when every per-type row is below that type's lowest-unannotated variant; the absence of a row means "no variants subject to that pipeline" → that pipeline is ignored. The child-model shape means adding a future pipeline type (e.g. SpliceAI, an alternate long-SV pipeline) is an enum value + a membership predicate, with no schema change, and correctly handles types that overlap rather than partition the variants.
Includes a backfill (via a
ManualOperationmigration) so the next annotation upgrade evaluates completeness correctly for existing VCFs.Full plan is committed to the repo:
claude/plans/split_uploaded_vcf_max_variant_short_sv_plan.md(permalink in a comment below).