0.8.0 staging: public API re-architecture & streamlining (breaking)

[Fieldnote-Echo]

Staging / tracking issue for the 0.8.0 cycle — a breaking re-architecture to streamline the public API into a cleaner, smaller, more consistent surface. Breaking changes are expected here (they are NOT permitted in 0.6.0).

Living checklist; items are added/triaged as we go.

Context — tiering principle & scope (drafted 2026-06-15)

Principle. Expose generously; the cost lives at only two boundaries — the stable (semver-locked) surface and persisted formats (each format is a permanent magic + loader + fuzz target + metadata-probe + "never break" commitment). So tier rather than cut: a small stable core plus a generous experimental tier (composed by opting into the feature). Adding to the stable surface later is non-breaking; removing is breaking — so keep the stable+persisted boundary stingy and let it accrete only what's profiled.

Scope — most re-tiering is NOT breaking and need not wait for 0.8.0. 0.2.0 (the only published release) shipped just Rank / RankQuant / Bitmap / SignBitmap (+ the deprecated *Index aliases). Everything added since 0.2.0 is unpublished, so re-tiering it is free now — it belongs in the pre-publish / 0.6.0 (#231) window, not the 0.8.0 breaking cycle. This issue (0.8.0) is only for changes that touch the 0.2.0-published surface.

Verified current state (2026-06-15, src/lib.rs):

• Already #[cfg(feature = "experimental")] (no semver cost; only the graduation decision remains): bucket_code (Expose reusable RankQuant bucket-code API for downstream evidence systems #220), const_weight_bitmap (Expose constant-weight bitmap overlap and null-tail helpers #222), contingency (Promote bucket-overlap contingency/projection math: dense (pairwise) + indexed (MultiBucketBitmap) #219), MultiBucketBitmap.
• Stable/ungated but added after 0.2.0 → re-tierable for free now (pre-publish / 0.6.0, not 0.8.0):
  • RankQuantFastscan — currently ungated/stable and unprofiled, plus the .ovfs/OVFS format. Demote to a gated, documented stability level before first publish — tracked in Feature-gate RankQuantFastscan with a documented stability level #84. Keep the OVFS magic reserved (don't reuse the bytes) so the format stays stable if it later graduates.
  • rank_io::{probe_index_metadata, IndexKind, IndexMetadata, IndexParams} — the metadata-probe is incomplete (OVFS probe deferred); don't ship it on the stable surface while incomplete → gate or #[doc(hidden)] until finalized.
  • RankQuantCapability + the b=8 path (Decide b=8 RankQuant evidence surface for OrdGraph/Ordscope use #221 / feat: capability-gated b=8 RankQuant (research/evidence width) (#221) #228) — research-grade width; confirm the stable promise is the capability enum only, not a perf guarantee.
  • TwoStageCandidatePolicy / SubsetScratch / rankquant_eval_search / CandidateBatch — release-hardened caller-owned two-stage primitives; review, likely keep stable.

Genuinely 0.8.0-breaking (touches the 0.2.0-published surface) — the real scope of this issue: the deprecated *Index alias removal, and any reshape of the four core types / the search_* family / SearchResults (all in the checklist below).

Format-governance rule (the expensive boundary): a type may be public/composable long before it earns a persisted format. Default = derive-at-load (no magic); add a new on-disk format only when a type is (a) stable and (b) load-time rebuild or mmap-ability is a real bottleneck. (Per the permanent read constraint below, existing magics — including legacy .tv* — never break.)

Candidate changes (to triage)

• Remove the deprecated pre-0.2 *Index type aliases (RankIndex / RankQuantIndex / BitmapIndex / SignBitmapIndex / …).
• Streamline the search_* family (search / search_asymmetric / *_subset / *_batched_serial / *_into / with_sign_probe) into a more consistent, discoverable API.
• Revisit the flat-buffer SearchResults representation as the stable result shape.
• Decide graduation of the experimental surface (stateless Contingency / Projection / bucket-code → stable; MultiBucketBitmap keep vs remove).
• Finalize the RankQuantFastscan public API + persistence surface (after its promotion lands).

Constraint — the read contract is permanent

The on-disk read contract never breaks: any file the crate has ever written must keep loading (current .ov* + legacy .tv*). Breaking changes in 0.8.0 apply to the API / write surface, not to historical file readability.

[Fieldnote-Echo]

Concrete item for this re-architecture: add OVFS (RankQuantFastscan) to the metadata-probe path (probe_index_metadata). It's deferred here because surfacing it needs a new IndexKind variant and IndexKind is not #[non_exhaustive] (adding a variant is breaking). Done-when: probe_index_metadata returns a Fastscan kind for .ovfs files (and ordvec-manifest recognizes it). A comment marking the deferral landed via the #233 follow-up PR.