feat(nvca): gate helm model caching behind a default-off feature flag (3.2)

[balajinvda]

Why

Backport of #1332 to the nvca 3.2 release line.

Helm model caching is gated only by CachingSupport, a broad shared gate.
There is no way to enable caching support on a cluster while keeping the Helm
model-cache path off, and no way to turn that path off on its own when it
misbehaves on a cluster.

What changed

Adds a HelmModelCaching feature flag, default off, as a sub-gate of
CachingSupport. storage.SelectHelmCacheBackend returns
HelmCacheBackendNone unless both flags are enabled.

SelectHelmCacheBackend is the single choke point: it is called once per
reconcile in internal/miniservice/reconcile.go, and its result drives both
caching branches. makeStorageRequests only creates a ModelCacheRequest for
the NVMesh, SharedFS, and Samba backends, and the ephemeral model-cache-init
container injection is gated on the Ephemeral backend. Returning None
disables both, so no other call sites needed changes.

Default off means existing clusters see no behavior change until the flag is
added to their feature gate list.

Differences from the main PR

Re-applied by hand rather than cherry-picked. This branch predates two changes
that landed on main, so the merge commit does not apply cleanly:

• SelectHelmCacheBackend on main takes a modelCacheStorageClass argument and
  its Samba fallback also requires the block class to exist. Here the signature
  is unchanged, so the doc comment keeps this branch's wording.
• TestSelectHelmCacheBackend_SambaClassLookupError does not exist on this
  branch, so it needed no flag update.

The gate itself, the flag declaration, and all new tests are identical to
#1332.

Customer Release Notes

Helm model caching is now controlled by a dedicated HelmModelCaching feature
flag, disabled by default, in addition to the existing CachingSupport flag.
Enable both to use the Helm model cache.

Plan Summary

Not applicable.

Usage

Add the flag to the cluster's feature gate list alongside CachingSupport:

CachingSupport,HelmModelCaching

Testing

• go build ./... is clean and go test ./pkg/storage/... ./pkg/featureflag/... ./internal/miniservice/... passes on this branch.
• gofmt -l reports nothing for the five changed files. Two other files on this branch (pkg/nvca/backendk8scache_test.go, internal/miniservice/status_byoo_test.go) are flagged by gofmt but are pre-existing and untouched here.

Tests carried over from #1332:

• TestHelmModelCachingFeatureFlag: asserts the declared default is off and
  reaches Enabled(), covers the +/- parse round trip, and asserts
  CachingSupport alone does not imply the sub-gate. Globals are restored with
  t.Cleanup.
• TestSelectHelmCacheBackend: two cases with nvcf-sc-30 present so the
  assertion is about the gate rather than a missing storage class, one for each
  flag off alone.
• TestMakeStorageRequests_BackendHandling: a HelmCacheBackendNone case with
  a valid cache spec, asserting no ModelCacheRequest is emitted.

QA: not required. Behavior is unchanged for any cluster that does not set the
new flag.

Notes

Per the nvca AGENTS.md, release-x.y branches take bugfixes only. This is a
feat by commit type but is behavior-neutral by default, so it is being
backported deliberately rather than by that convention.

Issues

Relates to #1331

Related Pull Requests

• feat(nvca): gate helm model caching behind a default-off feature flag #1332 (main)

Dependencies

None

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (1)

• main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f155d847-a97a-4cce-a67e-76ba914b7676

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}