Wire durable NVCA storage capability decisions from nvcf-sc

[balajinvda]

Context

PR #1334 publishes a small public storage catalog, JSON Schema, ConfigMap, strict loader, and tests. It is not wired into runtime selection. Each CSI provisioner entry contains only a provider ID, qualified PVC access modes, and regular/Helm model-cache transitions. Container cache is outside NVCA.

Required design

• Use live StorageClass/nvcf-sc as the sole provider-selection class and require Retain for durable cache storage.
• Find the catalog entry by exact provisioner. Transition code, not the flat mode list, owns writer-to-reader behavior.
• Keep the catalog minimal. Qualification evidence remains external and identifies the exact CSI/backend, StorageClass digest, Kubernetes/node matrix, and test run. A non-disabled transition is an operator assertion that this evidence applies.
• Add a namespaced ModelCacheBinding in the model-cache control namespace, keyed by (workflow, sharingDomain, cacheHandle).
• Keep binding spec immutable: provider, provisioner, transition, required modes, StorageClass/catalog digests, and deterministic shared-resource intent. Keep lifecycle, request references, realized data identity, and conditions in status.
• Persist a request selection with binding name/UID, deterministic reader intent, and a request finalizer before storage side effects. Use the API-assigned binding metadata.uid.
• Add binding references conditionally while Active. Retiring blocks new references. Request deletion removes reader resources and its binding reference before releasing the request finalizer.
• Serialize one active writer with a binding Lease. Use deterministic names and Get-before-Create; adopt only objects whose binding UID and immutable spec match.
• Implement provider-specific, no-copy cross-namespace reader views. Generic CSI cannot share one PVC across namespaces. Each PV must have a distinct CSI volumeHandle resolving to the same qualified backend data identity.
• Set readOnly: true on the PVC source and every model-cache volumeMount. Qualification must verify the actual mount is ro and that write operations fail for an identity proven writable on a baseline mount.
• Migrate legacy empty/coarse backends, NVMesh primary-PV markers, sharedfs writer-PVC markers, Samba backing-PVC markers, the class override, and NVMesh derived encryption classes without changing providers.
• On provider replacement, mark all bindings for the old StorageClass UID Retiring, drain to zero references, finish cleanup, recreate nvcf-sc, qualify, then resume.
• Fail closed for invalid catalogs, non-Retain classes, unknown provisioners/transitions, disabled workflows, and pre-side-effect drift. Never switch providers or delete retained data because of later drift.

Required tests

• catalog missing/malformed/unknown/disabled cases and transition mode requirements;
• same sharing domain and cache key converges on one binding; different workflows or domains do not share;
• optimistic-concurrency races, leader/agent restart, deterministic adoption, and mismatched-object rejection;
• request and binding finalizer ordering, stale-reference removal, active-reader cleanup, and provider retirement;
• exact legacy marker, class-override, and NVMesh encryption migration cases;
• one Lease writer, including competing Pods on one node;
• multi-namespace CPU/GPU readers resolving one provider data identity with checksum equality;
• both manifest read-only fields, observed ro mount flags, and denied create/append/rename/chmod/truncate/delete;
• explicit ROX versus read-only RWX evidence;
• restart, rescheduling, cancellation, failure injection, idempotency, cleanup, upgrade, and rollback;
• no source-sized clone, extra copy, or duplicate CSI handle.

Definition of done

• Runtime uses the public catalog and persists the binding/request state before storage side effects.
• External transitions remain disabled until their transition is implemented and the full workflow suite passes.
• Regular and Helm model-cache workflows are qualified independently.
• The target path selects through nvcf-sc; legacy state and NVMesh encryption are migrated explicitly.
• Operator documentation covers enablement, qualification evidence, and controlled provider replacement.

Performance qualification follows functional acceptance.

[balajinvda]

Foundation PR: #1334 publishes the public catalog, schema, ConfigMap, strict parser, and tests. It is intentionally unwired; this issue remains the runtime integration and cross-namespace design follow-up.