0.3.53: ship one GPU implementation — publish backend-cuda, remove backend-tornado

.gitignore

@@ -50,3 +50,7 @@ docs/content/modules/ROOT/attachments/javadoc/
 
 # Model cache directory
 **/.jvllm/
+
+# Agent worktrees and session scratch. These are embedded git repositories; git add -A
+# swept one into a release commit once.
+.claude/

CHANGELOG.md

@@ -4,6 +4,75 @@ All notable changes to models are documented here.
 
 ## [Unreleased]
 
+## [0.3.53] - 2026-10-07
+
+### Removed
+
+- **`backend-tornado` is removed.** Models ships one GPU implementation, and this was the weaker of
+  two. Measured on the same RTX 4090 and the same model on the same day: `backend-cuda` reached
+  exact token parity (1280 of 1280 token ids identical) and **31.26 tok/s** decode against a
+  **5.06 tok/s** Vector API control, with 344 ms readiness and no device errors. The TornadoVM arm
+  managed **6.49 tok/s** -- barely above the CPU path it exists to accelerate -- over **36 failed
+  `cuLaunchKernel` calls** (`CUDA_ERROR_LAUNCH_OUT_OF_RESOURCES`) that its report did not record,
+  with 32,171 ms readiness. It also could not be self-contained: the Maven artifact cannot carry
+  TornadoVM's device runtime, so an application had to install a matching distribution and launch
+  through its own launcher, which defeats "add a dependency and get acceleration". Evidence:
+  `benchmark-results/2026-10-07-tornado-removal`.
+- Removed with it, because they existed only to serve it: the `accelerator-profile` command of
+  `models-bench`, the TornadoVM benchmark arm in `models-accelerator-bench`, the `tornado-api` and
+  `tornado-runtime` dependencies, and the release precondition in `RELEASING.md` that required a
+  loader and parity gate on each NVIDIA profile under `models-accelerator-bench/results/`. That
+  precondition existed for `backend-tornado`; `backend-cuda`'s gates are
+  `cuda-kernel-gate --mode capability|parity|decode` and need no external runtime.
+- The KV-ridge experiment in `models-accelerator-bench` is unaffected, and the August measurement
+  records that name `backend-tornado` are kept as written -- they are what was measured then.
+
+### Added
+
+- `backend-cuda` is now published. It ships one `sm_80` PTX module serving every device of compute
+  capability 8.0 or above, carried in the jar under `META-INF/models/cuda/` with a SHA-256 the Java
+  loader recomputes. **It is opt-in and nothing activates it by accident**: there is no
+  `META-INF/services` entry, so no `ServiceLoader` discovers it, a consumer has to call
+  `CudaGgufBatchedMatrixKernel.open()` and inject the kernel, and `-Dmodels.cuda.disabled=true` is a
+  kill switch on top of that. Having the jar on the classpath does not route anything to a device.
+- `CudaRoutingCounters.declined(type, stage, reason, rows, cols)`, with the shape in the key, and
+  `declinedProjections` / `totalDeclinedProjections` in the gate report. A projection that answered
+  no to `isEligible` and left through the Java branch previously incremented nothing, so an empty
+  refusals map read as "everything ran on the device" when it only meant "nothing hit an explicit
+  refusal path".
+
+### Fixed
+
+- **The FFN gate and up projections never reached the device.** `CudaGgufBatchedMatrixKernel` did
+  not override `multiplyDual`, so `isDualEligible` inherited the SPI default of `false` and
+  `LlamaForwardPass.dualMatmulDispatch` sent both to `TensorOps.ggufDualMatmul` on every layer of
+  every token. On Granite 4.1 3B they are 8192x2560 each -- 53.3% of a layer's projection
+  arithmetic. The triple path for query, key and value was implemented; the dual path was not, and
+  nothing recorded the asymmetry.
+
+### Measured
+
+Both device gates pass, on an RTX 4090 at compute capability 8.9, Granite 4.1 3B Q4_K_M, 20 prompts:
+
+| gate | result | evidence |
+| --- | --- | --- |
+| G1 token parity | **passed** -- 1280 token ids identical | `benchmark-results/2026-10-07-g1-parity` |
+| G4 decode speed | **passed** -- 6.184x against a 3.00x gate | `benchmark-results/2026-10-07-g4-dualpath` |
+
+G4 moved from 1.788x to 6.184x, decode from 9.02 to 31.26 tok/s, with a control arm that moved
+0.2% and a byte-identical PTX module either side. Routing the two projections raised dispatch --
+launches 241 to 321, transfers 402 to 522 per decode step -- and it did not matter. The earlier
+reading of 1.788x as a dispatch ceiling was wrong; the binding constraint was the unimplemented
+path.
+
+**One host and one model.** G1 has no tolerance, so each qualifying hardware profile and each
+architecture family earns its own run before the claim generalises.
+
+The published surface changes in two ways and no others: `backend-cuda` is added and
+`backend-tornado` is removed. No remaining published module's Java behaviour changed -- the rest of
+`v0.3.52..HEAD` touches only benchmark applications and evidence.
+
+
 ## [0.3.52] - 2026-10-06
 
 ### Fixed

README.md

@@ -34,9 +34,10 @@ tokens. Models implements that pipeline on Java 25 and uses the Vector API for
 CPU SIMD execution:
 
 - `backend-java` executes every inference kernel in Java.
-- `backend-tornado` optionally compiles the Java Q4_0 projection kernels for a
-  qualified NVIDIA GPU. It keeps the Models graph in-process and falls back to
-  the Vector API when the device or artifact is not eligible.
+- `backend-cuda` optionally runs the K-quant projections and grouped-query decode
+  attention on a qualified NVIDIA GPU, through our own Rust kernels compiled to
+  PTX. It keeps the Models graph in-process and falls back to the Vector API when
+  the device or artifact is not eligible.
 - `backend-native` runs the same Java 25 and Vector API pipeline, substituting
   only selected, measured bottleneck kernels with a small Models-owned Rust
   library through Java's Foreign Function and Memory (FFM) API.
@@ -212,9 +213,11 @@ dependencies {
 }
 ```
 
-For qualified NVIDIA acceleration, add `backend-tornado` and launch with a
-matching TornadoVM PTX runtime. The default loader performs eager readiness and
-uses the Vector API when the GPU cannot safely retain the compiled plans. See
+For qualified NVIDIA acceleration, add `backend-cuda`. One `sm_80` PTX module
+ships inside the jar for every device of compute capability 8.0 or above, so no
+external runtime is installed; the path is opt-in through
+`CudaGgufBatchedMatrixKernel.open()` and falls back to the Vector API when the
+device or artifact is not eligible. See
 [Java GPU acceleration](https://integrallis.github.io/models/docs/models/current/gpu-acceleration.html).
 
 Use Apple's on-device system model on a supported Apple Silicon Mac:
@@ -375,7 +378,7 @@ documented in [Execution planning](https://integrallis.github.io/models/docs/mod
 | Model routing | `models-router` | adaptive selection and failover across in-process and hosted clients, with hard per-request capability and data-boundary requirements, without a provider SDK dependency |
 | Vector storage | `models-embedding` | optional bridge to `vectors` |
 | Apple on-device model | `backend-apple` | Apple Foundation Models through Java FFM |
-| Java GPU acceleration | `backend-tornado` | optional Java-authored Q4_0 projections on qualified NVIDIA GPUs |
+| NVIDIA GPU acceleration | `backend-cuda` | optional Rust-authored PTX K-quant projections and decode attention |
 
 These adapters are implemented and tested against the same backend contracts;
 they do not select hidden inference paths. Their framework dependencies are
@@ -409,7 +412,7 @@ RAG, Javadocs, and release testing.
 
 - [Executable Java notebooks](notebooks/README.md)
 - [Apple Foundation Models bridge](models-backend-apple/README.md)
-- [Java GPU acceleration](backend-tornado/README.md)
+- [NVIDIA GPU acceleration](backend-cuda/README.md)
 - [Native kernel backend](backend-native/README.md)
 
 ## Build

RELEASING.md

@@ -5,12 +5,25 @@ JReleaser signs and validates one Maven Central bundle, and the workflow creates
 the GitHub release.
 
 The publication allowlist contains `models-api`, `models-runtime`, `models`,
-`models-rag`, `models-semantic-order`, `backend-java`, `backend-tornado`, `backend-native`,
-`backend-apple`, `models-langchain4j`, `models-spring-ai`,
+`models-rag`, `models-semantic-order`, `backend-java`, `backend-native`,
+`backend-cuda`, `backend-apple`, `models-langchain4j`, `models-spring-ai`,
 `models-spring-boot-starter`, `models-embedding`, `models-audio`, `models-router`, and `models-decisions`. Benchmark
 applications, documentation tooling, and modules containing only package scaffolding are not
 published.
 
+`backend-cuda` publishes an **opt-in** artifact and nothing activates it by accident. There is no
+`META-INF/services` entry, so no `ServiceLoader` discovers it: a consumer has to call
+`CudaGgufBatchedMatrixKernel.open()` and inject the kernel, and `-Dmodels.cuda.disabled=true` is a
+kill switch on top of that. One `sm_80` PTX module serves every device of compute capability 8.0 or
+above, carried in the jar under `META-INF/models/cuda/` with a SHA-256 the Java loader recomputes.
+
+**Its numeric state must be stated in the release notes, not assumed from its presence.** G4 (decode
+speed) passes at 6.184x against a 3.00x gate, measured in
+`benchmark-results/2026-10-07-g4-dualpath`. G1 (exact token parity against the CPU path) is a
+separate gate with no tolerance, and a release note must say which way it last ran and on what
+hardware. Shipping the jar does not mean the device path is numerically verified; it means a caller
+can opt into it and read the gates.
+
 ## Cut a release
 
 1. Set a non-snapshot version in `gradle.properties`.
@@ -23,10 +36,12 @@ published.
 API numeric kernels. The release workflow builds and tests the Models-owned
 Rust kernels on every supported native platform and compiles the Apple
 Foundation Models bridge on macOS before staging the signed Maven artifacts.
-`backend-tornado` is an optional JVM artifact. The hosted release workflow verifies its Java
-fallback and publication shape. Before release, run the public loader and exact CPU/GPU output
-parity gate on each qualified NVIDIA hardware profile and retain the measurements under
-`models-accelerator-bench/results/`.
+Models ships **one** GPU implementation, `backend-cuda`. The TornadoVM arm was removed in 0.3.53:
+measured on the same RTX 4090 and the same model, `backend-cuda` reached exact token parity and
+31.26 tok/s decode where TornadoVM managed 6.49 tok/s over 36 failed `cuLaunchKernel` calls, and it
+required a separately installed runtime that the Maven artifact could not carry. The release
+precondition that pointed at `models-accelerator-bench/results/` went with it; `backend-cuda`'s
+gates are `cuda-kernel-gate --mode capability|parity|decode` and need no external runtime.
 
 The workflow uses the same Maven Central and GPG secrets as `mfcqi-java`:
 `MAVENCENTRAL_USERNAME`, `MAVENCENTRAL_PASSWORD`, `GPG_PUBLIC_KEY`,

backend-cuda/build.gradle.kts

@@ -241,3 +241,32 @@ tasks.withType<Test>().configureEach {
 tasks.named("check") {
     dependsOn(cargoTestHost, cargoClippy, verifyPtxArtifact)
 }
+
+// Coverage: the 0.80 bar published modules carry applies to everything here that a host can
+// execute, and two classes are exempted because they structurally cannot be.
+//
+// CudaDriver is the FFM binding to libcuda.so.1 -- every method is a downcall, so without a driver
+// there is nothing to cover. CudaGgufBatchedMatrixKernel's bulk is the dispatch path behind those
+// downcalls. Together they are 2,296 of the module's 2,491 missed instructions; the rest of the
+// module measures 0.86 covered without them, and the classes a host *can* reach are already well
+// past the bar -- CudaRoutingCounters at 391 of 403 instructions, Q8KActivations at 177 of 193.
+//
+// Their verification is hardware, not more off-device tests, and it is retained as evidence rather
+// than asserted: G1 token parity and G4 decode speed in benchmark-results/2026-10-07-g1-parity and
+// benchmark-results/2026-10-07-g4-dualpath, plus CudaQ6KDeviceParityTest, which launches the Q6_K
+// kernel against the CPU control at six widths and skips off-device. Lowering the global bar to let
+// this module in would have hidden a real gap in every other published module instead.
+tasks.named<JacocoCoverageVerification>("jacocoTestCoverageVerification") {
+    classDirectories.setFrom(
+        files(
+            classDirectories.files.map { directory ->
+                fileTree(directory) {
+                    exclude(
+                        "**/CudaDriver*.class",
+                        "**/CudaGgufBatchedMatrixKernel*.class",
+                    )
+                }
+            },
+        ),
+    )
+}

backend-native/src/main/rust/model-kernels/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "jmodels-kernels"
-version = "0.3.52"
+version = "0.3.53"
 edition = "2024"
 license = "Apache-2.0"
 publish = false