Experimental AI research system for verifier-backed optimization, systems research, and algorithm discovery.
Active Model investigates whether a frontier model can operate as an experimental researcher rather than merely as a coding assistant:
objective + evaluator
↓
reconnaissance → hypotheses → implementation → experiments
↓
measurement → failure analysis → next hypothesis
The emphasis is on externally checkable results.
094_time_decay_exponential_stabilization · SOL score 0.999092
Active Model attacked the benchmark from the public specification, reference implementation and evaluator, and reached a locally measured 0.997900 on this kernel — ≈**#8** on the board at the time — in its first 4 hours, before human research steering began.
Autonomy boundary. Active Model produced every implementation, experiment, measurement and attribution in this campaign. A human research lead set strategy and, after the first measured round, named specific techniques to try. The model implemented and adjudicated 13 such proposals against its own measurements — adopting 3 and refuting 10 with identified mechanisms — and independently found the largest single discrepancy of the campaign: a reproducibility discrepancy inherited from prior work, traced to its underlying mechanism.
The first 4 autonomous hours reached a locally measured 0.997900 — ≈#8 on the board at the time. Model-generated engineering under human research direction reached #1.
This distinction is intentional: I do not describe the complete run as autonomous. Nor was the autonomous phase exhausted — it had already diagnosed that discrepancy on its own and had 4 further candidates queued when steering began. Whether it would have reached #1 unaided is untested.
A later round, three days on, raised the score to 0.998647 and the margin over the previous leader from +0.000071 to +0.000154. Its candidates were generated, compiled and screened entirely offline, so a single 17-minute rental was enough to measure them. That round involved no technique-level human input.
Overtaken, then retaken. Days later another participant posted 0.999005, 358 ppm ahead, and the campaign's own "idea space exhausted" verdict became a verdict about a #2 kernel. A second campaign was opened with one instruction — start from the current kernel, do not copy the first campaign's attempts, bring it back to #1 — with the first campaign's kernel, tooling and dead-end registry as read-only inputs. Working autonomously within that directive, the model developed a substantially different kernel architecture that overcame a limitation the first campaign had treated as fundamental, validated it mathematically against a CPU model before renting any GPU, and incorporated it wherever measurement justified it. Its first official measurement, 0.999015, was already ahead — by 10 ppm, inside evaluator noise. Three written review rounds (~33 technique-level proposals, 5 adopted, the rest refuted on the same rental against byte-identical controls, each experiment's reading recorded before the rental started) took it to 0.999092, +87 ppm over the participant who had overtaken it, reproduced across four official evaluations. Cost: 43 B200 rentals, 12.2 GPU-hours, $45. Reported against interest: one adopted route gained consistently on our rentals but did not transfer to official evaluation, and the third round's highest-confidence proposal was predicted at 10–20 ppm and delivered 0.
- llama.cpp — PR #27478 (open): "ggml : speed up batch-1 CPU decode, align large allocations". Up to +15.29% end-to-end token-generation throughput on Ryzen 7 9700X (attention change alone +10.67%) and +9.22% on Neoverse-N1, measured on Qwen3-30B-A3B Q4_K_M at 8192-token context.
- zstd — three optimization PRs on encode/decode hot paths submitted upstream: #4729 · #4732 · #4733. Includes a decompression-hot-loop optimization removing a loop-carried memory dependency, improving decode throughput by +2.7–3.4% with GCC and +5.5–9.3% with Clang on Zen 5, with the same direction on Intel Raptor Lake.
- dav1d — two AV1 decoder optimizations submitted upstream. !1967 refmvs: collapse runs of identical temporal MVs: 77.5% of scanned temporal cells are bit-identical to their predecessor, so collapsing each run into one weighted call removes 77.5% of
add_temporal_candidate()invocations (39.6M → 8.9M) — +0.5–0.8% decode across x86-64 and AArch64, up to +2.96% on high-redundancy content. !1968 mc: avg_direct for full-pel compound blocks: for full-pel compound blocks the two-buffer scratch round trip reduces exactly to(a + b + 1) >> 1, eliminating it for 37.6–78.3% of compound plane-predictions — +0.9–2.8% depending on how strong the baseline SIMD path is. Both are exact by construction: bit-identical decoded output across 12 streams.
Autonomous research — the human supplies the objective, evaluator, constraints, tools and compute; the model chooses what to investigate and performs the technical search. Example: modded-nanogpt.
Human-steered research — the model still generates, implements and adjudicates the concrete solutions, but a human acts as a research lead: redirecting the search, questioning conclusions, and naming areas or techniques worth investigating. Example: SOL-ExecBench 094 after the autonomous phase.
The repository reports the two separately, because a result obtained with strategic steering demonstrates something different from a fully autonomous one.
- n=32 circle packing — improved by 4.45e-8 over Berthold et al. (Jan 2026), verified with the published verifier.
- n=21 rectangle packing — validated against the published AlphaEvolve result.
- n=26 · n=26 rectangle · n=27 rectangle — further circle-packing results.
- Spherical codes — new best-known S⁵ configurations for N=86 and N=98, registered with spherical-codes.org.
- Pegase AC-OPF — improved primal for the 13,659-bus European grid (386,108.81 → 386,106.54), accepted into MINLPLib.
- Lennard-Jones clusters — canonical minima matched for LJ38, LJ75 and LJ104.
- Santa 2025 tree packing.
See the individual result directories for evidence, verification procedures and caveats.
The common feature of the successful tasks is increasingly clear. It is not geometry, CUDA, or any particular optimization algorithm. It is the existence of:
- a concrete objective;
- a reliable evaluator;
- an experimental environment;
- enough freedom to modify the proposed solution;
- a feedback loop fast enough for repeated hypothesis testing.
Under those conditions, frontier models appear capable of substantially more open-ended technical search than one-shot code generation. How far that extends is the research question behind Active Model.
Claims here should be independently checkable through official benchmark leaderboards, upstream tests and CI, externally maintained verifiers, paired performance measurements with measured noise floors, exact-output checking, or independent benchmark maintainers.
Failed experiments are useful evidence too. The objective is not to make every run look successful, but to understand which research loops work.
This repository is primarily a results and verification archive. The full Active Model harness, operating rules and complete research traces are not currently public.
For actively competitive benchmarks the winning implementation may be withheld temporarily even when the verified result and methodology are public — the current #1 SOL-ExecBench kernel, the techniques it uses and the experiments that refuted the alternatives are not being released while the benchmark remains actively contested. The performance claim does not depend on them: the result is published by NVIDIA's own evaluator and leaderboard.
Selected implementation details can be shared privately for serious technical review or research collaboration.
autonomous research agents · AI for systems · model self-improvement through experimental feedback · GPU and CPU performance engineering · LLM training and inference optimization · algorithm discovery · verifier-guided mathematical and computational search
Matevž Kovačič — matevz.celje@gmail.com