Ralph is live on Bittensor mainnet (netuid 40) — and we renamed from Karpa

[bitzic]

Ralph is now running on Bittensor mainnet, netuid 40. If you've been following this project as Karpa, it's the same protocol with a clearer name — the org is now RalphLabsAI, the protocol repo is RalphLabsAI/ralph, and the site is ralphlabs.ai. Old links redirect.

This post is the plain-language version of what Ralph is, where it stands today, and what's next.

The core idea

Ralph treats open LLM-pretraining research as a protocol instead of a paper.

There is one canonical training recipe — public, versioned, and the thing the whole network works on. A miner's job is to improve it: propose a change (a learning-rate schedule, an initialization, a warmup, a data-mix tweak) as a patch against the recipe. Validators take that patch, train the patched recipe, and score it. When a patch wins, it's merged and the network crowns a new king recipe. Each king is a commit in a public lineage, so the recipe only ever moves forward — and anyone can read the diff that caused each step and the measurement that justified it.

Two design choices keep the score honest:

• A multi-scale ladder. Scoring isn't a single run. Each candidate is evaluated at three model scales (small → larger) plus a held-out private-hard slice of tasks. A patch that only overfits the public eval set doesn't survive the jump in scale or the private slice.
• Attested execution. A reported score is backed by an attestation that the run actually happened as described. The number you see corresponds to real compute, not a self-reported claim.

The result is a continuously improving recipe whose entire history — every change, every measured effect — is public and reproducible.

Where it stands today

• Mainnet is live on netuid 40. The validator sets weights on-chain; the king-change mechanism is the same one we exercised end-to-end on testnet.
• The evaluation ladder and the private-hard eval are implemented and tested, and the full baseline → ladder-eval pipeline has been validated on H100 (real DCLM eval bundle, a trained ~124M-param baseline at NanoGPT-Speedrun scale, and a dry-run of the multi-recipe driver).
• The class/code rebrand is mechanical and checkpoint-safe. Renaming the model classes does not touch the serialized checkpoint format — configs are stored by field name and state-dict keys are module paths, so existing checkpoints load unchanged. Back-compat aliases are kept for the old class names.

The open question we're testing in public

The thing that would invalidate the whole approach: what if a cheap, small-scale gate doesn't actually predict which recipes are better at larger scale? Then the selection signal is noise dressed as rigor.

So before running it, we publicly pre-registered a transfer-credibility test — a frozen analysis script, pinned reference models, and a commit tag anchored to Bitcoin via OpenTimestamps. The decision rule and thresholds are fixed in advance, and the raw results publish regardless of outcome, PASS or FAIL. The full methodology — reference set, statistic, gate, and slip clause — is in Discussion 04, and it's open for adversarial critique.

We'd rather discover cheaply that a small-scale gate doesn't transfer than canonize recipes that don't deserve it.

What's next

• Run the pre-registered transfer-credibility test and publish the raw result either way.
• On PASS: ralph-diffs. The second half of the project is a public knowledge corpus — a HuggingFace dataset where every accepted recipe change ships with its measured effect, openly licensed (ODC-BY with a permissive per-row license allowlist). The recipe is the artifact; the diffs are the knowledge that comes out of the network.
• On FAIL: a public post-mortem and a pre-scoped fallback. The protocol's credibility is downstream of every claim being falsifiable.

Get involved

• Read the recipe and the protocol: RalphLabsAI/ralph and RalphLabsAI/recipe.
• If you do pretraining or eval methodology: the transfer-credibility pre-registration (Discussion 04) is the best place to push back — reference set, choice of statistic, the dual gate, the bootstrap parameters, the slip-clause threshold. We want the critique before the verdict, not after.

Questions about the mechanism, the attestation model, or the eval ladder are welcome in the replies.