Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
7314334
Design: RL clean rebuild — two Qwen runs, ported v3-b shaped reward
aadarwal Jul 28, 2026
2bdac8e
Plan: RL clean rebuild implementation tasks
aadarwal Jul 28, 2026
ba1289b
Add rectangle-baseline computation to rl/common
aadarwal Jul 28, 2026
5ff5f3c
Measure boundary chamfer in the evaluator
aadarwal Jul 28, 2026
6fc1bc5
Port the v3-b shaped reward (iou_n) as policy shaped_v3b
aadarwal Jul 28, 2026
e55a84a
Spec: extend to the 2x2 run matrix (user direction)
aadarwal Jul 28, 2026
2532a4d
Default Track A to the shaped reward at G=8
aadarwal Jul 28, 2026
c96a331
Make the env shaped-reward test degenerate-target-robust
aadarwal Jul 28, 2026
028c219
Remove the studies/ceremony apparatus from rl/
aadarwal Jul 28, 2026
bdbb767
Two-run-matrix orchestrator with fixed cross-run probe eval
aadarwal Jul 28, 2026
0e1878b
Prompt contract v3: anisotropy scale note
aadarwal Jul 28, 2026
d7a760d
Document the clean training path
aadarwal Jul 28, 2026
97f81b1
Stages resume instead of refusing an existing log dir
aadarwal Jul 28, 2026
8749d53
Disable the SDK's pyqwest transport locally
aadarwal Jul 28, 2026
8aa0c3b
Chain on training weights; retry evaluator faults under contention
aadarwal Jul 29, 2026
177dba8
Branch campaigns, eval knobs, and the subprocess evaluation tier
aadarwal Jul 29, 2026
94ac281
Restore the Inkling runtime for the bounded L4 RL-only arm
aadarwal Jul 29, 2026
9214d63
Winner harvesting and winner-label SFT plumbing
aadarwal Jul 29, 2026
e10aea5
Harvest: status diagnostics and threshold/tokens flags
aadarwal Jul 29, 2026
8c1cd42
F1-F8 benchmark protocol evaluator (paper Table III: mean and best-of…
aadarwal Jul 29, 2026
812eb23
Iterative agent loop on the benchmark + chamfer asymmetry feedback
aadarwal Jul 29, 2026
9daff11
Publish hygiene: drop internal process docs, neutralize runs-root def…
aadarwal Jul 29, 2026
dca982d
Released-model evidence record + benchmark gallery composer
aadarwal Jul 29, 2026
706032d
Drop stored render gallery: regenerable from the record via make_gallery
aadarwal Jul 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,12 @@ The benchmark input fixtures (silhouette + footprint per device) are
`trajectories/` (cohort-pinned to the audited model set; `--all` widens to
later runs the trajectories also carry).

`training/qwen-rl-v2/` is the validated evidence for the released model
([`qpaig-mit/pixcell`](https://huggingface.co/qpaig-mit/pixcell), a GRPO-trained
LoRA for Qwen3.6-35B-A3B): the F1–F8 benchmark records and the
attempt–measure–revise deployment-loop record with every champion program;
the render gallery regenerates from it (`rl/track_a/make_gallery.py`). Its
README maps each number to its file.
`training/representation-curriculum-v1/` preserves superseded Qwen and Inkling
diagnostic runs. Those runs used the retired v1 direct prompt contract and are
not canonical baselines or paper results. Their protocol and artifacts remain
Expand Down Expand Up @@ -135,10 +141,12 @@ Every number in the paper's Results, mapped to its backing file.

### Sec V — verifier-derived training

The repository contains the training recipes under `rl/` and the derived
paper table and gallery under `arxiv/`. The full benchmark-attempt and
training-run ledgers are reserved for the companion training-artifact
release, as stated in the paper's data-availability paragraph.
The repository contains the training recipes under `rl/`, the derived
paper table and gallery under `arxiv/`, and the validated evidence for the
released model under `training/qwen-rl-v2/` (benchmark JSONs, deployment-loop
record, champion programs). Raw step-level training ledgers and
checkpoints stay outside the repository; the released adapter itself is
[`qpaig-mit/pixcell`](https://huggingface.co/qpaig-mit/pixcell).

## Curation notes (read before reusing)

Expand Down
38 changes: 38 additions & 0 deletions data/training/qwen-rl-v2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Qwen RL v2 — evidence for the released model

The validated record behind the released PixCell model
([`qpaig-mit/pixcell`](https://huggingface.co/qpaig-mit/pixcell)): a LoRA
adapter for `Qwen/Qwen3.6-35B-A3B` trained with GRPO alone (no SFT) on the
released curriculum L0→L4 at the non-thinking 4096-token operating point —
Run B of the campaign recipe in [`rl/`](../../../rl/README.md). Checkpoint of
record: `tinker://d8269c70-0dd9-5776-9cae-53e8b24e647a:train:0/sampler_weights/final`.

Per the repository rule, every number here was recomputed from artifacts by
the deterministic evaluator; nothing model-reported enters the record.

## Files

| Path | Protocol | Headline |
|---|---|---|
| `bench/base-nothink.json` | F1–F8, 8 attempts @ T=1.0, best-of-8 raw IoU | base model: 0/64 attempts executable |
| `bench/b-final.json` | same | released model: 39/64 executable, mean IoU 0.228, best-of-8 0.467 |
| `bench/a-post-l1.json`, `bench/d-post-l0.json` | same | mid-campaign controls (L0-SFT arm after L1; thinking arm after L0) |
| `agentloop/b-final.json` | attempt–measure–revise loop: 8 attempts, then 3 rounds × 4 revisions of the champion under number-free feedback | **mean champion IoU 0.491**; includes every champion program and per-round trajectory |

The agent loop is the deployment protocol — the released model is the
searcher inside that loop, and 0.491 is its number. The single-pass benchmark
is the ablation that shows what a bare forward pass gives up.

## Recompute

```bash
uv run python -m rl.track_a.benchmark_eval --model-path <checkpoint> --out bench.json
uv run python -m rl.track_a.agent_loop --model-path <checkpoint> --out agentloop.json
uv run python -m rl.track_a.make_gallery --champions agentloop.json --out-dir gallery/
```

The render gallery (per device: target | champion render | difference map)
is not stored here: `make_gallery` re-executes each stored champion program
from source and re-renders it against the frozen `data/benchmark` fixtures,
so it regenerates from this record alone, no checkpoint required. The
composite ships on the [model card](https://huggingface.co/qpaig-mit/pixcell).
281 changes: 281 additions & 0 deletions data/training/qwen-rl-v2/agentloop/b-final.json

Large diffs are not rendered by default.

62 changes: 62 additions & 0 deletions data/training/qwen-rl-v2/bench/a-post-l1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
{
"model_name": "Qwen/Qwen3.6-35B-A3B",
"model_path": "tinker://da5c0ccb-55cb-547c-aa0a-b5169801186a:train:0/sampler_weights/final",
"attempts": 8,
"temperature": 1.0,
"per_target": {
"final_1": {
"executable": 2,
"attempts": 8,
"mean_iou": 0.04667048709503287,
"best_iou": 0.30297820032053774
},
"final_2": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_3": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_4": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_5": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_6": {
"executable": 7,
"attempts": 8,
"mean_iou": 0.39151451399245063,
"best_iou": 0.6434092192544019
},
"final_7": {
"executable": 6,
"attempts": 8,
"mean_iou": 0.16893688302640955,
"best_iou": 0.45974476777216944
},
"final_8": {
"executable": 4,
"attempts": 8,
"mean_iou": 0.18510123992188554,
"best_iou": 0.46525473940028256
}
},
"overall": {
"executable": 19,
"total": 64,
"mean_iou": 0.09902789050447232,
"mean_best_of_k": 0.23392336584342394
}
}
62 changes: 62 additions & 0 deletions data/training/qwen-rl-v2/bench/b-final.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
{
"model_name": "Qwen/Qwen3.6-35B-A3B",
"model_path": "tinker://d8269c70-0dd9-5776-9cae-53e8b24e647a:train:0/sampler_weights/final",
"attempts": 8,
"temperature": 1.0,
"per_target": {
"final_1": {
"executable": 7,
"attempts": 8,
"mean_iou": 0.24696953452861198,
"best_iou": 0.43563673082449117
},
"final_2": {
"executable": 8,
"attempts": 8,
"mean_iou": 0.42891641561783983,
"best_iou": 0.5225050774837938
},
"final_3": {
"executable": 4,
"attempts": 8,
"mean_iou": 0.0815497234827478,
"best_iou": 0.21433973354480168
},
"final_4": {
"executable": 5,
"attempts": 8,
"mean_iou": 0.31383181845503727,
"best_iou": 0.6632708820396994
},
"final_5": {
"executable": 3,
"attempts": 8,
"mean_iou": 0.06311243742627864,
"best_iou": 0.24391132501379273
},
"final_6": {
"executable": 3,
"attempts": 8,
"mean_iou": 0.20205123415863877,
"best_iou": 0.6433213909378293
},
"final_7": {
"executable": 2,
"attempts": 8,
"mean_iou": 0.10999817415811014,
"best_iou": 0.48292264524221046
},
"final_8": {
"executable": 7,
"attempts": 8,
"mean_iou": 0.3748545833063247,
"best_iou": 0.5264526994671637
}
},
"overall": {
"executable": 39,
"total": 64,
"mean_iou": 0.22766049014169865,
"mean_best_of_k": 0.4665450605692228
}
}
62 changes: 62 additions & 0 deletions data/training/qwen-rl-v2/bench/base-nothink.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
{
"model_name": "Qwen/Qwen3.6-35B-A3B",
"model_path": null,
"attempts": 8,
"temperature": 1.0,
"per_target": {
"final_1": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_2": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_3": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_4": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_5": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_6": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_7": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_8": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
}
},
"overall": {
"executable": 0,
"total": 64,
"mean_iou": 0.0,
"mean_best_of_k": 0.0
}
}
62 changes: 62 additions & 0 deletions data/training/qwen-rl-v2/bench/d-post-l0.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
{
"model_name": "Qwen/Qwen3.6-35B-A3B",
"model_path": "tinker://b1eeca1b-b1c1-55b8-87c7-fd19a38a3088:train:0/sampler_weights/final",
"attempts": 8,
"temperature": 1.0,
"per_target": {
"final_1": {
"executable": 1,
"attempts": 8,
"mean_iou": 0.01881755829330524,
"best_iou": 0.15054046634644191
},
"final_2": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_3": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_4": {
"executable": 6,
"attempts": 8,
"mean_iou": 0.37353030776040386,
"best_iou": 0.6464024496792284
},
"final_5": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_6": {
"executable": 5,
"attempts": 8,
"mean_iou": 0.4082847723288638,
"best_iou": 0.764634685414781
},
"final_7": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
},
"final_8": {
"executable": 0,
"attempts": 8,
"mean_iou": 0.0,
"best_iou": 0.0
}
},
"overall": {
"executable": 12,
"total": 64,
"mean_iou": 0.10007907979782162,
"mean_best_of_k": 0.19519720018005643
}
}
Loading
Loading