diff --git a/arxiv/main.tex b/arxiv/main.tex index e2104089..a996e370 100644 --- a/arxiv/main.tex +++ b/arxiv/main.tex @@ -809,18 +809,20 @@ \subsection{Training with verifier-derived signals} policy optimization (GRPO) steps~\cite{grpo} without supervised demonstrations or critic-generated feedback. Each episode presents one maximum-visibility image, its physical footprint, the permitted DSL catalog, -and the source contract. The model returns one program in a non-thinking -4{,}096-token response. The evaluator applies the source gate, executes -accepted programs, renders their geometry at the target calibration, and -compares it with the reference. Syntax failures, source violations, execution -failures, and missing GDS outputs receive zero reward. Reference or evaluator -failures abort the run rather than entering the record as model errors. - -For a source-compliant executable program, let $J$ and $D$ denote IoU and -Dice, $c$ the symmetric boundary chamfer distance in micrometers, and $d$ the -footprint diagonal. The subscript $\mathrm{rect}$ denotes the result from a -solid rectangle filling the target's calibrated foreground bounding box. -Define +and the source contract. Following this, the model returns one program in a +non-thinking 4{,}096-token response, and the evaluator runs a process of +execution and comparison with the reference wherein syntax failures, source +violations, execution failures, and missing GDS outputs receive zero reward. +Moreover, reference or evaluator failures abort the run. The general motivation +behind such a direction is to have a signal that can cover the representation +space and determine whether small models can learn the representation. To that +extent, there is less focus on the manner in which signals for different types +of violations could be developed, as we focus on gauging learning capabilities. +To that end, for a source-compliant executable program, let $J$ and $D$ denote +IoU and Dice, $c$ the symmetric boundary chamfer distance in micrometers, and +$d$ the footprint diagonal. Here, the subscript $\mathrm{rect}$ denotes the +result from a solid rectangle filling the target's calibrated foreground +bounding box. Define \begin{align*} N(x;x_0) &= \max\left\{0,\frac{x-x_0}{1-x_0}\right\},\\ T(c) &= \exp\left[-\frac{c}{0.05d}\right],\\ @@ -844,13 +846,13 @@ \subsection{Training with verifier-derived signals} \end{equation} \end{subequations} The rectangle baseline removes geometric credit for filling the target -bounding box. The boundary term provides a signal for count and boundary +bounding box, while the boundary term provides a signal for count and boundary changes that may not yet improve overlap. -Training uses the Tinker API~\cite{tinker} with learning rate $10^{-5}$, +Training uses the Tinker API~\cite{tinker} throughout, with learning rate $10^{-5}$, temperature $1.0$, eight task groups per step, and eight rollouts per group. Within-group advantages are optimized with importance sampling and zero KL -penalty. Constant-reward groups are omitted. Run B spends 30 steps on each +penalty, and constant-reward groups are omitted. Run B spends 30 steps on each level from L0 through L4 and initializes every stage from the preceding checkpoint. After L0, each five-step cycle draws exactly 80\% of its task groups from the active level and 20\% from earlier levels. The stage sequence @@ -872,13 +874,12 @@ \subsection{Training with verifier-derived signals} produces 39 executable programs in 64 attempts, with mean IoU $0.228$ and mean best-of-eight IoU $0.467$ (Table~\ref{tab:rl}). The deployment loop uses an independent draw of eight initial attempts per target, retains the -strongest candidate, and gives it three rounds of four revisions. Feedback +strongest candidate, and gives it three rounds of four revisions. Here, feedback reports an error class, a closeness category, and whether material is missing or excessive without exposing numerical measurements. Mean champion IoU rises from $0.422$ after the initial attempts to $0.452$, $0.476$, and $0.491$ after the three revision rounds. This is one trained lineage evaluated -through repeated samples and revisions rather than an estimate of variation -across independent training runs. +through repeated samples and revisions. \input{table_rl.tex} @@ -950,14 +951,21 @@ \section{Conclusion and research contracts}\label{sec:conclusion} training artifact release. \begin{acknowledgments} -We thank the Thinking Machines Lab team for supporting the model-training -experiments with a research grant and access to the Tinker API. We also thank -the Google DeepMind team for graciously extending the Gemini API rate limits -used in the reconstruction experiments. +Artificial Intelligence, particularly Claude Opus 4.8, Fable 5, and ChatGPT +Codex 5.5 and 5.6 Sol, was substantively used in the drafting of this paper, +particularly in building the entirety of the codebase and contributing major +sections of the content. However, none of the ideas and motivations themselves +were proposed or directed by these models. The authors claim full +responsibility for the research and all associated claims and data presented +in this paper, all of which were created or certified by them. This work was supported in part by the NSF National Quantum Virtual Laboratory (NQVL:QSTD) under Award No.~\mbox{2533041} (ORAQL: Open-Stack Rydberg Atom Quantum Computing Laboratory) and by the NSF Center for Quantum Networks under Award No.~\mbox{EEC-1941583}. +We thank the Thinking Machines Lab team for supporting the model-training +experiments with a research grant and access to the Tinker API. We also thank +the Google DeepMind team for graciously extending the Gemini API rate limits +used in the reconstruction experiments. \end{acknowledgments} \bibliographystyle{apsrev4-2} diff --git a/assets/pixcell-paper.pdf b/assets/pixcell-paper.pdf index adf4eab2..c36905d5 100644 Binary files a/assets/pixcell-paper.pdf and b/assets/pixcell-paper.pdf differ