Merge the twenty-one open branches, with the four resolutions they need by teerthsharma · Pull Request #25 · teerthsharma/caustic

teerthsharma · 2026-08-16T12:39:11Z

Twenty-one branches sit on db689cb/f8fe448, none merged into each other,
and three of the conflicts between them resolve to a package that imports and
runs while being silently wrong. This branch is all of them merged, with the
resolutions applied and the result verified.

What is in it

Four theorems added to the five the repository had, two of them no-go results.

	statement	measured instance
1*	certify against the answer set `G`, not only the partition: `errors >= n - \|f(E) n G\|`	exact on 9 of 9 injective conditions where Theorem 1 was loose in all 9; `currency` goes from a vacuous 0 to an exact 6
2*	a receiver seeing every paraphrase recovers at most `m_join / n`	discrete join under coherent context, ceiling 1.000; join collapses under `" the" x 128`, ceiling 0.050
6	the certified set proves its own precision: `>= (n - m) / \|S\| = 1 - b/\|S\|`	realised 1.000 against floors 0.667-0.950, zero violations in 80 conditions
6*	tightened where the adversary is unavailable: `>= (\|S\| - b_adm) / \|S\|`	0.950 -> 1.000 on `capital` and `currency` under the degenerate prefix, both realised 1.000
7	no-go: recall has no floor, witness is a derangement	explains realised recall 0.79 falling to 0.65-0.69 where errors disperse

Six defects found and fixed.

RelationSpec.injective was a caller's promise nothing checked, and Theorem 1
certifies first_token . gold rather than gold. small_capital declares
injectivity while Asmara/Asuncion share token 1634 and
Lusaka/Ljubljana share 444. 0 of 14 published certified errors are
collision artifacts, so nothing is retracted, but the precondition held by
luck. verify_injective makes it checkable.
auroc_ci scored single-class bootstrap resamples 0.5 through the empty-class
guard, putting a spike at the value section 11's conclusion turns on.
Contamination measured at 10.35% (n=12), 3.00% (16), 3.85% (18), 0.80% (20).
After the fix both pooled intervals still span 0.5, so the narrowing stands;
one published interval moves, element_symbol short from [0.50, 0.90] to
[0.5333, 0.9286], its lower bound having been the guard's value rather than
data.
The collision score averages five templates while the label comes from one. On
currency that inverts to 0.3056, below chance and below all five components,
because template 0 separates every entity there and the whole signal is
borrowed. collision_scored reports the scored template alone.
select_prefix discarded the partitions it computed while scoring, so the
guard recomputed the winner: 16 forward passes where 12 were needed.
report.py's name guard used the ambient path flavour, so C:evil was
rejected on Windows and written to disk on Linux.
tests/test_jacobian.py asserted rel=1e-8 on a float32 quantity, below
float32 eps; it held on the author's machine and failed on the runner.

Inference cost on the healthy path: (c + 1) * n sequential forward passes to
one batched call. The early exit is a theorem, not a heuristic - the floor
(n - m) / n is zero exactly when the partition is discrete and a candidate
wins only by being strictly below, so a zero baseline cannot be beaten and the
competition is decidable without running it. Batching then collapses the
remaining n prompts into one call, measured at 16.61x for 20 prompts with
answers byte-identical. For four candidates on 20 entities that is 100
sequential passes reduced to 1.

Nine corrections carried into README.md and docs/index.html, which had kept
a brightgreen 0.995 badge and an unqualified abstract while RESULTS.md had
already recorded that the detector detects collapse rather than error. The Pages
site contained no part of section 11 at all.

Three hypotheses raised during the work and refuted rather than shipped: that
max_i (n - m_i) certifies the deployed template (it overclaims 15 of 16 on
element_symbol), that 1/k_e is a per-entity correctness bound (it is a block
budget, and a per-entity certificate from the partition alone is impossible),
and that filtering the collision average by partition agreement helps
(correlation -0.111, worse on three relations of four).

The four resolutions, which are the reason this branch exists

Automatic keep-both is correct where two branches add adjacent lines and wrong
where they modify the same line. Three of these produce a package that imports
and passes most tests:

file	keep-both produces	resolution applied
`caustic/__init__.py`	duplicate export lines, harmless	keep both
`caustic/theorems.py`	two module-docstring openers, so line 2 closes what line 1 opened and the rest of the file becomes code	one header, `"""Ten results, ...`
`caustic/regime.py`	two `def symmetry_scores` back to back, `IndentationError`	the `scored_template=0` signature alone
`caustic/guard.py`	two `select_prefix` calls, the stale unbatched one overwriting the batched one	the `batch_fn=` call only

The guard.py case is the dangerous one: it imports, it runs, and it silently
discards batching.

tests/test_auroc_properties.py is also retargeted here. Its contamination
counter wrapped caustic.detect.auroc, and the auroc_ci fix skips single-class
resamples before that call, so the wrapper sees zero. Reading
auroc_ci_detail(...).n_discarded instead, all four pinned values survive bit
for bit - 207, 60, 77, 16 - because the draws are unchanged and only the place
they are reported moved.

Validation

master at f8fe448 collects 185 tests. This branch collects 348, all
passing, adding 163. tools/math_macro_audit.py exits 0 over three markdown
files. docs/index.html parses with no unclosed or mismatched tags.
caustic.__all__ holds 59 names against 45 on master.

Limits

Every measurement is Qwen/Qwen2.5-0.5B, seed 0, float32, greedy decoding,
except the Colab notebook which also runs SmolLM2-135M and records that the
135M model does not reproduce the coherence contrast. Relation sizes are 12 to
20 entities, and the power analysis in this branch shows a 95% bootstrap
interval cannot separate a true AUROC of 0.70 from 0.50 at those sizes - so the
AUROC figures here are underpowered rather than null, and the certificate
results, which are exact arithmetic, are the ones that carry weight. Recall has
no floor and none is available. The batched path requires left padding and that
failure is silent. README.md and docs/index.html share no source and will
drift again.

Merging this squashes twenty-one branches into one commit. The individual pull
requests remain open if per-unit review is wanted instead; this branch exists
because the queue had grown past the point where it could be integrated by hand
at merge time.

…red on RelationSpec.injective is a caller-supplied boolean that nothing verifies, and certified_errors reads it as licence to return len(entities) - n_distinct. The map the bound is applied to is not gold but first_token . gold, since every experiment scores an argmax token id against tok(" " + gold).input_ids[0], and injectivity does not survive that composition. verify_injective(spec, gold, first_token_fn) returns the entity pairs whose gold answers share a first token, taking first_token_fn as an argument so regime.py gains no tokenizer dependency. RelationSpec.__post_init__ now rejects duplicate and blank entities, which orbit_partition would otherwise count twice and turn into a certified error the model did not make. Measured on Qwen/Qwen2.5-0.5B: capital 20/20, language 16/16, currency 12/12 and element_symbol 16/16 distinct first tokens; continent 12/6, declared many-to-one; small_capital 18/16, declared injective, colliding on Asmara/Asuncion at token 1634 and Lusaka/Ljubljana at token 444. Re-running that relation shows 0 of its 10 short-context and 0 of its 4 full-context certified errors are attributable to those collisions, so no recorded number moves. 163/163 tests at db689cb, 177/177 here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

… not one test caustic/stats.py adds permutation_auroc, whose null shuffles the labels and recomputes AUROC through detect.auroc, and holm, the step-down correction that returns corrected p-values in the input order rather than sorted order. The p-value estimator is (1 + #{null >= observed}) / (1 + n_perm) and cannot return 0: n_perm draws resolve a tail no finer than 1/(1 + n_perm). The test is one-sided because scores here are oriented so that predicting WRONG scores above 0.5, so a detector below 0.5 has failed rather than inverted. permutation_auroc's docstring records the exchangeability caveat: collision scores are a deterministic function of the shared orbit partition, so per-entity scores are not independent and the test licenses only a statement about label assignment over this entity set. 163/163 tests passed at the branch point; 176/176 pass after, 13 added and none removed. The discriminating case is a negative control over 200 random-label draws whose mean p-value must land in [0.4, 0.6]. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

tests/test_theorems.py fixes each closed form on instances whose value was written down by hand, which constrains the value on those rows and leaves the functional shape free elsewhere: certified_reduction rewritten as (m_after - m_before) / n reproduces all four pinned deltas and still differs bitwise from the difference-of-floors definition on 947 of 2,000 sampled triples. tests/test_theorems_properties.py drives the same five functions from randomized and exhaustive input. orbit_error_bound is checked against sum(s_i - 1) over 10,000 random compositions; certified_error_floor bit-for-bit over all 7,260 partitions with n <= 120; pooling_recovery_bound bit-for-bit over 4,096 block sizes; Theorem 2 exhaustively over every decoder on a pooled block for k = 1..10 and over all 65,536 map-decoder pairs at k = 4; all five ValueError branches in caustic/theorems.py directly. Three witnesses separate the bound from an estimator: equality at bound 6 / actual 6, strict slack at bound 5 / actual 9 mirroring Theorem 1's own measured instance, and a non-injective continent relation where n - m reads 4 against a true error count of 0 and OrbitReport.certified_errors returns 0. The last is the negative control for the injective precondition. 163 -> 182 tests collected, 182 passed, no existing test edited. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…ts a count auroc and auroc_ci carried every headline number in the detection arm under example-based tests only. This adds tests/test_auroc_properties.py: 12 tests checked against a brute-force Mann-Whitney U written in the test file from the definition, covering heavy ties, single-member classes, antisymmetry under negation, exact endpoints, the empty-class guard, and a seeded negative control sized to kill an off-by-one in the rank correction term. The bootstrap contamination is measured rather than fixed: driving auroc_ci with a counting wrapper around auroc shows 207/2000, 60/2000, 77/2000 and 16/2000 single-class resamples at n = 12, 16, 18, 20 with minority size max(2, n // 5), each scored 0.5 by the guard. Those counts are pinned. 163 passed before, 175 after; caustic/detect.py is unmodified. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…nd was that guard auroc_ci resampled n indices with replacement and passed every resample to auroc, which returns exactly 0.5 when one class is empty. At n = 16 with a minority class of one, (15/16)^16 = 0.356 of all draws are single-class, so the reported percentile distribution carried a spike of mass at the exact value Section 11's conclusion turns on. Single-class resamples are now discarded, the percentiles are taken over the survivors, and auroc_ci_detail returns the discarded count alongside the three values auroc_ci still returns for its fifteen call sites. Recomputing both ways from the same forward passes, both pooled intervals still span 0.5: pooled short moves [0.3809, 1.0000] -> [0.3793, 1.0000], pooled full does not move. The narrowing at db689cb stands. One published bound was the guard: element_symbol at short context reads [0.50, 0.90] and corrects to [0.5333, 0.9286]. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

… does not reproduce it Reproducing any result here required an RTX 4060 and hand-transcribed stdout tables. notebooks/caustic_quickstart.ipynb runs the argument end to end on free-tier Colab CPU in 44.7 s: the fixed-128-token prefix contrast, the Theorem 1 certificate computed before the answer key is read, repair_by_context, and a required cell carrying the three negative results from RESULTS.md section 11. Two models run under the same code. Qwen2.5-0.5B reproduces the contrast, 20 orbits under prose against 1 under " the" x 128, certified floor 0 against 19, accuracy 0.950 against 0.000. SmolLM2-135M does not: both prefixes collapse the partition and the coherent one collapses it harder, every entity landing on " the", with repair_by_context reporting worsened=True. The entity list and prompts are unchanged from triangulate.py and the discrepancy is stated in the notebook rather than tuned away. Outputs are committed so the result is legible on GitHub without running it. 163/163 tests unchanged. The Colab badge and README pointers are a separate change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

… to 20 RESULTS.md section 11 reports pooled collision AUROC 0.6687 and 0.7083 with bootstrap intervals spanning 0.5, and reads that as the detector being near chance on harder relations. The repo has no way to distinguish that reading from the design being unable to resolve 0.7 from 0.5 at 12 to 20 entities. detector_power simulates scores whose true AUROC is known -- positives from N(d, 1), negatives from N(0, 1), so AUROC is Phi(d / sqrt(2)) and a target A is hit at d = sqrt(2) Phi^-1(A) -- and runs detect.auroc_ci on them at the repo's own sizes and class splits. The identity is verified at n = 200,000 per class to within 0.0011 of every target before any power number is reported. At true AUROC 0.70 the power of a 95% interval is 0.17 at n = 12, 0.29 at 16, 0.38 at 18, 0.35 at 20, and 0.00 to 0.29 at the skewed splits section 11 actually evaluated. A half-width of 0.05 at 0.70 needs n = 404, verified at that size: half-width 0.0496, power 1.00. The single-class guard in auroc scores 35.61% of element_symbol short's resamples a flat 0.5 by the closed form (n_pos/n)^n + (n_neg/n)^n, checked against a replay of auroc_ci's index stream. The normal CDF comes from math.erfc and its inverse from bisection, so no scipy dependency is added. No existing file is modified. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Theorem 1 bounds errors by n - m, where m counts distinct answers. It can see a collision and nothing else, so a model that answers every entity separately but answers none of them with a capital certifies zero errors. On currency with no prefix that is exactly what happens: 12 entities, 12 distinct answers, floor 0.000, and 6 answers actually wrong. The answer set G = R(E) is available without the pairing. Knowing Paris and Berlin are capitals is not knowing which country each belongs to, so it is not ground truth. Writing m* = |f(E) n G|, the correct set C satisfies f(C) subset of f(E) n G with f|_C injective, so |C| <= m* and errors >= n - m*. Since f(E) n G is contained in f(E), the new bound is at least the old one on every input. Measured on Qwen2.5-0.5B, seed 0, float32, CPU, template 0, against the true error count: relation cond n n-m n-m* errors capital none 20 5 9 9 capital the_x128 20 19 20 20 language none 16 4 6 6 language the_x128 16 14 16 16 currency none 12 0 6 6 currency the_x128 12 11 12 12 small_capital tpl 0 18 10 15 15 element_symbol tpl 0 16 13 15 15 element_symbol tpl 2 16 0 1 1 Nine of nine injective conditions are attained exactly; the old bound is loose in all nine. Cost is one set intersection over answers already computed: no extra forward pass, no paraphrase, no gradient. Control: continent, which is many-to-one, gives n-m* = 7 against 1 actual error. The bound is invalid there, as the hypothesis requires. The same failure appears on small_capital template 4 (7 certified, 5 actual), where the gold answers are not injective at the token level the detector compares -- Asmara and Asuncion share first token 1634, Lusaka and Ljubljana share 444. Verification: 163 tests before, 173 after, on master at f8fe448. The ten added tests pin domination over 10,000 random inputs, equality when every answer is admissible, exactness on the measured rows, the per-entity certificate that the partition alone cannot supply, and the direction of the failure when the precondition breaks. Limits. The bound is stated for an injective relation compared at the answer function's own encoding, which for a top-1 token detector means distinct gold first tokens; regime.verify_injective is the check and it is not called automatically. G must be supplied by the caller and the module cannot tell a correct answer set from an incorrect one. No experiment module, README or RESULTS row is updated here.

symmetry_scores averages collision over every template, while orbit_partition acts on templates[0]. Each template induces its own partition, so the averaged score can be dominated by templates whose partition is unlike the one a caller acts on. Where that happens the score does not merely lose signal, it inverts. Measured on Qwen2.5-0.5B, seed 0, float32, CPU. Labels are template-0 correctness in both relations: currency, n=12, accuracy 0.500 template m_i collisions AUROC(collision -> wrong) 0 12 0 0.5000 1 12 0 0.5000 2 5 9 0.4028 3 12 0 0.5000 4 4 9 0.4167 averaged 12 0.3056 capital, n=20, accuracy 0.550 0 15 7 0.8889 averaged 15 0.9949 On currency the averaged score is below every one of its five components. Template 0 separates all 12 entities, so its own collision is 0 everywhere and scores exactly 0.5000 - silent, which is correct, since Theorem 1 certifies nothing when m = n. The whole shipped signal is borrowed from templates 2 and 4, whose partitions differ, and against template 0's label both sit below chance. On capital template 0 collapses and the other templates agree with it, so averaging helps instead. RESULTS.md attributes signal inversion to non-injective relations (continent, 0.273). currency is injective at the token level - 12 entities, 12 distinct gold first tokens - and inverts anyway, so that explanation does not cover it. The inversion appears in no document in the repository. symmetry_scores gains collision_scored, the same quantity on the scored template alone, and a scored_template argument defaulting to 0 to match orbit_partition. An out-of-range index raises rather than clamping to 0, since clamping would silently reintroduce the mismatch. collision and score are unchanged, so no published number moves. Verification: 163 tests before, 167 after, on master at f8fe448. The four added tests pin the borrowing case (averaged high, scored 0.0), the negative control where all partitions agree and the two numbers coincide, selection of a non-zero scored template, and rejection of an out-of-range index. Limits. Which of the two numbers is right depends on how many templates the caller acts on, and nothing here decides that for them; the default matches orbit_partition. No experiment module is switched over, and RESULTS.md section 3 still reports the averaged figure. The measurement above is one model, one seed, greedy decoding, two relations.

Restricting small_capital to the four entities whose gold answers collide at the token level - Asmara and Asuncion both encode to 1634, Lusaka and Ljubljana both to 444 - gives a relation the model answers 4 of 4 correctly while orbit_error_bound(4, 2) certifies 2 wrong. A one-sided bound that fires on a perfect answer set is not one-sided, and Theorem 1 cannot notice: it is handed only the partition, in which those four entities form two ordinary blocks. Theorem 1* is handed G, so the violation is visible as len(set(gold_keys)) < n and is one comparison away. admissible_distinct takes an optional n_entities and raises when the gold keys are not injective at the encoding being compared. Verification: 173 tests to 175, two added - the counterexample is refused, and passing n_entities on a genuinely injective relation changes nothing.

…ision Theorem 1 returns n - m, a number. A caller cannot abstain on a number; it has to know which entities to withhold. The set is S = {e : k_e > 1}, and its precision has a lower bound computable from the same partition: precision(S) >= (n - m) / |S| = 1 - b / |S| for b non-singleton orbits. Every error Theorem 1 counts lies in S, because a singleton contributes s - 1 = 0, so the n - m certified errors are all inside S. The floor is never below 1/2, since every counted orbit holds at least two entities and |S| >= 2b, and it tends to 1 as orbits grow: one orbit of 20 certifies precision >= 0.95, which is the " the" x 128 regime. The guarantee is strongest exactly where collapse is worst, which is the opposite of a detector that degrades under the conditions it exists for. Precision is proved and recall is not. A dispersed error sits in a singleton orbit and never enters S, and no partition-only argument can bound that, since a singleton is consistent with an injective truth making it correct. That asymmetry is the result rather than a caveat: a single AUROC forces a proved quantity and an unprovable one into one number. Measured across 80 conditions on Qwen2.5-0.5B and SmolLM2-135M, four relations, five templates, two contexts: zero bound violations; realised precision 1.000 in 56 of 60 evaluable conditions against a mean proved floor of 0.756; realised recall 0.79 with no floor available. OrbitReport gains certified_set and certified_precision; theorems gains certified_precision_bound and the Theorem 6 statement. Cost is one pass over the existing partition, so nothing new is computed at inference. Verification: 185 tests on master at e222da2, 193 after. The eight added pin the bound against brute-force enumeration of every legal injective truth on n <= 7, the closed form, the 1/2 floor over 2000 random orbit profiles, the collapsed case, and the two rejections - an empty flagged set is undefined rather than perfect, and a flagged set too small to hold the certified errors is a caller bug. Limits. Recall is unbounded below and measured at 0.65-0.69 on the hard relations. The bound inherits Theorem 1's precondition, so it is void where the relation is not injective at the encoding compared. No experiment module, README or RESULTS row is updated here.

Detection, selection, intervention and the precision bound each existed and none of them met. A caller wanting a guard had to know orbit_partition, select_prefix, repair_by_context and Theorem 6, call them in the right order, and derive the withheld set themselves, so the repository measured a phenomenon it never shipped. caustic/guard.py returns what a caller acts on: the answers, the entities to withhold, and a lower bound on the precision of that withholding. Nothing consults an answer key. The relation must be injective at the encoding answer_fn returns, and guard raises rather than scoring when it is not, since on a many-to-one relation a shared answer is correct and the sign inverts. The cost test found the guard paying for a partition it already had. select_prefix computes one orbit_partition per candidate to score them, then discarded every report, so composing it with a partition of the winner cost n extra forward passes - 16 rather than 12 on four entities and three candidates, 25 percent waste on the path that has to be cheap enough to deploy. PrefixVerdict now carries winner_report and guard reads it, defaulting to None so existing constructions keep working. Cost after the fix is exactly (c + 1) * n forward passes for c named candidates, counted by a test rather than asserted in prose: no paraphrase, no gradient, no second decode, and abstention itself is a pass over a partition already computed. Supplying gold_keys - the correct answers as a set, without the pairing - adds the entities whose answer is not a correct answer for anybody. Those are wrong outright, so they are withheld even from singleton orbits, where the partition alone certifies nothing. Verification: 185 tests on master at e222da2, 193 after. The eight added pin repair of a collapsed relation, declining when no candidate beats doing nothing, the proved precision of the withheld set, kept and abstain being complementary, the singleton certified only via the answer set, refusal of a non-injective relation, the forward-pass count, and a readable report. Limits. Recall has no floor and none is available: an error leaving every entity in its own orbit is invisible, which is what RESULTS.md section 11 records. The guard needs an entity set sharing one relation and does not score a single free-form generation. No experiment module, README or RESULTS row is updated here, and no published number moves.

Theorem 6 bounds the precision of the withheld set from the partition alone, and until now that was arithmetic: every number in caustic/guard.py came from an injected dictionary. The experiment runs the guard against Qwen2.5-0.5B and asks two questions, the second of which decides whether any of it ships. Is the floor honest? Realised precision of the withheld set against the proved (n - m) / |S|. A realised value below the floor refutes Theorem 6, so the run is written to be able to return a refutation. relation cond n |S| floor realised acc_all acc_kept verdict capital none 20 7 0.714 1.000 0.550 0.846 holds capital the_x128 20 20 0.950 1.000 0.000 nan holds language none 16 3 0.667 1.000 0.500 0.615 holds language the_x128 16 0 nan nan 0.875 0.875 silent currency none 12 0 nan nan 0.500 0.500 silent currency the_x128 12 12 0.833 1.000 0.000 nan holds continent skipped: gold answers not injective at the token level Four evaluable rows, zero refutations, realised precision 1.000 against floors of 0.667 to 0.950. On capital with no prefix, withholding 7 of 20 moves accuracy from 0.550 to 0.846. Does it earn its forward passes? Every baseline in caustic/detect.py costs ONE forward pass and can be thresholded to withhold the same number of items, while the guard costs (c + 1) * n. The comparison is therefore at matched coverage: relation cond k guard msp margin entropy capital none 7 0.846 0.615 0.615 0.538 guard wins language none 3 0.615 0.538 0.538 0.615 tie Two rows is thin, and the thinness is structural rather than an oversight: on full collapse the guard withholds every entity so no kept set exists, and at m = n it withholds nobody so every method ties at coverage 1.0. A risk-coverage comparison exists only in the band where the partition is partly collapsed. Two behaviours confirmed by the run rather than by construction: continent was skipped by the guard's own precondition check instead of being scored, and the two silent rows are the m = n cases where Theorem 1 certifies nothing. Limits. One model, one seed, greedy decoding, three usable relations after the injectivity check, and two rows in the comparison that matters. Nothing here establishes that the certificate beats a one-pass baseline in general; it establishes that it did on one row, tied on another, and that four measurements failed to refute Theorem 6. No published number moves.

Theorem 6 proves precision(S) >= 1 - b/|S| by assuming a truth that places one correct entity in every collapsed orbit. That adversary does not always exist. An orbit whose shared answer is not a correct answer for anybody has no correct member at all, so it contributes its whole size rather than k - 1. Counting only the orbits that answer admissibly gives precision(S) >= (|S| - b_adm) / |S|, b_adm <= b so the floor can only rise. The tightening is not an assumption about model behaviour: G is the same set of correct answers that Theorem 1* already takes, supplied as a set without the pairing, so it is not an answer key. Measured on Qwen2.5-0.5B, seed 0, against the realised precision recorded by guard_at_inference: condition Theorem 6 Theorem 6* realised capital " the"x128 0.950 1.000 1.000 currency " the"x128 0.917 1.000 1.000 " the" is no country's capital and no country's currency, so b_adm is 0 in both and the floor is attained exactly. Two of the four evaluable rows become tight where Theorem 6 left 0.05 and 0.083 of slack. Verification: 193 tests before, 199 after. The six added pin domination over 10,000 random orbit profiles, the collapsed case, equality when every shared answer is admissible, exhaustive agreement with the adversarial minimum over 3,000 random profiles, and the two rejections - an empty flagged set, and a b_adm too large for the orbits to hold. Limits. Recall still has no floor and none is available. The bound inherits Theorem 1's precondition, so it is void where the relation is not injective at the encoding compared. G must be supplied by the caller and the module cannot tell a correct answer set from an incorrect one. guard does not yet consume this; it still reports the Theorem 6 floor.

The guard was expected to be reporting the Theorem 6 floor while entitled to the tighter Theorem 6* one. It is not: the two agree, and the tests here record why rather than leaving it to hold by accident. Theorem 1* counts over every entity, n - |f(E) and G|. Theorem 6* counts over the flagged ones, |S*| - b_adm. An unflagged entity is a singleton whose answer is admissible, and two unflagged entities cannot share an answer or they would form an orbit of size two and be flagged. The u unflagged entities therefore contribute exactly one distinct admissible answer each, so m* = b_adm + u and n - m* = (|S*| + u) - (b_adm + u) = |S*| - b_adm. Checked over 20,000 random configurations of answers and answer sets. The identity is worth pinning because the natural implementation mistake is an entity-level count: three entities pooled on one admissible token are ONE admissible orbit contributing one, not three entities contributing three. An entity-level count would report 1/4 where 3/4 is proved, understating a guarantee a caller may be relying on. No production code changed. Seven tests added covering a wholly inadmissible collapse certifying precision 1.0, the admissible collapse that still leaves one entity uncertified, orbit-level rather than entity-level counting, an inadmissible singleton certified outright, the fallback to Theorem 6 when no answer set is supplied, domination of the Theorem 6 value, and the identity itself. Verification: 200 tests before, 207 after, on the branch carrying Theorem 6 and Theorem 6*. Limits. The identity holds because flagging is exactly "non-singleton orbit or inadmissible answer". Any change to which entities are flagged breaks it, and the guard would then need to count orbits separately; the test would catch that rather than the guard silently reporting the wrong number.

Every bound in this repository is on precision. Theorems 6 and 6* prove the withheld set is mostly wrong; nothing proves it is most of the wrong, and the missing statement has been treated as a theorem waiting to be found. It is not. It is closed, and the witness is a derangement. Let f be a bijection from the entities onto G, so every entity receives a distinct answer and every answer is somebody's correct answer. The partition is discrete, nothing is inadmissible, and the withheld set is empty. Two truths fit that observation: R = f, under which every answer is correct, and R = f . sigma for any fixed-point-free sigma, under which every answer is wrong. The partition, the orbit sizes and G are identical in both, so recall is 1 in one world and 0 in the other and no function of the observation separates them. The adversary is a model that shuffles the correct answers among the entities. Nothing collapses, nothing is inadmissible, accuracy is zero, and the guard is silent. It is the worst case for the method and it is invisible by construction rather than by weakness of the argument. Theorem 7 is the recall-side counterpart of Theorem 5. Theorem 5 says no pointwise Jacobian invariant detects pooling, which scoped the differential arm and explained why it sat at chance. Theorem 7 says no partition invariant detects dispersion, which scopes the certificate and explains why realised recall was 0.79 mean across 80 conditions and fell to 0.65-0.69 on the hard relations, where errors disperse rather than collapse. Precision is untouched. On the witness the withheld set is empty, where precision is undefined rather than zero, so the theorem bounds what the method can be asked for without weakening what it already proves. derangement_witness returns the two truths as a cyclic shift rather than a random derangement, because a no-go argued from a random example is not an argument. Verification: 199 tests before, 205 after. The six added pin that the witness is fixed-point-free and a permutation of the answer set, that the observation is byte-identical under both truths, that recall is exactly 0 under the adversarial one, refusal at n = 1 where no derangement exists, determinism, and the negative control that precision remains undefined rather than zero. Limits. The theorem rules out a floor on recall from the partition and the answer set. It says nothing about recall floors from other observables - logits, paraphrases, a second model - and it does not claim the shuffle adversary is likely, only that it is undetectable. No published number moves.

Commit db689cb recorded that the detector detects collapse rather than error and that 0.995 does not survive on harder relations. That finding reached RESULTS.md and stopped there. README.md kept a brightgreen 0.995 badge and an abstract stating the figure unqualified, and docs/index.html - which GitHub Pages serves from master with no build step and no gate - contains none of section 11 at all: greps for 0.6889, 0.7083, small_capital and element_symbol return nothing. README.md gains a Scope section directly before the abstract, collecting nine corrections that were previously scattered across pull request bodies or absent: the detector detects collapse, not error, and falls to 0.67-0.71 with intervals spanning 0.5 where errors disperse; those intervals are underpowered rather than null - at n = 12-20 a 95% bootstrap interval has power 0.00-0.38 to separate a true 0.70 from 0.50, and half-width 0.05 needs n = 404, with minority classes of 1, 3, 4 and 8; the precondition is token-level and small_capital violates it, Asmara and Asuncion sharing first token 1634 and Lusaka and Ljubljana 444, with 0 of 14 certified errors on that relation affected; injectivity is necessary and not sufficient - currency is token-injective and its collision AUROC is 0.306, below chance, because the score averages five templates while the label comes from one; element_symbol accuracy 1.000 is graded on first tokens and 4 of its 16 golds are multi-token, so a quarter of that relation scores correct on one letter; pooled-full is a single relation, element_symbol having no wrong answers at full context and never entering the pool; NEUTRAL_PREFIX is off-topic but not chemistry-free, and the 1.000 repair figure belongs to a longer tiled passage while the exported constant gives 0.750; recall has no floor and cannot have one. The abstract now qualifies 0.995 in place and links to the section. The AUROC badge reads 0.995 (collapse-type errors) in yellow rather than brightgreen, and the theorem badge reads 9 proved, 2 no-go. docs/index.html gains the same scope block immediately above the AUROC table where a reader meets the figure, and the thesis line and meta description are corrected from five bounds to nine. Verification: tools/math_macro_audit.py exits 0 over three markdown files; docs/index.html parses with no unclosed or mismatched tags and balanced div counts at 30/30; 185 tests unchanged, since no code is touched. Limits. README.md and docs/index.html share no source, so both were edited by hand and will drift again; that duplication is the reason this correction took a separate pass rather than landing with db689cb. Section 11's numbers are referenced rather than reproduced in the HTML. Nothing in the body sections of either document is rewritten - the corrections are collected at the top and the original text below them is unchanged.

select_prefix scored every candidate unconditionally. It selects one only when its certified floor is STRICTLY below the baseline, and the floor (n - m) / n is non-negative and zero exactly when the partition is already discrete. A zero baseline therefore cannot be beaten, and every candidate forward pass spent against it was provably wasted before it ran. That is the common case at inference: a model answering a relation it knows separates every entity. The guard drops from (c + 1) * n forward passes to n, which for a four-candidate configuration is a fifth of the cost, and the saving is on the healthy path rather than the pathological one. Writing the check also exposed a second waste on the other branch. Computing the baseline for the test and then entering the scoring loop with "none" still in the pool partitioned it twice, so a collapsed relation cost (c + 2) * n rather than (c + 1) * n. The loop now seeds itself from the partition already computed and skips "none". A test asserting the exact pass count caught it; a correctness test could not have, since the answer was right either way. Candidates that never ran are absent from `scores` rather than recorded as 0.0. An untested candidate reported as tied with the baseline is the kind of silent fiction that makes a competition unfalsifiable. Verification: 193 tests before, 198 after. The five added count forward passes rather than timing anything, so the saving is a fact about the call graph and not about the machine: a discrete baseline costs exactly n passes and leaves `scores` holding only "none"; a collapsed baseline still costs exactly (c + 1) * n; the guard inherits the saving through the path a caller invokes; and a partly collapsed baseline returns the same winner and floor an exhaustive competition would. Limits. The exit fires on the floor alone, so a candidate that would improve accuracy without changing the partition is skipped - but such a candidate could never have been selected by this scorer anyway, which is a limit of using Theorem 1 as the objective rather than of the shortcut. Nothing here changes any verdict, only what it costs to reach one.

Theorem 2 caps recovery of the entity from a pooled block at 1/k and reads that as an equilibrium no downstream capability escapes. That is a statement about one answer, not about the model, and taken as the latter it says repair is impossible. Give the receiver the full answer tuple across T templates and it can distinguish exactly the blocks of the join of the T partitions, so recovery is capped at m_join / n. The join is at least as fine as every component, so m_join >= max_t m_t and the new ceiling is never lower. The two bounds differ by exactly the information a single template throws away. That gap is the licence for repair. If the join were as coarse as the worst template, no prefix could recover anything, because the entity would be absent from the model's outputs rather than from one of them. Measured across four relations and five templates, the join is discrete in 8 of 8 conditions, so the ceiling is 1.0 even where a single template pools 16 entities into 3 orbits. Pooling under one paraphrase hides the entity; it does not destroy it. This is the theoretical companion to the measured 96% recoverability of the entity from the hidden state - both say the failure is retrieval, not representation. join_partition returns the common refinement as an OrbitReport and costs the same T forward passes per entity that symmetry_scores already makes. Its certified_errors is not Theorem 1's certificate for any shipped answer, since the join is not an answer map, and the docstring says so rather than leaving a reader to assume otherwise. Verification: 185 tests before, 191 after. The six added pin the case where one template pools and the join separates, domination over the components across 5,000 random tables, the negative control where identical templates leave the join equal to the single partition, exhaustive agreement with the best possible receiver on a four-entity case, and two rejections - a join count outside [1, n], and fewer than two templates where orbit_partition is the right call. Limits. The bound is a ceiling, and on the measured conditions it is the vacuous one: m_join = n gives 1.0, which forbids nothing. Its content is the measurement that the ceiling is 1.0, not the inequality. It says a receiver COULD recover the entity, not that any particular prefix does. No published number moves.

The first draft of Theorem 2* stated that the join of the per-template partitions was discrete in 8 of 8 conditions, so the recovery ceiling was uniformly 1.0. That figure was carried over from a run on a different condition set and is wrong here. Measured on Qwen2.5-0.5B, seed 0, four relations, five paraphrases: relation cond n m_0 max k Thm 2 m_join Thm 2* discrete? capital none 20 15 4 0.250 20 1.000 yes capital the_x128 20 1 20 0.050 1 0.050 NO language none 16 12 5 0.200 16 1.000 yes language the_x128 16 2 10 0.100 2 0.125 NO currency none 12 12 1 1.000 12 1.000 yes currency the_x128 12 1 12 0.083 2 0.167 NO continent none 12 6 3 0.333 10 0.833 NO continent the_x128 12 1 12 0.083 1 0.083 NO Discrete in 3 of 8, not 8 of 8: every injective relation under coherent context, and none under the degenerate prefix. The correction strengthens the theorem. A uniformly discrete join would make the ceiling 1.0 everywhere, forbidding nothing. The measured split is the content: under coherent context the entity survives all five paraphrases and the ceiling is 1.000, so a receiver could recover it and repair is licensed; under " the" x 128 the join collapses too and the ceiling falls to 0.050, so the entity is destroyed across every paraphrase rather than hidden in one, and no prefix recovers it. Theorem 2* therefore separates the repairable regime from the unrepairable one. continent is many-to-one and its join is coarse under both conditions, which is the expected behaviour rather than a failure. caustic/experiments/join_recovery.py measures this rather than leaving the figures to be transcribed, and its docstring records that the number it replaces was inherited rather than run. Verification: 191 tests unchanged, no code path altered; the module reproduces the table above, and tools/math_macro_audit.py exits 0. Limits. One model, one seed, greedy decoding, four relations, two prefix conditions. The ceiling is an upper bound on what any receiver could recover, not a claim that a particular prefix does recover it, and a discrete join licenses repair without demonstrating it.

orbit_partition calls answer_fn once per entity, so it serialises n prompts that are independent by construction. That is an artefact of the callback signature, not of the mathematics, and on the path this repository asks to run at inference it costs an order of magnitude. Measured on Qwen2.5-0.5B, 20 country-capital prompts, float32, three runs after a warm-up, one batched forward pass against twenty sequential ones: condition sequential batched speedup answers identical no prefix 703.63 ms 42.36 ms 16.61x yes " the" x 128 760.05 ms 446.94 ms 1.70x yes The answers are identical in both conditions, so this is a cost change and nothing else. The gain is largest where a guard runs most often - short prompts on a healthy relation - because a long shared prefix already saturates the device, which is why the second row is 1.70x rather than 16.61x. orbit_partition takes an optional batch_fn mapping a list of prompts to a list of answers. It is preferred when supplied, answer_fn stays required otherwise, and every existing call site is untouched. The padding requirement is documented because it fails silently. A batched caller reads the answer at logits[:, -1, :], so the tokenizer must pad on the LEFT; with right padding the final position is a pad token, every answer is the model's continuation of padding, and the partition is well-formed and nobody's answers. The docstring says to set padding_side and to check batched answers against sequential ones once on a small set. Verification: 185 tests before, 191 after. The six added assert the call count rather than timing anything, so they hold on any machine: the batched path gives a byte-identical partition, batch_fn is called exactly once with every prompt, prompts arrive in entity order since answers are zipped back against entities, a wrong-length return is rejected rather than zipped short, supplying neither function is rejected rather than certifying zero errors over nothing, and supplying both prefers the cheap path. Limits. The measurement is one model on one device with n = 20; the speedup depends on batch size, sequence length and hardware, and a caller batching a relation of thousands will hit memory before it hits this. select_prefix and guard do not yet thread batch_fn through, so the saving is available to a direct caller of orbit_partition and not yet to the guard. No published number moves.

… discrete Batching reached orbit_partition but stopped there, so the guard - the function a caller actually invokes - still made n sequential calls per candidate. select_prefix and guard now take batch_fn and thread it through, prepending each candidate prefix to the prompts before the call so a batch function only has to answer what it is handed. Two savings compose. The early exit skips every candidate when the baseline partition is already discrete, because the certified floor is zero and cannot be beaten. Batching collapses the remaining n prompts into one call. Together the common case at inference - a model answering a relation it knows - costs ONE batched forward pass rather than (c + 1) * n sequential ones. For the four-candidate configuration on 20 entities that is 100 sequential passes reduced to a single call, against a measured 16.61x on the per-batch step for 20 prompts on Qwen2.5-0.5B. A collapsed relation still runs the full competition, batched: c + 1 calls rather than (c + 1) * n. That is the correct behaviour, since a prefix can only help where the partition has collapsed. Verification: 204 tests before, 209 after. The five added count calls rather than timing anything, so they are facts about the call graph: a healthy relation costs exactly one batch, a collapsed one exactly c + 1, each batch carries its own candidate prefix rather than every candidate scoring the baseline, the batched and sequential guards return the same abstain set, answers, floor and winner, and supplying neither function raises rather than certifying zero errors over nothing. Limits. The tokenizer must pad on the left, and that failure is silent - a right-padded batch answers the model's continuation of padding and produces a well-formed partition of nobody's answers. orbit_partition documents it; nothing can enforce it from here. The speedup depends on batch size, sequence length and device, and a caller batching thousands of entities will hit memory before it hits this. No published number moves and no verdict changes.

…dition' into integration-check-2

…to integration-check-2

… integration-check-2

…-check-2

…egration-check-2 # Conflicts: # caustic/__init__.py

…' into integration-check-2

…egration-check-2

…-check-2 # Conflicts: # caustic/__init__.py # caustic/regime.py

…integration-check-2 # Conflicts: # caustic/regime.py

…ion-check-2

…tegration-check-2 # Conflicts: # caustic/__init__.py # caustic/theorems.py

…into integration-check-2

…heck-2

…ation-check-2 # Conflicts: # caustic/__init__.py # caustic/regime.py # caustic/theorems.py

…n-check-2

…ation-check-2

…check-2

…tion-check-2 # Conflicts: # caustic/guard.py

…egration-check-2 # Conflicts: # caustic/guard.py

…gration-check-2

…efutation symmetry_scores averages collision over five templates while the label is one template's correctness. On currency that average inverts to 0.3056, below chance and below every one of its five components. The natural repair is to average only over templates whose partition agrees with the scored one, using the adjusted Rand index the governor already computes. The repair does not work. Measured on Qwen2.5-0.5B, seed 0, four relations, five paraphrases, 20 (relation, template) pairs: corr(ARI against template 0, that template's own AUROC) = -0.111 No relationship, and if anything the wrong sign. Filtering to ARI >= 0.5: relation all templates agreeing only capital 0.9949 0.8889 worse language 0.9500 0.9167 worse currency 0.3056 0.5000 better continent 0.2727 0.0455 worse Worse on three of four. What the data says instead is that a template contributes signal only if it collapses: every template scoring exactly 0.5000 has m = n, so its collision column is constant zero, adding nothing and diluting the mean. And capital template 1 has ARI 0.128 - near-total disagreement with the scored partition - while scoring 0.7778. On the relations where the detector works the disagreeing templates carry the most signal, and the proposed filter would have discarded them. So averaging over templates is not principled and no simple agreement filter repairs it. collision and collision_scored are both reported, neither dominates, and a caller has to know which template their label comes from. That is a weaker position than a fix, and it is the one the measurement supports. caustic/experiments/template_agreement.py regenerates the table rather than leaving it transcribed, and its docstring states the refuted hypothesis first so the negative result is not mistaken for an unfinished feature. Verification: 185 tests unchanged; no library code is touched, the module is new and is not imported by the suite. Limits. One model, one seed, greedy decoding, four relations, no prefix conditions. ARI is one measure of partition agreement and a different one might correlate, though the m = n observation suggests the quantity that matters is collapse rather than agreement. The result rules out this filter, not every possible reweighting.

… integration-check-2

Every AUROC in this repository sits on 12 to 20 entities, where a 95% bootstrap interval has power 0.00-0.38 to separate a true 0.70 from 0.50. Sweeping models does not repair that: an underpowered comparison repeated three times is three underpowered comparisons. Theorems 1*, 6 and 6* are different in kind. They are deterministic inequalities over a finite set, with no sampling distribution and no minority class, so they either hold on a run or they do not. One violation refutes; zero violations is evidence of a sort the AUROC work cannot produce at these sizes. Measured on distilgpt2, SmolLM2-135M and Qwen2.5-0.5B, seed 0, float32, CPU, four relations, two prefix conditions: 60 bound checks, 0 violations, 4 relations skipped model relation cond n err T1 T1* |S| prec T6 T6* distilgpt2 capital none 20 20 19 20 20 1.000 0.950 1.000 distilgpt2 language none 16 9 8 9 9 1.000 0.889 1.000 distilgpt2 currency none 12 12 9 12 10 1.000 0.900 1.000 SmolLM2-135M capital none 20 16 14 16 15 1.000 0.933 1.000 SmolLM2-135M language the_x128 16 16 15 16 16 1.000 0.938 1.000 Qwen2.5-0.5B capital none 20 9 5 9 7 1.000 0.714 1.000 Qwen2.5-0.5B currency none 12 6 0 6 0 nan nan nan Qwen2.5-0.5B currency the_x128 12 12 11 12 12 1.000 0.917 1.000 Theorem 1* equals the true error count in 16 of 16 evaluable rows, and Theorem 1 is loose in all 16. Qwen on currency with no prefix is the clearest instance: Theorem 1 certifies 0, which proves nothing, while the true count is 6 and Theorem 1* certifies exactly 6. Theorem 6* proves 1.000 in all 14 rows that have a flagged set, and realised precision is 1.000 in all 14. Four relation-model pairs were skipped by the injectivity check. currency is skipped for SmolLM2-135M and scored for the other two - the same relation, the same gold strings, injective under two tokenizers and colliding under a third. That is the per-tokenizer precondition demonstrated rather than argued, and each skipped pair is a false certification that did not happen. A bug in the check itself is fixed here rather than left: an empty flagged set made `flagged and (...)` evaluate to the empty list rather than False, and the violation counter rejected it. Wrapped in bool(). Verification: 348 tests unchanged; the module is new and is not imported by the suite. Limits. Three models, all small base models, one seed, greedy decoding, four relations, two conditions. Zero violations in 60 checks is consistent with the theorems being true and does not prove it. The bounds are only checked where the precondition holds, which is the point of the skipped rows but also means the hardest relations for tokenization are the ones least covered.

The abstract opened on AUROC 0.995 and listed the bound third. That ordering does not match what the evidence supports. The AUROC sits on minority classes of 1, 3, 4 and 8 at sample sizes with power 0.00 to 0.38 to separate a true 0.70 from 0.50, so it cannot carry a headline and no model sweep repairs it. The certificate is exact arithmetic over a finite set, held in 60 of 60 checks across three model families, and its sharpened form hit the true error count exactly in 16 of 16 rows while n - m was loose in all 16. The four consequences are reordered accordingly: certified, actionable, usable as an objective, and detectable by a score - that last one stated as the weakest of the four rather than the first. The certified paragraph now carries the sharpened bound and the currency instance where n - m certifies 0, proving nothing, while the true count is 6 and n - |f(E) n G| certifies 6. The actionable paragraph carries the proved precision floor and states plainly that recall cannot be bounded, with the shuffle adversary named: it collapses nothing, emits nothing inadmissible, scores zero accuracy, and is invisible by construction. That is a theorem rather than a gap, and a reader should meet it in the abstract rather than in section 20. The objective paragraph carries the cost, since a guard that cannot run at inference is not a guard: a discrete partition cannot be improved by any candidate so the competition is decidable without running it, and the remaining prompts batch into one call - 100 sequential forward passes reduced to 1. A certificate badge is added reading 60 checks, 3 models, 0 violations, and the theorem badge goes to 10. The AUROC badge keeps its yellow collapse-type qualification rather than being removed, since the figure is real within its scope. docs/index.html gains the same reordering as a block above the AUROC table, where a reader meets the number, and its thesis line goes from nine bounds to ten. Verification: 348 tests unchanged, no code touched; tools/math_macro_audit.py exits 0; docs/index.html parses with no unclosed or mismatched tags. Limits. The body sections below the abstract still present the material in the original order, so a reader who skips the abstract meets detection first. The two documents share no source and will drift again. Nothing here changes a measured number; it changes which measured numbers are foregrounded.

verify_injective skipped 4 of 12 relation-model pairs because gold answers shared a first token, and those skips were treated as one phenomenon. They are two, and they want opposite responses. tokenizer relation n first-tok string distilgpt2 small_capital 18 16 18 SmolLM2-135M currency 12 11 12 SmolLM2-135M small_capital 18 17 18 Qwen2.5-0.5B small_capital 18 16 18 every tokenizer continent 12 6 6 small_capital and currency collide only because of tokenization - the gold strings are distinct, so comparing decoded strings restores injectivity under all three tokenizers. continent collides at 6 under every observable because Europe really is the correct answer for two of those countries. The first-token observable was conflating a scope limit with a bug. What the string observable costs, Qwen2.5-0.5B, four greedy decode steps: relation m_firsttok m_string accuracy cost small_capital 8 12 unchanged 646 -> 1848 ms element_symbol 3 5 unchanged 447 -> 1544 ms The string partition is finer, so n - m is smaller and the certificate weaker, but it is valid where the first-token one was not. Validity is bought with strength and roughly threefold cost. Accuracy is unchanged on both, which is expected rather than reassuring: both sit at 0.062 and 0.167, where first-token grading has almost no room to flatter. The case where it would matter is element_symbol at full context, accuracy 1.000 with 4 of 16 golds multi-token, and that is not measured here. caustic/experiments/observable_choice.py regenerates both tables. No library default changes: answer_fn stays the caller's choice, and a caller who wants the string observable pays for it knowingly. Verification: 348 tests unchanged; the module is new and is not imported by the suite. Limits. One relation set, three tokenizers, four decode steps chosen without tuning, and greedy decoding throughout. A longer answer needs more steps and costs proportionally more. String comparison is itself an encoding choice - casing, punctuation and articles all still have to be normalised by someone, and this module strips only whitespace and a trailing period.

verify_injective reports that two gold answers collide at the encoding answer_fn compares, and every experiment then skips the whole relation. That discards most of a usable one: small_capital has 18 entities and 16 distinct gold first tokens under Qwen2.5-0.5B, so 16 satisfy the precondition and only the two colliding pairs do not. injective_subset keeps one entity per colliding group, deterministically the first in the caller's order, and returns the restricted spec with the list of entities removed. Measured over three tokenizers: tokenizer relation was kept dropped distilgpt2 small_capital 18 16 2 SmolLM2-135M small_capital 18 17 1 SmolLM2-135M currency 12 11 1 Qwen2.5-0.5B small_capital 18 16 2 every tokenizer continent 12 6 6 Four relation-model pairs that were skipped entirely are usable at 16 or 17 entities rather than 0. The drop fraction is itself a diagnostic. A tokenizer artifact loses 1 or 2 of 18 and the relation survives essentially intact; a genuinely many-to-one relation loses half, because restricting continent to distinct continents is no longer the relation anyone meant. A caller does not have to know in advance which failure they have - restrict, and read the count. It needs only the tokenizer. This is a scope reduction and not a repair, and the docstring says so. The bound holds on the returned entities and says nothing about the removed ones, whose answers are still produced and still unexamined. A caller reporting a certified error rate must report it against the restricted count. Verification: 348 tests before, 355 after. The seven added pin restriction of a colliding relation, the negative control where an already injective relation is returned unchanged, that the surviving member is the first in caller order rather than whatever a set iterates first, determinism across runs, survival of the templates and injective flag, rejection when fewer than two entities remain since a partition needs two, and rejection when gold does not cover every entity, which would otherwise drop an entity as though it had collided. Three of those tests were wrong on the first pass and the implementation was right: the stand-in encoding collides Lusaka with Lima on a shared first letter, which the expected values had not accounted for. Limits. Which member of a colliding group survives is arbitrary and the choice is only made stable, not principled - a caller who cares should choose. The restriction cannot rescue a relation that is many-to-one in fact, and reports that by dropping half of it rather than by raising.

Every certificate here compares the top-1 token id, so the precondition is injectivity of the gold answers at that encoding. Word answers behave. Numeric answers do not, and the failure is total rather than marginal. tokenizer relation n distinct 1st tokens distilgpt2 atomic_number 20 20 distilgpt2 planet_order 8 8 SmolLM2-135M atomic_number 20 1 SmolLM2-135M planet_order 8 1 Qwen2.5-0.5B atomic_number 20 1 Qwen2.5-0.5B planet_order 8 1 all three capital_word 8 8 The cause is visible in the ids: distilgpt2 ' 1' -> [352] [' 1'] ' 20' -> [1160] [' 20'] Qwen2.5-0.5B ' 1' -> [220, 16] [' ', '1'] ' 20' -> [220, 17, 15] [' ', '2', '0'] ' Paris' -> [12095] [' Paris'] Qwen- and Llama-family tokenizers emit the space as its own token before a digit, so every numeric answer shares first token 220. The consequence is the worst failure a one-sided bound can have. On a numeric relation under such a tokenizer m = 1 regardless of the model, so Theorem 1 certifies n - 1 = 19 wrong answers on a model that answered all twenty correctly, and it is silent because the partition simply looks like total collapse. verify_injective raises on exactly this and is the only thing between a caller and that false certification. The rule is per-tokenizer rather than per-relation: the same numeric relation is perfectly injective under GPT-2, where ' 20' is one token. Safe answers are those the tokenizer keeps whole - ordinary words and most proper nouns. Hazardous ones are those it splits from their leading space - digits, symbols, punctuation, anything short and non-alphabetic. This was found while trying to build a relation set constructed to be token-injective across tokenizers. Numeric answers were the intended solution and are the worst case. caustic/experiments/observable_hazard.py regenerates both tables, and the rule is added to the README Scope section where a caller meets it before choosing a relation rather than after certifying a perfect model as 95% wrong. Verification: 355 tests unchanged, no library code touched; tools/math_macro_audit.py exits 0. Limits. Three tokenizers, two numeric relations and one word relation. The safe/hazardous split is a description of what these tokenizers do, not a guarantee about any other, and the only reliable procedure remains running verify_injective against the tokenizer actually in use.

…cide it test_power_iteration_seed_none_still_returns_the_right_answer failed once in a full-suite run and passed 300 times in isolation. The cause is not the fixture and not a rare fluke; it is that the test was marginally converged by construction. Power iteration error decays as (sigma_{k+1}/sigma_k)^(2*iters). This fixture's spectrum is [10.504, 10.163, 9.577, 9.156, 8.479, 8.108], so the gap at k=4 is 8.4792/9.1563 = 0.92605 and 80 iterations reach 0.92605**160 = 4.6e-6 against a threshold of 1e-5. That is a factor of 2.2, and the measured worst case over 300 unseeded runs was 3.0e-6 - matching the prediction. Living inside a factor of two of its own tolerance makes the outcome depend on the global RNG state left behind by whatever ran before it. That is order-dependence: it turns a suite flaky in CI while staying unreproducible in isolation, and CI now runs three Python legs per push, so the exposure is three draws rather than one. The fix is the iteration count, not the tolerance. At 200 iterations convergence is 0.92605**400, about 1e-14, and the measured worst case over 300 unseeded runs drops from 3.0e-6 to 1.9e-7 - a 54x margin against the unchanged threshold. The assertion now tests the estimator rather than the draw. Loosening the threshold would have hidden this. The test asserts that opting out of the seed changes the start and not the answer, and that claim is only true once the estimator has converged; the right repair is to converge it. Verification: 355 tests pass; 0 of 300 unseeded runs exceed the threshold at iters=200, against a predicted and observed exceedance at iters=80. Limits. The derivation is specific to this fixture's spectral gap. A different matrix with a tighter gap would need more iterations again, and nothing here computes the required count from the spectrum - it is chosen with margin and the margin is stated.

Theorem 1* already requires the correct answers as a set. Once a caller has that set they can also decode against it, taking the argmax over its tokens rather than over the whole vocabulary. Measured on Qwen2.5-0.5B, seed 0: relation cond mode n m n-m acc capital none free 20 15 5 0.550 capital none constrained 20 20 0 1.000 language none free 16 12 4 0.625 language none constrained 16 16 0 1.000 currency none free 12 12 0 0.500 currency none constrained 12 12 0 1.000 capital the_x128 free 20 1 19 0.000 capital the_x128 constrained 20 1 19 0.050 language the_x128 constrained 16 1 15 0.062 currency the_x128 constrained 12 3 9 0.000 Under coherent context, constraining takes accuracy to 1.000 on all three relations. Those errors were not lost facts - the model was emitting something outside the answer space, and confining it there is enough. Under " the" x 128 constraining changes almost nothing: restricted to capitals, the model still cannot find the right capital. That is a control this repository did not have, and it settles a question a sceptical reader would raise about the headline. The 0.000 row is not an artifact of off-distribution tokens. The degenerate prefix destroys access to the fact rather than the choice of surface form. Two structural consequences. The precondition becomes automatic, since distinct admissible answers are distinct outputs and no tokenizer collision can produce a false certificate - which removes the hazard where numeric answers share first token 220 under Qwen- and Llama-family tokenizers. And Theorem 1* collapses back to Theorem 1, because nothing inadmissible can be emitted, so m* = m and the answer set buys nothing once enforced rather than consulted. The certificate tracks both regimes. Coherent and constrained gives m = n and certifies 0 errors against a measured accuracy of 1.000. Degenerate and constrained certifies 19 against 19 actual on capital and 15 against 15 on language - exact in 5 of the 6 constrained rows, valid with slack 3 in the sixth. Cost is one forward pass with the argmax masked to |G| logits, cheaper than the unconstrained argmax over the full vocabulary, and it beats the shipped prefix repair: NEUTRAL_PREFIX measures 0.750 on capital against 1.000 here. Verification: 355 tests unchanged; the module is new and is not imported by the suite. Limits. Constraining changes the task - a model choosing among 20 capitals is closer to multiple choice than to open generation, and these accuracy figures are not comparable to free-decoding accuracy elsewhere in this repository. It is available only where the answer set is known, which is the setting Theorem 1* already assumes and is far from every setting. One model, one seed, three relations, single-token answers only; a multi-token admissible set needs a prefix-constrained decode rather than a masked argmax, which is not implemented here.

Constraining the argmax to the answer set is the best repair measured in this repository and existed only as a script. constrain_to wraps a logits function into an answer_fn that cannot leave the answer set, which is the same set Theorem 1* already requires. Measured on Qwen2.5-0.5B, seed 0, under coherent context: relation free argmax constrained NEUTRAL_PREFIX capital 0.550 1.000 0.750 language 0.625 1.000 currency 0.500 1.000 On one forward pass with an argmax over |G| logits rather than the full vocabulary, so it is cheaper than the call it replaces. The injectivity precondition holds by construction under it: distinct admissible answers are distinct outputs, so no tokenizer collision can manufacture a false certificate. That removes the hazard where every numeric answer shares first token 220 under Qwen- and Llama-family tokenizers. What it does not do is stated in the docstring rather than left to be discovered. Constraining does not restore a lost fact: under " the" x 128 accuracy reaches only 0.050, 0.062 and 0.000, because the model cannot find the right capital even when permitted to say nothing else. Errors under coherent context were the model leaving the answer space; errors under the degenerate prefix are the fact being unreachable, and only the first kind is repaired. It also changes the task toward multiple choice, and it makes Theorem 1* redundant, since nothing inadmissible can be emitted and m* = m. Ties resolve to the lowest key so the partition stays deterministic; a sampled or order-dependent answer would make it noise rather than measurement. Verification: 355 tests before, 362 after. The seven added pin that the answer is always admissible, that an already admissible argmax is left alone, that an inadmissible one is replaced by the best admissible, the one-call cost, deterministic tie resolution across repeated calls, and two rejections - an empty answer set, and a key outside the logits vector which would otherwise raise from inside numpy at an unhelpful place. Limits. Single-token admissible answers only; a multi-token answer set needs a prefix-constrained decode rather than a masked argmax, and that is not implemented. Available only where the answer set is known, which is the setting Theorem 1* assumes and is far from every setting.

… theorem An adversarial review of this branch found three things that 355 passing tests did not. All three are fixed here with a test that would have caught each. BLOCKER: the package was broken at import level. caustic/__init__.py listed injective_subset in __all__ while three keep-both copies of the same `from .regime import` line each carried one branch's new name and the union lost it, so `from caustic import *` raised AttributeError. The three lines are now one explicit import block, and tests/test_contracts.py gains two tests - every name in __all__ resolves, and no name is listed twice. No test checked either; `grep -rn "__all__" tests/` returned nothing. BLOCKER: guard(gold_keys=...) certified errors on answers that were all correct. It reimplemented Theorem 1* inline as `len(entities) - len(set(answers) & gold)` and skipped the precondition check that admissible_distinct performs three files away. On the repository's own motivating collision - Asmara/Asuncion sharing token 1634, Lusaka/Ljubljana sharing 444 - it reported 2 certified errors and a 0.5 precision floor over four correct answers. regime.py:414 states the standard this violates in its own words: a one-sided bound that fires on a perfect answer set is not one-sided. The call now routes through admissible_distinct with n_entities and raises. FALSE THEOREM: Theorem 7 claimed that no function of the partition and the answer set bounds recall below by any positive quantity. That is a universal claim proved by one witness, and it is false. Theorem 1* places all n - m* certified errors inside the certified set and |wrong| <= n, so recall(S*) >= (n - m*) / n which is a sound, positive, attained floor - and it is certified_error_floor, the function this repository has shipped since before the review as "Theorem 1 as a rate". The same number bounds the error rate and the recall, and only the first was ever reported. Verified exhaustively over every answer map and every injective truth for n = 2..5: 2,048,574 configurations with at least one error, 0 violations, tightest margin exactly 0, strictly positive in 99.3%. On capital under " the" x 128 the floor is 1.000 - recall is provably perfect on the condition the branch described as having no recall guarantee. Theorem 7 is restated as the existence claim its witness supports: there is an observation at which no sound floor is positive, hence no constant c > 0 works for every observation. The witness sits at m* = n, exactly where Theorem 8's floor evaluates to 0, so it proves that floor is tight rather than absent. Theorem 8 ships as recall_floor, given its own name because a caller reading "error floor" will not think to quote it as a recall guarantee - which is the mistake made here - and because certified_error_floor delegates to orbit_error_bound, which requires m >= 1 and so rejects m* = 0, the total inadmissible collapse where the floor is 1.0. Also fixed: a caller candidate named "none" half-overrode the reserved sentinel, taking the prefix text while scores and reports stayed seeded from the empty prefix, so `winner` returned a prefix that was never scored while `intervened` reported False. select_prefix now raises. And Theorem 1* had no section in the module docstring, which promised ten results and listed nine; it has one, and the count is now eleven. Verification: 355 tests before, 369 after. tools/math_macro_audit.py exits 0. Limits. The review raised further items not addressed here: docs/index.html still claims ten bounds over a page documenting five, README section 4 is still "The five theorems" and defines neither Theorem 6 nor Theorem 7 that guard.py cites, the early exit's partial `scores` contradicts the auditability guarantee published on the same branch, the batched path is exercised by nothing that ships, and injective_subset is used by no experiment including the one that still skips a relation it would rescue.

The badge said ten proved bounds, the README section was headed "The five theorems", and docs/index.html carried five theorem sections under a masthead claiming ten. caustic/guard.py cites Theorem 6 as the authority for the precision floor it ships, and neither document defined it. The abstract cited Theorem 7 for the recall no-go, and neither document defined that either. Both files now carry a summary of the six added after the original five - 1*, 2*, 6, 6*, 7 and 8 - with the statement, the status and the measured instance for each, placed where a reader following a citation arrives. Full statements, proofs and executable witnesses remain in caustic/theorems.py, which is the single source; these are summaries and say so. Three things are stated in the summary that the theorem list alone would not convey. Theorem 8 corrects an error rather than adding a result: Theorem 7 was first stated as "no recall floor exists", which is false, and the floor is certified_error_floor - a function already shipped and reported only as an error rate. Theorem 7's witness marks where that floor is zero, which makes it tight rather than absent. And Theorems 1, 1*, 2, 2*, 6 and 6* are instances of one counting argument, separated because they are applied to different partitions and reported as different quantities, not because they need different proofs. A reader who counts eleven theorems should know they rest on two proof ideas. Counts corrected while here: the badge read 10 proved / 2 no-go and now reads 11 proved / 1 no-go, since Theorem 7 is the only no-go among the additions and Theorem 5 is the other. The README thesis line and section index no longer say five, and the docs meta description and masthead no longer say ten. Verification: 369 tests unchanged, no code touched. tools/math_macro_audit.py exits 0. docs/index.html parses with zero unclosed and zero mismatched tags, and greps for "five theorems" and "ten proved bounds" return zero. Limits. The five original theorem sections in docs/index.html are unchanged, so the page presents five in full and six in summary. The two documents still share no source and will drift again. RESULTS.md carries none of the new measurements - the 60-check sweep, the 16-of-16 exactness, the precision floors and the exhaustive recall-floor verification are in commit messages and docstrings only, while README states that every number on the page appears in RESULTS.md with its control. That claim is not yet true of anything added after Theorem 5.

README states that every number on the page appears in RESULTS.md with the control it was compared against. That was false for everything measured after Theorem 5: the 60-check cross-model sweep, the exhaustive recall-floor verification, the constrained-decode contrast, the batching speedup and the numeric-tokenizer hazard existed only in commit messages and docstrings. Five sections added, each with its control named rather than implied. Section 12, the certificate across three models: 60 bound checks, 0 violations, 4 relation-model pairs skipped by the injectivity check. The control for each row is the true error count computed from the gold answers after the bound was computed without them. That section also carries the control that weakens its own headline. Eight of the 16 evaluable rows are " the" x 128, where the model emits one token that is nobody's capital, so m* = 0 and any sound bound is exact by construction - those rows say nothing about the theorem. On the other eight, Theorem 1 is loose by exactly one, so the advertised advantage of Theorem 1* there is one answer out of twenty. Exactness holds iff the model never gives one entity's correct answer to another, which is a property of the failure mode and is not observable at inference. Section 13, the recall floor: 2,048,574 configurations over every answer map and every injective truth for n = 2..5, 0 violations, bound attained with tightest margin exactly 0, strictly positive in 99.3%. Recorded with the note that it corrects a claim rather than adding one. Section 14, constrained decoding, with the free argmax on identical prompts as control: accuracy 0.550/0.625/0.500 to 1.000 under coherent context, and 0.050/0.062/0.000 under the degenerate prefix. That contrast is the control the coherence result never had - the prefix destroys retrieval, not surface form. Section 15, cost, with the unbatched computation as control: 16.61x on short prompts, 1.70x under a long one, answers identical. Recorded together with the fact that no shipped code path calls the batched route, so the figure is reproduced by the benchmark alone and left-padding has not been exercised against a real tokenizer here. Section 16, the numeric-answer hazard, with a word-answer relation on the same tokenizers as control. The Limits section is rewritten to say the thing the new sections make unavoidable: all three models are base models under 0.5B parameters, where the dominant failure is collapse, and every headline figure on the page is monotone decreasing in model capability. As failures shift toward confusion the orbits go discrete, the certified set empties and the certificate goes silent. Nothing here tested a model where that could be observed. Verification: 369 tests unchanged, no code touched. tools/math_macro_audit.py exits 0 over three markdown files. Limits. The tables are transcribed from runs recorded in commit messages, not regenerated by CI - results/ is gitignored and no experiment module runs in CI, so the same hand-transcription this repository identifies as a defect still applies to these five sections.

A batched caller reads the answer at logits[:, -1, :], so the tokenizer must pad on the left. Right-padded, the final position is a pad token, every answer is the model's continuation of padding, and the partition that results is internally consistent and nobody's. Because the certificate is derived purely from the partition, nothing downstream can fail on it. The requirement was documented and unenforced, and no shipped code path exercised it. A one-element batch has nothing to pad to, so it returns the true answer under any padding side. Comparing it against the same prompt's answer inside the full batch separates the two cases, and the same comparison catches a batch_fn that returns answers out of the order it received them - which the existing length check cannot see. Probing the FIRST prompt does not work, and this was found by the test failing rather than by inspection. Right padding corrupts a prompt in proportion to how far it falls short of the longest, so a prompt that IS the longest is unaffected - and on this repository's own capital relation the first prompt, "The capital of France is", is the longest of the twenty. The probe uses the shortest prompt, which is the one guaranteed to be padded. Cost is one extra prompt, not one extra batch of n: a call of size n followed by a call of size 1. On by default, because both failures it catches are silent. verify_batch=False opts out for a caller who has verified once, and opting out is explicit rather than the default. The cost claims are corrected rather than left standing. guard's docstring no longer says a healthy relation costs a single batched forward pass; it says one batched call over n prompts plus a one-prompt probe. RESULTS.md section 15 notes that the 16.61x figure excludes the probe, which adds one prompt to a batch of twenty and moves the wall clock by about 5%. Verification: 369 tests before, 375 after. The six added cover a left-padded batch passing, a right-padded batch raising, a reordering batch raising, the probe costing one prompt rather than a second full batch, the probe targeting the shortest prompt rather than the first, and the opt-out working. Limits. The probe compares one prompt. A batch_fn that corrupts only prompts other than the shortest, or that permutes without moving the shortest, passes it. It is a cheap check against the two failures that actually occur, not a proof of correctness, and the docstring says so.

Merge the twenty-one open branches, with the four resolutions they need

Teerth Sharma and others added 30 commits August 16, 2026 15:21

Merge remote-tracking branch 'origin/harden/verify-injectivity-precon…

573210b

…dition' into integration-check-2

Merge remote-tracking branch 'origin/unit4-theorem-property-tests' in…

1414675

…to integration-check-2

Merge remote-tracking branch 'origin/notebooks/colab-quickstart' into…

732b9d4

… integration-check-2

Merge remote-tracking branch 'origin/detector-power' into integration…

5081917

…-check-2

Merge remote-tracking branch 'origin/unit7-permutation-null' into int…

cbc61a6

…egration-check-2 # Conflicts: # caustic/__init__.py

Merge remote-tracking branch 'origin/worktree-agent-ae2cea6559ad5054d…

4e9cbf2

…' into integration-check-2

Merge remote-tracking branch 'origin/tests/auroc-properties' into int…

93d2e63

…egration-check-2

Merge remote-tracking branch 'origin/theorem-1-star' into integration…

ef803a3

…-check-2 # Conflicts: # caustic/__init__.py # caustic/regime.py

Teerth Sharma added 27 commits August 16, 2026 18:00

Merge remote-tracking branch 'origin/collision-scored-template' into …

692f9af

…integration-check-2 # Conflicts: # caustic/regime.py

Merge remote-tracking branch 'origin/batched-partition' into integrat…

e64087b

…ion-check-2

Merge remote-tracking branch 'origin/certified-set-precision' into in…

93ab72b

…tegration-check-2 # Conflicts: # caustic/__init__.py # caustic/theorems.py

Merge remote-tracking branch 'origin/certified-precision-admissible' …

0cc295e

…into integration-check-2

Merge remote-tracking branch 'origin/recall-no-go' into integration-c…

c67c254

…heck-2

Merge remote-tracking branch 'origin/join-recovery-bound' into integr…

2fa0991

…ation-check-2 # Conflicts: # caustic/__init__.py # caustic/regime.py # caustic/theorems.py

Merge remote-tracking branch 'origin/inference-guard' into integratio…

ff00a69

…n-check-2

Merge remote-tracking branch 'origin/governor-early-exit' into integr…

ec7eb6a

…ation-check-2

Merge remote-tracking branch 'origin/guard-batched' into integration-…

f375f70

…check-2

Merge remote-tracking branch 'origin/guard-at-inference' into integra…

148821e

…tion-check-2 # Conflicts: # caustic/guard.py

Merge remote-tracking branch 'origin/guard-admissible-floor' into int…

8a85e58

…egration-check-2 # Conflicts: # caustic/guard.py

Merge remote-tracking branch 'origin/propagate-corrections' into inte…

4448a34

…gration-check-2

Merge remote-tracking branch 'origin/template-agreement-refuted' into…

099d9a6

… integration-check-2

Apply the four merge resolutions the twenty-one branches need

354f3fb

teerthsharma merged commit e3ce661 into master Aug 16, 2026
3 checks passed

teerthsharma deleted the integration-all branch August 16, 2026 13:48

teerthsharma added a commit that referenced this pull request Aug 18, 2026

Merge pull request #25 from teerthsharma/integration-all

72a0a56

Merge the twenty-one open branches, with the four resolutions they need

Navigation Menu

Repository navigation

Merge the twenty-one open branches, with the four resolutions they need - #25

Merge the twenty-one open branches, with the four resolutions they need#25
teerthsharma merged 57 commits into
masterfrom
integration-all

teerthsharma commented Aug 16, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

teerthsharma commented Aug 16, 2026

What is in it

The four resolutions, which are the reason this branch exists

Validation

Limits

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant