Skip to content

Decide precision loss by symbol kind instead of by variable name - #71

Closed
nils-loose wants to merge 2 commits into
svcomp27/integrationfrom
fix/uf-detection-by-kind
Closed

nils-loose wants to merge 2 commits into
svcomp27/integrationfrom
fix/uf-detection-by-kind

Conversation

@nils-loose

Copy link
Copy Markdown
Contributor

What changes

DTOBuilder used to flag precision loss when any symbol in a branch constraint failed the name regex [A-Z].*_[0-9].*. It now decides from the kind each symbol was declared with.

  • SymbolRegistry. Each symbol is declared, where it is created, as one of: INPUT, VALUE (another symbolic value), ARRAY_BASE, MODEL_FUNCTION (the definition holds for every argument) or APPROXIMATE_MODEL_FUNCTION. A branch constraint is imprecise if it mentions an approximate function or a symbol nobody declared. An undeclared symbol is logged once.

  • Declarations:

    • Value.initSymbolic* declares VALUE.
    • SymbolicTraceHandler.addInput declares INPUT.
    • The array constructors declare ARRAY_BASE.
    • The String UFs (trimStart, trimEnd, commonPrefixLength, lastIndexOf) are MODEL_FUNCTION.
    • sin, cos, toLowerCase and equalsIgnoreCase are APPROXIMATE_MODEL_FUNCTION.
  • Approximate models record precision loss where they add constraints. This now happens explicitly (recordSymbolicPrecisionLoss()), not only when a branch mentions the function:

    • SinCosUF restricts its argument to the sample points.
    • The guarded toLowerCase restricts chars to printable ASCII.
    • lastIndexOf(int) with a symbolic char is approximate (non-BMP and negative chars are not modelled).

    These constraints are trace-global, so they reach every query.

  • Array base symbols get their own namespace ([I_alloc<n>, A_alloc<n>). An input array [I_<idx> can no longer share, and so constrain, an allocated array's base symbol.

Why this is sound: a precise symbol may still be under-constrained, for example a fresh value, an array base outside the stored range, or a partially defined function. That only over-approximates: UNSAT stays a proof. A model that does not reach its branch is a divergence, and an unexpected divergence already downgrades SAFE. Anything that may exclude real behaviour stays a precision loss.

Measured

  • Setup: --sv-comp, --mem15g, 300 s, no SA. 1189 runs, compared with baseline-results.jsonl.
  • Affected set (188 runs):
    • every non-correct KB run with precision-loss-false-positive-var-name, precision-loss-uf or array-base-symbol-collides-with-input;
    • every expected-VIOLATION run with precision loss in the baseline;
    • every expected-SAFE run that is not SAFE and has precision loss.
  • Regression set: all 1001 baseline-correct runs (the change touches every run's DTO).

Result: +8 / -0, net +8, no new wrong verdicts. The one wrong verdict, AlbersProjection_true (SAFE -> VIOLATION), is the same as in the baseline.

task prop base new pts
jbmc-regression/StringCompare04 NRE unknown safe +2
jbmc-regression/StringIndexMethods03 NRE unknown safe +2
jbmc-regression/StringIndexMethods05 NRE unknown safe +2
jdart-regression/array-iteration01 NRE unknown safe +2

Soundness check:

  • All 11 expected-VIOLATION runs where precision loss blocks a false SAFE keep the flag and stay UNKNOWN: coral*/Optimization* (sin/cos, sampled Math) and the two lastIndexOf(int) autostubs, which are now flagged by the explicit record.
  • The flag was dropped on 13 runs, all already correct (9 VIOLATION, 4 SAFE).
  • No run gained a precision-loss flag, so no undeclared symbol turned up.

Tests

  • New SymbolRegistrySpec covering:
    • input and array base are precise, and the base name is outside the input namespace;
    • trim UFs are precise;
    • sin is a precision loss and records it;
    • an undeclared UF is a precision loss;
    • a later declaration does not weaken an input or an approximate one.
  • Full executor test suite: 8735 tests, 0 failures.

Limits

  • Most remaining KB runs with these causes are still blocked by context loss (collections, toLowerCase, StringBuffer) or by timeouts.
  • The registry is a static map. It is not reset between threads. Names are unique per run.

🤖 Generated with Claude Code

nils-loose and others added 2 commits October 18, 2026 21:38
DTOBuilder flagged a branch constraint as imprecise when one of its
symbols did not match the name pattern [A-Z].*_[0-9].*. Array base
symbols ([I_0) and the exact UFs of the String models (trimStart,
trimEnd, commonPrefixLength, lastIndexOf) failed it, so SAFE was
downgraded for constraints that are encoded exactly.

Symbols are now declared with their kind where they are created
(SymbolRegistry): inputs, other symbolic values, array base symbols,
exact model functions and approximate model functions. A constraint
is imprecise if it mentions an approximate model function or a symbol
nobody declared.

Array base symbols get their own namespace (<prefix>_alloc<n>), so an
input array of the same prefix and index cannot share them.

lastIndexOf(int) with a symbolic char records precision loss: chars
outside the BMP, and negative ones, are not modelled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sin/cos, toLowerCase and equalsIgnoreCase are approximate model
functions. sin/cos restrict their argument to the sample points, and
the guarded toLowerCase restricts chars to printable ASCII. Both
constraints reach every query, so they record precision loss where
they are added, not only when a branch mentions the function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
SinCosUF.java: deleted as in #80 (its trace-global sample constraint gave
a false SAFE); #71's eager precision loss there goes with it.
StrategyService.py: solve_branch takes both #74's extra_exprs and #80's
approximate_rlimit, on #80's sliced query.
SVCompDriver.py: retries (#74) and first queries both get the
approximate-query cap (#80); a solution keeps #74's record and #80's
query-based guess flag.
SymbolicExplorer.py: both sets of options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
Combining #71 and #80: each per-call math_<f>_<n> function is declared
APPROXIMATE_MODEL_FUNCTION in the SymbolRegistry, so a branch over one is
a precision loss by kind, not only as an undeclared symbol. The registry
test builds its function through SampledMath, as SinCosUF is gone, and
the divergence tests pass #80's approximate_rlimit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
_extract_variables: one copy (#82 cherry-picks #72's patch).
StrategyService.py, Database.py: #82 walks the Database's tree under the
lock; visit_tree passes the solved and #74's retired branches, in #72's
'with lock' style.
SVCompDriver.py: #82's shallow snapshot of the target node for
check_divergence, with #74's solution record and #80's guess flag; the
ingest-failure (#72) and stack-overflow (#78) checks read the Database
without copying the tree.
DTOBuilder.java: #82's FormulaDumper with #71's SymbolRegistry check.
DFS.py: #82 logs the per-node line only with an SA graph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nils-loose added a commit that referenced this pull request Oct 19, 2026
EqualsIgnoreCaseUF, ToLowerCaseUF: deleted as in #90 (replaced by exact
models); #71's registry entries go with them. UFHandler keeps no models,
as #80 removed SinCosUF too. StringValue.java: keep #71's SymbolRegistry
import.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nils-loose nils-loose mentioned this pull request Oct 20, 2026
@nils-loose

Copy link
Copy Markdown
Contributor Author

Included in dev through #92 (merge commit 534c182), together with the fixes for interactions between PRs. This PR stays as the record of the change and its measured flips.

@nils-loose nils-loose closed this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant