Skip to content

Add a portability audit script (npm run portability) - #58

Merged
jameskfry merged 1 commit into
mainfrom
feat/portability-audit-script
Oct 8, 2026
Merged

jameskfry merged 1 commit into
mainfrom
feat/portability-audit-script

Conversation

@jameskfry

Copy link
Copy Markdown
Owner

Summary

Turns the one-off portability audit into a repo script: scripts/portability-audit.cjs (npm run portability). Given expressions, or a source tree to extract them from, it tells which ones do not mean the same thing in PHP (Symfony) and in JavaScript. It needs no data.

npm run build
npm run portability -- --extract ./src ./config      # find the expressions in a code base
npm run portability -- rules.json --fail-on ordinary # a list; exit 1 in CI when something is not portable
npm run portability -- rules.txt --php-symfony /path/to/symfony-expression-language

How it works

An expression has no data, so every leaf (a variable, a property chain like object.getOwner(), a call like is_granted('X')) becomes a placeholder, its type is inferred from how it is used (an operand of < is a number, the left of matches a string...), and the expression is evaluated with the 'portable' semantics on values of three kinds:

Regime Values A failure means
ordinary typical values of the right type not portable whatever the data
edge 0, "", "0", [], null portable unless such a value shows up
mixed values of the wrong type portable as long as the types are right

The report gives the share of portable expressions per regime, the causes, which expressions would give a different result under 3.0's default than under 2.x's rules (the number to read before upgrading), and what could not be parsed. --php-symfony also evaluates every sample with Symfony's own PHP and lists differences.

What it found on real input

Run on 691 expressions extracted from 242 open-source projects (306 audited, the rest are a bare name or a single call):

portable
typical values 80.7%
typical and edge values 72.5%
every regime (incl. wrong types) 8.5%
  • 57 of the 59 expressions that are not portable with typical values are one pattern: two objects with equal contents compared with == / === (the ownership check object.getOwner() == user). PHP says true; 2.x said false; the 3.0 default says true.
  • Comparison with Symfony's PHP: 55,080 / 55,080 samples identical, and Symfony's parser and this library agree on parse/reject for every one of the strings.

(A correction while building this: my first run treated a value tested for truthiness (name or alias) as an object, which hid the "" / "0" cases where PHP and JS disagree. Fixed here; it moved "typical and edge" from 78.5% to 72.5%.)

Review notes

  • The core is importable and takes the library as a parameter (createAuditor(lib)), so the tests run it against the sources, no build needed. --lib points the CLI at any build, e.g. an installed package.
  • The extractor is regex-based on purpose (no PHP/YAML parsers): it finds expressions in the contexts where Symfony puts them, and anything that doesn't parse is reported, not silently dropped. Writing the tests found a gap ('@=service(...)' in a quoted YAML list) that the first version missed; fixed.
  • Pure Node, no dependencies. Not published in the npm package (scripts/ is in .npmignore).

Test plan

  • npm test: 558 passed (58 new: reading lists, extraction from 15 syntaxes and a real tree, classification of typical expressions, the report, every CLI path and exit code)
  • With Symfony's checkout (the CI php job runs these): 153 passed, including the comparison with PHP
  • Coverage run, tsc --strict, builds + smoke tests, npm audit --omit=dev
  • Exercised by hand: stdin (-), --lib on the installed 3.0.0-beta.1, a pre-3.0 library refused, --help
  • CI (renames the php job to "PHP against Symfony's own code": the aggregate CI check is unchanged)

🤖 Generated with Claude Code

scripts/portability-audit.cjs (npm run portability) tells which expressions
of a code base do not mean the same thing in PHP (Symfony) and in JavaScript,
from the expressions alone, without any data.

- Reads expressions from a JSON or text file, from stdin, or extracts them from
  source code (--extract): PHP attributes and annotations, YAML / XML
  configuration (security:, condition:, guard:, @=...), evaluate() calls.
- Every leaf of an expression (variable, property chain, function call) becomes
  a placeholder whose type is inferred from how it is used; the expression is
  evaluated with the "portable" semantics on values of three kinds: typical
  values, edge values (0, "", "0", [], null) and values of the wrong type.
- Reports the share of portable expressions per regime, the causes, which
  expressions would give another result with the rules of 2.x than with the
  default ones of 3.0, and what could not be parsed.
- --fail-on <regime> sets the exit code for CI, --json writes the report,
  --seed makes it reproducible, --lib audits an installed package.
- --php-symfony <dir> also evaluates every sample with Symfony's own PHP
  (scripts/portability-audit.php) and lists the answers that differ.

The core is importable and takes the library as a parameter, so the tests run
it against the sources; the PHP comparison is opt-in like the other PHP checks
and runs in the CI job that already has PHP and Symfony.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@jameskfry
jameskfry merged commit ed77d0e into main Oct 8, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant