Install the portability audit as a command; publish an allowlist of files

.github/workflows/ci.yml

@@ -60,7 +60,7 @@ jobs:
 
   build:
     # The sources are tested above; this runs what is published: the Babel build (lib/) and the minified browser bundle (dist/).
-    name: Build and smoke-test the bundles
+    name: Build, smoke-test the bundles and the package
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
@@ -77,6 +77,9 @@ jobs:
 
       - run: npm run test:build
 
+      # npm pack, then install the tarball into a clean project and run what it ships (the entry point, the bundle, the command)
+      - run: npm run test:package
+
   types:
     name: TypeScript definitions
     runs-on: ubuntu-latest

CHANGELOG.md

@@ -1,10 +1,5 @@
 # Changelog
 
-## Unreleased
-
-- `scripts/portability-audit.cjs` (`npm run portability`) finds the expressions of a code base and tells which ones do not mean the same thing in PHP and in JavaScript, and
-  can compare the answers of this library with Symfony's own PHP for them (see the README).
-
 ## 3.0.0
 
 This release brings the library back in line with Symfony's ExpressionLanguage, closes a hole that let expressions read
@@ -43,6 +38,8 @@ Everything below is explained, with before/after and how to keep the old behavio
   backslash. **Backslashes meant for a regex must be doubled** (`"/^\\d+$/"`).
 - **`matches`**: any delimiter, modifiers `i m s u` only (`g` is rejected), `SyntaxError` for an invalid pattern,
   `null` matches as an empty string.
+- **`..` (range)** counts down (`3..1` is `[3, 2, 1]`), follows PHP's `range()` for numeric strings and characters (`"1".."3"` is `[1, 2, 3]`, `"a".."e"` are the letters), refuses an array,
+  and a range of more than 10 million items is a `ValueError` (it used to try to build it). The subject of `matches` cannot be an array (a `TypeError`, as in Symfony).
 - **`in` / `not in`** throw when the right operand is not an array or hash (`"a" in "abc"` used to work).
 - **`min()` / `max()`** accept an array like PHP's and throw like PHP's on empty or non-array input
   (`min([3, 1])` was `NaN`).
@@ -62,6 +59,8 @@ Everything below is explained, with before/after and how to keep the old behavio
 
 ### ✨ New
 
+- **`expression-language-portability`**, a command of the package (`npx expression-language-portability`): finds the expressions of a code base, tells which ones do not mean the
+  same thing in PHP and in JavaScript, and can compare this library's answers with Symfony's own PHP for them. `--fail-on` makes it usable in CI. See the README.
 - **`CompiledExpressionLanguage`**: compile expressions ahead of time and evaluate them without parsing. `dumpCompiled()`
   generates **JavaScript** (default; as an ES module, a CommonJS module or a bare expression) **or PHP** (`{ target: 'php' }`),
   in the format Symfony's own `CompiledExpressionLanguage` loads. See the README.
@@ -90,6 +89,11 @@ Everything below is explained, with before/after and how to keep the old behavio
 - An unterminated string followed by another quoted string was mis-lexed.
 - "Did you mean ...?" was never offered for unknown function names.
 
+### 📦 Package contents
+
+The package now publishes an allowlist of files (`files` in `package.json`, replacing `.npmignore`): `lib`, `dist`, `bin`, this changelog and the upgrade guide. The compiled tests,
+CI workflows, build configuration and examples are no longer in the tarball (286 KB instead of 602 KB). Nothing under `lib/` or `dist/` moved, so imports of the package are unaffected.
+
 ### Still different from Symfony
 
 Numbers are doubles (`1 === 1.0`, integers above 2^53 are rounded), regexes are ECMAScript's, and a hash and a class instance are both objects.

README.md

@@ -555,15 +555,17 @@ some 1,700 ns uncompiled).
 ## Auditing your expressions for portability
 
 If the same expressions run on a server and in a browser, you want to know which of them depend on the rules of a language.
-`scripts/portability-audit.cjs` tells you, from the expressions alone: it needs no data.
+The package installs a command that tells you, from the expressions alone: it needs no data.
 
 ```bash
-npm run build                                       # the script audits the build in lib/ (or --lib <path>)
-npm run portability -- --extract ./src ./config     # find the expressions in your source code
-npm run portability -- rules.json                   # ...or give a list: a JSON array of strings, or one expression per line
-npm run portability -- rules.txt --fail-on ordinary # exit with 1 when an expression is not portable, to use it in CI
+npm install --save-dev expression-language
+npx expression-language-portability --extract ./src ./config      # find the expressions in your source code
+npx expression-language-portability rules.json                    # ...or give a list: a JSON array of strings, or one expression per line
+npx expression-language-portability rules.txt --fail-on ordinary  # exit with 1 when an expression is not portable, to use it in CI
 ```
 
+It audits the version of the package it comes with. From a checkout of this repository, run `npm run build` and then `npm run portability -- <the same arguments>`.
+
 `--extract` reads PHP attributes and annotations (`#[IsGranted(expression: ...)]`, `@Security(...)`, `new Expression(...)`, `Assert\Expression`),
 YAML and XML configuration (`security:`, `condition:`, `guard:`, `@=...`) and `evaluate()` / `compile()` calls in JavaScript.
 
@@ -623,6 +625,7 @@ Run the tests with `npm test`. The pull request checks ([ci.yml](.github/workflo
 | `npm run test:coverage` | the same tests with the sources instrumented (code that `compile()` serialises with `Function#toString()` must stay out of it: see the `istanbul ignore` hints) |
 | `npm run test:types` | the TypeScript definitions (`types/test-types.ts`, `tsc --strict`) |
 | `npm run build:all && npm run test:build` | the Babel build and the minified browser bundle, which are what gets published |
+| `npm run test:package` | `npm pack`, then the tarball installed into a clean project: what is in it, the entry point, the bundle, the `expression-language-portability` command (needs the network) |
 
 The `CI` workflow also audits the runtime dependencies (`npm audit --omit=dev`) and evaluates the PHP that `dumpCompiled()` generates with Symfony's own loader, against a Symfony commit pinned in `SYMFONY_EXPRESSION_LANGUAGE_SHA`: move it on purpose, together with the fixtures.
 

scripts/portability-audit.cjs → bin/portability-audit.cjs

@@ -20,7 +20,8 @@
  *
  * Usage:
  *
- *   node scripts/portability-audit.cjs [options] [file ...]
+ *   npx expression-language-portability [options] [file ...]      (once the package is installed)
+ *   node bin/portability-audit.cjs [options] [file ...]            (from a checkout of the repository)
  *
  *   file                    a JSON file (an array of strings, or of {expr: "..."}), or a text file with one expression per line
  *                           (blank lines and lines starting with # are ignored). "-" reads the standard input.
@@ -35,8 +36,8 @@
  *   --php-symfony <dir>     also evaluate the samples with Symfony's own PHP and compare (needs php, and a checkout of
  *                           symfony/expression-language; see the README, "Contributing")
  *   --php <binary>          the php to use (default: php)
- *   --lib <path>            the expression-language build to audit (default: ../lib, run `npm run build` first); point it to
- *                           node_modules/expression-language to audit the installed package
+ *   --lib <path>            the expression-language build to audit (default: the one this script comes with: the installed package, or
+ *                           lib/ of a checkout, which `npm run build` makes)
  *   --help
  *
  * Exits with 0, 1 (see --fail-on, or a disagreement with PHP) or 2 (wrong usage).

package.json

@@ -4,6 +4,17 @@
   "description": "Javascript implementation of symfony/expression-language",
   "main": "lib/index.js",
   "types": "dist/index.d.ts",
+  "bin": {
+    "expression-language-portability": "bin/portability-audit.cjs"
+  },
+  "files": [
+    "bin",
+    "dist",
+    "lib",
+    "!lib/**/__tests__",
+    "CHANGELOG.md",
+    "UPGRADE-*.md"
+  ],
   "browser": "dist/expression-language.min.js",
   "repository": {
     "type": "git",
@@ -18,7 +29,8 @@
     "test:coverage": "jest --coverage",
     "test:types": "tsc --noEmit --strict --target es2020 --moduleResolution node --esModuleInterop --skipLibCheck types/test-types.ts",
     "test:build": "node scripts/smoke-build.cjs lib/index.js dist/expression-language.min.js",
-    "portability": "node scripts/portability-audit.cjs"
+    "test:package": "node scripts/check-package.cjs",
+    "portability": "node bin/portability-audit.cjs"
   },
   "jest": {
     "testPathIgnorePatterns": [

scripts/check-package.cjs

@@ -0,0 +1,114 @@
+#!/usr/bin/env node
+/*
+ * Checks the package the way people receive it: packs it, looks at what is in the tarball, installs the tarball into a clean project,
+ * and runs what it ships (the entry point, the browser bundle, and the command of the "bin" entry).
+ *
+ * The tests run on the sources and smoke-build.cjs on the built files; this is the only one that goes through `npm pack`, so it is the
+ * one that notices a file that is no longer published (the `files` of package.json), a bin entry that points nowhere, or a command that cannot run.
+ *
+ * Run `npm run build:all` first. Needs the network: the dependencies of the package are installed from the registry.
+ *
+ *   node scripts/check-package.cjs
+ */
+'use strict';
+
+const childProcess = require('child_process');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+
+const root = path.join(__dirname, '..');
+const manifest = require(path.join(root, 'package.json'));
+
+const MUST_CONTAIN = [
+    'package.json', 'README.md', 'CHANGELOG.md', 'UPGRADE-3.0.md', 'LICENSE.txt',
+    'lib/index.js', 'dist/expression-language.js', 'dist/expression-language.min.js', 'dist/index.d.ts',
+    // the PHP helper is run by the command, it has to travel with it
+    'bin/portability-audit.cjs', 'bin/portability-audit.php',
+];
+// Nothing of the repository's tooling, and nothing a working copy leaves lying around (a coverage report, local settings of an editor):
+// the `files` of package.json is an allowlist, and this is what proves it holds
+const MUST_NOT_CONTAIN = [
+    /^src\//, /^scripts\//, /^types\//, /^examples\//, /^\.github\//, /^node_modules\//, /^\.idea\//, /^\.claude\//, /^coverage\//,
+    /__tests__/, /^rollup\.config/, /^jest-reporter/, /^tsconfig/, /^\.babelrc/, /^\.npmignore/, /^\.env/,
+];
+
+const failures = [];
+const check = (ok, message) => {
+    console.log(`  ${ok ? 'PASS' : 'FAIL'} ${message}`);
+    if (!ok) {
+        failures.push(message);
+    }
+
+    return ok;
+};
+const run = (command, args, options = {}) => childProcess.spawnSync(command, args, {encoding: 'utf8', maxBuffer: 1 << 28, ...options});
+
+const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'check-package-'));
+try {
+    console.log('The tarball');
+    const packed = run('npm', ['pack', '--json', '--pack-destination', directory], {cwd: root});
+    if (!check(packed.status === 0, 'npm pack succeeds')) {
+        console.log(packed.stderr);
+        process.exit(1);
+    }
+    const [info] = JSON.parse(packed.stdout);
+    const files = info.files.map((file) => file.path);
+    const tarball = path.join(directory, info.filename);
+
+    for (const file of MUST_CONTAIN) {
+        check(files.includes(file), `contains ${file}`);
+    }
+    for (const pattern of MUST_NOT_CONTAIN) {
+        check(!files.some((file) => pattern.test(file)), `contains nothing that matches ${pattern}`);
+    }
+    for (const [name, target] of Object.entries(manifest.bin || {})) {
+        check(files.includes(target.replace(/^\.\//, '')), `the bin entry "${name}" points to a file of the tarball (${target})`);
+    }
+
+    console.log('\nInstalled into a clean project');
+    const project = path.join(directory, 'project');
+    fs.mkdirSync(project);
+    fs.writeFileSync(path.join(project, 'package.json'), JSON.stringify({name: 'check-package', version: '1.0.0', private: true}));
+    const install = run('npm', ['install', tarball, '--no-audit', '--no-fund', '--ignore-scripts'], {cwd: project});
+    if (!check(install.status === 0, 'npm install of the tarball succeeds')) {
+        console.log(install.stderr);
+        process.exit(1);
+    }
+
+    const node = (code) => run(process.execPath, ['-e', code], {cwd: project});
+    const entry = node(`const l = require('${manifest.name}'); process.stdout.write(String(l.ExpressionLanguage && new l.ExpressionLanguage().evaluate('1 + 2')))`);
+    check(entry.status === 0 && entry.stdout === '3', 'require() of the package works and evaluates an expression');
+
+    const bundle = node(`const l = require('${manifest.name}/dist/expression-language.min.js'); process.stdout.write(String(new l.ExpressionLanguage().evaluate('1 + 2')))`);
+    check(bundle.status === 0 && bundle.stdout === '3', 'the minified browser bundle works');
+
+    check(fs.existsSync(path.join(project, 'node_modules', manifest.name, manifest.types)), `the types are there (${manifest.types})`);
+
+    for (const name of Object.keys(manifest.bin || {})) {
+        const executable = path.join(project, 'node_modules', '.bin', name);
+        if (!check(fs.existsSync(executable), `the command ${name} is installed`)) {
+            continue;
+        }
+
+        const list = path.join(project, 'rules.txt');
+        fs.writeFileSync(list, 'price > 10\nobject.owner == user\n');
+
+        const help = run(executable, ['--help'], {cwd: project});
+        check(help.status === 0 && help.stdout.includes('Portability audit'), `${name} --help`);
+
+        const audit = run(executable, [list, '--samples', '40'], {cwd: project});
+        check(audit.status === 0 && /audited\s+: 2/.test(audit.stdout) && audit.stdout.includes('NOT PORTABLE with ordinary values: 1'), `${name} audits a list of expressions`);
+
+        const failing = run(executable, [list, '--samples', '40', '--fail-on', 'ordinary'], {cwd: project});
+        check(failing.status === 1, `${name} --fail-on exits with 1 when an expression is not portable`);
+
+        const usage = run(executable, [], {cwd: project});
+        check(usage.status === 2, `${name} without arguments is a usage error (exit code 2)`);
+    }
+} finally {
+    fs.rmSync(directory, {recursive: true, force: true});
+}
+
+console.log(failures.length ? `\n${failures.length} check(s) failed.` : '\nAll checks passed.');
+process.exit(failures.length ? 1 : 0);

src/__tests__/PortabilityAudit.test.js

@@ -3,7 +3,7 @@ import os from "os";
 import path from "path";
 import * as library from "../index";
 
-const audit = require("../../scripts/portability-audit.cjs");
+const audit = require("../../bin/portability-audit.cjs");
 
 // the library under audit is the sources: no build needed
 const auditor = (options = {}) => audit.createAuditor(library, {samples: 120, ...options});
@@ -231,6 +231,34 @@ describe('the report', () => {
     });
 });
 
+describe('the command that the package installs', () => {
+    const manifest = JSON.parse(fs.readFileSync(path.join(__dirname, '../../package.json'), 'utf8'));
+    const root = path.join(__dirname, '../..');
+
+    test('the bin entry points to the script', () => {
+        expect(manifest.bin).toEqual({'expression-language-portability': 'bin/portability-audit.cjs'});
+        expect(fs.existsSync(path.join(root, manifest.bin['expression-language-portability']))).toBe(true);
+    });
+
+    test('the script is a node program, and is executable in the repository', () => {
+        const file = path.join(root, 'bin/portability-audit.cjs');
+
+        expect(fs.readFileSync(file, 'utf8').startsWith('#!/usr/bin/env node\n')).toBe(true);
+        expect(fs.statSync(file).mode & 0o111).not.toBe(0);
+    });
+
+    test('the PHP helper it runs sits next to it, and the directory is published', () => {
+        expect(fs.existsSync(path.join(root, 'bin/portability-audit.php'))).toBe(true);
+        // `files` is an allowlist: what it does not name is not published
+        expect(manifest.files).toEqual(expect.arrayContaining(['bin', 'lib', 'dist']));
+        expect(fs.existsSync(path.join(root, '.npmignore'))).toBe(false);
+    });
+
+    test('the default library is the build next to the script, so the installed package audits itself', () => {
+        expect(audit.parseArguments([]).lib).toBe(path.join(root, 'lib', 'index.js'));
+    });
+});
+
 describe('the command line', () => {
     const run = (args, extra = {}) => {
         const out = [];