feat: add highlight.js compatibility harness

CLAUDE.md

@@ -92,7 +92,7 @@ harness. See `scripts/README.md` for the full recipe and how to add a library.
 - Each library has a config at `scripts/libs/<lib>.sh` (pinned repo/ref, bundle entry, prepare hook, verdict). The runner clones the pinned ref, bundles with a pinned esbuild, prepends the shared JS-only Node-globals shims (`scripts/node-shim.js` for `process`/`console`, `scripts/node-buffer-shim.js` for `Buffer`/`TextEncoder`/`TextDecoder`) — never baked into jsse's globals, so test262 is unaffected — runs on jsse, and cross-checks the test count against Node. Caches live under `/tmp/jsse-libtests/<lib>/`.
 - The shim stubs are guarded so they are inert on Node, letting `--node` run the identical bundle as the reference.
 - **`Buffer`** is a pure-JS subclass of `Uint8Array` (`node-buffer-shim.js`), needing zero new engine object kinds; `instanceof Uint8Array` holds. Self-verifying fixtures live in `scripts/shim-fixtures/`; run them (jsse + Node cross-checked) with `./scripts/run-shim-fixtures.sh`.
-- **Currently green (cross-checked vs Node):** `decimal.js` (22,624), `big.js` (47,456; ~7 min — heavy arbitrary-precision arithmetic on the tree-walker), `acorn` (13,507), `prismjs` (2,563).
+- **Currently green (cross-checked vs Node):** `decimal.js` (22,624), `big.js` (47,456; ~7 min — heavy arbitrary-precision arithmetic on the tree-walker), `acorn` (13,507), `prismjs` (2,563), `highlight.js` (731; ~30 min for the full auto-detection competition set).
 - **`bignumber.js`** is wired but blocked on a jsse bug it surfaced (strict-mode `return <call-returning-non-object>` in a constructor returns that value instead of `this`, jsse#238); it goes green once that engine bug is fixed.
 - **`luxon`** is wired at 3.7.2 with an exact 1,152-test Node cross-check. Node is green; jsse currently passes 1,045, with the remaining Intl/system-zone gaps tracked in jsse#262–#265.
 - **`moment`** is wired at 2.30.1 with an exact 3,871-test Node cross-check. Node is green with 162,868 assertions; jsse executes every test and records 198 visible failing assertions, tracked in jsse#311.

scripts/README.md

@@ -227,6 +227,7 @@ assertions).
 | `lodash` | 4.17.21 | ✅ 6,794 (cross-checked) | QUnit via the shared harness; a few tests skipped on jsse — see below |
 | `ajv` | v8.17.1 | ⚠️ 5,466 / 5,480 (Node: 5,480) | ~4 min; four codegen option variants across drafts 6, 7, 2019-09, and 2020-12; residuals tracked in #274 and #275 |
 | `prismjs` | v1.30.0 | ✅ 2,563 (cross-checked) | token streams for ~290 grammars |
+| `highlight.js` | 11.11.2 | ✅ 731 (cross-checked) | 536 markup + 195 auto-detection fixtures across 192 grammars; ~30 min |
 | `js-sha256` | v0.11.1 | ✅ 916 (cross-checked) | Pure-JS SHA-224/SHA-256 and HMAC vectors; string, Buffer, TypedArray, and ArrayBuffer inputs |
 | `luxon` | 3.7.2 | ⚠️ 1,045 / 1,152 | exact count cross-checked; Node is 1,152 / 1,152; blocked on #262–#265 |
 | `moment` | 2.30.1 | ⚠️ 198 failing assertions across 3,871 tests | exact registered-test count cross-checked; Node is green with 162,868 assertions; residual tracked in #311 |
@@ -245,6 +246,28 @@ JSSE and Node execute all 2,563 fixtures. The cross-check requires both engines
 to report the same fixture count, so engine-specific skips cannot masquerade as
 a successful run.
 
+### highlight.js markup and auto-detection fixtures
+
+`scripts/gen-highlightjs-entry.js` registers all 192 built-in grammars from the
+pinned source tree and embeds its filesystem fixtures into one deterministic
+bundle. The 536 markup fixtures run in highlight.js debug mode and compare the
+generated HTML byte-for-byte with upstream's expected output after the same
+whitespace trimming as its Mocha suite.
+
+The auto-detection corpus contributes 195 more cases after applying upstream's
+`autoDetection()` filter to its 198 inputs (G-code, properties, and plain text
+opt out).
+Each eligible input competes against the complete grammar set, exercising the
+relevance-scoring state machine rather than a single-language fast path. This
+is the expensive half of the run: roughly 30 minutes on jsse's tree-walker.
+
+Upstream currently comments out its dynamic auto-detection assertions, and
+eight ambiguous samples are won by a different grammar when all languages
+compete. The generator records Node's winners for pinned 11.11.2 and runs the
+public production mode for detection; debug mode exposes an upstream Nix
+zero-width assertion on otherwise valid inputs. Both engines therefore compare
+against the same fixed Node oracle and still report the exact 731-case count.
+
 ### Luxon
 
 Luxon's 58 Jest files are statically bundled by `gen-luxon-entry.js`; the

scripts/gen-highlightjs-entry.js

@@ -0,0 +1,164 @@
+#!/usr/bin/env node
+'use strict';
+
+// Generate a filesystem-free highlight.js entry from its upstream markup and
+// auto-detection fixtures. Run from a highlight.js checkout; the sole argument
+// is the output path.
+
+const fs = require('fs');
+const path = require('path');
+
+const root = process.cwd();
+const output = process.argv[2];
+
+if (!output) {
+  console.error('usage: gen-highlightjs-entry.js <output>');
+  process.exit(2);
+}
+
+/** @param {unknown} value */
+function serialize(value) {
+  // U+2028/U+2029 are valid JSON but unsafe in pre-JSON-superset JS parsers.
+  return JSON.stringify(value)
+    .replace(/\u2028/g, '\\u2028')
+    .replace(/\u2029/g, '\\u2029');
+}
+
+const languagesRoot = path.join(root, 'src', 'languages');
+const markupRoot = path.join(root, 'test', 'markup');
+const detectRoot = path.join(root, 'test', 'detect');
+
+// The detect directories are samples rather than active assertions upstream;
+// with every built-in grammar competing, these are Node's winners for the
+// pinned 11.11.2 sources where the directory name does not win.
+const detectExpectedOverrides = {
+  'javascript/sample1.txt': 'gml',
+  'json5/default.txt': 'pgsql',
+  'smalltalk/default.txt': 'nix',
+  'sql/default.txt': 'pgsql',
+  'vala/default.txt': 'csharp',
+  'xml/default.txt': 'django',
+  'xml/groovy-julia.txt': 'django',
+  'xml/js.txt': 'django',
+};
+
+const languageIds = fs.readdirSync(languagesRoot)
+  .filter(fileName => fileName.endsWith('.js'))
+  .map(fileName => fileName.slice(0, -3))
+  .sort();
+
+const markupFixtures = [];
+for (const language of fs.readdirSync(markupRoot).sort()) {
+  const languageDir = path.join(markupRoot, language);
+  if (!fs.statSync(languageDir).isDirectory()) continue;
+
+  for (const fileName of fs.readdirSync(languageDir).sort()) {
+    if (!fileName.endsWith('.expect.txt')) continue;
+
+    const expectedPath = path.join(languageDir, fileName);
+    const sourcePath = expectedPath.replace(/\.expect\.txt$/, '.txt');
+    if (!fs.existsSync(sourcePath)) {
+      throw new Error(`Missing markup source: ${path.relative(root, sourcePath)}`);
+    }
+
+    markupFixtures.push([
+      `${language}/${fileName.slice(0, -11)}`,
+      language,
+      fs.readFileSync(sourcePath, 'utf8'),
+      fs.readFileSync(expectedPath, 'utf8').trim(),
+    ]);
+  }
+}
+
+const detectFixtures = [];
+for (const language of fs.readdirSync(detectRoot).sort()) {
+  const languageDir = path.join(detectRoot, language);
+  if (!fs.statSync(languageDir).isDirectory()) continue;
+
+  for (const fileName of fs.readdirSync(languageDir).sort()) {
+    const filePath = path.join(languageDir, fileName);
+    if (!fs.statSync(filePath).isFile() || fileName === '.DS_Store') continue;
+
+    const fixtureName = `${language}/${fileName}`;
+    detectFixtures.push([
+      fixtureName,
+      language,
+      fs.readFileSync(filePath, 'utf8'),
+      detectExpectedOverrides[fixtureName] || language,
+    ]);
+  }
+}
+
+const imports = languageIds
+  .map((id, index) => `import language${index} from '../src/languages/${id}.js';`)
+  .join('\n');
+const registrations = languageIds
+  .map((id, index) => `hljs.registerLanguage(${serialize(id)}, language${index});`)
+  .join('\n');
+
+const entry = `'use strict';
+
+// Generated by jsse/scripts/gen-highlightjs-entry.js from pinned upstream fixtures.
+import hljs from '../src/highlight.js';
+${imports}
+
+${registrations}
+hljs.debugMode();
+
+const MARKUP_FIXTURES = ${serialize(markupFixtures)};
+const DETECT_FIXTURES = ${serialize(detectFixtures)};
+
+let passed = 0;
+let failed = 0;
+
+function fail(kind, name, expected, actual) {
+  failed++;
+  console.log('FAIL ' + kind + ' ' + name);
+  console.log('  expected: ' + JSON.stringify(expected));
+  console.log('  actual:   ' + JSON.stringify(actual));
+}
+
+for (let i = 0; i < MARKUP_FIXTURES.length; i++) {
+  const fixture = MARKUP_FIXTURES[i];
+  try {
+    const actual = hljs.highlight(fixture[2], { language: fixture[1] }).value.trim();
+    if (actual === fixture[3]) passed++;
+    else fail('markup', fixture[0], fixture[3], actual);
+  } catch (error) {
+    fail('markup', fixture[0], fixture[3], String(error && error.stack || error));
+  }
+}
+
+// The upstream auto-detection suite is currently commented out, and the Nix
+// grammar throws a zero-width debug assertion for some otherwise valid inputs.
+// Exercise the production API here; Node's winners for the pinned fixtures are
+// recorded below after the initial oracle run.
+hljs.safeMode();
+let detectTotal = 0;
+for (let i = 0; i < DETECT_FIXTURES.length; i++) {
+  const fixture = DETECT_FIXTURES[i];
+  // This is the filter used by highlight.js's (currently disabled) dynamic
+  // auto-detection test registration. Some grammars deliberately opt out.
+  if (!hljs.autoDetection(fixture[1])) continue;
+
+  detectTotal++;
+  try {
+    const actual = hljs.highlightAuto(fixture[2]).language;
+    if (actual === fixture[3]) passed++;
+    else fail('detect', fixture[0], fixture[3], actual);
+  } catch (error) {
+    fail('detect', fixture[0], fixture[3], String(error && error.stack || error));
+  }
+}
+
+const total = MARKUP_FIXTURES.length + detectTotal;
+console.log('HighlightJS: ' + passed + ' passed, ' + failed + ' failed, ' + total + ' total');
+if (failed) process.exit(1);
+`;
+
+fs.mkdirSync(path.dirname(path.resolve(output)), { recursive: true });
+fs.writeFileSync(output, entry);
+console.log(
+  `Generated ${output} with ${markupFixtures.length} markup and ` +
+  `${detectFixtures.length} auto-detection fixture inputs.`
+);

scripts/libs/highlight.js.sh

@@ -0,0 +1,35 @@
+# highlight.js — markup and language auto-detection fixtures across all 192
+# built-in grammars. The generated entry registers the source grammars and
+# embeds every fixture so neither engine needs a filesystem API at runtime.
+
+LIB_REPO="https://github.com/highlightjs/highlight.js.git"
+LIB_REF="11.11.2"
+LIB_ENTRY="test/jsse-entry.js"
+LIB_ESBUILD_PLATFORM="neutral"
+LIB_EXPECT_COUNT="731" # 536 markup + 195 auto-detection inputs (three grammars opt out)
+LIB_TIMEOUT="3600"
+
+lib_prepare() {
+  # The source core imports deep-freeze-es6. The distributed package normally
+  # bundles it, so install only that pinned dependency instead of the full
+  # build/test toolchain.
+  node -e "const p=require('./package.json'); p.devDependencies={}; p.scripts={}; require('fs').writeFileSync('package.json', JSON.stringify(p,null,2)+'\n')"
+  npm install --no-save --ignore-scripts --no-audit --no-fund deep-freeze-es6@3.0.2
+  node "$SCRIPT_DIR/gen-highlightjs-entry.js" "$LIB_ENTRY"
+}
+
+lib_verdict() {
+  local out="$1" rc="$2" line passed failed total
+  line="$(grep -oE 'HighlightJS: [0-9]+ passed, [0-9]+ failed, [0-9]+ total' "$out" | tail -1 || true)"
+  if [ -z "$line" ]; then echo "FAIL 0"; return 1; fi
+  if [[ "$line" =~ HighlightJS:\ ([0-9]+)\ passed,\ ([0-9]+)\ failed,\ ([0-9]+)\ total ]]; then
+    passed="${BASH_REMATCH[1]}"
+    failed="${BASH_REMATCH[2]}"
+    total="${BASH_REMATCH[3]}"
+    if [ "$rc" -eq 0 ] && [ "$failed" -eq 0 ] && [ "$passed" -eq "$total" ] && [ "$total" -gt 0 ]; then
+      echo "PASS $total"; return 0
+    fi
+    echo "FAIL $total"; return 1
+  fi
+  echo "FAIL 0"; return 1
+}