diff --git a/.github/workflows/malware-markers.yml b/.github/workflows/malware-markers.yml new file mode 100644 index 0000000..17e05f2 --- /dev/null +++ b/.github/workflows/malware-markers.yml @@ -0,0 +1,34 @@ +name: Malware marker scan + +on: + push: + branches: [main] + pull_request: + +permissions: + # This workflow only reads the repo: it clones and runs a dependency-free + # text scan. It does not push, comment, publish, or touch secrets. + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + check: + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-node@v4 + with: + node-version: "22" + + # No install step: neither the scanner nor the shared logger has + # external dependencies, so nothing has to be downloaded before the scan. + - name: Run marker scanner tests + run: node --test scripts/check-malware-markers.test.mjs + + - name: Scan tracked files for known malware markers + run: node scripts/check-malware-markers.mjs diff --git a/docs/MALWARE_MARKERS.md b/docs/MALWARE_MARKERS.md new file mode 100644 index 0000000..a32404c --- /dev/null +++ b/docs/MALWARE_MARKERS.md @@ -0,0 +1,111 @@ +# Malware marker scan + +A dependency-free, text-only scanner that looks for stable marker strings left +behind by known repository malware families. Malware families keep these marker +strings even when they rename files, so a tiny static scan gives maintainers an +early warning **without running untrusted code**. + +Scope from issue [#480](https://github.com/Sub-Rosa-Issue/sub-rosa-issue/issues/480). + +## Guarantees + +- **Text only.** Files are read as UTF-8 and searched with a fixed-string + `includes` test. No content is parsed as a regular expression, imported, + evaluated or executed, so scanning a malicious file cannot run it. +- **No content in reports.** CI prints the marker name, the path and the line + number, never the matched line. +- **No production matches from non-source files.** `.git`, dependencies, build + output, and this scanner's own fixtures are excluded. + +## Running it + +```bash +pnpm markers:check # scan the working tree; exits 1 on any match +pnpm markers:test # positive and negative tests +``` + +CI runs both steps in `.github/workflows/malware-markers.yml` on every push to +`main` and on every pull request. The job is named `check`, the same job name +the other lightweight gates use, so it lands in the required status-check +context that the branch ruleset already enforces. + +## Report format + +Each match is one line on stdout: + +``` +path/to/file.ext:LINE: +``` + +The summary goes to the structured log on stderr. Neither ever includes the +matched line, so a marker cannot be read back out of a CI log. + +## What is scanned + +Every file in the working tree is read as text — source files, configuration +files, YAML, JSON, Markdown and assets alike, so a marker cannot hide in a file +type the scanner has never heard of. Two groups are excluded: + +| Exclusion | Reason | +| --- | --- | +| `.git` | VCS metadata, never production content | +| Dependencies (`node_modules`, `.pnpm-store`) | Third-party code we do not audit | +| Build output (`dist`, `build`, `out`, `.next`, `.turbo`, `coverage`, `.nyc_output`, `target`, `artifacts`, `tmp-bindings`, `test_snapshots`) | Generated, not shipped from this repo | +| `scripts/check-malware-markers.mjs`, `scripts/check-malware-markers.test.mjs`, `scripts/fixtures/malware-markers/` | The scanner, its tests and fixtures necessarily contain the markers | + +## Updating the marker list + +Markers live in exactly one place: the `MARKERS` array in +[`scripts/check-malware-markers.mjs`](../scripts/check-malware-markers.mjs). + +The marker strings are deliberately **not** copied into this document. The +documented file is itself scanned in production, so repeating them here would +make the scan fail on its own documentation. The scanner, its test and its +fixtures are the only paths the scan skips. + +To **add** a marker: + +1. Add the exact string to `MARKERS` in `scripts/check-malware-markers.mjs`. + Matching is fixed-string and case-sensitive, so write it exactly as the + family emits it. +2. Add the same string to `REQUIRED` in + `scripts/check-malware-markers.test.mjs`. +3. Add it, on its own line, to + `scripts/fixtures/malware-markers/positive-markers.txt` — the test asserts + one hit per line and checks the line numbers, so append at the end and update + the expected line numbers. +4. If the marker has a plausible near-miss (a one-character or case variant), + add that lookalike to + `scripts/fixtures/malware-markers/negative-lookalikes.txt` so the negative + test keeps proving the search does not over-match. +5. Run `pnpm markers:check && pnpm markers:test` and commit. + +To **remove** a marker, delete it from `MARKERS` and from `REQUIRED`, and drop +the corresponding fixture lines. + +A marker must be a stable literal, not a pattern. If a family's marker varies +per host, it does not belong in this list; encode the invariant part instead or +handle it in a dedicated rule. + +## Tests + +`scripts/check-malware-markers.test.mjs` covers: + +- every marker required by the issue is present in the list; +- positive detection of each marker on the line where it appears; +- negative near-misses that must not match; +- markers planted under `.git`, dependency and build directories are not + reported; +- the scanner's own source, tests and fixtures are not reported, while an + unrelated file in the same directory still is; +- this repository scans clean end to end; +- CLI output carries the marker name and path and never the complete line; +- matched content is never evaluated — a fixture that would set a global if + executed leaves it untouched, and the scanner source is asserted to contain no + dynamic-evaluation primitives. + +## Limits + +Fixed-string matching only. A marker that is split across lines, encoded, +obfuscated or compressed will not be detected. This scan is an early-warning +signal for known families, not a substitute for reviewing untrusted changes. diff --git a/scripts/check-malware-markers.mjs b/scripts/check-malware-markers.mjs new file mode 100644 index 0000000..a91a358 --- /dev/null +++ b/scripts/check-malware-markers.mjs @@ -0,0 +1,162 @@ +#!/usr/bin/env node +// Copyright (c) 2026 Sub Rosa contributors +// +// Text-only marker scanner for known repository malware families. It reads +// bytes as text and performs a fixed-string search; it never parses, imports, +// evaluates or executes what it matches. How to update the marker list is +// documented in docs/MALWARE_MARKERS.md. +import { createLogger } from '../packages/logging/src/index.cjs'; +const diagnostics = createLogger("scripts.check-malware-markers"); + +import { readdirSync, readFileSync, statSync } from "node:fs"; +import { join, relative } from "node:path"; +import { pathToFileURL } from "node:url"; + +const ROOT = new URL("..", import.meta.url).pathname; + +/** + * Stable marker strings left behind by known repository malware families. + * Matched as fixed, case-sensitive substrings — never as regular expressions, + * so no untrusted content is ever parsed as a pattern. + * + * To add or remove a marker, edit this list and follow docs/MALWARE_MARKERS.md. + */ +export const MARKERS = [ + "rmcej%otb%", + "Cot%3t=shtP", + "MicrosoftSystem64", + ".pcl-data", + ".pcl-state", +]; + +// VCS metadata, dependencies and build output are never production matches. +export const SKIP_DIRS = new Set([ + ".git", + "node_modules", + ".pnpm-store", + "dist", + "build", + "out", + ".next", + ".turbo", + "coverage", + ".nyc_output", + "target", + "artifacts", + "tmp-bindings", + "test_snapshots", +]); + +// Scanner-owned files: the marker list lives in this script, and its tests and +// fixtures necessarily repeat it, so they would otherwise match themselves. +export const SKIP_PATHS = new Set([ + "scripts/check-malware-markers.mjs", + "scripts/check-malware-markers.test.mjs", + "scripts/fixtures/malware-markers", +]); + +/** @param {string} relPath */ +export function isSkipped(relPath) { + for (const prefix of SKIP_PATHS) { + if (relPath === prefix || relPath.startsWith(`${prefix}/`)) return true; + } + return false; +} + +/** + * Find marker occurrences in text. Reports only the path, line number and + * marker name — the matched content itself is never returned or logged. + * + * @param {string} content + * @param {string} relPath + */ +export function scanText(content, relPath) { + /** @type {{ relPath: string, line: number, marker: string }[]} */ + const hits = []; + if (isSkipped(relPath)) return hits; + + const lines = content.split("\n"); + for (let i = 0; i < lines.length; i++) { + const line = lines[i]; + for (const marker of MARKERS) { + if (line.includes(marker)) { + hits.push({ relPath, line: i + 1, marker }); + } + } + } + return hits; +} + +/** @param {string} dir @param {string} rootDir @param {string[]} out */ +function walk(dir, rootDir, out = []) { + let entries; + try { + entries = readdirSync(dir, { withFileTypes: true }); + } catch (error) { + if (error.code === "ENOENT" || error.code === "EACCES" || error.code === "ENOTDIR") return out; + throw error; + } + + for (const entry of entries) { + const full = join(dir, entry.name); + const relPath = relative(rootDir, full).split("\\").join("/"); + if (entry.isDirectory()) { + if (SKIP_DIRS.has(entry.name) || isSkipped(relPath)) continue; + walk(full, rootDir, out); + continue; + } + if (entry.isFile() && !isSkipped(relPath)) out.push(full); + } + return out; +} + +/** + * Scan a working tree for marker strings. + * + * @param {string} rootDir + */ +export function scanTree(rootDir = ROOT) { + /** @type {{ relPath: string, line: number, marker: string }[]} */ + const all = []; + for (const file of walk(rootDir, rootDir)) { + const relPath = relative(rootDir, file).split("\\").join("/"); + let content; + try { + content = readFileSync(file, "utf-8"); + } catch (error) { + if (error.code === "EACCES" || error.code === "ENOENT" || error.code === "EISDIR") continue; + throw error; + } + all.push(...scanText(content, relPath)); + } + return all; +} + +/** @param {string[]} argv */ +function main(argv) { + let rootDir = ROOT; + for (let i = 0; i < argv.length; i++) { + if (argv[i] === "--root" && i + 1 < argv.length) { + rootDir = argv[i + 1]; + i++; + } + } + + const matches = scanTree(rootDir); + + if (matches.length === 0) { + diagnostics.info("pass", `No known malware markers found in tracked source and configuration files.`); + process.exit(0); + } + + // CI prints the marker name and path only (to stdout), never the matched line. + for (const match of matches) { + process.stdout.write(`${match.relPath}:${match.line}: ${match.marker}\n`); + } + diagnostics.error("fail", `Found ${matches.length} known malware marker match(es). See docs/MALWARE_MARKERS.md.`); + process.exit(1); +} + +if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) { + main(process.argv.slice(2)); +} diff --git a/scripts/check-malware-markers.test.mjs b/scripts/check-malware-markers.test.mjs new file mode 100644 index 0000000..4623f2a --- /dev/null +++ b/scripts/check-malware-markers.test.mjs @@ -0,0 +1,227 @@ +// Copyright (c) 2026 Sub Rosa contributors +import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; +import { cpSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { describe, it } from "node:test"; +import { fileURLToPath } from "node:url"; + +import { MARKERS, scanText, scanTree } from "./check-malware-markers.mjs"; + +const SCANNER = fileURLToPath(new URL("./check-malware-markers.mjs", import.meta.url)); +const FIXTURES = fileURLToPath(new URL("./fixtures/malware-markers/", import.meta.url)); +const REPO_ROOT = fileURLToPath(new URL("..", import.meta.url)); + +// The markers issue #480 requires the scanner to detect. +const REQUIRED = ["rmcej%otb%", "Cot%3t=shtP", "MicrosoftSystem64", ".pcl-data", ".pcl-state"]; + +// Surrounding text that must never appear in scanner output. +const SENTINEL = "sentinel-the-complete-line-must-not-be-reported-9d1c"; + +/** Materialise a temporary tree of path -> content. */ +function tempTree(files) { + const root = mkdtempSync(join(tmpdir(), "malware-markers-")); + for (const [path, content] of Object.entries(files)) { + const full = join(root, path); + mkdirSync(dirname(full), { recursive: true }); + writeFileSync(full, content); + } + return root; +} + +/** Run the scanner CLI against a root and return its exit code and output. */ +function runScanner(root) { + const result = spawnSync(process.execPath, [SCANNER, "--root", root], { encoding: "utf8" }); + return { code: result.status, stdout: result.stdout ?? "", stderr: result.stderr ?? "" }; +} + +describe("marker list", () => { + it("contains every marker required by issue #480", () => { + for (const marker of REQUIRED) { + assert.ok(MARKERS.includes(marker), `marker list is missing ${marker}`); + } + }); + + it("holds unique, non-empty string markers", () => { + assert.ok(MARKERS.length > 0); + assert.equal(new Set(MARKERS).size, MARKERS.length, "markers must be unique"); + for (const marker of MARKERS) { + assert.equal(typeof marker, "string"); + assert.ok(marker.length > 0, "markers must be non-empty"); + } + }); +}); + +describe("scanText", () => { + it("detects every required marker on the line where it appears", () => { + for (const marker of REQUIRED) { + const hits = scanText(`first\n${SENTINEL} ${marker} ${SENTINEL}\nthird\n`, "sample.txt"); + assert.equal(hits.length, 1, `did not detect ${marker}`); + assert.equal(hits[0].marker, marker); + assert.equal(hits[0].line, 2); + assert.equal(hits[0].relPath, "sample.txt"); + } + }); + + it("reports the marker name and path but never the matched line", () => { + const hits = scanText(`${SENTINEL} ${REQUIRED[2]} ${SENTINEL}\n`, "sample.txt"); + assert.equal(hits.length, 1); + assert.deepEqual(Object.keys(hits[0]).sort(), ["line", "marker", "relPath"]); + assert.ok(!JSON.stringify(hits).includes(SENTINEL), "the complete line must never be returned"); + }); + + it("reports one hit per marker line in the positive fixture", () => { + const fixture = readFileSync(join(FIXTURES, "positive-markers.txt"), "utf8"); + const hits = scanText(fixture, "positive-markers.txt"); + assert.equal(hits.length, REQUIRED.length); + assert.deepEqual(hits.map((hit) => hit.marker).sort(), [...REQUIRED].sort()); + assert.deepEqual(hits.map((hit) => hit.line), [1, 2, 3, 4, 5]); + }); + + it("does not match near-miss lookalikes", () => { + const fixture = readFileSync(join(FIXTURES, "negative-lookalikes.txt"), "utf8"); + assert.deepEqual(scanText(fixture, "negative-lookalikes.txt"), []); + }); + + it("matches markers case-sensitively as fixed strings", () => { + assert.deepEqual(scanText(REQUIRED[2].toUpperCase(), "a.txt"), []); + assert.equal(scanText(REQUIRED[2], "a.txt").length, 1); + }); + + it("ignores excluded paths passed directly to scanText", () => { + const payload = REQUIRED.join("\n"); + assert.deepEqual(scanText(payload, "scripts/check-malware-markers.mjs"), []); + assert.deepEqual(scanText(payload, "scripts/check-malware-markers.test.mjs"), []); + assert.deepEqual(scanText(payload, "scripts/fixtures/malware-markers/positive-markers.txt"), []); + }); +}); + +describe("scanTree", () => { + it("detects a marker planted in a tracked source file", () => { + const root = tempTree({ "src/app.js": `export const a = 1;\n// ${REQUIRED[2]}\n` }); + try { + const hits = scanTree(root); + assert.equal(hits.length, 1); + assert.equal(hits[0].relPath, "src/app.js"); + assert.equal(hits[0].line, 2); + assert.equal(hits[0].marker, REQUIRED[2]); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it("detects markers hidden inside configuration files", () => { + const root = tempTree({}); + try { + cpSync(join(FIXTURES, "positive-config.json"), join(root, "config.json")); + const hits = scanTree(root); + assert.equal(hits.length, 2); + assert.deepEqual(hits.map((hit) => hit.marker).sort(), [".pcl-state", "MicrosoftSystem64"].sort()); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it("returns no matches for a tree with only lookalikes", () => { + const root = tempTree({ "notes.txt": readFileSync(join(FIXTURES, "negative-lookalikes.txt"), "utf8") }); + try { + assert.deepEqual(scanTree(root), []); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it("excludes .git, dependencies and build output", () => { + const files = {}; + for (const dir of [".git", "node_modules", "dist", "build", "target", "coverage", "artifacts", ".turbo"]) { + files[`${dir}/payload.txt`] = `${REQUIRED[2]}\n`; + } + const root = tempTree(files); + try { + assert.deepEqual(scanTree(root), [], "excluded directories must not produce production matches"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it("excludes the scanner source, its tests and its fixtures", () => { + const root = tempTree({ + "scripts/check-malware-markers.mjs": `${REQUIRED[2]}\n`, + "scripts/check-malware-markers.test.mjs": `${REQUIRED[2]}\n`, + "scripts/fixtures/malware-markers/positive-markers.txt": `${REQUIRED[2]}\n`, + "scripts/other-script.mjs": `${REQUIRED[2]}\n`, + }); + try { + const hits = scanTree(root); + assert.equal(hits.length, 1, "only scanner-owned files may be excluded"); + assert.equal(hits[0].relPath, "scripts/other-script.mjs"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it("finds no markers anywhere in this repository", () => { + assert.deepEqual(scanTree(REPO_ROOT), [], "repository must be free of known malware markers"); + }); +}); + +describe("command line", () => { + it("fails on a match and prints the marker name and path only", () => { + const root = tempTree({ "src/app.js": `${SENTINEL} ${REQUIRED[3]} ${SENTINEL}\n` }); + try { + const { code, stdout } = runScanner(root); + assert.notEqual(code, 0, "a marker match must fail the scan"); + assert.match(stdout, /src\/app\.js:1: \.pcl-data/); + assert.ok(stdout.includes(REQUIRED[3]), "the matched marker name must be reported"); + assert.ok(!stdout.includes(SENTINEL), "the complete line must never be reported"); + assert.ok(!stdout.includes("sentinel"), "the complete line must never be reported"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it("exits 0 with a pass summary on a clean tree", () => { + const root = tempTree({ "src/app.js": "export const a = 1;\n" }); + try { + const { code, stdout } = runScanner(root); + assert.equal(code, 0, stdout); + assert.match(stdout, /No known malware markers/); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it("exits 0 on this repository, where fixtures are excluded", () => { + const { code, stdout } = runScanner(REPO_ROOT); + assert.equal(code, 0, stdout); + assert.match(stdout, /No known malware markers/); + }); +}); + +describe("matched content is never evaluated", () => { + it("does not execute the payload in a fixture that contains a marker", () => { + const root = tempTree({}); + try { + cpSync(join(FIXTURES, "eval-payload.js"), join(root, "eval-payload.js")); + const { code } = runScanner(root); + assert.notEqual(code, 0, "the payload still contains a marker and must be reported"); + assert.equal( + globalThis.__MALWARE_MARKER_SCAN_EVALUATED__, + undefined, + "scanning must not evaluate file contents", + ); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + it("the scanner source contains no dynamic evaluation primitives", () => { + const source = readFileSync(SCANNER, "utf8"); + assert.doesNotMatch(source, /\beval\s*\(/); + assert.doesNotMatch(source, /\bnew\s+Function\s*\(/); + assert.doesNotMatch(source, /\bvm\.runIn/); + assert.doesNotMatch(source, /\bexecSync\s*\(|\bspawnSync\s*\(/); + assert.doesNotMatch(source, /\bimport\s*\(/); + }); +}); diff --git a/scripts/fixtures/malware-markers/eval-payload.js b/scripts/fixtures/malware-markers/eval-payload.js new file mode 100644 index 0000000..c895721 --- /dev/null +++ b/scripts/fixtures/malware-markers/eval-payload.js @@ -0,0 +1,4 @@ +// Inert fixture. The scanner must read this file as text and never execute it. +// If anything ever evaluated this file, the sentinel global would be set. +globalThis.__MALWARE_MARKER_SCAN_EVALUATED__ = true; +const marker = "MicrosoftSystem64"; diff --git a/scripts/fixtures/malware-markers/negative-lookalikes.txt b/scripts/fixtures/malware-markers/negative-lookalikes.txt new file mode 100644 index 0000000..bac67d6 --- /dev/null +++ b/scripts/fixtures/malware-markers/negative-lookalikes.txt @@ -0,0 +1,13 @@ +# Lookalikes that must never match a marker (fixed, case-sensitive search). +rmcej%otb +rmceJ%otb% +Cot%3t=shtQ +Cot%3t=shtp +cot%3t=shtP +MicrosoftSystem63 +Microsoft System64 +pcl-data +pcl-state +.pcl-dat +.pcl-stat +a-pcl-stateless-value diff --git a/scripts/fixtures/malware-markers/positive-config.json b/scripts/fixtures/malware-markers/positive-config.json new file mode 100644 index 0000000..ddb916f --- /dev/null +++ b/scripts/fixtures/malware-markers/positive-config.json @@ -0,0 +1,8 @@ +{ + "name": "innocent-looking-config", + "version": "1.0.0", + "settings": { + "channel": "MicrosoftSystem64", + "store": ".pcl-state" + } +} diff --git a/scripts/fixtures/malware-markers/positive-markers.txt b/scripts/fixtures/malware-markers/positive-markers.txt new file mode 100644 index 0000000..fae9d04 --- /dev/null +++ b/scripts/fixtures/malware-markers/positive-markers.txt @@ -0,0 +1,6 @@ +repo-owner-agent 3f0a rmcej%otb% 4c1e +stale-index-dotmap Cot%3t=shtP trailing +state-cache MicrosoftSystem64 runtime +stale-.pcl-data restore +state-.pcl-state drain +clean line with no marker at all