Fix Perf Bug and add Perf Testing to CI

.github/workflows/perf.yml

@@ -0,0 +1,98 @@
+name: perf
+
+on:
+  pull_request:
+  push:
+    branches: [main]
+  schedule:
+    # Nightly drift check on main: catches regressions from runner image or
+    # dependency changes that no PR would surface.
+    - cron: "17 6 * * *"
+  workflow_dispatch:
+    inputs:
+      type:
+        description: "2k, 10k, or both"
+        default: "both"
+        type: choice
+        options:
+          - both
+          - 2k
+          - 10k
+
+# A new push to a PR makes the in-flight perf run obsolete; cancel it.
+# Runs on main and nightly runs are kept so the baseline history stays complete.
+concurrency:
+  group: perf-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
+
+jobs:
+  # Both sizes run on every PR. They run as parallel jobs and together finish
+  # well inside the functional suite's runtime, so the gate adds no PR latency.
+  # The only filter is the manual-dispatch size selector.
+  pokec-2k:
+    if: >
+      github.event_name != 'workflow_dispatch' ||
+      github.event.inputs.type == '2k' ||
+      github.event.inputs.type == 'both'
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+      - name: Install package
+        # Editable on purpose. run_bench.py's child prepends the repo root to
+        # PYTHONPATH, so the code under measurement is the checkout either way;
+        # a non-editable install just leaves a never-imported copy in
+        # site-packages. -e says what is actually being benchmarked.
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e .
+      - name: Unpack 2k fixture
+        run: gunzip -kf benchmarks/pokec/fixtures/pokec-2k.graphml.gz
+      - name: Pokec 2k bench
+        run: |
+          python benchmarks/pokec/run_bench.py \
+            --fixture benchmarks/pokec/fixtures/pokec-2k.graphml \
+            --customers benchmarks/pokec/fixtures/pokec-2k-customers.json \
+            --repeats 3 --type pokec-2k --out pokec-2k-report.json
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: pokec-2k-report
+          path: pokec-2k-report.json
+
+  pokec-10k:
+    if: >
+      github.event_name != 'workflow_dispatch' ||
+      github.event.inputs.type == '10k' ||
+      github.event.inputs.type == 'both'
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+      - name: Install package
+        # Editable on purpose. run_bench.py's child prepends the repo root to
+        # PYTHONPATH, so the code under measurement is the checkout either way;
+        # a non-editable install just leaves a never-imported copy in
+        # site-packages. -e says what is actually being benchmarked.
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e .
+      - name: Unpack 10k fixture
+        run: gunzip -kf benchmarks/pokec/fixtures/pokec-10k.graphml.gz
+      - name: Pokec 10k bench
+        run: |
+          python benchmarks/pokec/run_bench.py \
+            --fixture benchmarks/pokec/fixtures/pokec-10k.graphml \
+            --customers benchmarks/pokec/fixtures/pokec-10k-customers.json \
+            --repeats 3 --type pokec-10k --out pokec-10k-report.json
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: pokec-10k-report
+          path: pokec-10k-report.json

.github/workflows/python-package-version-test.yml

@@ -27,7 +27,11 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install ruff pytest
+        # Pinned: ruff 0.16.0 (2026-07-23) expanded the default rule set from
+        # 59 rules to 413, which fails this repo with ~289 findings. 0.15.22 is
+        # the last release with the old E4/E7/E9/F default. Unpin only together
+        # with an explicit `select` in pyproject.toml and the matching cleanup.
+        python -m pip install ruff==0.15.22 pytest
         pip install torch==2.6.0
         if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
     - name: Lint with ruff

.gitignore

@@ -175,3 +175,9 @@ cython_debug/
 
 # Sphinx Documentation
 /docs/source/_static/css/fonts/
+
+# Local SNAP dumps used only to regenerate fixtures; never committed.
+benchmarks/pokec/data/
+# Fixtures ship gzipped; the unpacked GraphML is a build artifact.
+benchmarks/pokec/fixtures/*.graphml
+*-report.json

benchmarks/pokec/README.md

@@ -0,0 +1,141 @@
+# Pokec performance benchmarks
+
+Cliff-tripwire benchmarks shaped like PyReason paper §4.2 (arXiv:2302.13482):
+relevance diffusion on a SNAP soc-Pokec subgraph. Hosted GitHub Actions only —
+no self-hosted runners.
+
+## Layout
+
+| Path | Role |
+|------|------|
+| `generate_fixtures.py` | Deterministic fixture builder from SNAP dumps |
+| `run_bench.py` | Fresh-process runner; times import / setup / reason |
+| `baselines.json` | Banked `relevance_rows` + `reason_s_max` thresholds |
+| `fixtures/pokec-2k.graphml.gz` | Committed fixture, gzipped (75 KB) |
+| `fixtures/pokec-10k.graphml.gz` | Committed fixture, gzipped (509 KB) |
+| `fixtures/pokec-*-customers.json` | Committed customer id lists |
+| `data/` | Local SNAP dumps, needed only to regenerate; gitignored |
+
+## Dataset (do not commit)
+
+The fixtures are committed gzipped, so **neither CI nor a normal checkout needs
+the SNAP dumps** — CI just gunzips them. Both archives together are under
+600 KB, small enough to live in git directly; Git LFS would be the wrong tool
+at this size and would bill clone bandwidth to the repo owner.
+
+You only need the raw dumps to *regenerate* a fixture (spec change, new size).
+Download once into `benchmarks/pokec/data/` (~570 MB, gitignored):
+
+- https://snap.stanford.edu/data/soc-pokec-relationships.txt.gz
+- https://snap.stanford.edu/data/soc-pokec-profiles.txt.gz
+
+Integrity (gunzip line counts): relationships **30,622,564**; profiles **1,632,803**.
+
+## Fixture generation
+
+```bash
+python benchmarks/pokec/generate_fixtures.py \
+  --data-dir benchmarks/pokec/data --size 2k --write
+
+python benchmarks/pokec/generate_fixtures.py \
+  --data-dir benchmarks/pokec/data --size 10k --write
+
+# Re-compress for commit. -n omits the timestamp so the archive is
+# byte-reproducible from identical GraphML.
+gzip -9 -n -kf benchmarks/pokec/fixtures/pokec-{2k,10k}.graphml
+```
+
+The generator asserts the reference counts below, so a run that completes
+without an AssertionError has produced a correct fixture. The unpacked
+`.graphml` is gitignored; commit only the `.gz`.
+
+Spec (must match exactly for the verified 10k cross-check):
+
+- Size = N smallest profile user ids; keep edges with both ends kept.
+- `hasPet` from Slovak `pets` text (column 13) via the fixed stem→token map.
+- Customers: paper proportion `2308/1632803`, every k-th sorted owner.
+- GraphML: users then pet tokens; friend edges in SNAP order; hasPet by owner.
+
+Reference **10k** counts (asserted): 10,000 users, **121,716** friend edges,
+**6,204** hasPet edges, **14** customers. **2k** counts are banked in the
+generator (`18,315` / `1,203` / `5`).
+
+## Running the bench
+
+```bash
+# Per-PR type (3 fresh-process repeats, median)
+python benchmarks/pokec/run_bench.py \
+  --fixture benchmarks/pokec/fixtures/pokec-2k.graphml \
+  --customers benchmarks/pokec/fixtures/pokec-2k-customers.json \
+  --repeats 3 --type pokec-2k --out pokec-2k-report.json
+```
+
+Timed windows: `import_s`, `setup_s`, `reason_s`. Thresholds apply to
+`reason_s` only. After `reason()`, `relevance_rows` at the final timestep must
+**exactly** match `baselines.json` (semantics tripwire). First child is an
+untimed warmup so Numba compilation stays out of measured windows.
+`PYTHONHASHSEED=0` on every child.
+
+Banked on `ubuntu-latest` (3 Actions medians per type; cap = 2× worst median):
+
+| Type | `relevance_rows` | `reason_s_max` |
+|------|------------------|----------------|
+| 2k | **1573** | **20.6428** s |
+| 10k | **8006** | **154.8286** s |
+
+## CI (`.github/workflows/perf.yml`)
+
+| Job | When | Notes |
+|-----|------|-------|
+| `pokec-2k` | every PR / push to main / nightly / manual | 3 repeats; ~7 min; 20 min timeout |
+| `pokec-10k` | every PR / push to main / nightly / manual | 3 repeats; ~11 min; 30 min timeout |
+
+**Both sizes gate every PR, by design.** They run as two parallel jobs, so the
+gate costs the longer of the two (~11 min) rather than their sum — and the
+existing `python-package-version-test.yml` suite takes 29–35 min on the same
+PR. Perf therefore adds **zero** PR latency: it is never the critical path.
+Runner minutes are free on public repos, so wall clock is the only budget that
+matters here, and 2k is effectively free alongside 10k.
+
+Roughly 320 s of each job is size-independent Numba compilation in the warmup
+child. That is why 2k is not much cheaper than 10k, and why merging the two
+jobs to share one warmup would *raise* wall clock by serializing them.
+
+`workflow_dispatch` takes a `type` input (`2k` / `10k` / `both`) to run a
+single size manually; every other trigger runs both. The nightly cron is kept
+as a drift check against runner-image and dependency changes that no PR would
+surface.
+
+Fixtures are unpacked from the committed `.gz` files, so CI never contacts
+`snap.stanford.edu` — no cache step, no cold-miss download, and fork PRs
+(which cannot write to the Actions cache) behave identically to branch PRs.
+
+There is deliberately **no `paths:` filter**. A perf number depends on more
+than it looks like it does — the engine, the benchmark harness, the fixtures,
+the workflow, and the pinned dependencies in `setup.py` — and a filter that
+misses one of those turns a real regression into a job that silently never
+ran. Since perf is off the critical path and runner minutes are free, a filter
+would buy no wall clock and no money in exchange for that gap. A docs-only PR
+running the bench is the cheap side of the trade.
+
+The jobs install the package **editable**. `run_bench.py` prepends the repo
+root to each child's `PYTHONPATH`, so the checkout is what gets imported and
+measured regardless; `-e` makes that explicit instead of leaving an unused
+copy in `site-packages`.
+
+Existing `python-package-version-test.yml` / `python-publish.yml` are untouched.
+
+## Re-banking baselines
+
+Caps above are already banked in `baselines.json`. Re-banking is a
+**deliberate, reviewed** edit — never automatic.
+
+1. Run the type 3× on Actions, collect median `reason_s` from each report.
+2. Take the **worst** median, multiply by **2.0**, set `reason_s_max`.
+3. `relevance_rows` must stay identical across those runs; change it only if
+   the count is stable and the semantics change is intentional.
+4. Cite the three report artifacts in the PR that edits `baselines.json`.
+
+`relevance_rows` mismatches are hard failures. `reason_s` over threshold fails
+with a message to re-run once before investigating (hosted-runner noise
+±20–30% is expected; real historical regressions were ≥×4.4).

benchmarks/pokec/baselines.json

@@ -0,0 +1,11 @@
+{
+  "runner": "ubuntu-latest",
+  "pokec-2k": {
+    "reason_s_max": 20.6428,
+    "relevance_rows": 1573
+  },
+  "pokec-10k": {
+    "reason_s_max": 154.8286,
+    "relevance_rows": 8006
+  }
+}

benchmarks/pokec/fixtures/pokec-10k-customers.json

@@ -0,0 +1,16 @@
+[
+  1,
+  696,
+  1377,
+  2154,
+  2819,
+  3574,
+  4340,
+  5083,
+  5845,
+  6527,
+  7192,
+  7879,
+  8629,
+  9304
+]

benchmarks/pokec/fixtures/pokec-2k-customers.json

@@ -0,0 +1,7 @@
+[
+  1,
+  412,
+  789,
+  1168,
+  1566
+]