A gated research harness: the scaffolding around a research project, with the project removed. It exists to make the expensive mistakes — an uncommitted result, a submission nobody can map back to code, a metric that a surrogate computation quietly manufactured — fail at commit or submit time instead of in a paper.
Marked as a GitHub template repository: Use this template (or the git clone
below) starts a project with the harness already wired.
What you get:
- Commit gates (
framework/hooks/pre-commit, blocking): code-size limits, oracle-graded test mirrors, per-file coverage clearance, docstring contracts, duplication, a single parameter registry, mutation evidence, surrogate-distortion anti-patterns, static types over the staged blob, device transfers, memory bottlenecks, instruction references, the pinned acceptance battery, an sbatch provenance contract, and index freshness. Two advisory censuses (hot path, abstract cost) print findings without ever blocking. - Ref gates (
framework/hooks/reference-transaction,framework/hooks/pre-merge-commit): a ref move lands only when the new commit's tree was armed by the battery that graded it, so--no-verify, a rewritten branch and an ungated merge are refused rather than discovered later. - Nothing that can only shrink is allowed to grow: the mirror-alias table (MIR003), the loop-waiver table (COV005) and the duplication baseline (CPD002) are ratchets — an addition is refused, a removal passes.
- Tests that cannot be theater: a mirror must parse to behavioral assertions and assert against something other than itself (HNS068/HNS070), every staged test file is graded the same way, and mutation evidence is stamped with the digest of the test surface it was scored against, so a verdict cached against older tests can neither be recorded nor believed.
- Submit gates (
framework/gates/sbatch_gate.sh, strict by default): dirty tree, contract, execution paths,local_gate, stub smoke, GPU canary,sbatch --test-only. - One place for every project token (
framework/harness.py): slug, env-var prefix, gate-code prefix, source/test roots, interpreter, cluster names, cache and gate-state directories. - A worked example (
research/params/,research/probe/,experiments/example_plan/,scripts/smoke/plan_run_smoke.sh) that clears every gate, including mutation evidence.
| Required | Python ≥ 3.11, git. The floor is declared once (PYTHON_FLOOR, requires-python) and every hook, smoke driver and submit gate refuses below it. |
| Hook tooling | Declared once in pyproject.toml ([project.optional-dependencies]): pip install -e ".[dev]" pulls pytest, coverage, mutmut, ruff, rope, pyflakes, vulture, detect-secrets, mypy, pyright. |
| Optional | dvc and mlflow (pip install -e ".[docs]") for the documentation-chain gate; without them the gate refuses with the install hint instead of passing vacuously. The advisory tools degrade quietly: the duplication gate reports a note when the jscpd binary (npm) is absent, and the pre-push steps for vulture/detect-secrets/lint-imports run only when those binaries exist. |
| Not needed | A GPU, a scheduler, or any network service to run the harness itself. |
git clone https://github.com/MaBarov/research-template my-project && cd my-project
# One command: venv at the floor, the dev extra, hooks, wiring check, example suite.
python scripts/setup/bootstrap.py # add --dry-run to see the plan first
.venv/bin/python -m framework.indexing.cli --output INDEX.mdThe bootstrap resolves its interpreter as .venv, then --interpreter, then
python3.13…python3 on PATH, and refuses anything below the floor. To drive
the steps yourself:
python3 -m venv .venv # 3.11+; use python3.11/3.12 if your python3 is older
.venv/bin/python -m pip install -e ".[dev]"
python scripts/setup/setup_framework.py # core.hooksPath = framework/hooks
python scripts/setup/check_framework_wiring.py
.venv/bin/python -m pytest tests -q # the example suiteEvery project token lives in framework/harness.py. The rename is one command,
which refuses a dirty worktree or an occupied target, moves the source tree and
its mirror, rewrites the tokens in every tracked text file, then fails unless no
old token survived:
python scripts/setup/init_project.py --slug myproj --prefix MYPROJ --code MYX [--dry-run]
python framework/harness.py --get slug # verify the new identityADOPTING.md documents the identity keys, the manual steps (pyproject.toml
tables, [tool.<slug>.contracts]) and the ledgers the gates refresh themselves.
python -m pytest framework/tests tests -q # 829 tests: gate contracts + example suite
bash framework/hooks/pre-commit # the commit battery, on the staged tree
bash framework/gates/sbatch_gate.sh slurm/template.sbatch --dry-runOn a checkout without .venv the hook needs the floor interpreter explicitly:
RESEARCH_PYTHON=/path/to/python3.11 bash framework/hooks/pre-commit| Gate | Blocks | Command |
|---|---|---|
| Antipatterns | commit | python framework/gates/check_antipatterns.py --strict FILE... |
| Sbatch contract | commit, submit | bash framework/gates/check_sbatch_contract.sh slurm/job.sbatch |
| Test mirrors | commit | python framework/gates/checks/check_test_mirror.py --strict --staged FILE... |
| Coverage | commit | python framework/gates/check_coverage.py --mode staged --staged FILE... |
| Code size | commit | python framework/gates/limits/check_limits.py --mode all |
| Docstring contracts | commit | python -m framework.indexing.cli --strict-contracts --staged-files FILE... |
| Duplication | commit | python framework/gates/checks/check_duplication.py --strict --staged FILE... |
| Parameters | commit | python framework/gates/params/check_param_duplicates.py --strict FILE... |
| Mutation evidence | commit | python framework/gates/mutation/check_mutation.py --mode staged --staged FILE... |
| Distortion | commit | python framework/gates/distortion/check_distortion.py --strict --staged FILE... |
| Static types | commit | python framework/gates/typecheck/check_types.py --mode staged --staged --strict FILE... |
| Instruction refs | commit | python framework/gates/instructions/check_instruction_refs.py --strict --staged AGENTS.md |
| Ref provenance | any ref move | framework/gates/freshness/ref_guard.py --state prepared (fed by git's reference-transaction hook) |
| Pinned battery | commit | python -m pytest -m pinned -q framework/tests tests |
| Device transfers | commit | python framework/gates/transfers/check_device_transfers.py --mode staged --staged --strict FILE... |
| Memory bottlenecks | commit | python framework/gates/memory/check_memory_bottlenecks.py --mode staged --staged --strict FILE... |
| Hot path (advisory) | never (warns) | python framework/gates/hotpath/check_hotpath.py --mode all |
| Abstract cost (advisory) | never (warns) | python framework/gates/abstract_cost/check_abstract_cost.py --mode all |
| Index freshness | commit | python -m framework.indexing.cli --output INDEX.md --check |
| Submit gate | submit | bash framework/gates/sbatch_gate.sh slurm/job.sbatch |
Every gate is fail-closed and none of them takes an override flag. Ledgers under
framework/ (coverage_evidence.json, mutation_evidence.json,
loop_waivers.json, jscpd-baseline.json) record evidence, not exemptions:
they are refreshed by re-running the gate, never edited by hand.
cp slurm/template.sbatch slurm/my_job.sbatch # keep the provenance guard
$EDITOR experiments/my_run/run.py # the payload
$EDITOR scripts/smoke/run_smoke.sh # its smoke driver
bash framework/gates/sbatch_gate.sh slurm/my_job.sbatch --dry-run
bash framework/gates/sbatch_gate.sh slurm/my_job.sbatch ARG...The gate refuses a dirty tree at submit time and the job refuses again at runtime; a run always carries the commit it executed.
Smoke drivers resolve their interpreter through
scripts/smoke/lib/interpreter.sh: SMOKE_PYTHON, then the checkout's
.venv, then framework/harness.py, then python3.13…python3. An
interpreter below the declared floor (3.11) is refused — a smoke that passes on
an unsupported Python is a false signal — so on a bare host run
SMOKE_PYTHON=/path/to/python3.11 bash scripts/smoke/<name>_smoke.sh.
framework/ harness.py, gates/{antipattern,checks,coverage,distortion,
freshness,limits,mutation,params,provenance}, hooks/, indexing/
research/ your package (worked example: params/, probe/)
tests/research/ mirror tests, one per source module
scripts/ entry points, setup/, slurm_queue/, smoke/
slurm/ job scripts; template.sbatch is the one to copy
experiments/ run entry points grouped by experiment
third_party/ pinned vendored trees (never edited in place)
Not a framework, not a pipeline runner, and not a dataset or model registry: it
is the policy layer — what must be true before a result is allowed to count —
plus the small amount of machinery that enforces it. Research code goes in
research/; the harness only asks it to be sized, tested, mirrored, contracted
and submitted honestly.