Preserve experiment artifact hashes with offline verifier and CI

.github/workflows/evidence-integrity.yml

@@ -0,0 +1,23 @@
+name: Offline evidence integrity
+
+on:
+  pull_request:
+  push:
+    branches: [master]
+
+permissions:
+  contents: read
+
+jobs:
+  artifact-contract:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - name: Compile offline evidence utility
+        run: python -m py_compile tools/evidence_manifest.py
+      - name: Verify synthetic artifact creation and tamper detection
+        run: python -m unittest discover -s tests -v

README.md

@@ -266,6 +266,28 @@ This is retained as a historical record of an interpretation that was later with
 
 Where available, raw chronological evidence is preferable to prose verdicts because it preserves the evolution of the hypotheses.
 
+## Offline artifact integrity
+
+A small **standard-library-only** helper, `tools/evidence_manifest.py`, creates an explicit SHA-256 manifest for files collected in an authorized, local experiment. It does **not** send traffic, collect packets, or infer which network component produced an observation.
+
+From the directory containing the artifact paths:
+
+```bash
+python tools/evidence_manifest.py \
+  --experiment controlled-path-001 \
+  --artifact captures/control.pcap \
+  --artifact captures/stimulus.pcap \
+  --out run-manifest.json
+
+python tools/evidence_manifest.py --verify run-manifest.json
+```
+
+The example presumes those two capture files already exist. The program refuses missing, non-regular, absolute, and outside-directory artifacts and will not overwrite an existing manifest. It stores each file's relative path, size, SHA-256, timestamp of manifest creation, and an **analyst-supplied**, explicitly non-validated hypothesis status. The default status is `unverified`.
+
+A successful `--verify` checks whether the files still match their recorded bytes. It **does not establish source authenticity, capture completeness, clock accuracy, attribution, causal interpretation, or legal chain of custody**. The artifact schema covers integrity, not yet the full measurement-record fields listed below.
+
+CI runs synthetic creation, tampering, and invalid-path checks completely offline. It does not execute the raw-packet research scripts or run against live middleboxes.
+
 ## Requirements
 
 The scripts primarily use:
@@ -347,7 +369,7 @@ Change one independent variable at a time whenever possible.
 - Absence of a reset is not equivalent to successful application-layer communication.
 - The tools do not yet share one normalized experiment schema.
 - There is no automated replay harness against a controlled middlebox matrix.
-- There is no current CI pipeline or hardware-independent test suite.
+- CI now covers the standalone artifact-integrity helper; network experiments still lack hardware-independent regression coverage.
 - Historical filenames and comments can still reflect the exploratory language used during development.
 
 ## Intended use

tests/test_evidence_manifest.py

@@ -0,0 +1,55 @@
+"""Offline artifact-integrity tests. No packets are transmitted or captured."""
+import hashlib
+import json
+from pathlib import Path
+import tempfile
+import unittest
+
+from tools.evidence_manifest import build_manifest, verify_manifest
+
+
+class EvidenceManifestTests(unittest.TestCase):
+    def test_create_and_verify_local_evidence(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            base = Path(temporary)
+            (base / "control.bin").write_bytes(b"controlled synthetic capture")
+            manifest = build_manifest("controlled-lab-001", ["control.bin"], base)
+            self.assertEqual(manifest["analyst_status"], "unverified")
+            self.assertEqual(manifest["status_basis"], "analyst_supplied_not_automatically_validated")
+            self.assertEqual(manifest["artifacts"][0]["sha256"], hashlib.sha256(b"controlled synthetic capture").hexdigest())
+            self.assertEqual(verify_manifest(manifest, base), 1)
+            self.assertEqual(json.loads(json.dumps(manifest))["experiment_id"], "controlled-lab-001")
+
+    def test_tampered_artifact_fails(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            base = Path(temporary)
+            path = base / "test.bin"
+            path.write_bytes(b"before")
+            record = build_manifest("test-tamper", ["test.bin"], base)
+            path.write_bytes(b"after")
+            with self.assertRaisesRegex(ValueError, "Evidence mismatch"):
+                verify_manifest(record, base)
+
+    def test_missing_or_outside_artifact_rejected(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            base = Path(temporary)
+            for name in ("missing.bin", "../other.bin", "/tmp/example.bin"):
+                with self.subTest(name=name):
+                    with self.assertRaises(ValueError):
+                        build_manifest("test-paths", [name], base)
+
+    def test_duplicate_paths_rejected(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            base = Path(temporary)
+            (base / "x.bin").write_bytes(b"x")
+            with self.assertRaisesRegex(ValueError, "Duplicate"):
+                build_manifest("test-duplicate", ["x.bin", "x.bin"], base)
+
+    def test_wrong_manifest_version_is_not_silently_accepted(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            with self.assertRaisesRegex(ValueError, "Unknown"):
+                verify_manifest({"format": "unknown", "artifacts": []}, Path(temporary))
+
+
+if __name__ == "__main__":
+    unittest.main()

tools/evidence_manifest.py

@@ -0,0 +1,115 @@
+#!/usr/bin/env python3
+"""Create and verify a local artifact manifest; never performs network probes.
+
+The analyst supplies the experiment status. A matching hash does not authenticate
+the capture source, establish a chain of custody, or validate an interpretation.
+"""
+import argparse
+from datetime import datetime, timezone
+import hashlib
+import json
+from pathlib import Path
+import sys
+
+FORMAT = "northguard.reverse-observer.artifacts.v1"
+STATUSES = ("unverified", "observed", "falsified_under_tested_conditions", "structurally_invalid")
+
+
+def safe_artifact(base: Path, name: str) -> Path:
+    rel = Path(name)
+    if not name or rel.is_absolute() or ".." in rel.parts or rel == Path("."):
+        raise ValueError(f"Artifact path must be relative and remain inside the working directory: {name!r}")
+    full = base / rel
+    if full.is_symlink() or not full.is_file():
+        raise ValueError(f"Artifact must be an existing regular, non-symlink file: {name!r}")
+    # Also reject a symlinked parent directory, even when the file itself is regular.
+    if not full.resolve().is_relative_to(base.resolve()):
+        raise ValueError(f"Artifact escapes the working directory: {name!r}")
+    return full
+
+
+def digest(path: Path) -> tuple[str, int]:
+    h = hashlib.sha256()
+    count = 0
+    with path.open("rb") as stream:
+        for chunk in iter(lambda: stream.read(1024 * 1024), b""):
+            h.update(chunk)
+            count += len(chunk)
+    return h.hexdigest(), count
+
+
+def build_manifest(experiment_id: str, artifacts: list[str], base: Path, status: str = "unverified") -> dict:
+    if not experiment_id.strip() or not artifacts:
+        raise ValueError("A non-empty experiment ID and at least one artifact are required")
+    if status not in STATUSES:
+        raise ValueError(f"Unsupported analyst status: {status}")
+    if len(artifacts) != len(set(artifacts)):
+        raise ValueError("Duplicate artifact paths are not permitted")
+
+    entries = []
+    for name in sorted(artifacts):
+        path = safe_artifact(base, name)
+        checksum, size = digest(path)
+        entries.append({"path": Path(name).as_posix(), "sha256": checksum, "bytes": size})
+
+    return {
+        "format": FORMAT,
+        "experiment_id": experiment_id,
+        "created_at_utc": datetime.now(timezone.utc).isoformat(),
+        "analyst_status": status,
+        "status_basis": "analyst_supplied_not_automatically_validated",
+        "artifacts": entries,
+    }
+
+
+def verify_manifest(manifest: dict, base: Path) -> int:
+    if manifest.get("format") != FORMAT:
+        raise ValueError("Unknown artifact manifest format")
+    artifacts = manifest.get("artifacts")
+    if not isinstance(artifacts, list) or not artifacts:
+        raise ValueError("Manifest must contain artifacts")
+    seen = set()
+    for item in artifacts:
+        if not isinstance(item, dict) or not isinstance(item.get("path"), str):
+            raise ValueError("Malformed artifact entry")
+        name = item["path"]
+        if name in seen:
+            raise ValueError("Duplicate artifact in manifest")
+        seen.add(name)
+        actual_hash, actual_bytes = digest(safe_artifact(base, name))
+        if actual_hash != item.get("sha256") or actual_bytes != item.get("bytes"):
+            raise ValueError(f"Evidence mismatch: {name}")
+    return len(artifacts)
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Offline artifact integrity records for authorized lab measurements")
+    parser.add_argument("--experiment", help="Analyst-defined experiment identifier")
+    parser.add_argument("--artifact", action="append", default=[], help="Artifact path relative to current directory (repeatable)")
+    parser.add_argument("--status", choices=STATUSES, default="unverified", help="Analyst-supplied hypothesis status")
+    parser.add_argument("--out", type=Path, help="New manifest path; never overwritten")
+    parser.add_argument("--verify", type=Path, help="Verify an existing manifest against its local artifacts")
+    args = parser.parse_args()
+    try:
+        if args.verify:
+            if args.experiment or args.artifact or args.out:
+                parser.error("--verify must be used alone")
+            manifest = json.loads(args.verify.read_text(encoding="utf-8"))
+            count = verify_manifest(manifest, Path.cwd())
+            print(f"ARTIFACT_HASHES_MATCH: {count} file(s); interpretation not verified")
+        else:
+            if not args.experiment or not args.out:
+                parser.error("creation requires --experiment, --artifact, and --out")
+            result = build_manifest(args.experiment, args.artifact, Path.cwd(), args.status)
+            with args.out.open("x", encoding="utf-8") as output:
+                json.dump(result, output, indent=2)
+                output.write("\n")
+            print(f"ARTIFACT_MANIFEST_CREATED: {args.out}")
+        return 0
+    except (OSError, ValueError, KeyError, json.JSONDecodeError) as exc:
+        print(f"ARTIFACT_MANIFEST_ERROR: {exc}", file=sys.stderr)
+        return 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())