Skip to content

MEAS-001: granular resource receipts (RSS/VRAM/CPU/IO/faults/energy) for bench, CRUX, dogfood, CI ledger #4522

Open

Description

@noahgift

Operator (2026-09-27): our instrumentation must show not only timing but granular resource usage.

Audit (cop, 2026-09-27): today every receipt we have is timing-only.

  • apr bench --json (crates/apr-cli/src/commands/bench.rs:356-395) emits no resource fields.
  • Build ledger peak_rss_mb is null in 1084 of 1095 receipts, and free_disk_gb is null too.
  • The serve-bench MemoryResults/EnergyResults hold estimates or constants.
  • CRUX D-03 --report-peak-memory is a draft.
  • CiAssertions.max_memory_mb is dead code.
  • The dogfood C1 RSS check only WARNs.
  • No resource number is gated.

Rows. Every row is measured, not estimated, and every field names its source:

  • R1 (aprender-52): apr bench/qa/profile JSON gains resources{}:
    • peak_rss_bytes (getrusage ru_maxrss)
    • cpu_user_s and cpu_sys_s
    • minor and major page faults
    • voluntary and involuntary ctx switches
    • threads_peak
    • io read and write bytes (/proc/self/io)
    • vram_peak_bytes (NVML/cuMemGetInfo high-water, sampled)
    • energy_j (RAPL, when readable; else null with a reason)
      Also: close CRUX D-03 and replace the serve-bench estimates with real readings.
  • R2 (aprender-5d): gates.
    • Wire CiAssertions.max_memory_mb plus a VRAM ceiling into perf-matrix.yaml, with contract thresholds.
    • Turn dogfood C1 RSS from WARN into FAIL.
    • Case table: a planted over-budget run must go RED.
  • R3 (infra-33): CI build ledger reads on-runner cgroup v2 (memory.peak, cpu.stat, io.stat, memory.events) per job, so no field stays null. The arbiter logger's cpu_seconds must match the spec.

Done when a real receipt from nightly and from an rc shows every field non-null (or null with a stated reason), and one gate has been proven RED by mutation.

Activity

  1. added this to the 0.70.1 milestone on Sep 27, 2026
  2. modified the milestones: 0.70.1, 0.71.0 on Oct 4, 2026
  3. noahgift commented on Oct 14, 2026

    @noahgift
    MemberAuthor

    merged into #5305 (master ticket for epic #3598; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.

  4. noahgift commented on Oct 14, 2026

    @noahgift
    MemberAuthor

    Moved: this sub-ticket is now a row in #5322 (Speed measurement harness: resource receipts, observability and the ollama/llama.cpp ratio), split out of #5305 by theme.

  5. modified the milestones: 0.71.0, 0.73.0 on Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priorityowner:la-0.71Owned by the 0.71 look-ahead slotsize:MK̂ 3-4 h

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions