Skip to content

M3: collector interface, event bus, process collector - #3

Merged
dylanpatriarchi merged 1 commit into
mainfrom
feat/m3-process-collector
Aug 1, 2026
Merged

dylanpatriarchi merged 1 commit into
mainfrom
feat/m3-process-collector

Conversation

@dylanpatriarchi

Copy link
Copy Markdown
Owner

Milestone 3 — collector interface, event bus, process collector

Stack: 2 of 6 · base main · next: M4 file collector

Educational project — read-only and defensive. Collectors observe; nothing here opens a handle for writing, signals another process, or loads code into one.

Argus now observes a live host.

collector.nim — one abstract base

start / stop / privileges / available / unavailableReason. A CollectorSet reports every collector it could not start, so a gap in coverage is stated rather than implied.

Each collector declares the privilege it wants and what it can still see without it. That's what makes "least privilege" checkable instead of a claim. Surfaced as argus doctor, which runs before anything is collected:

$ argus doctor          # on macOS
process — unavailable
    reason: no /proc on this system; the process collector needs a Linux-style procfs
    privilege: read /proc/<pid>/exe (root or CAP_SYS_PTRACE for other users' processes) (optional)
        for: resolve the real executable path; without it Argus falls back to argv[0],
             which the process itself controls and can therefore lie about
No collectors can run here. Argus would observe nothing.

bus.nim — bounded queue between collection and detection

A collector reading kernel telemetry must never wait on a rule evaluation, so the queue never blocks a producer. Past its ceiling it drops — and counts every drop. The reasoning is in the module: a monitor that OOMs the host it was watching has done more harm than the events it missed, but silent loss isn't acceptable either.

procfs.nim — /proc parsing as pure functions

The stat parser scans to the last ), because a process may legally be named (evil) thing and splitting on whitespace would then read ppid out of the process name. The real uid is the first of the four columns in status — taking the wrong one misattributes every setuid process. Both have tests.

process.nim — the polling collector

The tree root is a parameter, not a hard-coded /proc, so the exact production code path runs against a fixture directory in the tests. No mocks.

  • Pid reuse is detected via the kernel start time and emits exit-then-start, rather than silently attributing a new process's actions to a dead one.
  • An exe that could only come from argv[0] is flagged in raw.exe_source, since the process controls that string.
  • The first scan establishes a baseline and reports nothing — otherwise every already-running process would look like it just started.

Polling rather than netlink is a deliberate trade, documented in the module and the design doc: no privilege beyond reading /proc, at the cost of missing processes shorter-lived than the interval. The interface is the same either way, so netlink can be dropped in later without the engine noticing.

Design doc updated where the implementation diverged

start() takes the EventBus rather than an emit closure — a closure captures a GC'd environment, which cannot cross a thread boundary in Nim. The doc says so rather than quietly not matching the code.

Verification

90 new tests, 386 total, all green — including concurrent producers hammering the bus and a threaded collector running end-to-end into the engine.

One real bug found while testing, worth flagging for review: casting a pointer back to a ref inside a thread yields an un-incremented reference that ORC destroys at scope exit. That over-decrement corrupted the heap and crashed in an unrelated place much later. Those bindings are now {.cursor.}, and the reasoning is written down in bus.nim so the next collector doesn't repeat it.

Argus now observes a live host. Collectors are the only code that knows what
a platform's telemetry looks like; everything they emit is normalized.

- collector.nim: one abstract base (start/stop/privileges/available/
  unavailableReason) plus a CollectorSet that reports every collector it
  could NOT start, so a gap in coverage is stated rather than implied.
  Each collector declares the privilege it wants and what it can still see
  without it — that is what makes 'least privilege' checkable rather than a
  claim. Exposed as 'argus doctor'.

- bus.nim: bounded, lock-protected queue between collection and detection.
  A collector reading kernel telemetry must never wait on a rule, so the
  queue never blocks a producer. Past the ceiling it drops and COUNTS the
  drop: a monitor that OOMs the host it was watching has done more harm
  than the events it missed, but silent loss is not acceptable either.

- procfs.nim: /proc parsing as pure functions. The stat parser scans to the
  LAST ')' because a process may legally be named '(evil) thing' and
  splitting on whitespace would then read ppid out of the process name.
  The real uid is the first of the four columns in status.

- process.nim: polling collector. The tree root is a parameter, not a
  hard-coded /proc, so the exact production code path runs against a
  fixture directory in tests — no mocks. Pid reuse is detected via the
  kernel start time and emits exit-then-start rather than silently
  attributing a new process's actions to a dead one. An exe that could only
  come from argv[0] is flagged, since the process controls that string.

Polling rather than netlink is a deliberate trade, documented in the module
and in docs/design.md: no privilege beyond reading /proc, at the cost of
missing processes shorter-lived than the interval. The interface is the same
either way, so netlink can be dropped in later without the engine noticing.

docs/design.md updated where the implementation diverged from the proposal:
start() takes the EventBus rather than an emit closure, because a closure
captures a GC'd environment that cannot cross a thread boundary in Nim.

90 new tests (386 total), including concurrent producers against the bus and
a threaded collector end-to-end into the engine.

One real bug found and fixed while testing: casting a pointer back to a ref
inside a thread yields an un-incremented reference that ORC destroys at scope
exit. That over-decrement corrupted the heap and crashed somewhere unrelated.
Such bindings are now {.cursor.}, with the reasoning written down in bus.nim
so the next collector does not repeat it.
@dylanpatriarchi
dylanpatriarchi merged commit 4bc4166 into main Aug 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant