M3: collector interface, event bus, process collector - #3
Merged
Merged
Conversation
Argus now observes a live host. Collectors are the only code that knows what
a platform's telemetry looks like; everything they emit is normalized.
- collector.nim: one abstract base (start/stop/privileges/available/
unavailableReason) plus a CollectorSet that reports every collector it
could NOT start, so a gap in coverage is stated rather than implied.
Each collector declares the privilege it wants and what it can still see
without it — that is what makes 'least privilege' checkable rather than a
claim. Exposed as 'argus doctor'.
- bus.nim: bounded, lock-protected queue between collection and detection.
A collector reading kernel telemetry must never wait on a rule, so the
queue never blocks a producer. Past the ceiling it drops and COUNTS the
drop: a monitor that OOMs the host it was watching has done more harm
than the events it missed, but silent loss is not acceptable either.
- procfs.nim: /proc parsing as pure functions. The stat parser scans to the
LAST ')' because a process may legally be named '(evil) thing' and
splitting on whitespace would then read ppid out of the process name.
The real uid is the first of the four columns in status.
- process.nim: polling collector. The tree root is a parameter, not a
hard-coded /proc, so the exact production code path runs against a
fixture directory in tests — no mocks. Pid reuse is detected via the
kernel start time and emits exit-then-start rather than silently
attributing a new process's actions to a dead one. An exe that could only
come from argv[0] is flagged, since the process controls that string.
Polling rather than netlink is a deliberate trade, documented in the module
and in docs/design.md: no privilege beyond reading /proc, at the cost of
missing processes shorter-lived than the interval. The interface is the same
either way, so netlink can be dropped in later without the engine noticing.
docs/design.md updated where the implementation diverged from the proposal:
start() takes the EventBus rather than an emit closure, because a closure
captures a GC'd environment that cannot cross a thread boundary in Nim.
90 new tests (386 total), including concurrent producers against the bus and
a threaded collector end-to-end into the engine.
One real bug found and fixed while testing: casting a pointer back to a ref
inside a thread yields an un-incremented reference that ORC destroys at scope
exit. That over-decrement corrupted the heap and crashed somewhere unrelated.
Such bindings are now {.cursor.}, with the reasoning written down in bus.nim
so the next collector does not repeat it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Milestone 3 — collector interface, event bus, process collector
Stack: 2 of 6 · base
main· next: M4 file collectorArgus now observes a live host.
collector.nim— one abstract basestart/stop/privileges/available/unavailableReason. ACollectorSetreports every collector it could not start, so a gap in coverage is stated rather than implied.Each collector declares the privilege it wants and what it can still see without it. That's what makes "least privilege" checkable instead of a claim. Surfaced as
argus doctor, which runs before anything is collected:bus.nim— bounded queue between collection and detectionA collector reading kernel telemetry must never wait on a rule evaluation, so the queue never blocks a producer. Past its ceiling it drops — and counts every drop. The reasoning is in the module: a monitor that OOMs the host it was watching has done more harm than the events it missed, but silent loss isn't acceptable either.
procfs.nim—/procparsing as pure functionsThe stat parser scans to the last
), because a process may legally be named(evil) thingand splitting on whitespace would then readppidout of the process name. The real uid is the first of the four columns instatus— taking the wrong one misattributes every setuid process. Both have tests.process.nim— the polling collectorThe tree root is a parameter, not a hard-coded
/proc, so the exact production code path runs against a fixture directory in the tests. No mocks.exethat could only come fromargv[0]is flagged inraw.exe_source, since the process controls that string.Polling rather than netlink is a deliberate trade, documented in the module and the design doc: no privilege beyond reading
/proc, at the cost of missing processes shorter-lived than the interval. The interface is the same either way, so netlink can be dropped in later without the engine noticing.Design doc updated where the implementation diverged
start()takes theEventBusrather than anemitclosure — a closure captures a GC'd environment, which cannot cross a thread boundary in Nim. The doc says so rather than quietly not matching the code.Verification
90 new tests, 386 total, all green — including concurrent producers hammering the bus and a threaded collector running end-to-end into the engine.
One real bug found while testing, worth flagging for review: casting a pointer back to a
refinside a thread yields an un-incremented reference that ORC destroys at scope exit. That over-decrement corrupted the heap and crashed in an unrelated place much later. Those bindings are now{.cursor.}, and the reasoning is written down inbus.nimso the next collector doesn't repeat it.