Repository navigation
Instrument host and enforcement capability health #116
Description
Activity
- addedFeatureA specific missing product behavior.A specific missing product behavior.Operator experience and releasesConfiguration, services, recovery, observability and reproducible releases.Configuration, services, recovery, observability and reproducible releases.P1Core supported-product work to prioritize after the working local alpha.Core supported-product work to prioritize after the working local alpha.Source reviewedCurrent source evidence checked; no new runtime reproduction claimed by backlog creation.Current source evidence checked; no new runtime reproduction claimed by backlog creation.In progressImplementation is active.Implementation is active.
on Sep 24, 2026 - added a commit that references this issue
on Sep 28, 2026 - addedQueuedSelected for upcoming work; implementation has not started and dependencies may remain.Selected for upcoming work; implementation has not started and dependencies may remain.In progressImplementation is active.Implementation is active.and removedIn progressImplementation is active.Implementation is active.QueuedSelected for upcoming work; implementation has not started and dependencies may remain.Selected for upcoming work; implementation has not started and dependencies may remain.
on Oct 2, 2026 Progress through #408, #409 and #410 updates the remaining scope.
Executor RSS, descriptor and state-filesystem observations are exported. The payment-owned settlement snapshot reads durable pending orders and transfer states, including obligations retained while payments are disabled. These are backlog observations, not proof of payment.
Local drop gauges count actual ingress/egress TCX_DROP verdicts and associated socket-buffer lengths. They are current-counter totals exported as maxima; restarts can lower them. Socket buffers may contain multiple wire packets. These gauges do not measure all denied traffic or establish continuous enforcement across supported paths.
The new disclosure completion bound measures from the oldest newly acknowledged key's scheduled due time to receipt of verified durable-commit coverage, on the sender's current-generation schedule clock. It includes the reply path and skipped epochs, separately from dispatcher backlog. Duplicate replies do not refresh samples; actual sample age expires after one minute. Missing receipts, failed writes, stale sessions and observed clock invalidation remain unavailable. Retired-chain timing stays unknown. This is not calibrated one-way latency or an indefinite archive-retention promise.
At
8755920, focused Linux race tests and pinned generation passed. The leased SQLite round-trip fixture checks held commits, lost replies, duplicate coverage and failed storage. Candidate CI and guest-source CI record integrated validation.Keep #116 open for independently calibrated clock uncertainty and complete denied-traffic/continuous-enforcement observations. Collection remains bounded, uses fixed labels and omits incomplete numeric observations instead of reporting misleading zeros.
Merged as 47bb370. All 17 checks passed on both the exact candidate and identical merged main, including main guest-source validation.
Problem
Packet counter selection, schedule lifecycle and resource cleanup have internal state and logs, but operators cannot collect stable metrics for actual enforcement mode, host pressure or disclosure freshness.
Proposed change
Expose low-cardinality host memory/FD/disk signals plus capability mode, denied traffic, clock uncertainty, disclosure lag and remaining schedule lifetime only where authoritative subsystem observations exist. Add settlement backlog instrumentation by consuming the payment status contract.
Acceptance criteria
Current evidence
PR #356 merged the supported observation slice in
8e3eb292a386f5d50aa3e63520ceb8652b3dabd4. The existing authenticated metrics endpoint now aggregates selected packet-counter modes, attribution reasons, kernel clock readiness/error estimates and announced schedule lifetime. Dispatcher RSS, descriptors and state-filesystem capacity were already available. Missing, malformed, stale, future-dated or disconnected reports remain unknown; incomplete numeric summaries are omitted.All 16 jobs passed in candidate CI 36979225833; the actual merged tree is identical to that candidate. Focused metrics tests passed under the race detector on supported Linux. Pinned Prometheus health, availability and storage fixtures passed. Collection remains bounded, uses fixed labels and introduces no I/O under the registry lock or enforcement changes. See the metrics contract.
Remaining proposal scope: denied-traffic observations, independently measured clock uncertainty, actual end-to-end disclosure delivery lag and settlement backlog still lack the corresponding authoritative exporter contracts. Executor process RSS/descriptor/disk observations are also not exported. Existing observations must not be renamed or treated as these missing measurements. Packet enforcement validation remains coordinated with #67; settlement instrumentation stays with the payment status work. This issue remains open for those gaps.
Dependencies
Related work
These are integration points, not prerequisites for starting this issue.
Validation must use owned local fixtures on supported Linux environments. Record the implementing merge request and relevant test results before closing this issue.
Imported from GitLab issue 115. Originally opened 2026-09-10. Historical GitLab links may require access to the original project.