feat(netproxy,diagnose): record network prompts nobody answered in time - #260
Open
Retr0MrWave wants to merge 2 commits into
Open
feat(netproxy,diagnose): record network prompts nobody answered in time#260Retr0MrWave wants to merge 2 commits into
Retr0MrWave wants to merge 2 commits into
Conversation
6 tasks
While a prompt is open the connection is held, and the requesting tool is
subject to its own timeout — the opencode Skainet plugin aborts model
discovery after 3s, while answering a dialog measurably takes longer. The
tool gives up, no verdict is ever reached, and no net.decision is written.
`omac diagnose` then reported "0/1 connection(s) blocked · Nothing was
blocked by the network policy" for a run that had silently failed.
The dialog is deliberately detached from the request context
(internal/netprompt builds its timeout from context.Background()), so an
open prompt outlives the request that raised it and dies with the process.
Detection therefore belongs at teardown: Filter.DrainAbandonedPrompts
records everything still in flight when the proxy closes, as a new
net.prompt_abandoned audit event carrying how long the prompt was open.
diagnose gains a matching axis: the abandoned count in the status line, a
problem hint naming the host, the wait, and the remedy (pre-allow the host
— answering faster is not reliable), and suppression of the now-misleading
"nothing was blocked" note. Runs with no abandoned prompts are unchanged.
Verified on Linux/bwrap, no human interaction needed (the client times out
on its own):
omac sandbox run --profile <prompt-enabled> --audit-log /tmp/a.jsonl \
-- curl -s --max-time 1 https://example.com # exit 28
diag log: net ABANDONED example.com:443 (prompt raised 964ms ago, no
verdict — the requesting tool stopped waiting)
audit: net.prompt_abandoned example.com 443 waited_ms=964
omac diagnose:
last run · filtered mode · 0/0 connection(s) blocked ·
1 prompt(s) abandoned
• 1 network prompt(s) were raised but never answered in time
example.com:443 gave up after 964ms …
Fix: add the host to network.allow_domain …
Deliberately not included: making a bare `omac sandbox run` write the
central audit trail (issue checklist item 4). It would couple sandboxrun to
launcher-config loading, which it does not do today — worth its own change.
go build ./... && go test ./... pass except TestIntegrationWorktreeKnownLimitations
and TestIntegrationWorkflowInterpretersRunnable, which fail identically on
clean main on this host (local toolchain paths absent from the default profile).
Closes #257
Refs #241
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Ilia Zhuravok <ilia.zhuravok@tngtech.com>
…e up Review findings on this branch. The first is the substantive one: the original commit only detected prompts still open at shutdown, so the headline case from #257 — a short-timeout client gives up, the user answers seconds later, the run continues — still produced a plain allow and no abandoned record, and diagnose still printed "Nothing was blocked". Nothing in the proxy observes the requester leaving (both clientSourceCtx and the dialog ctx derive from context.Background()), so instead of inferring a disconnect, the prompt's open duration is now recorded on the decision itself: net.decision carries waited_ms for prompt-sourced verdicts, and diagnose reports any answered after >=3s (the shortest client budget observed in the wild — the opencode Skainet plugin's model discovery) as a likely silent failure. Both halves now suppress the misleading "nothing was blocked" note. Also fixed: - Server.Close drained before the listener and connections were torn down, clearing the coalescing map while requests could still arrive: a follow-up request for the same host would raise a second dialog and go unrecorded. The drain is now last. - A drained prompt answered later emitted a net.decision too, so one request appeared as both abandoned and allowed. promptWait carries a drained flag and the resolution path skips the emit (logging why). - abandonedPromptHints emitted one uncapped detail line per entry, which under --run all spans the whole persistent log. Now aggregated per host:port with a repeat count and capped at 8, mirroring maxBlockedShown. - docs: net.prompt_abandoned added to the audit event-type list, plus the waited_ms field on prompt decisions. Verified: late-answer path (was invisible, now a problem finding): net.decision waited_ms=5200 source=prompt -> "1 prompt(s) were answered after 3.0s or more — a short-timeout tool may have failed anyway chat.model.tngtech.com:443 was held for 5.2s …" and no "Nothing was blocked by the network policy" drain path still works (curl --max-time 1, no human interaction): net.prompt_abandoned example.com waited_ms=967 go test ./internal/netproxy/ -race passes. Refs #257 Refs #241 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Ilia Zhuravok <ilia.zhuravok@tngtech.com>
mwtng
force-pushed
the
feat/abandoned-prompt-audit
branch
from
August 27, 2026 17:42
fd0a7b9 to
102da62
Compare
Contributor
|
I just rebased and reran the verification. I did not do a review, yet. But probably one should have a look at the changes to the documentation and if they are really necessary in such detail. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue: Closes #257 · Refs #241
What
net.prompt_abandonedaudit event for a network prompt that was raised but never resolved.net.decisionevents now carrywaited_ms— how long the dialog was open before it was answered.omac diagnosereports both, and no longer claims "Nothing was blocked by the network policy" for a run where either happened.Why
While a prompt is open the connection is held, and the requesting tool is subject to its own timeout. The opencode Skainet plugin aborts model discovery after 3s; answering a GUI dialog measurably takes longer (5s measured here, with the dialog already in focus). So the tool gives up, the fetch fails silently, and the allow you then grant applies to a request nobody is waiting for.
Both halves of that were invisible:
net.decisionis ever written.omac diagnosereported0/1 connection(s) blocked · Nothing was blocked by the network policyfor a run that had silently failed.This is not an edge case: the compiled-in default profile (
internal/sandboxprofile/resolve.go:23) has noallow_domain, so on a fresh install every host prompts on first contact.How
internal/netprompt/prompt.go:312builds its timeout fromcontext.Background()), so an open prompt outlives the request that raised it and dies with the process. Detection therefore belongs at teardown:Filter.DrainAbandonedPromptsrecords whatever is still in flight when the proxy closes. It runs last inServer.Close, after the listener and connections are gone — draining earlier clears the coalescing map while requests can still arrive, which would raise a second dialog for a host already being asked about.clientSourceCtxis alsocontext.Background()-derived), so for the answered-too-late case the prompt's open duration is recorded on the decision instead, and diagnose flags anything ≥3s (promptLatencyThresholdMS— the shortest client budget observed in the wild).promptWaitcarries adrainedflag so one request cannot appear as both abandoned and allowed.host:portwith a repeat count and capped at 8, mirroringmaxBlockedShown— otherwise--run allemits one line per occurrence across the whole persistent log.Verification
go build ./...,go vet ./internal/...clean.go test ./internal/netproxy/ -raceclean.go test ./...passes exceptTestIntegrationWorktreeKnownLimitationsandTestIntegrationWorkflowInterpretersRunnable, which fail identically on cleanmainon this host (local toolchain paths absent from the default profile).Drain path, end to end, no human interaction (the client times out on its own):
Late-answer path (the case the first commit missed — a
waited_ms=5200prompt-sourced allow):…and no "Nothing was blocked" in either case. Runs with neither condition are unchanged (regression test asserts the note still fires).
The second commit is review fixes; the late-answer detection, the
Closeordering, the double-report suppression and the detail cap each have a test.Follow-up
omac sandbox runwrite the central audit trail. It only gets an auditor when the parent passes--audit-log(internal/sandboxrun/run.go:144), and wiring it would couplesandboxrunto launcher-config loading — worth its own change.🤖 Generated with Claude Code