feat: a payload leaves the spool only when the backend has it (ADR 0075)

[kfeofantov]

Adds the transmit path. A backend exists and receives payloads; this agent had
none at all, so nothing had ever travelled between the two. That gap is the
point of the change — two systems built toward one document diverge in the
places the document does not cover, and only a running agent talking to a
running backend finds them.

What it does

One setting, backend.baseURL, with no default: absent means ship nothing,
which is exactly today's behaviour. POST {base}/v1/kubernetes/{kind}, one
payload per request, the body the spool file verbatim, {kind} read out of the
payload's own bytes and checked against the registry (ADR 0022).

Nothing is read out of a response but its status — the body is drained so the
connection can be reused and discarded unparsed. ADR 0001 forbids the backend to
send anything the agent acts on; this is where that stops being a promise about
the backend and becomes a property of the agent.

A file is deleted only after a 2xx, and only if it is still the file that was
sent. A superseding writer replaces a file by rename while its predecessor is
in flight, so deleting by name alone would drop a newer payload that was never
offered, and nothing downstream could notice.

Answer	What happens
2xx	Delivered. The file goes.
429, 5xx, network, timeout, anything unrecognized	Deferred. File stays, round ends, backoff grows.
400	Held: counted malformed, never offered again.
413, or past the agent's own ceiling	Held: counted too_large, never offered again.
401, 403	Counted unauthorized. Shipping halts for the life of the process.

An unrecognized status is deliberately transient: a base URL with a typo answers
404 to every payload, and treating that as permanent would empty the spool
against a configuration mistake.

The two decisions the ADR was asked to record

A permanently rejected payload is held, not dropped. Both options lose it —
the spool is bounded and nothing re-reads it — so the choice is between losing
it now and losing it in a day. The day buys the bytes: a 400 means the agent and
the backend disagree about a schema, and finding exactly that is what this stage
is for. backend-requirements.md §5 already permitted either. The mark is keyed
by file identity, so the next pass writing under the same name clears it.

The 4 MiB ceiling is a placeholder, chosen to sit under limits proxies
commonly impose rather than to describe anything either side measured. It is not
a chunking threshold and must not become one (ADR 0027 §4). What it waits for is
a stated protocol limit, which belongs in the registry as a property of a kind
beside cadence — ADR 0073 already built the place. Until then the too_large
counter is what makes the constant's wrongness visible.

Ordering, and ADR 0027 §2

One goroutine, one request in flight. The spool holds one file per natural key,
so a shipper that never has two requests open never has two versions of a key in
flight. That closes the obligation ADR 0027 recorded against a shipper that did
not exist, and backend-requirements.md §4 has stated since then.

Collection does not depend on shipping

Run returns only on cancellation, halted or not: in cmd/agent a lifecycle
task that returns cancels the agent, so a backend refusing this agent's identity
would otherwise stop collection. An unreachable, slow or hostile backend changes
nothing about what the agent collects or writes, and the tests hold that line.

Counters

collection_coverage gains a shipping block — delivered, deferred,
unreadable, halted, and rejected carrying the three reasons — plus
backend_configured in its configuration shape (a switch, never the address;
tested).

It does not reuse intake_rejections. That block carries the same three
words about the opposite direction: what this agent's own receiver refused from
a node inside the cluster (ADR 0067, wired through internal/nodeintake and
already exposed as node_reports_rejected_total). Merging them would leave
neither answerable.

/metrics gains shipments_total{outcome}, shipments_rejected_total{reason}
and shipping_halted — the last because a halted agent ships nothing, so the
coverage payload carrying the same fact never arrives. No new label names.

Deliberately not here

No TLS, certificates, enrollment, renewal or CA pinning, and no invented
identity — no cluster id in a payload, no header naming the sender. Identity
is a property of the connection and arrives with mTLS (ADR 0005, ADR 0008); a
field standing in for it now would have to be torn out of the schema, the
handlers and the stored keys. The backend attributes everything to one
configured source and knows it. Nothing is stubbed.

The chart value and README/security.md say plainly that this stage is plain
HTTP for a backend the operator controls.

Still open

ADR 0027 §4's other deferral — container_restarts and pod_disruptions are
byte-identical whether a window is final or sampled part way through. It was
deferred because there was no ingest to ask; there is one now, and the deadline
was always "before a backend stores these kinds". It is a payload change and a
conversation, not a transport change. Recorded in ADR 0075's Not addressed
here.

Evidence

Run against a clean git archive HEAD extraction, not the working tree:

go build ./...                                    ok
go test -race ./...                               ok
go vet -tags e2e ./...                            ok
golangci-lint run                                 0 issues
buf lint && buf breaking --against origin/main    ok
make chart-lint                                   1 chart linted, 0 failed

e2e has not been run yet; the chart change is cluster-facing, so it should run
against kind before merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KeZECJXTDHL6Nts95eKMJA