feat(transport): authenticate SSH-carried mesh sessions

[hartsock]

What

• Rebase the SSH transport on the authenticated-delivery provenance from fix(bus): require authenticated delivery provenance #81 and expected-responder reply binding from fix(bus): bind replies to expected responders #82, without duplicating either policy in the SSH crate.
• Treat OpenSSH as a carrier only. Before any envelope can be dispatched, both endpoints exchange bounded, versioned hellos and prove live possession of their AgentKey over a fresh, role-separated transcript containing both exact certificate chains, both nonces, protocol/version, transport, session parameters, and the initiator's exact expected responder.
• Wrap every post-authentication envelope in a session-bound, directional, monotonic record signed by the authenticated AgentKey. Cross-session replay, reflection, record reordering, tampering, and cut-over injection therefore fail before bus admission.
• Construct AuthenticatedPeer exclusively from the certificate whose leaf verifies the peer's handshake proof. Envelope-controlled fields never establish or overwrite carrier provenance; fix(bus): require authenticated delivery provenance #81 independently requires the envelope signer to equal that authenticated carrier before mutating replay state.
• Add the production SshTransport: loopback-only forwarding/listening, reusable exact-peer sessions, authenticated reply routes with safe fallback, bounded concurrency/queues/frames, cancellation-safe record I/O, deterministic close behavior, and process/task cleanup.
• Make pinned host verification the secure default (StrictHostKeyChecking=yes with a dedicated known-hosts file). Retain accept-new only as an explicitly selected bootstrap-TOFU policy and document it as such.
• Isolate and supervise the OpenSSH subprocess: absolute executable and identity paths, isolated SSH configuration, no agent/control-socket/password fallback, continuously drained bounded stderr, actionable exit status, authentication/read/write deadlines, and kill/reap on failure.
• Update the SSH ADR, crate/root READMEs, CI, and release publication order. The unrelated Iroh downgrade is removed; Iroh remains at 0.98.2.

The connection lifecycle is structurally OpenSSH carrier -> unauthenticated mesh session -> mutually authenticated mesh session -> signed record stream. The unauthenticated typestate has no envelope API and cannot mint direct provenance.

Test plan

• just check
• just cov-ci — 92.54% workspace line coverage (75% required)
• 41 SSH crate unit tests
• Real ephemeral sshd + /usr/bin/ssh -W integration test on macOS
• Workspace-wide fmt, all-target clippy with -D warnings, unit/integration tests, and doc tests
• Independent adversarial review and diff-quality review; no remaining blocker/high finding

The adversarial suite covers mutual authentication and Bus request/reply; exact authenticated caller context; missing private key; copied certificate; captured-proof replay against fresh nonces; role/proof reflection; independent transcript-field tampering; malformed, oversized, truncated, and stalled handshake/record input; strict host-pin success and unknown/changed-host failure; unauthorized SSH key; foreign mesh root; wrong expected responder; wrong reply signer; cross-session/direction/counter replay; record tampering; partial-read/write cancellation; repeated traffic and session reuse; shutdown races; and stale reply routes.

The central regression uses the real SSH transport adapter: a Mallory-authenticated session carries Alice's captured envelope, #81 rejects carrier != signer before nonce/sequence mutation, and the exact envelope is then accepted over Alice's authenticated session.

Adversarial self-review

1. AuthenticatedPeer is permitted only after the peer certificate validates under the configured generation context and user root, the initiator's exact responder pin matches, and the certificate leaf strictly verifies the peer's role-separated proof over the fresh ordered transcript.
2. No envelope-controlled byte influences that identity: ordinary envelope bytes cannot be read until the authenticated typestate exists, and provenance is copied unchanged from its proof-verified certificate.
3. Mallory cannot submit Alice's signed envelope as Mallory: even a valid Mallory session record reaches fix(bus): require authenticated delivery provenance #81 as carrier Mallory and signer Alice, so admission fails before replay, sequence, handler, or reply state changes.
4. A captured handshake cannot authenticate a second connection because either honest endpoint contributes a fresh 32-byte nonce to the signed transcript. Records additionally bind the resulting transcript ID and strict counters.
5. Reflection fails because hellos require opposite roles, proof preimages include the role byte, and records include direction.
6. Before authentication succeeds, only bounded handshake buffers and local socket/process/task/semaphore resources are created. No Inbound, bus replay/sequence state, handler state, or remote reply state is mutated.
7. A halfway failure reaches the single authentication deadline, consumes/drops the unauthenticated stream, kills/reaps an outbound SSH child, and emits no inbound delivery. There is no unbound fallback. Cancellation during record I/O poisons that half.
8. Each leaf signs PROOF_DOMAIN || role || BLAKE3(TRANSCRIPT_DOMAIN || len(IHello) || IHello || len(RHello) || RHello). Each application record signs RECORD_DOMAIN || transcript_id || direction || counter || len || BLAKE3(raw_envelope_bytes).

Out of scope

• Binding the OpenSSH account, authorized key, or SSH session ID to the AgentKey. System ssh -W does not expose that evidence; SSH access and inner mesh identity are two separate gates, not co-bound factors.
• Detecting a transparent byte-for-byte live relay. Fresh proofs and session-signed records stop captured-transcript replay and post-handshake cut-over/injection, but a live forwarding relay requires SSH channel-binding evidence or an in-process SSH design such as spike(transport): russh dual-role SSH proxy round-trip — operator → airship bastion → newt embedded sshd (cert auth both hops, no-PTY subsystem) #49.
• Dynamic generation/revocation-policy changes on an existing transport. Callers close and rebind with new session parameters.
• A policy mapping SSH principals to mesh principals, or responder allowlists beyond the existing same-user trust model and exact outbound target pin.
• Treating the six-word dock mnemonic as an anti-MITM SAS; first-contact ceremony remains owned by design: mesh exposes decision surfaces (first-contact pinning, caveat grants) — no TUI in library crates #65/enrollment & delegation: SAS pairing ceremony + chain-to-principal pinning (ceremony-contract §5.4/§2.1) #66.

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_6f945c2f-9c2c-44c5-85a7-db059a82abb3)

[hartsock]

Adversarial security review — feat/transport-ssh @ 0e13576

Reviewed against one question: does an SSH-arrived request carry the same verified-caller session binding as a QUIC-arrived one?

No — and it cannot, by construction of the v1 design. It is weaker in two independent ways.

Green CI was not treated as evidence; findings below are traced in code and, where marked, confirmed with a standalone repro built against this branch's crates (outside the branch, then deleted).

Why QUIC's binding is real and SSH's is unreachable

The QUIC binding is a genuine proof-of-possession, and it is nearly free because of a coincidence: agent-mesh-transport/src/identity.rs:10-38 — iroh's EndpointId is an ed25519 public key, so the AgentKey is the QUIC session key. agent-mesh-bus/src/bus.rs:828 (envelope_matches_session) therefore compares the signer against a key the TLS handshake proved possession of, enforced at bus.rs:894 inside accept_conn — inside the transport, not at the seam.

SSH has no such coincidence. The SSH keypair is a different key, and the ssh -W subprocess design gives the Rust process no access to the SSH session identifier / exchange hash, so channel binding is not merely unimplemented — it is unreachable from this architecture. The ADR concedes it at Decision 5.

The app-level handshake cannot substitute: agent-mesh-transport/src/handshake.rs:41-46 — HelloMsg { cert_chain } is a bare cert with no signature over a fresh challenge, and identity.rs:34 says so outright ("the app-level handshake itself signs nothing"). A captured Hello is replayable verbatim.

And the seam imposes no obligation: bus.rs:606-624 spawn_accept_loop is explicitly transport-agnostic, and Inbound (agent-mesh-bus/src/transport.rs:37-44) has no field for a transport-authenticated peer identity. RequestContext is then reconstructed from the envelope alone (agent-mesh-bus/src/inbox.rs:282-285). That is authority reconstruction: session-bound verified caller → signed blob → accepted caller identity.

R1 ssh recv_envelope(stranger-rooted envelope) -> ACCEPTED  signer_user=0d9016fd21c3 (hub_user=37c65be2390f)
R2 inbox dispatch of relayed envelope -> handler_ran=true reply=Ok(true)
R2 RequestContext.caller_agent_fp = 724f2f5125cc  (alice=724f2f5125cc mallory=1249e74c7914)
R3 same envelope twice -> Some(Replay)
R4 max-length prefix + 16 bytes of body -> reader parked holding a 16 MiB zeroed buffer

Important mitigating fact: nothing depends on this crate. It implements no Transport, has no server side, and no crate imports it. There is no SSH ingress today and therefore no live exploit. The defect is that the shipped primitives, used exactly as the ADR instructs, produce the degraded path — and the ADR asserts they don't.

Findings

F1 — HIGH, confirmed. The "transport-agnostic overlay" claim is false; two admission checks live in the transport and neither is reproduced.
docs/decisions/ssh_transport.md:30-34 claims the overlay is "unchanged and transport-agnostic by design." Two checks are transport-resident: (1) auto-team admission, agent-mesh-transport/src/handshake.rs:117-133 ensure_trustable — peer's user_fp must equal ours else DifferentUser, fail-closed; and (2) session binding at bus.rs:894. SignedEnvelope::verify() (agent-mesh-protocol/src/envelope.rs:114-134) does not do the first — it checks only chain self-consistency, payload CID, and agent signature, so any stranger's self-generated UserKey produces an envelope it accepts. agent-mesh-transport-ssh/src/framing.rs:61-84 recv_envelope reproduces neither. R1 proves it. On QUIC that peer never gets past the handshake.

F2 — HIGH, confirmed. Relay equals impersonation; exact replay is blocked, first-delivery relay is not.
R2: Mallory re-frames Alice's verbatim, unmodified signed envelope onto her own channel. It is accepted, dispatched, and the handler's RequestContext.caller_agent_fp is Alice's. Nothing records who delivered the bytes. I tried to refute this and partly succeeded — R3 shows the nonce cache (agent-mesh-bus/src/replay.rs:58-71) rejects an exact duplicate the hub has already seen. What is not blocked is relay of an envelope the hub has not yet seen: the MITM case, a cross-hub capture, or an envelope harvested from a compromised peer's outbox. Two secondary effects of SequenceTracker::check_and_advance (replay.rs:111-124, monotonic with forward jumps allowed): the relayed copy wins and the victim's own copy is then rejected as Replay, and a relay at a high sequence permanently wedges the victim's identity on that hub. Given newt-agent#1643's dock grant is a location-scoped bearer record, a relayed dock request accepted as its signer is exactly the failure the bearer model cannot absorb. (newt-mesh/src/service.rs:61 still uses context-free handle_requests, so this is a live-fire risk on the next change, not this instant.)

F3 — HIGH, confirmed. accept-new is TOFU and is not fail-closed; the claim appears four times.
Asserted in the PR body/commit message, ssh_transport.md:73, process.rs:24-26, and most starkly error.rs:24-27 ("Fail-closed: we never silently accept an unknown host key"). Falsified against OpenSSH 9.6p1 with this PR's exact flag set:

=== accept-new, empty known_hosts (first contact) ===
Warning: Permanently added '127.0.0.1' (ED25519) to the list of known hosts.
nobodyxyz@127.0.0.1: Permission denied (publickey).      <- proceeded to auth
=== StrictHostKeyChecking=yes, same first contact ===
No ED25519 host key is known ... and you have requested strict checking.
Host key verification failed.                             <- actually fail-closed

accept-new silently trusts an unknown host key under BatchMode and proceeds to offer the spoke's key to whoever answered. Capability required: DNS/routing/ARP control, or simply answering first for the hub name before the spoke's first connection — which for a fresh spoke is every deployment. Gain: termination of the channel and therefore the plaintext framed envelope stream both ways, plus (per F2) onward relay with no session binding to stop it. The other half of the claim is supported — accept-new does refuse a changed key — but see F4, that alarm is discarded. Note this repo's own newt-agent/docs/design/ssh-ca-trust-root.md already specifies OpenSSH certificates with the human's key as CA; a @cert-authority line plus StrictHostKeyChecking=yes removes the TOFU window with zero new code.

F4 — MEDIUM, confirmed. stderr is /dev/null; every SSH-layer security signal is destroyed, and the comment claims the opposite.
process.rs:109-111 says diagnostics are surfaced "via tracing instead" — there is no tracing call anywhere in the crate (tracing, anyhow, serde, async-trait are all declared-but-unused). The host-key-changed banner, Permission denied, Host key verification failed all go to /dev/null; the child's exit status is never inspected. Consequently SshTransportError::Auth and ::HostKey are never constructed anywhere — dead variants documenting protections the code cannot detect. Every SSH failure surfaces identically as Ssh("read len: early eof"), so the one detection F3's mitigation does provide (changed host key = active MITM) is indistinguishable from a peer hangup.

F5 — MEDIUM, confirmed. Unexplained iroh downgrade 0.98.2 → 0.98.1, workspace-wide, in a security PR.
Cargo.lock moves iroh back a patch (der and pkcs8 move forward). Nothing in the PR needs iroh; the new crate doesn't depend on it. CI runs --locked, so it shipped green and silent. It affects the QUIC transport that carries the actual session binding, and newt-agent consumes agent-mesh by path dep, so it propagates downstream. Please justify or revert (cargo update -p iroh --precise 0.98.2). Otherwise the lock churn is clean: one package added, zero removed, no new third-party crates.

F6 — MEDIUM, confirmed. Decision 3's server side does not exist and cannot work as specified.
ssh_transport.md:91-97 has the mesh process binding 127.0.0.1:<mesh_port> with sshd forwarding direct-tcpip to it, "zero new server code." There is no TcpListener anywhere in the repo — the mesh listener is iroh/QUIC over UDP (agent-mesh-transport/src/endpoint.rs:138). direct-tcpip is TCP; ssh -W to that port connects to nothing. The shape of the eventual answer is already bad: whoever writes that listener will reach for framing::recv_envelope, which authenticates nothing but the signature, at which point any local uid on the hub can inject envelopes with no SSH auth, no auto-team check and no session binding — the SSH gate becomes decorative for local processes. "Fine on a single trusted host" (line 102) is doing unearned work on a host whose purpose is hosting other operators' docked agents.

F7 — LOW/MED, confirmed. Framing: no read timeout, 16 MiB parked per stalled peer.
Stated plainly because I attacked it and mostly failed: read_exact handles partial reads correctly, u32::from_be_bytes cannot overflow, len as usize is safe, and the cap check precedes the allocation. Decode logic is byte-identical to agent-mesh-transport/src/stream.rs:52-72 — no decoder drift. What is real: vec![0u8; len] commits up to 16 MiB on a 4-byte prefix with no read timeout anywhere in the crate (R4). The QUIC path has handshake and idle timeouts in front of it; this path has neither. Conditioned on F6's listener existing, it is remotely reachable pre-authentication.

F8 — REFUTED, reported for the record. Subprocess argv is not option-injectable.
SshTarget::ssh_args (process.rs:60-83) places every attacker-influenceable field where getopt consumes it as an option value, and fuses user/host into one trailing user@host element. user = "-oProxyCommand=touch /tmp/pwned; false" → consumed as -o's value, no destination remains → usage error, nothing executed. host = "h -oProxyCommand=..." → hostname contains invalid characters. Non-exploitable residuals worth hardening anyway: no field validation on the pub fields (a future caller taking a hub name from a docks.d record has no guard); no -- terminator (OpenSSH 9.6 honors it — free hardening); Command::new("ssh") resolves via PATH; and no -F control, while man ssh notes -W's implied ClearAllForwardings "can be overridden in the configuration file" — a UserKnownHostsFile /dev/null in the invoking user's ssh_config would combine with accept-new to make every connection a permanent first-contact accept.

F9 — MEDIUM, confirmed. The tests are vacuous for the security claim.
8/8 pass and not one exercises a refusal of hostile input. framing.rs:114-142 is two happy-path round-trips plus an oversized-prefix test — a bounds check, not the trust mechanism. env.verify(), the single control the ADR rests on, is never tested negatively. process.rs:206-234 join(" ")s the argv and asserts substrings; ssh_args_are_fail_closed asserts the literal "StrictHostKeyChecking=accept-new" is present — it names itself fail-closed while asserting the presence of the TOFU flag, encoding F3's false claim as a passing test.

Missing hostile tests: tampered signature refused; CID-mismatch refused; stranger-UserKey envelope refused (currently accepted, R1); envelope relayed on a channel not owned by its signer refused (currently accepted, R2); truncated body after a valid prefix fails bounded rather than hanging (currently hangs, R4); argv construction under adversarial fields; exit-status/stderr surfacing distinguishing auth failure from host-key change.

Documentation residue

agent-mesh-transport-ssh/Cargo.toml:10 describes the crate as "built on russh," and error.rs:4-6/13-15 narrate "the russh SSH stack." There is no russh. That residue from the abandoned in-process design is likely why the ADR's security narrative describes protections the subprocess implementation does not have.

Recommendation: NEEDS-CHANGES

Not BLOCK — there is no ingress and no exploitable path in the running mesh today, and the framing and argv code are more careful than expected (F7 and F8 both partly refuted). Not LAND-WITH-RESIDUALS — the PR ships documentation asserting security properties the code does not have, into a repo whose merged Field Guide already tells readers "who is asking is transport-authenticated, not self-declared." Landing as written puts a contradicting authority into the record for the next implementer, and that implementer is the one who wires up the ingress.

To lift, in descending order of necessity:

1. Correct the claims. Remove "fail-closed" wherever it describes accept-new (PR body, commit message, ADR L73/L77, process.rs:24-26, error.rs:24-27). Rewrite L30-34 to state that auto-team admission and session binding are transport-resident and this carriage reproduces neither. Rewrite Decision 5's rationale — "the two layers are independent" is the assumption that fails. Fix the russh references; drop or implement the dead error variants.
2. Make the seam enforce it, so no transport can forget. Add a transport-authenticated peer identity to Inbound and move the envelope_matches_session comparison out of accept_conn into the seam, with an explicit "unbound" variant refused by default. Today the check lives in an implementation; it belongs in the contract. This is the one change that makes the whole finding class structurally impossible.
3. Give SSH a real binding before any ingress lands. Since the subprocess carriage cannot do SSH channel binding, add a signed-challenge handshake at the mesh layer (hub sends a fresh nonce, spoke signs it with the AgentKey) as a precondition for producing a bound RequestContext. This also fixes the QUIC handshake's own replayability and makes the check portable. Port ensure_trustable alongside it.
4. Fix the host-key story per this repo's own design: StrictHostKeyChecking=yes + pinned UserKnownHostsFile + @cert-authority. If TOFU must remain for bootstrap, make it an explicit per-target opt-in, not the hardcoded default.
5. Stop discarding the alarms. Pipe stderr, drain it on a task, log via the already-declared tracing, inspect exit status, and construct Auth/HostKey from real signatures.
6. Justify or revert the iroh downgrade.
7. Add the seven hostile tests, with the stranger-rooted and relayed-envelope cases as acceptance evidence, plus a read timeout, --, and field validation.

Items 1, 6 and 7 are cheap and can land with this PR. Items 2–4 are the real gate and should block any commit that gives this crate an ingress path — the moment a Transport impl or a loopback listener appears, F1/F2/F3/F6 compose into a working impersonation of any agent whose envelope an attacker can observe once.

[hartsock]

Addendum — three corroborating points from the agent-mesh side

Contributed by the session that authored the Field Guide (#76); posting them here because two materially change the recommendations above and one would have made the suggested tests vacuous.

1. The Field Guide already documents this as a standing residual — the SSH carriage doesn't introduce it, it inherits it and removes the only thing that closed it.

The auto-team handshake by itself is characterised as "a filter, not a proof": the Hello cert is checked without proof-of-possession and is not bound to conn.remote_id(). What upgrades a QUIC-arrived request from filtered to proven is #75's envelope_matches_session exact-compare of the envelope signer pubkey against the iroh TLS session key. That is the whole delta.

So the finding above can be stated more sharply than I did: an SSH-arrived request is not merely unbound, it is left holding exactly the residual the Field Guide already flags — and any responder authorizing on "same UserKey" alone is trusting a cert that may have been harvested or relayed. This also means F1 and F2 are not new risks introduced by this PR so much as the removal of the one mechanism that mitigated a known one, which is a stronger reason to fix it at the seam (recommendation 2) rather than per-transport.

2. There is an in-house first-contact ceremony that already rejects TOFU — mirror it rather than inventing one.

Regarding F3: newt-agent#1643's docks.d design deliberately does not accept-new. It uses an operator-root-gated signed registry, a 6-word SAS ceremony for first contact, and a generation bump on revoke that drops live sessions. That is the established pattern for exactly the problem StrictHostKeyChecking=accept-new is being used to paper over here, and it is stronger than the @cert-authority suggestion I made in recommendation 4 because it also gives you revocation with live-session teardown.

Recommendation 4 should therefore read: adopt the docks.d ceremony shape for first contact, with StrictHostKeyChecking=yes plus a pinned UserKnownHostsFile as the SSH-layer expression of it. Inventing a third trust-establishment mechanism for this transport would be the wrong move when the repo already has one that fails closed.

3. Warning for anyone implementing the hostile tests in F9 — do not ride the in-memory transport.

InMemoryTransport / MeshNet skip env.verify(). A test built on them therefore cannot demonstrate signature enforcement: it will pass whether or not verification happens, which is precisely the vacuous-test shape flagged in F9.

So the two acceptance tests — stranger-rooted envelope refused and relayed envelope refused — must run against a real transport, or carry an explicit tamper assertion that fails if verification is skipped. Written naively on InMemoryTransport they would go green while proving nothing, and would then stand in the record as evidence that the hole is closed.

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_62f1a830-d2a4-425d-8c11-8231a0c5d41f)