Skip to content

feat(sidecar): evm-logical-digest task + shared S3 emission seal + task panic isolation#211

Merged
bdchatham merged 4 commits into
mainfrom
feat/evm-logical-digest-gate
Jun 20, 2026
Merged

feat(sidecar): evm-logical-digest task + shared S3 emission seal + task panic isolation#211
bdchatham merged 4 commits into
mainfrom
feat/evm-logical-digest-gate

Conversation

@bdchatham

Copy link
Copy Markdown
Contributor

Adds the full-keyspace digest gate (sei-chain#3611's seidb evm-logical-digest) as a discrete sidecar task — the per-segment boundary seal that closes the touched-key comparator's cold-state blind spot (a key migrated wrong and never touched again is invisible to per-block Layer 2). Plus three seams the systems-engineering review called for. No "ShadowResultProducer" abstraction — that's deferred to the 3rd producer (YAGNI).

What's here

  • sidecar/s3/emit.go — one S3 emission helper (StreamGzipNDJSON/StreamGzipJSON/StreamGzipFunc), collapsing 3 duplicated gzip-pipe paths. Twofold integrity seal: an aws-chunked SHA-256 wire checksum over the compressed body (io.Pipe streaming/backpressure preserved) + an uncompressed-payload SHA-256 surfaced via EmitResult for out-of-band verification. result_compare/result_export refactored onto it (no behavior change).
  • sidecar/enginerecover() in runTask turns a handler panic into a failed TaskResult (+ seictl_task_panics_total) instead of crashing the sidecar.
  • sidecar/tasks/evm_logical_digest.go — the discrete task: shells out to seidb for flatkv + memiavl (semantic + translator), asserts both backends' opened version == height (fail-closed — no wrong-height false match), parses the FINAL_DIGEST/per-bucket contract, publishes an EndpointDigestRecord. axes_proved deliberately omits balance (the semantic account digest zeroes it — that axis stays the per-block comparator's job).

Cross-review (systems-engineer + idiomatic-reviewer) — applied

  • Symmetric memiavl version assertion (the flatkv-only check left a wrong-height false-match hole if seidb clamps to the nearest snapshot).
  • recover() inside the s3 writer goroutine — a panic there (e.g. MarshalJSON over chain data) runs on a task-spawned goroutine outside the engine's handler recover; converted to a returned error so the upload aborts (no truncated-but-valid object) and the process survives.
  • Dropped the empty-by-construction uncompressed_sha256 from the published record (a record can't carry the hash of its own bytes; the seal is out-of-band in the log/TaskResult).
  • comment-precision fixes (memory bound is the uploader part-pool, not "gzip window"; S3 checksum is per-part-composite for multipart) + a version:-line length guard.

Notes

  • seidb is shelled out to (configurable seidbPath), not vendored. #3611 also needs a one-line registration fix (EvmLogicalDigestCmd() isn't in seidb's root AddCommand) — flagged to the author.
  • Trigger is the out-of-band task API; no controller/CRD change (consistent with the avoided ResultExportConfig one-way door).
  • One-way-door surfaces for confirmation before a consumer reads them: the task-type string "evm-logical-digest", the param field names, and the EndpointDigestRecord schema.
  • GOWORK=off go build ./... clean; go test ./sidecar/... green (incl. new memiavl-version-mismatch + writer-panic regression tests); gofmt -s clean.

🤖 Generated with Claude Code

…sk panic isolation

Add the offline full-keyspace digest gate (sei-chain#3611's seidb
evm-logical-digest) as a discrete sidecar task — the per-segment boundary
seal that closes the touched-key comparator's cold-state blind spot. Plus
three seams the systems-engineering review called for; no producer
abstraction (extract at the 3rd producer, not the 2nd).

- sidecar/s3/emit.go: single S3 emission helper (StreamGzipNDJSON/JSON/Func),
  collapsing 3 duplicated gzip-pipe paths. Twofold integrity seal: an
  aws-chunked SHA-256 wire checksum over the compressed body (streaming
  preserved) + an uncompressed-payload SHA-256 surfaced via EmitResult for
  out-of-band verification. result_compare/result_export now call it.
- sidecar/engine: recover() in runTask converts a handler panic into a failed
  TaskResult (+ seictl_task_panics_total) instead of crashing the sidecar.
- sidecar/tasks/evm_logical_digest.go: the discrete task. Shells out to seidb
  for flatkv + memiavl (semantic + translator), asserts BOTH backends'
  opened version == requested height (fail-closed; no wrong-height false
  match), parses the FINAL_DIGEST/per-bucket contract, and publishes an
  EndpointDigestRecord. axes_proved deliberately omits balance (the semantic
  account digest zeroes it — that axis stays the per-block comparator's job).

Cross-reviewed (systems-engineer + idiomatic): symmetric memiavl version
assertion, a recover() inside the s3 writer goroutine (a panic there escapes
the engine's handler recover), and the empty-by-construction uncompressed
hash dropped from the published record (its seal is out-of-band).

seidb is shelled out to (configurable path), not vendored. Trigger is the
out-of-band task API; no controller/CRD change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cursor

cursor Bot commented Jun 20, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
New stable task type/param/schema contracts and integrity-sensitive digest publishing; S3 upload refactor touches export/compare paths though behavior is intended unchanged aside from checksums and clearer error classification.

Overview
Adds an evm-logical-digest sidecar task that runs seidb evm-logical-digest on flatkv and memiavl at a given height (default semantic and translator normalizations), fails closed if either backend’s opened version ≠ requested height, and publishes gzipped EndpointDigestRecord verdicts to S3 with the uncompressed SHA-256 logged out-of-band.

Introduces sidecar/s3/emit.go as the single streaming gzip→S3 path (pipe backpressure, S3 SHA-256 checksum on upload, reader-verifiable hash over pre-gzip bytes). result-export and result-compare drop their duplicated pipe/gzip code in favor of these helpers; export also separates RPC/producer errors from S3 failures when the upload pipe breaks mid-stream.

The task engine now recover()s handler panics into failed TaskResults and exposes seictl_task_panics_total; the emit writer goroutine has its own recover so marshal panics abort the upload instead of crashing the sidecar.

Reviewed by Cursor Bugbot for commit bd67199. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread sidecar/tasks/result_export.go
Comment thread sidecar/tasks/evm_logical_digest.go
…gbot)

exportPage routed every StreamGzipFunc error through ClassifyS3Error, so a
collectResults failure (block_results RPC, marshaling, ctx cancel) surfaced as
a misleading S3 access error. Capture the producer error and return it as-is;
ClassifyS3Error now wraps only a genuine upload failure.

Also document that runSeidb leaves cmd.Stderr nil so Output() captures stderr
into ExitError.Stderr (verified: the existing stderrTail already works — the
second Bugbot finding was a false positive).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bdchatham

Copy link
Copy Markdown
Contributor Author

Addressed both Bugbot findings in 073ff7b:

  • Bug 1 — RPC failures mislabeled as S3 errors — fixed. exportPage now captures the producer error and returns it unclassified; ClassifyS3Error wraps only a genuine upload failure. A block_results RPC / marshal / ctx-cancel failure is no longer surfaced with misleading S3 hints.
  • Bug 2 — seidb stderr never captured — verified false positive. cmd.Output() populates *exec.ExitError.Stderr precisely when cmd.Stderr is nil (which it is here), so stderrTail already surfaces seidb's stderr. Confirmed empirically — the same Output() pattern over sh -c 'echo "seidb: snapshot not found" >&2; exit 1' yields ExitError.Stderr = "seidb: snapshot not found\n". Added a clarifying comment so the (correct) reliance on that stdlib behavior is explicit rather than looking like an omission.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 073ff7b. Configure here.

Comment thread sidecar/tasks/result_export.go
bdchatham and others added 2 commits June 20, 2026 10:31
…es (Bugbot)

The prior fix returned collectErr whenever set, which dropped ClassifyS3Error's
Retryable hint when S3 failed mid-stream (the upload closes the pipe, surfacing
a wrapped write error in collectErr while uploadErr holds the real S3 cause).

Tag the sink-write path in collectResults with errSinkWrite so exportPage can
tell a genuine producer fault (RPC/marshal/ctx -> return as-is) from a
downstream sink/S3 fault (-> ClassifyS3Error on uploadErr, retaining Retryable).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Regression tests for the twice-flagged path: a block_results RPC fault is
returned unclassified; a genuine upload failure keeps its ClassifyS3Error
(Operation S3) treatment.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bdchatham

Copy link
Copy Markdown
Contributor Author

Third finding (S3 errors misreported on export) — fixed in bd67199, good catch. The prior fix over-corrected: returning collectErr whenever set dropped ClassifyS3Error's Retryable hint when S3 failed mid-stream. Now collectResults' sink-write path is tagged (errSinkWrite), so exportPage distinguishes a genuine producer fault (RPC/marshal/ctx → returned as-is) from a downstream sink/S3 fault (→ ClassifyS3Error on uploadErr, retaining Retryable). Locked with two regression tests covering both directions.

@bdchatham bdchatham merged commit a43ba3f into main Jun 20, 2026
3 checks passed
@bdchatham bdchatham deleted the feat/evm-logical-digest-gate branch June 20, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant