Skip to content

Add payload_format raw: send the NATS payload as the event data (#143) - #154

Merged
bpacholek merged 1 commit into
mainfrom
fix-143-raw-payload
Sep 27, 2026
Merged

bpacholek merged 1 commit into
mainfrom
fix-143-raw-payload

Conversation

@bpacholek

Copy link
Copy Markdown
Contributor

Summary

Closes #143.

An event's data was always the JSON envelope {"topic","payload","time"}. Clients that only want the payload had to unwrap it, plain-text payloads arrived as escaped JSON strings, and the envelope cost 120–150 bytes of max_event_size.

Changes

  • New directive payload_format envelope | raw, default envelope.
  • Raw frames. The event's data is the NATS payload itself, one data: line per line of it, which EventSource joins with \n again. An empty payload is one empty data: line and still dispatches an event. id: stays last (M11-11: Delivery visibility nits — id-first frames vs eager SSE parsers, invisible intentional skips, unattributable slow-consumer drops #107). The connected and reset events keep their JSON data.
  • Unsendable payloads. SSE data is UTF-8 text, and a carriage return ends a line. A payload that is not valid UTF-8, or holds a carriage return, is dropped, logged, and counted as nuts_messages_dropped_total{reason="raw_not_text"}. An oversized payload keeps its raw_payload reason.
  • Plumbing. Parsed in caddyfile.go, validated and defaulted in provision.go. Documented in the README (listing, payload_format section, event format, the metrics row), CONFIGURATION, the website, metrics help and the CHANGELOG. A new /raw route in Caddyfile.test.

Tests

  • Unit. TestFormatMessageEvent_RawPayload: exact frames for single-line, multi-line, trailing-newline, empty and non-ASCII payloads; drops for CR, CRLF and invalid UTF-8; oversize precedence; the drop's metric and log.
  • Fuzz. FuzzFormatMessageEventMatchesReference now also varies raw mode, with an independent reference. It adds a round-trip property: whenever raw mode keeps a message, what EventSource makes of the frame is exactly the payload. 30 s ran clean.
  • Integration. TestHandler_RawPayloadFormat, against a real server: a multi-line payload arrives intact, a CR payload is dropped, and the stream goes on.
  • Directive. Caddyfile full-config DeepEqual, validation rows and a JSON row.
  • Functional. The scenario "Events carry the raw payload" on /raw; 33/33. The full suite passes under -race; lint is clean.
  • Mutation.
    • gremlins: caddyfile.go 40/0, provision.go 207/0 (16 not covered, the baseline's), serve.go 215/0.
    • 7 hand mutants, all killed. They cover the UTF-8 check, the CR check, the line loop's advance (killed by timeout), the drop's log case, the Caddyfile assignment, the default, and the validation case.
  • Example stack. /raw?topic=numbers gives data: {"number":50}; /events gives the envelope.

🤖 Generated with Claude Code

An event's data was always the JSON envelope {"topic","payload","time"},
so clients that only want the payload had to unwrap it, and plain-text
payloads arrived as escaped JSON strings.

payload_format raw writes the NATS payload itself: one data line per
line of it, which EventSource joins with "\n" again (an empty payload is
one empty data line and still dispatches an event). SSE data is UTF-8
text, and a carriage return ends a line. So a payload that is not valid
UTF-8, or holds a carriage return, cannot be sent: it is dropped, logged
and counted as nuts_messages_dropped_total{reason="raw_not_text"}. An
oversized payload keeps its raw_payload reason. The default, envelope,
changes nothing.

Closes #143

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@bpacholek
bpacholek merged commit 7f20db0 into main Sep 27, 2026
5 checks passed
@bpacholek
bpacholek deleted the fix-143-raw-payload branch September 27, 2026 23:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optional raw payload mode: the NATS payload as the SSE data, without the JSON envelope

1 participant