Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ jobs:

steps:
- name: Check out repository
uses: actions/checkout@v6
uses: actions/checkout@v7

- name: Set up Go
uses: actions/setup-go@v6
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
Expand Down Expand Up @@ -105,7 +105,7 @@ jobs:
run: make test-functional

- name: Upload coverage to Codecov
uses: codecov/codecov-action@v6
uses: codecov/codecov-action@v7
with:
files: ./coverage.out
flags: unittests
Expand All @@ -124,7 +124,7 @@ jobs:

steps:
- name: Check out repository
uses: actions/checkout@v6
uses: actions/checkout@v7

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
Expand Down Expand Up @@ -172,12 +172,12 @@ jobs:

steps:
- name: Check out repository
uses: actions/checkout@v6
uses: actions/checkout@v7
with:
fetch-depth: 0

- name: Set up Go
uses: actions/setup-go@v6
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
Expand Down Expand Up @@ -240,7 +240,7 @@ jobs:

steps:
- name: Check out repository
uses: actions/checkout@v6
uses: actions/checkout@v7

- name: Set up QEMU
uses: docker/setup-qemu-action@v4
Expand Down Expand Up @@ -334,7 +334,7 @@ jobs:

- name: Install Cosign
if: startsWith(github.ref, 'refs/tags/v')
uses: sigstore/cosign-installer@v3
uses: sigstore/cosign-installer@v4
with:
# Match the pin in release.yml so both signing surfaces
# (docker image here, checksums.txt in release.yml) use the
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/fuzz.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,10 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v7

- name: Set up Go
uses: actions/setup-go@v6
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/mutation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v7

- name: Set up Go
uses: actions/setup-go@v6
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
Expand All @@ -45,7 +45,7 @@ jobs:

- name: Cache gremlins binary
id: cache_gremlins
uses: actions/cache@v5
uses: actions/cache@v6
with:
path: ~/go/bin/gremlins
key: gremlins-${{ runner.os }}-${{ steps.gremlins_version.outputs.version }}
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,12 +62,12 @@ jobs:
timeout-minutes: 20
steps:
- name: Check out repository
uses: actions/checkout@v6
uses: actions/checkout@v7
with:
fetch-depth: 0

- name: Set up Go
uses: actions/setup-go@v6
uses: actions/setup-go@v7
with:
go-version-file: go.mod
cache: true
Expand All @@ -87,7 +87,7 @@ jobs:
uses: anchore/sbom-action/download-syft@v0

- name: Install Cosign for checksum signing
uses: sigstore/cosign-installer@v3
uses: sigstore/cosign-installer@v4
with:
# Pin cosign itself, not just the installer action. The rest
# of the supply-chain tooling pins tightly (golangci-lint
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/website.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:
timeout-minutes: 15
steps:
- name: Check out repository
uses: actions/checkout@v6
uses: actions/checkout@v7

- name: Set up Ruby
uses: ruby/setup-ruby@v1
Expand All @@ -49,7 +49,7 @@ jobs:
# 22, not 20: the pinned Wrangler below requires Node >= 22. Keep in step with
# website/Dockerfile so `make website-build` still mirrors this build.
- name: Set up Node
uses: actions/setup-node@v4
uses: actions/setup-node@v7
with:
node-version: '22'
cache: npm
Expand Down
78 changes: 58 additions & 20 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,23 @@ to replace it.
sequence after a NATS reconnect or consumer loss; a client whose writes
stop completing is disconnected at `write_timeout` and resumes via
`Last-Event-ID`.
- `shared_subscriptions`: connections caught up with the live stream share
one consumer per topic set, and move to their own when they fall behind.
- A stream watch: stream-info reads, on every stream request and every 10 s,
notice a JetStream stream recreated or restored under open SSE streams.
Those streams end with a `reset` event whose id, `0`, makes EventSource
replay the new stream (`stream_recreated`, `stream_rewound`).
- NATS liveness: pings every `nats_ping_interval` (20 s) declare a server
that stopped answering stale within a minute. Until then, a stream-info
read that times out makes NUTS ping, and while the ping goes unanswered,
requests are told to retry at once.
- Event shaping: `event_type` names events (`message`, `topic`, or
`header <name>`), `payload_format raw` sends the NATS payload itself, and
`event_id_format sequence_time` makes ids `<sequence>-<unix ns>`, so that
a cursor from a replaced stream is noticed.
- Retry hints: `sse_retry` sends `retry:` with each handshake, and
`transient_retry` sets the average of the jittered delay NUTS asks for
after transient failures.
- NATS authentication (credentials file, token, user/password) and TLS / mTLS
to the NATS server.
- Optional first-party subscriber JWT auth (`subscriber_jwt_key`,
Expand All @@ -44,10 +61,13 @@ to replace it.
- Per-frame write deadlines (`write_timeout`, 30 s by default).
`dispatch_timeout` is deprecated and has no effect.
- Probes: `live_path` (`/livez`), `ready_path` (`/readyz`), and the legacy
`health_path` (`/healthz`).
`health_path` (`/healthz`); `health_details` adds the NATS server and the
stream's details to the readiness JSON.
- Hub discovery via `Link: <url>; rel="nuts"` when `hub_url` is set.
- Prometheus metrics (`nuts_*`) registered via `promauto`, surfaced through
Caddy's `/metrics` handler.
- A dependency-free JavaScript helper for replay-aware pages,
[example/nuts-client.js](example/nuts-client.js).

A current and complete directive list with defaults, JSON field names,
validation rules, and operational notes is in
Expand Down Expand Up @@ -82,7 +102,7 @@ Tests live alongside the source:
| [serve_test.go](serve_test.go) | Request parsing, replay planning, formatting, and stream helper unit tests without live NATS. |
| [serve_integration_test.go](serve_integration_test.go) | `ServeHTTP` end to end on an embedded JetStream server: streaming, replay cursors, write failures, probes, hub discovery, heartbeats, topic prefixes, NATS restarts. |
| [hardening_test.go](hardening_test.go) | Request hardening: CORS, connection, topic and event-size limits, probe paths, replay caps and windows, oversized cursors. |
| [consumer_test.go](consumer_test.go) | Feed unit tests (ordering, oversize drops, backpressure, failure and stop paths) with a fake iterator; recovery of a deleted consumer. |
| [consumer_test.go](consumer_test.go) | Feed unit tests (ordering, oversize drops, backpressure, failure and stop paths) with a fake iterator; recovery of a deleted consumer; the stream watch (generations, recreation and rewind detection, the poll's continuity check) and the `reset` frame. |
| [shared_test.go](shared_test.go) | Shared-subscription mechanics (ring, gap-free joins, fall-behind, lifecycle) and end-to-end behaviour; the delivery contract tests also run in shared mode. |
| [helpers_test.go](helpers_test.go) | Helper edge cases: topic and cookie validation, JSON, URL redaction, SSE writes and deadlines, NATS error classification. |
| [metrics_test.go](metrics_test.go) | Each metric moves with the event it counts; registration on Caddy's registry. |
Expand All @@ -92,7 +112,7 @@ Tests live alongside the source:
| [caddy_integration_test.go](caddy_integration_test.go) | Caddy-in-the-loop tests via `caddy.Load` (access logs, HTTP metrics, the metrics endpoint). |
| [performance_test.go](performance_test.go) | `TestPerformance_*` confidence tests and benchmarks. |
| [fuzz_test.go](fuzz_test.go) | Fuzz targets holding the topic, filter and cookie validators to their contracts in both directions, and the subject matchers to a reference; the nightly Fuzz workflow runs every target. |
| [docs_test.go](docs_test.go) | Keeps metric names in README, website, docs and ops files in step with `metrics.go`, and the nightly fuzz matrix in step with the `Fuzz` functions. |
| [docs_test.go](docs_test.go) | Keeps metric names in README, website, docs and ops files in step with `metrics.go`, the defaults documented in `docs/CONFIGURATION.md` in step with `Provision`, and the nightly fuzz matrix in step with the `Fuzz` functions. |
| [testutil_test.go](testutil_test.go) | Helpers shared by more than one test file, by theme: embedded NATS servers (and restarts on the same port), handler set-up, response writers, SSE stream readers and assertions, fakes for the stream path, metric and log readers, subscriber JWTs. A helper used by one file lives in that file. |
| [functional_test/](functional_test/) | [Godog](https://github.com/cucumber/godog) BDD tests against a real Docker Compose stack. |
| [features/](features/) | Gherkin `.feature` files driving the Godog suite. |
Expand Down Expand Up @@ -265,35 +285,52 @@ that gap before review starts, instead of leaking it into review load.

## CI and release surfaces

CI is GitHub Actions ([.github/workflows/ci.yml](.github/workflows/ci.yml),
[.github/workflows/release.yml](.github/workflows/release.yml),
[.github/workflows/website.yml](.github/workflows/website.yml)).
CI is GitHub Actions:

| Workflow | Runs on |
| --- | --- |
| [ci.yml](.github/workflows/ci.yml) | Every pull request and every push to `main`/`master` or a `v*` tag. |
| [release.yml](.github/workflows/release.yml) | `v*` tags. |
| [website.yml](.github/workflows/website.yml) | `v*` tags, and `workflow_dispatch` for a manual redeploy. |
| [fuzz.yml](.github/workflows/fuzz.yml) | Nightly at 04:00 UTC: every fuzz target. |
| [mutation.yml](.github/workflows/mutation.yml) | Weekly on Sunday at 03:00 UTC: the full gremlins run. |

Actions are pinned by major version (`@v7`), and Dependabot proposes the
bumps.

PRs run, in this order:

1. `gofmt`, `go mod tidy` diff check, golangci-lint, unit tests with coverage,
focused race tests, `go vet`, the example JavaScript client's tests,
`govulncheck`.
the whole unit package under `-race`, `go vet`, the example JavaScript
client's tests (`make test-js`), `govulncheck`.
2. Functional test matrix (`nats:2.10-alpine`, `nats:2.12-alpine`,
`nats:2.14-alpine`, `nats:2.15-alpine`) and a 3× functional stress pass.
`nats:2.14-alpine`, `nats:2.15-alpine`), a 3× functional stress pass, and
one functional pass under `-race`.
3. Coverage upload to Codecov.
4. Production Docker image build + `caddy adapt` validation + Trivy scan +
SBOM (SPDX JSON) artifact.
5. GoReleaser config check + snapshot dry run when release-relevant files
change ([.goreleaser.yml](.goreleaser.yml), workflows, `Makefile`,
`Dockerfile`).

Pushes to `main`/`master` and `v*` tags additionally:
Pushes to `main`/`master` additionally build the multi-arch (`linux/amd64`,
`linux/arm64`) image, scan it with Trivy before and after the push, and push
it as `idcttech/nuts:latest` with provenance and SBOM attestations.

`v*` tags additionally:

1. Build, scan and push the image the same way as `idcttech/nuts:<tag>`
(a tag never moves `latest`), then sign it keylessly via Cosign + GitHub
OIDC.
2. Once CI has passed on the tagged commit, GoReleaser publishes the GitHub
release: archives, their SBOMs, and `checksums.txt` signed with Cosign
(`checksums.txt.sig`, `checksums.txt.pem`).
3. Build the [website/](website/) Jekyll site and deploy it to Cloudflare at
`https://idct.tech/nuts`. Docs changes merged to `main` go live with the
next tag.

1. Build and push the multi-arch (`linux/amd64`, `linux/arm64`)
`idcttech/nuts` image with provenance and SBOM attestations.
2. On tag: scan the pushed image by digest with Trivy and sign all tags
keyless via Cosign + GitHub OIDC.
3. Update the Docker Hub description from [DOCKERHUB_README.md](DOCKERHUB_README.md).
4. GoReleaser publishes archives and SBOMs for the tag.
5. Build the [website/](website/) Jekyll site and deploy it to Cloudflare at
`https://idct.tech/nuts` ([website.yml](.github/workflows/website.yml); also
runnable on demand via `workflow_dispatch`).
No workflow updates the Docker Hub description; its text is in
[DOCKERHUB_README.md](DOCKERHUB_README.md).

Release surfaces and verification commands are documented in
[docs/RELEASE.md](docs/RELEASE.md). Operator-facing release notes follow the
Expand All @@ -306,7 +343,8 @@ The published artefact set:
- `idcttech/nuts:<version>` Docker image (multi-arch, non-root uid 10001,
Cosign-signed, SBOM-attested for tag releases). `:latest` is mutable — pin
to a concrete tag in production.
- GitHub release archives with SHA-256 checksums and SBOMs.
- GitHub release archives with SBOMs, and SHA-256 checksums signed with
Cosign.
- Source build via `go build ./cmd/caddy` or `xcaddy build --with
github.com/ideaconnect/nuts`.

Expand Down
40 changes: 40 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,46 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.6.0] - 2026-09-28

A development release on the 0.x line. SSE streams on a JetStream stream that
was recreated or restored no longer go silent, and requests no longer wait on
a NATS server that stopped answering. New optional directives name events,
send raw payloads, put the message's time in event IDs, add readiness details
and set the retry hints, and `example/nuts-client.js` gives pages
replay-aware subscriptions. Nothing in an existing configuration has to
change.

### Upgrading

**Operator impact.** Upgrade urgency: **high** if you recreate or restore
JetStream streams while NUTS serves them. Open SSE streams skipped the
replaced stream's messages until it passed their old position, while
heartbeats kept them looking healthy (#133). **Medium** otherwise.

- **Config changes:** none required. Every new directive is optional, and
its default keeps 0.5.0's behaviour. Two changes you may notice:
- NUTS pings the NATS server every 20 seconds (`nats_ping_interval`)
instead of every two minutes, so a server that stops answering is
declared stale within a minute (`disconnected from NATS`,
`stale connection`).
- SSE streams on a replaced stream end with a `reset` event, and
EventSource replays the new stream from its start. Bound those replays
with `replay_max_messages` or `replay_window`; the
`NutsStreamReplacedUnderClients` rule in `ops/prometheus-alerts.yml`
alerts on them.
- **Deployment checks:** each handler reads its stream's info every 10
seconds. The readiness JSON names servers, clusters and limits only with
`health_details`, which public probe paths should leave off.
- **Rollback:** back to 0.5.0 is safe unless `event_id_format
sequence_time` was on. 0.5.0 does not parse `<sequence>-<unix ns>`
cursors: it answers such a `?last-id=` with `400` and ignores such a
`Last-Event-ID`, so those clients start from new messages.

**Compatibility.** Caddy 2.11.x (built with v2.11.4), Go 1.26.8,
nats-server 2.10 or newer (2.14.7 or newer for multi-topic subscriptions),
as in 0.5.0.

### Added
- `example/nuts-client.js`, a dependency-free ES module wrapping
EventSource for replay-aware pages (#146). It keeps the cursor of every
Expand Down
Loading
Loading