[Silthus]

Note

Replaced by #609 (Python tier subtraction) and #610 (worker metric and delivered event). This PR mixed lanes that upstream's merge rules keep apart, so it was split in two. Same code, carved by file.

Problem

A team's sending tier grows with the email it sends through its own domain.
Sandbox sends go out through a shared PostHog identity, but they still record email_sent, which the tier history and the team's sending allowance read.
So sandbox sends could lift a team's own tier, or use up its allowance.
This PR keeps sandbox sends out of both, and gives sandbox delivery its own event.

Part of Silthus/personal#199 (sandbox delivery, slice 3 of 3 there). Stacked on #605.
Replaces #454.

Impact

Grow Workflows, activation rate: sandbox sends stay a separate funnel step, and the tier keeps measuring a team's own sending.
No production team sees this yet: WORKFLOWS_SANDBOX_SENDER_ENABLED stays off.

Merge Danger

Warning

🟡 Merge risk: medium · two-way door (merge-risk rubric)

• The tier sweep subtracts email_sandbox_sent from email_sent per day, and the allowance per workflow, floored at zero. No sends record email_sandbox_sent until the sandbox switch is on, so tier results do not change before then.
• The SES webhook in the CDP API now takes the sandbox sender. It captures one more event for deliveries on SES_SANDBOX_CONFIGURATION_SET, and does nothing new while that setting is empty.
• Own-domain sends record the same metrics as before.

Production lines: 114 changed (git diff --numstat against #605, tests and docs excluded).

Changes

Where	Before	After
Worker, successful non-test sandbox send	email_sent	email_sent and email_sandbox_sent
Tier history (_history_from_daily)	Counts email_sent	Counts email_sent minus email_sandbox_sent per day, floored at zero
Team sending allowance	Counts email_sent	The same subtraction per workflow, floored at zero
SES webhook, delivery on the sandbox configuration set	No event	workflows sandbox email delivered with the recipient count, test sends included

• email_sent stays, so the workflow's metrics tab still shows sandbox sends.
• Sandbox bounces and complaints still count toward the tier's rates.
• The webhook captures the event only for a signed tracking code.

How did you test this code?

• Lane gate on HEAD: tsc passes. eslint, oxlint, prettier and ruff pass on the changed files. semgrep --config .semgrep/rules/security/ reports nothing on the lines this PR adds.
• Red on feat(workflows): skip sandbox sends while the shared ses tenant is paused #605's code: 2 sandbox send cases miss email_sandbox_sent, 4 webhook cases capture no delivered event, and 5 tier and allowance tests count sandbox sends.
• Green on HEAD: email.service.test.ts, email-tracking.service.test.ts, sandbox-email-sender.test.ts, rate-limiter.service.test.ts, sandbox-sender-state.service.test.ts and ses.test.ts; Django test_email_sending_tier.py, test_email_reputation.py and test_workflow_email_health.py.
• Test seams:
  • EmailService.executeSendEmail with a sandbox integration: the metrics it records.
  • The SES webhook route on CdpApi, with the SNS signature check stubbed: the delivered event.
  • _history_from_daily and the allowance endpoint, with ClickHouse app metrics seeded.
• Proofs owed (fleet proof gate):
  • Review: GPT-6.1 Sol, 1 round, no findings.
  • Live run: gap. Not run within the lane's time cap; the tests drive sends against real Postgres and Redis with the SES client stubbed. Owed before the sandbox switch goes on.
  • Prod shape: gap, accepted until read-only production access arrives.
  • CI: gap. Fork CI cannot run on Silthus/posthog.

Test rationale: the tier tests cover per-day subtraction, because a sandbox excess on one day must not erase own sends on another, and sandbox feedback must still count. The webhook tests cover the configuration set and the signed-code check, which decide whether an event fires.

Release status

• This change is behind a feature flag and is not available to users

Docs update

docs/internal/workflows-email-sending-tiers.md documents the subtraction, the delivered event, and that the switch waits for this metric.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (implementation). Design pass and review by Codex, GPT-6.1 Sol.

Ported from the reviewed source PR above onto #605 as a wayfinder ticket lane. The worker metric is new here: the source stack recorded it in a PR this stack does not carry. Skills: tdd, /writing-tests, pr-description, merge-risk, qa-swarm design pass.
No duplicate: this replaces the source fork PR named above, which stays open until the stack exists.
Public artifact: all test data is invented.

Decisions

Question	Answer	Who	Where
Sandbox delivery metric	Keep email_sent for the metrics tab, add email_sandbox_sent, and subtract it in the tier and the allowance	🤖 Agent	ticket

Involvement: no grilling rounds; design pass (1 lane, no findings); review (GPT-6.1 Sol, 1 round, no findings); proof captured as test output.

🤖 Generated with Claude Code