[Silthus]

Problem

A team's sending tier grows with the email it sends through its own domain.
Sandbox sends go out through a shared PostHog identity, but they record email_sent, which the tier history and the team's sending allowance read.
So sandbox sends could lift a team's own tier, or use up its allowance.
This PR makes the tier and the allowance subtract sandbox sends.

Part of Silthus/personal#199 (sandbox delivery). Stacked on #608.
Replaces the Python half of #606, which mixed Python with Node. Originally #454.

Stack

• Slice V3 of the sandbox sender rollout: a paused shared SES tenant stops sandbox sends, and sandbox sends leave the sending tier. Slice order: feat(workflows): mirror the sandbox ses tenant state in django #607 → feat(workflows): skip sandbox sends while the shared ses tenant is paused #608 → feat(workflows): subtract sandbox sends from the sending tier #609 → feat(workflows): record sandbox deliveries as their own metric #610.
• Base: workflows-sandbox-delivery/tenant-pause-gate (feat(workflows): skip sandbox sends while the shared ses tenant is paused #608). Lands after feat(workflows): skip sandbox sends while the shared ses tenant is paused #608 and every PR below it.
• Production lines: 17 (tests, stories, generated types, migrations and docs excluded).
• Rebuilt on 2026-10-28 on the current master, in the new slice order. Test imports merged with master.
• Merged on 2026-10-31. No conflicts and no change to its own lines.
• Lane gate on 07b56bedb66: tier and reputation suites 92 passed, ruff clean. Pre-push hogli ci:preflight --strict passed. Review: GPT-6.1 Sol (high) read the conflict resolutions of the master merge, no findings.
• Proof on 07b56bedb66:
  • Live run on 2026-10-31 at the slice tip 1bb35bd788e: a real Django server, cdp-api and email worker, with SES, the SNS certificate host, MX and personhog stubbed. The tier history left the sandbox sends out (sent=0) and still counted their bounces and complaints.
  • Proof gate: fails only on the ci gap (exit 3). Fork CI cannot run Semgrep or the Depot jobs, and the lane check counts the layers below, because it diffs against master.

Impact

Grow Workflows, activation rate: the tier keeps measuring a team's own sending once sandbox sends exist.
No production team sees this yet: WORKFLOWS_SANDBOX_SENDER_ENABLED stays off.

Merge Danger

Warning

🟡 Merge risk: medium · two-way door (merge-risk rubric)

• The tier sweep subtracts email_sandbox_sent from email_sent per day, and the allowance per workflow, floored at zero.
• Safe to deploy alone: no code records email_sandbox_sent until the worker metric (feat(workflows): record sandbox deliveries as their own metric #610) deploys, so the subtraction is zero and tier results do not change.
• Own-domain sends count as before.

Production lines: 17 changed (git diff --numstat against #608, tests and docs excluded).

Changes

Where	Before	After
Tier history (_history_from_daily)	Counts email_sent	Counts email_sent minus email_sandbox_sent per day, floored at zero
Team sending allowance	Counts email_sent	The same subtraction per workflow, floored at zero

• Sandbox bounces and complaints still count toward the tier's rates.

How did you test this code?

Live run on the slice tip

• Live run of the V3 tip 7e95863f7a0 on a PostHog preview. Stubs: a local SES endpoint that records each call, and example.com seeded as a valid MX domain; a local SNS signing certificate, so a signed delivery webhook verifies; the trigger event produced straight onto the CDP events topic; personhog pointed at the persons database.

• SES tenant DISABLED, then the Django sync: the mirror row reads DISABLED, and the next sandbox send is skipped with "the sandbox sender is paused right now". ENABLED and a sync: the next send goes out. The worker picked up each change without a restart.

• A live workflow send on the sandbox sender recorded email_sent and email_sandbox_sent once each, the pair the sending tier subtracts.

• A signed SES Delivery notification for that message on the sandbox configuration set returned 200 and captured workflows sandbox email delivered. The same envelope with a forged signature returned 403.

• Lane gate on HEAD: the pre-push hogli ci:preflight --strict passes. The lane classifier puts this PR on the Python side only.

• Red: on feat(workflows): skip sandbox sends while the shared ses tenant is paused #608's code, 4 tier history cases in test_email_sending_tier.py and 1 allowance case in test_email_reputation.py fail.

• Review fix: the allowance case fails on a mutant that subtracts sandbox sends team-wide.

• Green on HEAD: test_email_sending_tier.py, test_email_sending_tier_notifications.py, test_email_reputation.py and test_workflow_email_health.py, 128 passed.

• Test seams: the tier sweep over seeded daily metrics, and the email reputation API's allowance.

• Proofs owed (fleet proof gate):

  • Review: GPT-6.1 Sol, 1 round, 1 finding fixed in f7e9df3170c (the allowance case now pins the per-workflow floor); final read: no findings.
  • Live run: passed at the stack tip (69afe3e4b1f), recorded on every delivery head. One real email worker process (cdp-cyclotron-worker-email) on scratch Postgres, with real Redis, Kafka and ClickHouse: a member received the email, a non-member was skipped, a paused tenant was skipped, the daily team cap held, and each delivered send recorded email_sandbox_sent, which the tier's own ClickHouse read then left out. Stubbed: the SES endpoint (a local HTTP stub behind the real SESv2 client), the MX verdict for the reserved example.com domain, and the tenant-state rows the Django sync would write.
  • Merge-queue lanes: CI computes this check against master, so it reads as mixed until the Node layers below land. Land the stack bottom-first, not with one trunk merge on the top layer.
  • Prod shape: gap, accepted until read-only production access arrives. The sandbox tenant is not configured in production.
  • CI: gap. Fork CI cannot run on Silthus/posthog.

Test rationale: the regressions that matter are sandbox sends lifting a team over the use bar, one day's sandbox excess erasing another day's own sends, and sandbox bounces disappearing from the rates.

Release status

• This change is behind a feature flag and is not available to users

Docs update

docs/internal/workflows-email-sending-tiers.md documents the tier subtraction.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (implementation). Review by Codex, GPT-6.1 Sol.

Split out of #605 and #606 so each PR passes upstream's merge rules: a migration does not share a PR with nodejs/, and Python does not share a PR with Node. The code is carved by file from those reviewed heads without changes; the stack's tip has the same tree as #606.
Skills: tdd, /writing-tests, pr-description, merge-risk.
Public artifact: all test data is invented.

Decisions

Question	Answer	Who	Where
Slice order of the sandbox sender stack	Option B: the Channels row moves into V1 so a team has a sender to test from; V1's worker PRs land first	👍 Accepted default	map
How to split #605 and #606	Four fresh single-commit branches off #604, in deploy order: Django mirror, worker gate, Python tier, worker metric; #605 and #606 close	🤖 Agent	ticket
How a mixed-lane stack lands	Bottom-first; CI checks lanes against master, so each layer passes once the layers below it land	🤖 Agent	ticket
Where the docs lines go	Each half documents its own behavior; the tip's doc matches #606	🤖 Agent	ticket
Sandbox delivery metric	Keep email_sent for the workflow metrics tab, add email_sandbox_sent, and subtract it in the tier per day and in the allowance per workflow, floored at zero	🤖 Agent	ticket

Involvement: no grilling rounds; design pass carried from #605 and #606 (the code is unchanged); review (GPT-6.1 Sol, 1 round and a final read, 1 finding fixed); proof captured as test output and a live run; master merge on 2026-10-31: review of its conflict resolutions (GPT-6.1 Sol, no findings) and a fresh live run at the slice tip.

🤖 Generated with Claude Code