Skip to content

feat(workflows): test templates with the sandbox sender - #628

Draft
Silthus wants to merge 4 commits into
workflows-sandbox-product/channels-rowfrom
workflows-sandbox-product/test-sends
Draft

Silthus wants to merge 4 commits into
workflows-sandbox-product/channels-rowfrom
workflows-sandbox-product/test-sends

Conversation

@Silthus

@Silthus Silthus commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Replaces #624. GitHub marked #624 merged into its old base when the stack was reordered, so this draft carries the same branch.

Problem

Stack

Impact

Pays into Workflows activation through the sandbox funnel step: a team without a verified domain can send itself a template test before it sets up its own sender.

Merge Danger

Note

🔵 Merge risk: low · two-way door (merge-risk rubric)

  • Every change sits behind workflows-sandbox-sender; with the flag off the modal lists the same senders and prefills the recipient as on master.
  • The send request changes only when the sandbox sender is selected, and the worker still decides delivery.

Changes

  • With workflows-sandbox-sender on, "From" lists the sandbox sender after the verified own senders, with a "Sandbox" tag and the note "Delivers only to verified members of your organization." when it is selected.
  • With no verified own sender, the modal preselects the sandbox sender.
  • "Send to" suggests verified teammates, signed-in user first, and still accepts any typed address. It starts empty instead of prefilled, so the suggestions show.
  • With the sandbox sender selected:
    • An address outside the suggestions shows a warning with "Add your own sender". It does not block the send; the worker decides who receives it.
    • Typing your own address while it is unverified asks you to verify it.
    • "Send test email" waits while the member list loads ("Checking your organization members").
  • A sandbox test sends without the template's custom From name, address and Reply-To, which the sandbox sender ignores.
  • With the flag off, the modal behaves as on master.
  • Provisioning stays on Channels (feat(workflows): show the managed sandbox sender in channels #602): the modal lists the sandbox sender once Channels created it.

Production lines (tests, stories and inline kea typegen excluded): 263, from git diff --numstat against #602.

How did you test this code?

Live run on the slice tip

  • Live run of the V1 tip 1c08de0b9b4 on a PostHog preview. Stubs: a local SES endpoint that records each call, and example.com seeded as a valid MX domain.
  • Channels created the sandbox sender. A test send to a verified member was accepted and reached SES with the sandbox tenant, its configuration set, the fixed From address and the footer.
  • A test send to an address outside the organization was skipped before SES.
Send test email suggests verified members Warning for an address outside the organization Test email sent to a member
Send test email suggests verified members Warning for an address outside the organization Test email sent to a member

Lane gate on the head:

  • Typecheck (pnpm --filter=@posthog/frontend typescript:check, on a Fleet box): no errors.
  • Jest: the TemplateLibrary suites, 71 passed.
  • Oxfmt and Oxlint on the changed files: no errors. Pre-push hogli ci:preflight --strict: 0 failures.
  • CI: not run. Fork CI cannot run because the fork lacks the GitHub App secret.

Test seams:

  • messageTemplateTestSendLogic (kea logic with mocked integrations, members and invocation APIs): which senders the modal lists and preselects, flag on and off, the recipient suggestions and warnings, and the request a sandbox test sends.
  • SendTestEmailModal (rendered with the real logic): the unverified-self prompt, the member loading state, and picking or typing a recipient through the field, which only the DOM shows.

Test rationale:

  • messageTemplateTestSendLogic.test.ts, new sandbox sender block: preselection, ordering, flag-off drop of a stale selection, member suggestions when the flag lands on an open modal, the stripped From and Reply-To, the outside-address warning (case-insensitive, unverified and unknown members excluded), and the hold while members load. Plus one case for the empty recipient with the flag on. Each failed before the change. The nearest existing test, "defaults the sender to the first verified email integration", still guards the flag-off list.
  • SendTestEmailModal.test.tsx: the unverified-self prompt beats the outside-address warning; the field shows a skeleton while members load and the empty hint after; a picked suggestion and a typed address (Enter) both reach the send request. The last case fails when the field's onChange is disconnected from the logic. No rendered test covered this modal before.

Proof (recorded with fleet proof on the head):

  • UI walk on a seeded local preview, flag on, one verified own sender, the sandbox sender, two verified members:

    From menu Recipient suggestions Address outside the suggestions
    Sender menu Suggestions Outside warning
  • Live run on the same preview: a test from the sandbox sender to a suggested member sent from: {"integrationId": <sandbox>} and replyTo: "", without the template's custom From. The worker skipped the send (no delivery route yet), and the API still answered success; reporting that skip is TS1.

  • The head adds only a rendered test, so both reuse the evidence from a0bef769cf4.

  • Prod shape: gap, accepted until read-only production access arrives. The change is frontend-only behind the flag.

  • CI: gap, fork CI cannot run (see above).

Reviews (GPT-6.1 Sol, high):

  • Round 1 on a0bef769cf4, two findings:
    • The modal offers a sender the worker cannot deliver yet. Rejected: the flag turns on only after the worker's sandbox route, per docs/internal/workflows-email-sending-tiers.md, and the workflow email step already offers the same sender under the same flag.
    • No test drove the new recipient field. Fixed in b37800cb0c6.
  • Final read on b37800cb0c6: no findings.
  • Design pass before opening: one minor finding (members loaded on every flag update), fixed in a0bef769cf4.

Release status

  • This change is behind a feature flag and is not available to users

Docs update

None.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (porting); review by GPT-6.1 Sol

  • Ported from feat(workflows): test templates with the sandbox sender #451 (f1e44aa7116), keeping only the sandbox sender and suggestion parts. Left out: skip reporting (TS1), sandbox provisioning from the modal and its retry banner (Channels owns provisioning), and the worker changes (nodejs/, its own lane).
  • Skills: tdd, writing-tests, pr-description.

Decisions

Question Answer Who Where
Test fixture domain Reserved example.com instead of a real domain 👍 Accepted default map
Slice order of the sandbox sender stack Option B: the Channels row moves into V1 so a team has a sender to test from; V1's worker PRs land first 👍 Accepted default map
Provisioning from the test-send modal None; Channels provisions the sandbox sender, the modal lists it once it exists 🤖 Agent Silthus/personal#202
Skip reporting from #451 Left to its own PR (TS1), for every sender 🤖 Agent Silthus/personal#202
Recipient outside the suggestions Warn, do not block; the worker enforces who receives a sandbox send 🤖 Agent Silthus/personal#202
When members load When the modal is open with the flag on, through a subscription, not on every flag update (design pass) 🤖 Agent Silthus/personal#202

🤖 Generated with Claude Code

Silthus and others added 4 commits October 28, 2026 09:28
The template test send lists the sandbox sender last, with a Sandbox tag
and the members-only note, and preselects it when the project has no
verified own sender. With the workflows-sandbox-sender flag on, the
recipient field suggests verified teammates, signed-in user first, and
warns about addresses outside them. A sandbox test drops the template's
custom From and Reply-To, which the sandbox sender ignores.

Refs Silthus/personal#202

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Members load when the modal opens with the sandbox sender flag on, or
when the flag turns on while the modal is open. Other feature flag
updates no longer reload the member list.

Refs Silthus/personal#202

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… domain

The test send modal is now the first place that shows the note, so it
defines it. The unverified member fixture uses example.com.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant