Skip to content

feat(workflows): validate sandbox sender use at save time - #603

Draft
Silthus wants to merge 4 commits into
workflows-sandbox-product/email-step-pickerfrom
workflows-sandbox-product/save-validation
Draft

Silthus wants to merge 4 commits into
workflows-sandbox-product/email-step-pickerfrom
workflows-sandbox-product/save-validation

Conversation

@Silthus

@Silthus Silthus commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Problem

Stack

Impact

Keeps the sandbox funnel step honest: a sandbox send only happens where the trial allows it.

Merge Danger

Caution

🔴 Merge risk: high · two-way door (merge-risk rubric)

  • The validation runs on every workflow and hog function save, flag or not. A bug could reject valid saves for any team.
  • Limited by: it only raises when a sender in the step resolves to the managed sandbox row; the email-sender lookup is shared and cached per request; API tests cover each allowed and rejected case.

Changes

  • Saving a workflow or hog function rejects the sandbox sender outside workflow email steps and test sends, with an error that says to pick a sender on an own verified domain.
  • Broadcasts cannot use it, and converting a workflow to a broadcast refuses when it uses it.
  • Inside an email step it must be the only sender, without custom From, name or Reply-To, and the project must have the flag.
  • An email step saved unchanged keeps saving, so a flag change does not block unrelated edits. The exemption holds only while the step stays an email step. A step changed to a generic function step is checked again.
  • A stored destination that holds the sandbox sender can still be disabled or deleted. Broadcasts get no exemption, because a broadcast can never send through the sandbox sender.
  • Sender detection reads a non-empty rotation alone, as the worker does, so a stale integrationId next to an own rotation does not block a save.
  • Mechanical: the email-sender domain cache moved into one per-request integration cache shared by both checks.

Production lines (tests excluded): 223, from git diff --numstat against #629.

How did you test this code?

Live run on the slice tip

  • Live run of the V2 tip 7f851f2a9dc on a PostHog preview. Stubs: a local SES endpoint that records each call, and example.com seeded as a valid MX domain.
  • The email step's sender picker lists the sandbox sender with its tag.
  • Switching a step from its own sender to the sandbox sender asks first. After the confirm, the saved step reads from: {integrationId: <sandbox>} and an empty reply-to, and a test send from it reached SES.
  • With the flag on, saving a broadcast or a generic function step on the sandbox sender returns 400 with its message, and a workflow email step saves. With the flag off, all three save and the picker hides the sandbox sender.
  • Shadow diff on d233d18682a: the same save requests against this PR's base and head. Only the sandbox cases outside workflow email steps changed, from accepted to 400; every other case returned the same response.
Flag on: the step offers the sandbox sender Flag off: the picker hides the sandbox sender
Flag on: the step offers the sandbox sender Flag off: the picker hides the sandbox sender

Lane gate on the head, run on a Fleet box:

  • Backend: the sandbox validation and sender suites, broadcast conversion, hog flow API, email action and draft publish suites, and products/cdp/backend/test/test_validation.py: 896 passed.
  • Hog function API suites (products/cdp/backend/api/test/): 333 passed.
  • After the review fixes: the sandbox validation suite with test_validation.py: 207 passed; the hog flow API suite: 594 passed.
  • Python types (uv run mypy --cache-fine-grained .): no issues, on the stack head.
  • Pre-push hogli ci:preflight --strict: 0 failures.
  • CI: not run. Fork CI cannot run because the fork lacks the GitHub App secret.

Test rationale:

  • test_sandbox_sender_validation.py: each place the sandbox sender is allowed or rejected, rotation with other senders, overrides, the flag, unchanged steps, and test sends. Carried from feat(workflows): validate sandbox sender use at save time #430; no existing test covered the sandbox sender.
  • The same file, review cases: disabling and deleting a stored destination, a stale integrationId beside an own rotation, and an email step changed to a generic function step with the sender unchanged. Each failed before its fix; the last returned 200.
  • test_broadcast_conversion.py: conversion refuses a workflow on the sandbox sender.
  • test_validation.py: one assertion follows the shared cache rename.

Proof (fleet proof, on a preview of the stack):

  • Live run: changing a stored email step into a generic function step returns 400. A stored destination with the sandbox sender can be disabled and deleted (200). A new destination with it returns 400.
  • Shadow diff, feat(workflows): show the managed sandbox sender in channels #602 against this head, with the same stored rows: 21 cases unchanged, 5 changed, all named by the goal.
Case Before After
New broadcast, destination, rotation or name override with the sandbox sender 201 400 with the fix in the message
Unchanged resave of a stored broadcast with the sandbox sender 200 400
Own-sender saves, stored sandbox workflows and destinations, stale-fallback rotation 200 or 201 unchanged
  • Prod shape: gap, accepted under a standing rule until read-only production access is available. Production access would show how many stored broadcasts the strict rule touches.
  • Review: round 1 by GPT-6.1 Sol, three findings. The stored-destination 400 and the rotation merge are fixed. Templated sender ids are rejected for this PR, see Decisions. Final read 1 found that the unchanged exemption skipped the step-type rule, now fixed. Final read 2 found nothing.

Release status

  • No feature flag controls this change

Docs update

docs/internal/workflows-email-sending-tiers.md names the save-time rules.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Claude Opus 5.5 (carving); review by GPT-6.1 Sol

Decisions

Question Answer Who Where
Flag gate on the save checks Gate only the sandbox branch; the rotation fix and the shared integration cache stay unflagged 👍 Accepted default map
Slice order of the sandbox sender stack Option B: the Channels row moves into V1 so a team has a sender to test from; V1's worker PRs land first 👍 Accepted default map
One sender-ID parser for validation and broadcast conversion (design pass) Kept two: sharing would import a private CDP helper into Workflows, and conversion reads lenient stored drafts 🤖 Agent Silthus/personal#200 (resolution)
Hog-templated sender ids (review round 1) Not checked at save time; the worker skips every sandbox send today, and the delivery stack must enforce the rules at send time on the resolved sender 🤖 Agent Silthus/personal#200 (resolution)
Stored broadcasts with the sandbox sender Stay strict: an unchanged resave returns 400, because a broadcast can never send through it 🤖 Agent Silthus/personal#200 (resolution)
Unchanged-sender exemption (final read 1) Holds only while the step stays an email step 🤖 Agent Silthus/personal#200 (resolution)

Involvement: no grilling rounds; one design pass (GPT-6.1 Sol); one review round and two final reads (GPT-6.1 Sol); proof captured.

🤖 Generated with Claude Code

Silthus and others added 4 commits October 28, 2026 09:54
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…working

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nges type

A stored email step that turns into a generic function step no longer keeps
its sandbox sender as an unchanged value, so the step-type rule rejects it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Projects without workflows-sandbox-sender skip the sandbox sender
checks and their integration lookup, so the save path stays as it was
for everyone else.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus
Silthus force-pushed the workflows-sandbox-product/channels-row branch from 1f4ddaf to ed38f61 Compare October 28, 2026 10:00
@Silthus
Silthus force-pushed the workflows-sandbox-product/save-validation branch from 356d02f to d233d18 Compare October 28, 2026 10:00
@Silthus
Silthus changed the base branch from workflows-sandbox-product/channels-row to workflows-sandbox-product/email-step-picker October 28, 2026 10:00
Silthus added a commit that referenced this pull request Oct 28, 2026
Silthus added a commit that referenced this pull request Oct 28, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant