Skip to content

feat(workflows): tie email events to the send and version that caused them - #123902

Open
mayteio wants to merge 1 commit into
masterfrom
posthog/workflows-email-event-lineage
Open

mayteio wants to merge 1 commit into
masterfrom
posthog/workflows-email-event-lineage

Conversation

@mayteio

@mayteio mayteio commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

This is a copy of #123128, opened from a branch in this repository so it can get a PostHog Review pass. The diff is unchanged.

Problem

  • Teams that send workflow emails cannot join an open, click or bounce to the email that caused it.
  • No $workflows_email_* event carries a per-send reference.
  • $workflows_email_sent carries no workflow version.
  • So nobody can compare how two versions of a workflow email perform, in insights or through the MCP.
  • A transactional email status lookup, stacked on feat(messaging): add a single transactional email send endpoint #123556, reads $workflow_invocation_id from these events.

This is the first of two PRs. The second (#123129) attributes unsubscribes to the send that carried the link.

Warning

🟡 Merge risk: medium · one-way door (merge-risk rubric)

  • Per-version email metrics, the per-version billable count, and conversion attribution move for an email that renders before a republish and sends after it. Unversioned metrics, usage, and billed amounts do not change.
  • The new event properties land in customer projects. Once published, they are a contract a revert does not take back.
  • Focused tests cover each version path, and a cross-version run shows queued steps resume in both directions during a deploy.

Changes

Send reference:

  • $workflows_email_sent and $workflows_email_failed now carry $workflow_invocation_id, the run that sent the email.
  • With $workflow_action_id, that names the step and run of every send.
  • An accepted SES send also carries $email_message_id, the id SES returned.
  • Every SES callback event (delivered, opened, clicked, bounced, blocked, failed) carries the same $email_message_id and $workflow_invocation_id.
  • So a callback joins to its exact sent event.
  • No email address or token enters a new property. Both new ids are opaque.

Version:

  • The sent event from a workflow step now carries $workflow_version. A plain hog function send carries none.
  • The version is the one the step rendered under. A republish between render and send no longer moves the email to the new version.
  • The tracking code had the same flaw and now reads the same version. SES callbacks already took $workflow_version from the signed tracking code.
  • The step builder records that version once in the step state. A step saved before this change gets the live version, as on master.
  • The email_sent, email_failed and email_untracked metrics and the email billable count use that version too.
  • Before, they took the live version while the callbacks took the rendered one, so one send split across two versions.
  • A conversion after an email attributes to the version the email rendered under. Push steps keep the live version, which their metrics still count under.
  • Self-optimising workflows (Self-optimizing workflows: suggestion loop tracking #92154) reads per-version app metrics. A send that spans a republish now counts under one version instead of two.
  • No in-app notice: only sends that span a republish move, and the old split was wrong.

Not in this PR:

  • A send id minted before the send, carried in the signed tracking code and the unsubscribe link. It needs a two-phase rollout.
  • The skip metrics (email_suppressed, email_bounce_prevented, message_frequency_capped) still count under the live version, as push metrics do.
  • A conversion watcher keeps the version the run started on, as on master. A separate PR handles it.

How did you test this code?

Test rationale: each new test fails if the matching property drops off its event. The test seams from #123128:

  • EmailService.executeSendEmail, asserting the captured sent event, plus a new row for a workflow step with a version.
  • The same send table asserts the version on the email_sent metric. The untracked-send test asserts it on email_untracked.
  • The SES webhook endpoint over HTTP, asserting the bounce event body that reaches internal capture.
  • SesWebhookHandler.handleWebhook, asserting the message id on each callback type.
  • The flow function builder: a step rendered under version 2 resolves to version 2 after a republish to 3. A step saved before the field existed resolves to the live version.
  • The flow function step handler: an email rendered under version 2 and run after a republish to 3 attributes conversions and billing to version 2. Push and fetch steps keep their old behavior.

Run on this branch, on top of master at 2e6bb0a:

  • Jest over email-tracking.service.test.ts, email.service.test.ts, helpers/ses.test.ts, actions/hog_function.test.ts and hogflow-executor.service.test.ts.
  • 399 of 402 tests passed. Three failed with ECONNREFUSED to maildev (127.0.0.1:1025 and :1080), which this sandbox does not run.
  • The three are should send an email, should include tracking code in the email with distinct_id, and the executor email queue round-trip. This diff does not touch them.
  • Prettier and ESLint pass on the 13 changed files.
  • tsc --noEmit -p nodejs reports no errors in the changed files. Its 31 errors are missing native session replay modules, unrelated to this diff.

Not run:

Release status

  • No feature flag controls this change

Docs update

None. No existing doc lists the $workflows_email_* properties.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: PostHog Desktop (Claude Code), Claude Opus 5.5

🤖 Generated with Claude Code


Created with PostHog Desktop

… them

Add $workflow_invocation_id to $workflows_email_sent, $workflows_email_failed
and every SES callback event, and $email_message_id to accepted SES sends and
their callbacks, so each email event joins to the run and send that caused it.

The sent event from a workflow step now carries $workflow_version. The step
records the version it rendered under, and the send, tracking code, email
metrics, billable count and conversion attribution all use that version, so a
send that spans a republish no longer splits across two versions.

Copied unchanged from #123128 so it can get a PostHog Review pass from a branch
in this repository.

Co-authored-by: Reichenbach, Michael <755327+Silthus@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: e72a2c10-f145-4f7c-8754-a4fcc06a18be
@mayteio mayteio self-assigned this Oct 27, 2026
@trunk-io

trunk-io Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

✨ Submitted to Merge by @mayteio. It will be added to the merge queue once all branch protection rules pass. See more details here.

mayteio commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

@PostHog review --deep

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

✅ Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

@mayteio mayteio added the posthog-review Run a Standard PostHog Review label Oct 27, 2026 — with PostHog

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed c11a2ae: no findings.

This Standard review started from the posthog-review label. Comment @posthog review --deep for a Deep review, or @posthog help to see all commands.

@posthog posthog Bot removed the posthog-review Run a Standard PostHog Review label Oct 27, 2026
@mayteio
mayteio marked this pull request as ready for review October 27, 2026 10:46
@mayteio mayteio added the stamphog Request AI approval (no full review) label Oct 27, 2026 — with PostHog
@parameterai

parameterai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Risk: No findings

This change pins workflow email events, metrics, and billing attribution to the workflow version an email actually rendered under (stored once in step state), and adds opaque send-reference properties ($workflow_invocation_id, SES $email_message_id) to sent/failed events and SES callback events. I traced every new data path — tracking-code minting, SNS-verified webhook properties, captured-event properties, and app-metric version stamping — and found no new attacker-reachable input, trust-boundary weakening, or cross-tenant exposure; all new event fields are opaque internal ids gated by the existing team-level engagement-capture opt-in.

Sentinel reviewed c11a2ae · Review settings

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approved — this change needs a human reviewer.

Re-adding the stamphog label gives the same result unless the changed files change.

stamphog can't auto-review this pull request because two gates refused it. The deny-list gate matched billing, since the change touches nodejs/src/cdp/services/hogflows/billing-utils.ts and the email billable count. The tier gate classified it as T2-never (210 lines across 13 files, a single-area feature), a tier that is never auto-reviewed. The size gate passed, so splitting is not needed to get under the ceiling. The author should ask a human reviewer to look at it, and could separate the billing-related changes from the rest so a reviewer can assess them on their own.

  • posthog[bot] reviewed the current head.
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✗ matches: billing
size ✓ 87L, 8F substantive, 210L/13F incl. docs/generated/snapshots — within ceiling
tier ✗ classified as T2-never: T2-never (210L, 13F, single-area, feat)
stamphog 2.6.1 .stamphog/policy.yml @ unknown · reviewed head c11a2ae

@stamphog stamphog Bot removed the stamphog Request AI approval (no full review) label Oct 27, 2026
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team October 27, 2026 10:47
@trunk-io

trunk-io Bot commented Oct 27, 2026

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

View Full Report ↗︎ ⋅ Docs

mayteio added a commit that referenced this pull request Oct 27, 2026
GET /api/projects/:project_id/messaging/emails/:message_id/ returns { id, status, created_at, status_at, reason } for a transactional send.

- queued: the send API records the run before it answers 202, so a real send has a status from the start.
- sent: the email worker's record of every email the provider accepts (message_assets).
- skipped: the run finished without a send record, after a short grace for the record to arrive.
- failed: the run failed, or a $workflows_email_failed event exists.
- delivered, bounced, complained: from $workflows_email_* events keyed by $workflow_invocation_id, which need engagement capture and the change in #123902.

Reads go through the cdp facade (fetch_invocation_lifecycle) and the workflows facade (fetch_sent_email_asset), scoped to the project's transactional workflows so each ClickHouse read stays on its sort key. Reads have their own rate-limit buckets and need hog_flow:read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: e72a2c10-f145-4f7c-8754-a4fcc06a18be
mayteio added a commit that referenced this pull request Oct 27, 2026
…failed

The status now comes from two records PostHog writes itself: the run the send API records before it answers, and the worker's record of an email the provider accepted. queued while the run waits or its send record is on the way, sent once the send record exists, failed when the run errored (reason is the error kind) or finished without sending (reason not_sent).

The checks before the send API answers already refuse every email the worker would skip, so a separate skipped status and its log read are dropped. Delivered, bounced and complained are dropped too: they need engagement capture, depend on #123902, and come from events anyone with the project token can capture. They can come back once provider outcomes have trusted storage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: e72a2c10-f145-4f7c-8754-a4fcc06a18be
mayteio added a commit that referenced this pull request Oct 27, 2026
GET /api/projects/:project_id/messaging/emails/:message_id/ returns { id, status, created_at, status_at, reason } for a transactional send.

- queued: the send API records the run before it answers 202, so a real send has a status from the start.
- sent: the email worker's record of every email the provider accepts (message_assets).
- skipped: the run finished without a send record, after a short grace for the record to arrive.
- failed: the run failed, or a $workflows_email_failed event exists.
- delivered, bounced, complained: from $workflows_email_* events keyed by $workflow_invocation_id, which need engagement capture and the change in #123902.

Reads go through the cdp facade (fetch_invocation_lifecycle) and the workflows facade (fetch_sent_email_asset), scoped to the project's transactional workflows so each ClickHouse read stays on its sort key. Reads have their own rate-limit buckets and need hog_flow:read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: e72a2c10-f145-4f7c-8754-a4fcc06a18be
mayteio added a commit that referenced this pull request Oct 27, 2026
…failed

The status now comes from two records PostHog writes itself: the run the send API records before it answers, and the worker's record of an email the provider accepted. queued while the run waits or its send record is on the way, sent once the send record exists, failed when the run errored (reason is the error kind) or finished without sending (reason not_sent).

The checks before the send API answers already refuse every email the worker would skip, so a separate skipped status and its log read are dropped. Delivered, bounced and complained are dropped too: they need engagement capture, depend on #123902, and come from events anyone with the project token can capture. They can come back once provider outcomes have trusted storage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: e72a2c10-f145-4f7c-8754-a4fcc06a18be
trunk-io Bot pushed a commit that referenced this pull request Oct 27, 2026
GET /api/projects/:project_id/messaging/emails/:message_id/ returns { id, status, created_at, status_at, reason } for a transactional send.

- queued: the send API records the run before it answers 202, so a real send has a status from the start.
- sent: the email worker's record of every email the provider accepts (message_assets).
- skipped: the run finished without a send record, after a short grace for the record to arrive.
- failed: the run failed, or a $workflows_email_failed event exists.
- delivered, bounced, complained: from $workflows_email_* events keyed by $workflow_invocation_id, which need engagement capture and the change in #123902.

Reads go through the cdp facade (fetch_invocation_lifecycle) and the workflows facade (fetch_sent_email_asset), scoped to the project's transactional workflows so each ClickHouse read stays on its sort key. Reads have their own rate-limit buckets and need hog_flow:read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: e72a2c10-f145-4f7c-8754-a4fcc06a18be
trunk-io Bot pushed a commit that referenced this pull request Oct 27, 2026
…failed

The status now comes from two records PostHog writes itself: the run the send API records before it answers, and the worker's record of an email the provider accepted. queued while the run waits or its send record is on the way, sent once the send record exists, failed when the run errored (reason is the error kind) or finished without sending (reason not_sent).

The checks before the send API answers already refuse every email the worker would skip, so a separate skipped status and its log read are dropped. Delivered, bounced and complained are dropped too: they need engagement capture, depend on #123902, and come from events anyone with the project token can capture. They can come back once provider outcomes have trusted storage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: e72a2c10-f145-4f7c-8754-a4fcc06a18be

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant