Skip to content

feat(skills): add send-transactional-email skill - #433

Closed
Silthus wants to merge 14 commits into
PostHog:mainfrom
Silthus:feat/send-transactional-email-skill
Closed

Silthus wants to merge 14 commits into
PostHog:mainfrom
Silthus:feat/send-transactional-email-skill

Conversation

@Silthus

@Silthus Silthus commented Oct 17, 2026 •

Copy link
Copy Markdown

Problem

A coding agent asked to "send a welcome email" has no PostHog skill to reach for. PostHog can send a transactional email from code (the transactional email API, in beta), but nothing tells an agent to verify a sender first, keep the email as a template in the app's code, and wire the send into the server.

With this skill, the agent verifies a sender, writes the email as a local template in code, and sends it with posthog.emails.send({ templateKey, subject, html, templateData }). The first send uploads the template. Rendered content and templates made in PostHog (templateId) are the other two forms.

Tracking: Silthus/posthog#476, spec Silthus/posthog#485 (section "Agent skill").
Source and changes: Silthus/posthog#511, spec Silthus/posthog#502 revision 4.
Flat send contract: PostHog/posthog#120921, slice 20 of the Resend parity spec (Silthus/posthog#544).

Impact

Pays into Grow Workflows through organizations that send email from code (not measured yet: it needs the send source on each send, PostHog/posthog#120628). In scope: transactional email. Expected move: medium, up. Run cost: n/a.

Merge Danger

Warning

🟡 Merge risk: medium · two-way door (merge-risk rubric)

  • The skill teaches the flat contract of Transactional email SDK contract for posthog-python and posthog-node posthog#120921, which no released SDK has yet: posthog-node's code form (PostHog/posthog-js#5312 and its follow-ups) and the server's code form on /messaging/send/email/. Released before them, an agent writes calls that fail.
  • The build fetches https://posthog.com/docs/workflows/transactional-email-api.md, which answers 404 until PostHog/posthog.com#21306 publishes. Until then a clean release build fails for every skill.
  • Merge after the docs page is live and the posthog-node release with the code form is out. A revert removes the skill from the next release.

Summary

Adds the send-transactional-email skill next to the instrument-* skills (context/skills/omnibus/send-transactional-email/). It ships the transactional email API docs page as its reference, and walks nine steps in order:

  1. Detect the platform, and check the PostHog MCP is connected and has the upload and send tools.
  2. Check for a verified sender with integrations-list, and verify an unknown one once instead of guessing.
  3. Set one up with the sender tools (Domain Connect link or DNS records, verify until success, bounded), or send the person to the PostHog UI when those tools are missing.
  4. Pick the form: a local template (the default), rendered content such as React Email output, or a template made in PostHog by templateId.
  5. Write the local template in code: a subject and an html string, {{ variables.* }} only, a key of letters, digits, - and _. The agent never uploads it: the first send does.
  6. Stop when the API is not enabled: a send by templateKey answers 404 with not_found or no code.
  7. Have the person create a phs_ key with hog_flow:write, read from POSTHOG_PROJECT_SECRET_API_KEY.
  8. Write the send with posthog-node (appHost behind a proxy, from only on a verified sender), or over HTTP to /messaging/send/email/ with an Idempotency-Key. Over HTTP, a 409 template_outdated leads to an upload by key with if_code_managed: true and a resend with the same key. Retries reuse the key; job runners pass maxRetries: 0 and a key derived from the email's purpose.
  9. Branch on error.code, with the contract's codes and the 409 for a template edited in PostHog.

"Change an email the app already sends": the agent edits the local template in code. The first send with the changed content uploads it. A template_superseded warning points to its fix.

"Inside the PostHog Wizard": questions go through wizard_ask, the secret key goes to the env file through set_env_values and never shows in the run, PostHog tools run through posthog_exec, and a missing tool is inconclusive because the Wizard's login may lack their scopes. Two [ABORT] reasons match the abort cases of PostHog/wizard#1487.

Evidence

Flat contract (slice 20), agent run on a fixture app. Claude Code on Sonnet 5.5 with only this skill, built from c626ddf with the docs page stubbed (it answers 404 until #21306 publishes). The app is the SvelteKit app of PostHog/wizard-workbench#2221. Prompt: add a welcome email after signup, with the MCP steps marked as done and no commands run.

// src/lib/posthog_mailer.ts (agent's diff, excerpt)
const { error } = await posthog.emails.send({
  to: user.email,
  distinctId: user.id,
  templateKey: "welcome",
  subject: welcomeEmail.subject,
  html: welcomeEmail.html,
  templateData: { first_name: firstName, company_name: companyName, account_url: accountUrl },
  idempotencyKey: `welcome:${user.id}`,
})
  • The template is a module with subject and html strings that use {{ variables.* }}.
  • No templateFile, uploadTemplate or template: remains in the diff, and no MCP upload ran.
  • Red is the old STEP 8, which taught templateFile() verbatim. No red agent run was made.

Bundle (test seam: the context-mill build). scripts/lib/tests/send-transactional-email-skill.test.js is unchanged and passes at 161a6fd. pnpm test: 29 files, 220 tests pass at c626ddf.

Review. One GPT-6.1 Sol review round over this PR and #1487, then a final read. Three findings on this skill were fixed in db4baf8 and 161a6fd: hidden tools inside the Wizard, the Wizard tool names, and replyTo needing no verified sender.

Gaps

  • No live send for the flat contract: posthog-node's code form and the server's code form are still in draft.
  • The fixture run marked the MCP steps as done. Sender verification and the first upload by a send are not run.
  • The Wizard section is not run inside the Wizard. That needs the skill in a published menu and Wizard CI credentials.
Earlier evidence: the file-based shape (templateFile, MCP upload), up to 8b236c3

Bundle (test seam: the context-mill build). scripts/lib/tests/send-transactional-email-skill.test.js builds the skill through generateSkill with fetch stubbed for the docs URL, and asserts SKILL.md is named omnibus-send-transactional-email, lists references/transactional-email-api.md, and that file holds the docs page.

  • Red before the skill: AssertionError: expected undefined to be defined.
  • Green after. Removing shared_docs fails it.
  • pnpm test: 29 files, 220 tests pass. pnpm build emits dist/skills/omnibus-send-transactional-email.zip (SKILL.md, references/transactional-email-api.md, references/COMMANDMENTS.md).

Search ranking (test seam: the MCP learn -s scorer). The PostHog MCP's own SkillCatalog.searchResults from services/mcp, over the released agent-skills-latest skills.zip with this build's omnibus skills overlaid the way .github/actions/download-context-mill-skills does:

Prompt Before (context-mill v1.57.0) After
send a welcome email from my Node app not returned (1. consuming-endpoints-from-client-code) rank 1
send a password reset email not returned (1. signals-scout-workflows) rank 1
send a signup confirmation email not returned (1. designing-email-templates) rank 1
transactional email not returned (1. designing-email-templates) rank 1
send an email from code not returned (1. designing-email-templates) rank 1

Local agent session. Claude Code on Sonnet 5.5, a clean config, only this skill and a local PostHog MCP. Django and the CDP worker run from the server stack with the flag on, and the scratch Node app uses posthog-node from #5312. The project starts with no sender and the MCP has no sender tools, so this exercises the fallback path.

Transcript excerpt, skill at 5a1f8e1
Turn 1: "Send new users a welcome email when they sign up. We use PostHog, and its MCP server is connected."
> Skill send-transactional-email
> exec search workflows-upload-email-template|workflows-send-email|integrations-
  < [... "integrations-list", ..., "workflows-send-email", "workflows-upload-email-template"]
> exec call integrations-list {"kind":"email"}
  < count: 0
> Write emails/welcome.html   (<title>Welcome to Acme, {{ variables.first_name }}!</title>)
> Edit src/posthog.js         (secretKey, enableLocalEvaluation: false)
> Edit src/signup.js          (posthog.emails.templateFile(..., { name }) and emails.send after signup)
> .env.example                (POSTHOG_PROJECT_SECRET_API_KEY=)
Reply: "The project has no verified email sender ... the MCP here doesn't have the tools to create one.
In PostHog, go to Workflows → Channels, add an email sender on your domain ... Tell me when it shows as verified."

(person verifies hello@example.com and sets the phs_ key in .env)

Turn 2: "Done ... jane@example.com is my test inbox, so you can run the signup demo once as the test."
> exec call integrations-list {"kind":"email"}      < 2,email,hello@example.com
> exec call workflows-upload-email-template {"key":"welcome","name":"Welcome","subject":"Welcome to Acme, {{ variables.first_name }}!", "preheader":..., "html":<whole file>, "source":{"repository":"github.com/example/acme-app","path":"emails/welcome.html","ref":"12a4e9a..."}}
  < result: created, managed_by: code
> npm run signup-demo

Django:
PUT  /api/projects/7/messaging_templates/by_key/welcome/ -> 201
POST /api/projects/@current/messaging/send/email/        -> 202
POST /api/projects/@current/messaging/send/email/        -> 202   (second run, see below)

Dev mail catcher:
{"from": ["hello@example.com"], "to": ["jane@example.com"], "subject": "Welcome to Acme, Jane!"}
{"from": ["hello@example.com"], "to": ["jane@example.com"], "subject": "Welcome to Acme, Jane!"}
  • The MCP upload matched the file byte for byte, so each SDK send was one request with no re-upload.
  • The agent ran the demo a second time to read output it had truncated, and said so. 3d85758 adds the rule that prevents it: save a test run's output to a file and never rerun to read it.
  • An earlier run at fa8aeb1 hit a harness bug (pnpm pulled @posthog/core from npm) and the agent stopped after one failed send instead of retrying.

Source and changes (#511), local agent sessions. Same stack as above, from spec/code-managed-templates 07d7e0e, with no CDP worker (nothing sends). Each run gets its own project with hello@example.com verified. The scratch repo is a monorepo with the app in apps/web, and its origin is https://x-access-token:ghp_…@github.com/example/acme.git with a fake token. Red runs use the skill at 965a656, green runs this branch.

Scenario Red (965a656) Green
Set up the welcome email from the repo root github.com/example/acme, apps/web/emails/welcome.html, HEAD same
Same, agent started in apps/web same same
"Add a line under the greeting" edits the file, no upload edits the file, no upload, "the first send after you deploy uploads it"
"... and make it live in PostHog right away", then "undo that line" uploads both times, no warning uploads both times, warns that every running copy sends it, and explains why the undo needs an upload

No run called an update, patch or design tool. Sonnet 5.5 already did the right thing with the old wording, so this is a proof that the behavior holds, not a red-green test. The new text adds the running-copy warning and the undo rule.

Green at c128595: upload, early publish, undo, and the template API
Turn 1: "Send new users of the web app a welcome email when they sign up. ... Don't send a test email."
> exec call workflows-upload-email-template {"key":"welcome","name":"Welcome",...}
  < result: created, managed_by: code
    source: {repository: github.com/example/acme, path: apps/web/emails/welcome.html, ref: ad86435b7521cff0dbb52a917644428b79a4cc0e}
Reply: "... Leaked token: your git origin URL has an access token embedded in it. I left it out of the upload ..."

Turn 2: "Add a line under the greeting ... and make it live in PostHog right away."
> Edit apps/web/emails/welcome.html
> exec call workflows-upload-email-template {...}   < result: updated
Reply: "... Every welcome email sent from now on, from any running copy of the app, will include the new line."

Turn 3: "Actually, undo that line. People should get the email exactly as it was before."
> Edit apps/web/emails/welcome.html
> exec call workflows-upload-email-template {...}   < result: updated, content_hash v1:cee89c39… (the first upload's)
Reply: "I uploaded explicitly because the app's templateFile() doesn't re-upload a reverted file on its own."

GET /api/projects/18/messaging_templates/ (personal API key)
{"key": "welcome", "managed_by": "code", "source_repository": "github.com/example/acme", "source_path": "apps/web/emails/welcome.html", "source_ref": "ad86435b7521cff0dbb52a917644428b79a4cc0e", "content_hash": "v1:cee89c391f5a62815bd33e69302080e6a0e1fb18454ae0c2416af4283e6364ac"}
  • Across the 9 proof projects, no stored source_repository holds a token, a user name or a scheme.
  • The ranking table above is unchanged at 8b236c3: the same five rank-1 results with the same scores. The skill's description did not change.
  • pnpm test (220 passed) and pnpm build pass at 8b236c3. The bundle test needed no change: no test asserts the skill's wording.
  • The last commit, 8b236c3, rewords two sentences after the green run and has its own final read.

Gaps

  • Sender verification through the MCP sender tools: not run. #116863–#116865 are not on the server branch, so the session used the UI fallback with a sender verified locally.
  • The ranking seam is a proof script, not a committed test: the MCP scorer lives in PostHog/posthog.
  • The full proof run is at 5a1f8e1. Later commits change wording the session did not reach: the rerun rule (3d85758), verifying an unknown sender (711998d, 965a656), and the Next.js notes.
  • The production MCP learn catalog (skills.zip) carries no context-mill skill today, instrument-* included. The table shows the overlaid catalog that agent hosts load.
  • pnpm security-scan:skills could not triage locally (no LLM provider), so it flagged 413 matches across other skills. None is in this skill.
  • Upstream workflows don't run on this fork PR until a maintainer approves them.
  • #511: the change section's two later paths are not run, because workflows-request-email-template-change (#513) and Edit with your agent (#503) don't exist yet.
Earlier summary

Adds the send-transactional-email skill next to the instrument-* skills (context/skills/omnibus/send-transactional-email/). It ships the transactional email API docs page as its reference, and walks the spec's nine steps in order:

  1. Detect the platform, and check the PostHog MCP is connected and has the upload and send tools.
  2. Check for a verified sender with integrations-list, and verify an unknown one once instead of guessing.
  3. Set one up with the sender tools (Domain Connect link or DNS records, verify until success, bounded), or send the person to the PostHog UI when those tools are missing.
  4. Write the template as an HTML file: <title> is the subject, values come from {{ variables.* }}, never person.*.
  5. Upload it with workflows-upload-email-template, passing source: repository from git remote get-url origin as host/owner/name with any token dropped, path from the repository root, and ref from git rev-parse HEAD. posthog-node fills the same fields on its own uploads. sender_unavailable leads to a verify, then Make default.
  6. Stop when the API is not enabled: the tools are hidden, or a 404 without a code or with not_found.
  7. Have the person create a phs_ key with hog_flow:write, read from POSTHOG_PROJECT_SECRET_API_KEY.
  8. Write the send: posthog.emails.send with posthog.emails.templateFile(...) in Node.js (Next.js and edge covered), the send route by key plus an upload where the project deploys elsewhere. Sends are not idempotent: no retries, ask before any test send.
  9. Branch on error.code.

A section, "Change an email the app already sends", covers later changes:

  • With the repository, the agent edits the file. It never uses PostHog's update, patch or design tools, and leaves the template managed by code.
  • With templateFile(), the first send from the changed code uploads it. The agent uploads early only when the person asks, and says every running copy sends the new version.
  • An undo to content PostHog published before needs an upload, because a send with an older hash gets the newer content.
  • By key, the deploy step uploads the change, or the agent uploads it after the change.
  • Without the repository, the agent names the file or calls workflows-request-email-template-change where the MCP has it.

The MCP serves context-mill skills as resources. Agent hosts that overlay context-mill onto the posthog skills, like the Tasks sandbox and PostHog Desktop, see it as send-transactional-email.

Decisions

Question Answer Who Where
The send shape the skill teaches The flat contract of #120921: templateKey with templateData for a local template; it replaces templateFile(), the template field and data 👤 Human PostHog/posthog#120921
Who uploads a local template The first send, never the agent; the MCP upload step and the deploy-time upload go, and so does the source link 👤 Human Spec: Resend parity, decision 3
A missing tool inside the Wizard Inconclusive: take the UI path for the sender, and only a send's 404 shows the API is off 🤖 Agent pending approval
Scopes for the Wizard run None added; the skill works without the sender tools 🤖 Agent pending approval
Earlier decisions (file-based shape)
Decision Who Where
The skill name, place and nine steps 👤 Human Spec #485, "Agent skill"
An agent with the repository edits the file; "request a change" is only for agents without it 👤 Human Spec #502, revision 4
Local proof uses the UI fallback with a locally verified sender, and records MCP verification as a gap 👤 Human #476
Ranking is measured with the MCP's SkillCatalog over the overlaid catalog, because the MCP learn catalog has no context-mill skills 🤖 Agent #476
The skill's description doesn't widen to change requests; routing those is #515's job 🤖 Agent #511
An early upload of a templateFile() change happens only when the person asks 🤖 Agent #511
sender_unavailable leads to a verify (which sets an unset default), then Make default: no MCP tool sets the default sender 🤖 Agent #476
The upload passes the <title>, preheader and whole file, so the SDK's first send needs no re-upload 🤖 Agent #476

Involvement: earlier work had two Opus 5.5 review rounds and six final reads. The slice 20 work had one GPT-6.1 Sol review round and one final read, with live agent proof captured on a fixture app.


Companion: PostHog/posthog#118927 reserves the name so a posthog skill cannot shadow it.

Written by Claude Code (Opus 5.5).

🤖 Generated with Claude Code

Silthus and others added 5 commits October 17, 2026 03:02
Teaches a coding agent to send one transactional email from server code
with PostHog: verify a sender through the PostHog MCP first, keep the
template as an HTML file in the repository, upload it by key, and send
it with posthog-node or the HTTP API. Ships the transactional email API
docs page as its reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Check the PostHog MCP before any sender work, never recreate an existing
sender, bound verification polling, set the default sender through a
verify, ask before any test send, and cover Next.js and edge runtimes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus and others added 4 commits October 17, 2026 08:34
… file

The MCP upload passes source.repository from the origin remote without
credentials, source.path from the repository root and source.ref from HEAD.
A new section sends a change to a code-managed template to its file and
leaves the upload to posthog-node or the deploy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reverted file goes live only through an upload, because a send with an
older published hash gets the newer content. Any send from the changed
code publishes it. A by-key change is uploaded after the change. An agent
without the repository tells the person or requests the change. The SDK
always sends a path, so only repository and ref keep their stored value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A revert to published content needs an upload only for templateFile().
A by-key change waits for the deploy step when there is one. A test
through workflows-send-email publishes nothing, and an agent without
the repository shares the PostHog Code task link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus and others added 2 commits October 22, 2026 03:51
The skill now writes a local template in the app's code and sends it
with templateKey, subject, html and templateData. The first send
uploads it, so the MCP upload, templateFile() and the deploy-time
upload by key go. Rendered content and templates made in PostHog
(templateId) are the other two forms.

Sends carry an idempotency key and retry with it. Job runners pass
maxRetries: 0 and a key derived from the email's purpose. STEP 9
lists the contract's codes, including the 409 for a template edited
in PostHog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Inside the Wizard the agent asks through wizard_ask, writes the secret
key with set_env_values, reaches PostHog tools through posthog_exec,
and ends a blocked run with an [ABORT] reason the Wizard's
transactional-email program matches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus and others added 3 commits October 22, 2026 04:38
The Wizard's login may lack the workflows and integrations scopes, so a
hidden tool there no longer stops the run as "API not enabled". The
Wizard section names its tools as the runtime shows them, and replyTo
no longer needs a verified sender.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…test send

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant