Agent skill: verify a sender, upload the template from the repo, send by key #476

Activity

1. Silthus

   added

   wayfinder:implement

   on Oct 14, 2026
2. Silthus

   marked this as blocked on Oct 14, 2026

   • posthog-node: emails.send from a key or an HTML file, synced by content hash, returning { data, error } #474
3. Silthus

   added a parent issue on Oct 14, 2026

   • Map: send code-managed email templates with one PostHog call #461
4. Silthus

   marked this as blocked by 2 issues on Oct 14, 2026

   • Docs: transactional email API page for code-managed templates, an anchor per error code #475
   • Bring the agent sender-verification tools to PostHog/posthog #466
5. Silthus

   changed the title [-]Agent skill: send transactional email, verifying a sender first[/-] [+]Agent skill: verify a sender, upload the template from the repo, send by key[/+] on Oct 16, 2026
6. Silthus

   marked this as blocked by 2 issues on Oct 16, 2026

   • Email templates: upload a template by key from code #487
   • Email templates: one send route that names the template in the body, with a content hash #488
7. Silthus

   self-assigned this

   on Oct 17, 2026
8. Silthus commented on Oct 17, 2026

   

   Silthus

   on Oct 17, 2026

   OwnerAuthor

   More actions

   Runtime: T3 child task
   Task: node:delegated-task:command%3Amcp%3A955e36b3-3986-469c-b816-09cf656780af%3Adelegate-task%3Awf-email-impl-476-2
   Cast: Opus 5.5, high. Cap 240 minutes.
9. Silthus commented on Oct 17, 2026

   

   Silthus

   on Oct 17, 2026

   OwnerAuthor

   More actions

   Resolution

   Shipped: two draft PRs, both linked to the T3 thread, neither marked ready.

   • PostHog/context-mill#433, branch Silthus:feat/send-transactional-email-skill against main. Reviewed head 965a656ea1d4f7b6612186d2a8b0575842302dee. fleet proof gate passes (HEAD reviewed in final read 3).
   • PostHog/posthog#118927, branch Silthus:chore/reserve-send-transactional-email-skill against master. Reviewed head 5845c44a0de2921942814f9e10b05e679450e928. fleet proof gate passes (HEAD reviewed in final read 1).

   What shipped

   • The skill, at context/skills/omnibus/send-transactional-email/ (config.yaml, description.md). It ships the docs page as references/transactional-email-api.md, and consumers see it as send-transactional-email. It walks the spec's nine steps in order:
     • The MCP check and the platform.
     • A sender check with integrations-list. An unknown sender is verified once rather than guessed.
     • Sender setup with the feat(workflows): let agents set up an email sending domain via mcp PostHog/posthog#116863 tools, with polling bounded and the SPF, DMARC and 403 rules. When those tools are missing or refused, the person goes to the PostHog UI.
     • An HTML template with a <title> subject and {{ variables.* }}.
     • The upload with workflows-upload-email-template and source. sender_unavailable leads to a verify, then Make default.
     • A stop when the flag is off: the tools are hidden, or a 404 without a code or with not_found.
     • A phs_ key read from POSTHOG_PROJECT_SECRET_API_KEY.
     • The send: posthog.emails.send with templateFile(..., { name }), including Next.js and edge. Other languages use the send route by key plus an upload where the project deploys.
     • Errors handled by error.code.
     • It promises no idempotency, and the agent asks before any test send.
   • Name reservation: the name is added to reserved_names.py (build lint) and bundled_skills.py (skills store). The store list also gains instrument-metrics, which had drifted. The three docs that list the omnibus skills are updated.

   Proof

   Bundle seam. scripts/lib/tests/send-transactional-email-skill.test.js builds the skill with fetch stubbed for the docs URL. It asserts the name, the reference listing and the reference content.

   red:   AssertionError: expected undefined to be defined   (no omnibus-send-transactional-email)
   green: Tests  1 passed (1)
   mutation (shared_docs removed): expected '---\nname: omnibus-send-transactional…' to contain '`references/transactional-email-api.m…'
   

   On 965a656: pnpm test 220 passed, and pnpm build emits omnibus-send-transactional-email.zip.

   Ranking seam. It uses the PostHog MCP's own SkillCatalog.searchResults (services/mcp), over the released agent-skills-latest skills.zip with context-mill overlaid the way download-context-mill-skills does it.

   Prompt	Red (context-mill v1.57.0)	Green (965a656)
   send a welcome email from my Node app	not returned	rank 1 (152 vs 75)
   send a password reset email	not returned	rank 1
   send a signup confirmation email	not returned	rank 1
   transactional email	not returned	rank 1
   send an email from code	not returned	rank 1

   Local agent session. The stack:

   • Django and the CDP worker from spec/code-managed-templates e52feb3ccd9, with the flag forced on, private personhog, and an SES Valkey.
   • The local MCP dev server, with FEATURE_FLAG_OVERRIDES.
   • posthog-node from posthog-js#5312 5a896e92.
   • Claude Code on Sonnet 5.5, with a clean config and only this skill and the MCP.

   The project started with no sender, and the MCP had no sender tools.

   Transcript excerpt, skill at 5a1f8e1

   Turn 1: "Send new users a welcome email when they sign up. We use PostHog, and its MCP server is connected."
   > Skill send-transactional-email
   > exec search workflows-upload-email-template|workflows-send-email|integrations-   < upload and send tools present, no integrations-email-*
   > exec call integrations-list {"kind":"email"}                                      < count: 0
   > Write emails/welcome.html, Edit src/posthog.js (secretKey), Edit src/signup.js (templateFile + emails.send), .env.example
   Reply: "... the MCP here doesn't have the tools to create one. In PostHog, go to Workflows → Channels ... Tell me when it shows as verified."
   
   (seed script stands in for the UI verification of hello@example.com; phs_ key set in .env)
   
   Turn 2: "Done ... you can run the signup demo once as the test."
   > exec call workflows-upload-email-template {"key":"welcome","name":"Welcome",...,"source":{"repository":"github.com/example/acme-app","path":"emails/welcome.html",...}}
     < result: created, managed_by: code
   > npm run signup-demo   (run twice: see below)
   
   PUT  /api/projects/7/messaging_templates/by_key/welcome/ -> 201
   POST /api/projects/@current/messaging/send/email/        -> 202
   POST /api/projects/@current/messaging/send/email/        -> 202
   
   Dev mail catcher:
   {"from": ["hello@example.com"], "to": ["jane@example.com"], "subject": "Welcome to Acme, Jane!"}
   {"from": ["hello@example.com"], "to": ["jane@example.com"], "subject": "Welcome to Acme, Jane!"}
   

   • The agent's first send landed in the dev mail catcher. The upload matched the file byte for byte, so each send was one request.
   • The agent reran the demo to read output it had truncated, which sent a second email, and it reported this. 3d85758 adds the rule that prevents it.
   • The full evidence is in ~/.local/state/fleet/proof/PostHog/context-mill/433/965a656…/live-run/.

   Name reservation, with a scratch product skill named send-transactional-email:

   upstream/master: hogli lint:skills -> OK: 164 product skill(s) ... passed; store refuses send-transactional-email: False, instrument-metrics: False
   this branch:     hogli lint:skills -> ERROR: 'send-transactional-email' is owned by PostHog/context-mill ...; store refuses both: True
   

   test_build_skills.py and test_bundled_skills.py pass, and the pre-push hogli ci:preflight --strict reported 0 failures.

   Review

   Opus 5.5 at high effort, in fresh subagents, recorded with fleet proof review-start and review-finish.

   • Round 1, FAIL: one high (the merge order on the unpublished docs page, now in the PR's risk callout) and five mediums, all fixed: a disconnected MCP read as flag-off, the sender_unavailable contradiction, unbounded polling, a test send without asking, and the Next.js path. Of the lows, five were fixed. Two were rejected: the version number can't be named until Think through the Elements tab PostHog/posthog#5312 releases, and a test that the two name lists match would cross the isolated products.skills boundary.
   • Final read 1, FAIL: one medium, fixed. It asked instead of verifying an unknown sender.
   • Final read 2, FAIL: one medium, fixed. A member's 403 on verify had no route. One low was rejected, because feat(workflows): expose read-only email sending readiness over mcp PostHog/posthog#116865 adds config.verified to integrations-list.
   • Final read 3: PASS. Two optional lows were left as they are, with a reason.

   Calls made

   • Ranking measure: the MCP learn catalog (skills.zip) carries no context-mill skill today, instrument-* included. The PostHog MCP serves context-mill skills only as resources. So the ranking ran the MCP's own scorer over the overlaid catalog that agent hosts load. It is a proof script, not a committed test, because the scorer lives in PostHog/posthog.
   • Default sender: no MCP tool sets one. The first verified sender becomes the default, and a successful verify sets the default when none is set, so sender_unavailable leads to a verify and then Make default.
   • The upload passes the <title>, preheader and whole file, so posthog-node's first send needs no re-upload. templateFile(..., { name }) keeps the name from drifting to the subject.
   • Local docs: the build fetches the docs URL, which still answers 404. The local build used a seeded doc cache built from posthog.com#21306's c86c7a6.
   • The proof agent ran on Sonnet 5.5 as a typical user agent. The run is at 5a1f8e1. The later commits are wording the session did not reach.
   • Harness bug: pnpm 12 ignored the package.json overrides and pulled @posthog/core from npm. A first run hit network_error, and the agent stopped without retrying. The install was fixed and the proof rerun.
   • Scope: besides the two name lists, the posthog PR updates the three docs that list the names and adds instrument-metrics to the store list.
   • Assignee: setting it on chore(skills): reserve the send-transactional-email skill name PostHog/posthog#118927 failed. Silthus lacks the permission upstream.
   • context-mill uses pnpm (pnpm-lock.yaml), although its AGENTS.md shows npm commands.

   Gaps

   • MCP sender verification: not run. feat(workflows): let agents set up an email sending domain via mcp PostHog/posthog#116863–feat(workflows): expose read-only email sending readiness over mcp PostHog/posthog#116865 are not on the server branch, so the session took the UI fallback with a locally verified sender.
   • Merge order: context-mill#433 waits on docs(workflows): add the transactional email API page PostHog/posthog.com#21306 being published (a clean build 404s until then), posthog-js#5312 being released, and the server stack feat(messaging): upload, send and lock code-managed email templates PostHog/posthog#118924.
   • pnpm security-scan:skills: no local LLM triage was available, so it flagged 413 matches across other skills. None is in this skill.
   • Fork CI: upstream workflows on the fork PRs need maintainer approval. Only the Wiz scanners ran (pass).

   Follow-ups

   • MCP learn catalog: the published skills.zip doesn't overlay context-mill skills, so learn -s in the PostHog MCP never returns instrument-* or this skill. Overlaying them in ci-agent-skills.yml would make the ranking proof hold in production.
   • posthog-node version: name the release that ships posthog.emails in the skill once Think through the Elements tab PostHog/posthog#5312 releases.
   • Ownership: context-mill's README ownership table and CODEOWNERS could name the Workflows team for this skill.
   • Docs page: outputFileTracingIncludes is under experimental in Next.js 14 and earlier, and the docs page doesn't say so.
10. Silthus

    closed this as completedon Oct 17, 2026