Skip to content

feat(analytics): set detected email providers on the person - #1493

Draft
Silthus wants to merge 5 commits into
PostHog:mainfrom
Silthus:feat/detected-email-providers
Draft

Silthus wants to merge 5 commits into
PostHog:mainfrom
Silthus:feat/detected-email-providers

Conversation

@Silthus

@Silthus Silthus commented Oct 26, 2026 •

Copy link
Copy Markdown

Problem

The Wizard's codebase scan already finds the email services a user's code sends through, such as Resend, SendGrid or Amazon SES. That fact stays in the scan and never reaches the person, so feature flags, surveys and the Workflows onboarding in the PostHog app can't tell that this user already sends email from their code.

With this PR, the logged-in person gets wizard_detected_email_providers (the detected kinds, such as ["Resend"]) and wizard_detected_email_provider_at (when the scan found them), on every program that scans: the default integration flow, interactive or CI, wizard warehouse and wizard self-driving. A wizard --signup run now identifies the account it creates, so its events and these properties land on that account's person too.

Impact

Pays into Grow Workflows through Started orgs (orgs that create their first workflow). Not measured yet: an insight in the internal project counts people with wizard_detected_email_providers per week and how many of them reach Workflows, start its onboarding, create an email sender, or launch a workflow or broadcast. In scope: email, the Messaging category. Expected move: small, up. On its own this PR only adds the targeting signal; a follow-up such as a "We found Resend in your code" card in Workflows uses it. Run cost: n/a.

Merge Danger

Note

🔵 Merge risk: low · one-way door (merge-risk rubric)

  • Released Wizards write two new properties onto real person profiles in our analytics project. A revert stops new writes, but the values already set stay until a $unset.
  • Everything else is additive: reportCodebaseScan gains one call behind its existing consent, login and failed-scan checks, and Analytics gains setPersonProperties. No existing event or $identify payload changes.
  • wizard --signup runs now call identifyUser for the new account, as OAuth logins already do, so their events merge onto that person instead of staying anonymous. A failed user read keeps the signup going.

Changes

  • Which providers: emailProviderPersonProperties (src/services/codebase-scan/email-providers.ts) reads the run's codebase scan and keeps the tool catalog kinds of email services: AwsSes, Beehiiv, BigMailer, Brevo, CampaignMonitor, ConvertKit, Drip, Elasticemail, EmailOctopus, Flexmail, Loops, Mailchimp, MailerLite, MailerSend, Mailgun, Mailjet, Plunk, Postmark, Resend, SendGrid, Shipmail, Smaily, SparkPost, Tinyemail. Kind names match PostHog's live /api/wizard/tool_catalog/.
  • When it goes out: reportCodebaseScan, the one place a scan is cleared to leave the machine, sets both properties with analytics.setPersonProperties when it shares the scan: sharing granted, telemetry on, logged in with a known user, and the scan did not fail. A login without a user (a signup whose user read failed, or an API key without user:read) has no person to set them on, so it sends nothing. The scan-share latch keeps it to once per run. Login identifies the user first, so the $set lands on the logged-in person. A scan with no email provider sends nothing, so the person keeps what an earlier run found.
  • The client: Analytics.setPersonProperties sends posthog-node's $set event on the run's identified id.
  • Signup runs: askForProvisioningSignup (src/tui/auth/project-data.ts) reads the new account's user with the provisioned token, which already holds user:read, identifies it, and returns it as the login's user. Before, a signup run never identified anyone, so every event and the $set stayed on an anonymous person. A failed read is captured with captureException and the signup continues unidentified.

Test plan

Lane gate on 7e96c5d: pnpm typecheck green, prettier and eslint clean on the changed files, and the tests of every touched module (codebase-scan, detection, shared/utils, tui/auth, the integration, warehouse, self-driving programs and the program runner) green. Full gate on 7e96c5d: pnpm lint (0 errors) and pnpm test (build plus the whole suite) green locally. CI: the Wiz scans pass; Build & Test, Semgrep and the commit check wait for a maintainer to approve workflows for this fork PR.

Test seams:

  • reportCodebaseScan(store) on a CI session after a real scan of a temp project against the fixture catalog, with analytics mocked at @utils/analytics: a Resend + Stripe project, reported twice after login, sets ["Resend"] and the scan time once; a Stripe-only project, a declined-sharing session, and a login without a user set nothing.
  • Analytics over a mocked posthog-node client: after identifyUser, setPersonProperties sets the properties on the identified id.
  • getOrAskForProjectData({ signup: true }) with provisioning and the API mocked: a successful signup identifies the new account and returns it as the login's user; a failed user read still returns the signed-up credentials, with no user.

Proofs per qa-swarm PROOF.md:

  • Live run (saved for 7e96c5d), two runs of the real code with a fetch preload that logged each analytics request and answered it locally, so nothing reached the real project:
    • Signup: the real logIn with the TUI's OAuth provider on a --signup session, over real HTTP to a local server that answers the provisioning API and /api/users/@me/ and serves the live tool catalog. Analytics sent $identify on the new account's id, then $create_alias, then one $set with ["Resend"] on that id.
    • CI-shaped session: the real reportCodebaseScan, SessionStore and Analytics scanning a Next.js app with the resend package. A report before login sent nothing about the person; after login, three reports sent exactly one $set on the logged-in id.
  • Gap: no real account, OAuth login or ingestion. This box has no PostHog key, a real signup would create an account, and the client's analytics key and host are fixed. The $set is a plain person update on the identified id, with no merge involved.
  • Prod shape (gap, accepted by standing rule until 2026-11-02): the scan reads customer code; the share of scanned projects with an email provider stays unknown until read-only production access is set up.
  • Post-deploy watch: owed after merge.
Captured requests, signup live run on 7e96c5d (trimmed)
provisioning:  POST account_requests, POST oauth/token, POST resources, GET /api/users/@me/ (Bearer <provisioned token>)
$identify      signup-new-user  {"$set": {"email": "new@example.com", "name": "New"}}
$create_alias  signup-new-user  {"alias": "<run anonymous id>"}
wizard: auth complete  signup-new-user
$set           signup-new-user  {"$set": {"wizard_detected_email_providers": ["Resend"], "wizard_detected_email_provider_at": "<scan time>"}}

LLM context

Built by Claude Code (Opus 5.5) from a handoff in Michael's Workflows onboarding thread.

Question Answer Who Where
Should --signup runs identify the new account? Yes: read the new user with the provisioned token and identify it, as OAuth does, so the property reaches the new account. Replaces leaving signup runs anonymous as a follow-up. 👤 Human comment
Email providers only, or a general wizard_detected_services? Email providers only. A general list costs the same to send but has no consumer yet, and the scan report already carries every detected kind. 🤖 Agent comment
Which kinds count as email providers? Services whose main job is sending email, marketing email included. Test inboxes (Mailtrap, Mailosaur) and multi-channel platforms (ActiveCampaign, Omnisend, Customer.io, Klaviyo, Braze, Iterable, Courier, Knock, OneSignal) stay out. 🤖 Agent comment
Where does the property get set? When the scan is shared after login, on the logged-in person, for every program that scans. Replaces the earlier $set on the pre-login warehouse sources detected event, which only the interactive integration flow sends. 🤖 Agent comment
What happens on a run with no email provider? No $set, so the person keeps what an earlier run found. 🤖 Agent comment

Involvement: no grilling rounds, no prototypes, a design pass, a cross-family review, a QA swarm panel (7 lenses across GPT-6.1 Sol and Opus 5.5, including the author's Team perspective) a narrow second round on its fixes, and two final reads, live-run proof captured.

🤖 Generated with Claude Code

The codebase scan already finds email-sending services such as Resend.
Set them on the person as wizard_detected_email_providers and
wizard_detected_email_provider_at, through $set on the
`warehouse sources detected` event. A $set captured before login is set
again at identify, so it reaches the person who uses the PostHog app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silthus and others added 3 commits October 26, 2026 19:05
Set the person properties when the scan is shared after login, so CI,
headless, warehouse and self-driving runs set them too, and drop the
pre-login replay in Analytics. Count Campaign Monitor, ConvertKit,
beehiiv and Drip as email providers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read the new account's user after provisioning and identify it, as the
OAuth login does, so a signup run's events and the detected email
providers land on its person. A failed read keeps the signup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A login without a user, such as a signup whose user could not be read,
has no person to set them on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Silthus
Silthus marked this pull request as ready for review October 26, 2026 19:34
@Silthus
Silthus requested a review from a team as a code owner October 26, 2026 19:34
@Silthus

Silthus commented Oct 26, 2026

Copy link
Copy Markdown
Author

Note

🤖 Claude Opus 5.5 responding on behalf of Michael

  • 👤 Should --signup runs identify the new account? Yes, read the new user with the provisioned token and identify it, as OAuth does, so the property reaches the new account.
  • 🤖 Email providers only, or a general wizard_detected_services? Email providers only. A general list has no consumer yet, and the scan report already carries every detected kind.
  • 🤖 Which kinds count as email providers? Services whose main job is sending email, marketing email included. Test inboxes and multi-channel platforms (ActiveCampaign, Omnisend, Customer.io, Klaviyo, Braze, Iterable, Courier, Knock, OneSignal) stay out.
  • 🤖 Where does the property get set? When the scan is shared after login, on the logged-in person, for every program that scans.
  • 🤖 What happens on a run with no email provider? No $set, so the person keeps what an earlier run found.

@Silthus
Silthus marked this pull request as draft October 28, 2026 04:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant