[Silthus]

Important

Stacks on #118980 (spec/template-agent-edit), which stacks on #118924. GitHub can't base a PR on a fork branch, so this targets master. Review starts after stack A's tip, 0960e3de116: git diff 0960e3de116...spec/template-request-change.

Problem

A person who can't reach a code-managed email template's repository has no way to get the template changed.
The template page is read-only (#118924), and #118980 only hands the person a prompt for their own coding agent.
A marketer without a coding agent, PostHog AI in the sidebar, and the Slack bot can't start the change.

Caution

🔴 Merge risk: high · one-way door (merge-risk rubric)

• A new route and MCP tool start a billable PostHog Code task with write access to the customer's GitHub repository. The task's credits and the pull request it opens outlive a revert.
• Limits: everything sits behind workflows-transactional-email-api (off). Project secret API keys and calls from inside PostHog Code are refused, callers need hog_flow:read and task:write plus editor access to workflows, and the task opens a pull request that a person merges. PostHog never saves, uploads or syncs the template.

Changes

On a code-managed template, "Edit with your agent" gains Run with PostHog Code. Only flagged uploads make a template code-managed, so nobody sees it today.

• Request a change. POST /api/projects/{id}/messaging_templates/{id}/request_change/ with {instruction} starts a PostHog Code task. The task edits the template's file in its GitHub repository and opens a pull request. The response is 201 {task_id, task_url, repository}.
• The task prompt is the person's sentence, the same "About this file" block the copied prompt uses, and the rule to change only this file and leave the upload to after the merge.
• Who may call it: sessions, personal API keys and OAuth tokens with hog_flow:read and task:write, and editor access to workflows. Project secret API keys get 403. A call from inside a PostHog Code sandbox gets posthog_code_not_available, so PostHog AI and the Slack bot hand the person the template link instead.
• Errors use the feat(messaging): upload, send and lock code-managed email templates #118924 envelope (code, detail, why, fix, docs_url): template_not_managed_by_code, template_repository_missing, template_repository_unsupported, github_repository_not_connected, posthog_code_not_available, instruction_missing, posthog_code_run_failed.
• The button is shown only when PostHog Code is available to the organization. It says why it can't run: no workflow editor access, no repository, GitHub not connected, the change not described, or longer than 2,000 characters. It ignores a second click while the request runs. A refusal shows detail and fix in the popover, or as a toast when the popover was closed. Success shows a toast and a task link in the notice.
• Agents never update or patch a code-managed template, and never rebuild and upload it from PostHog. With the repository in their workspace they edit the file. Inside PostHog Code they share the template link. Any other agent calls the new MCP tool workflows-request-email-template-change. The designing-email-templates skill, the tool texts, the 403 immutable fix and the sidebar page context say this in the same order.
• Events: message template change run requested and … run failed (button), message template change requested and … change refused (server, with surface). No instruction text, path or repository in properties.
• Mechanical: regenerated OpenAPI types and MCP tool definitions; tach.toml lets messaging use the tasks facade.

Captures from the real app built from this branch at 0c3c8b87b51, on a scratch project with invented content (Hedgebox, example.com):

Change described, Run with PostHog Code next to Copy prompt (1440px)	Same at 560px, about the scene width with the side panel open

Refused: the route's detail and fix in the popover (560px)	Started: toast and task link in the notice (1440px)

Reading order

Ticket	Commits	What
Silthus/posthog#513	804a0b87d6c	Route, PostHog Code task, error codes, MCP tool, 403 immutable fix text
Silthus/posthog#514	60fb98f06a5	"Run with PostHog Code" in the popover
Silthus/posthog#515	fbbbe6ac4f1, df551ee9fb6	Eval harness; skill, page context, tool text, evals
Merges	495bca44dec, 85da5752479	Stack A into this branch
Review round 1	6a8e0376690, e00a29226a4, 29a52f31796	hog_flow:read scope, agent texts point at change_prompt_context, .. paths unknown, refusal event; button access and length reasons, toast when closed; eval reads the page's real context
Review round 2	fc5dbc9aa98, 0c3c8b87b51	Refusal event on every refusal, one wording for when an agent edits the file, upload rule in the upload tool; skill chip not dismissible, code-point length

How did you test this code?

Live run, Django built from this branch on a scratch database, with the flag forced on and create_and_run_task wrapped to pass start_workflow=False. That creates a real Task row and starts no sandbox. The Desktop access, credits and GitHub repository checks ran for real. At 0c3c8b87b51:

• A personal API key started a task. The task got repository example/app, create_pr, read-only PostHog scopes, and the full description. The template's hash and its workflow's version stayed unchanged.
• A project secret key with task:write, a key without task:write, and a call with the sandbox header got 403. A template made in PostHog got template_not_managed_by_code.
• In the real app (Chromium, 1440 and 560 px): the button starts one task on a double click, a repository the GitHub integration can't reach shows detail and fix in the popover, and success shows the toast and the notice link.

Shadow diff, base 0960e3de116 against HEAD, the same requests on identical scratch databases. Upload, list, GUI edits and release answer byte for byte the same. The only differences are the new route, the new 403 immutable fix text, and a source path with .. now being unknown in the prompt.

Test rationale:

• test_template_change_request.py (APIBaseTest, templates made by the real upload route, the tasks facade mocked at its function boundary as the spec asks): the exact task arguments and description, every refusal code, the server events, project secret keys and tokens without hog_flow:read or task:write refused, and the template and workflow unchanged.
• test_code_managed_template_lock.py: the exact 403 immutable fix, including a path that could carry an instruction and one that leaves the repository.
• test_template_change_prompt.py: a .. path is unknown.
• messageTemplateLogic.test.ts (API mocked at HTTP): the exact request body, one request on a double click, each disabled reason, availability, success, refusal in the popover and as a toast.
• templateAgentContext.test.ts: the page context, a closed chip on one template keeping every other page's context (through the context store), and the eval's sidebar fixture rendered from the real builders.
• Eval harness and scorer tests, including an upload by key counted as touching the template.

Local gate on HEAD: messaging backend, eval harness, MCP tool definitions and repo invariants 1,615 passed; mypy repo-wide clean; frontend tsgo clean; TemplateLibrary jest 112 passed; MCP typecheck, tool schema snapshots and tool-name lint; hogli build:openapi committed; pre-push ci:preflight --strict 0 failures.

Gaps:

• No real PostHog Code run. This devbox has no PostHog Code sandbox provider and no GitHub App credentials, so no task ran and no pull request opened against the scratch repository. The task's arguments and description are proven at the facade boundary and by the real Task row.
• The evals have never run. products/workflows/evals/eval_code_managed_templates.py is written, discovered and unit-tested, but this devbox has no LLM gateway, Braintrust or Modal credentials. Run it where they exist, from a checkout of this branch in a flox shell:

  hogli evals eval_code_managed_templates                   # green: all three cases (Docker)
  hogli evals eval_code_managed_templates --provider modal  # same on Modal
  # red on the skill before #515:
  git checkout 85da5752479 -- products/workflows/skills/designing-email-templates/SKILL.md
  hogli evals eval_code_managed_templates --eval slack_message_names_code_managed_template
  git checkout HEAD -- products/workflows/skills/designing-email-templates/SKILL.md

• Prod shape is not measured until production read access exists (2026-11-02).
• CI on this fork PR waits for a maintainer to approve the run.

Release status

• This change is behind a feature flag and is not available to users

Docs update

The docs section on changing a code-managed template, with the request-change error anchors, ships with PostHog/posthog.com#21306 (Silthus/posthog#507).

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Claude Code, Opus 5.5 wrote the code and the review fixes.

Built as three tickets of spec Silthus/posthog#502 (map Silthus/posthog#461), each test-first with /tdd, then reviewed as one PR with a QA swarm because it adds a billable task with repository write access to a public API and an MCP tool. Skills: /improving-drf-endpoints, /implementing-mcp-tools, /writing-evals, /writing-kea-logics, /writing-tests, /writing-user-facing-copy, /writing-pr-descriptions. Every screenshot and fixture uses invented data (example.com, Hedgebox). CodeRabbit is retired on this repository, so it did not run.

Decisions

Question	Answer	Who	Where
Does "Run with PostHog Code" ship, and do PostHog AI and Slack route code-managed changes to a PR?	Yes: one server action, "request a change", serves the web button, agents and any MCP client (Card 4 → b, widened)	👤 Human	spec #502, revision 3
Who edits a code-managed template?	The agent with the repository edits the file. "Request a change" is only for agents and people without it (spec #502, revision 4)	👤 Human	#511
May a call from inside a PostHog Code sandbox start a task?	No: the tasks product's rule holds, so request_change refuses it	👤 Human	map #461, Sandbox rule
What do PostHog AI and the Slack bot do for a code-managed template?	They hand the person the template link, where Run with PostHog Code opens the pull request	👤 Human	#507
Does the eval gap block the PR?	No: the evals are recorded as a gap with the run command	👤 Human	#515
Where is the task's prompt built?	The same server builder as the copied prompt, plus the change rules; no second builder	🤖 Agent	spec #502
Which PostHog scopes does the task get?	Read-only: it changes a file and opens a pull request, so it can't change the template in PostHog	🤖 Agent	#513
How is an agent's sentence tied to the file?	A sentence that doesn't start with the starter gets In `<path>`, change what this request asks for: in front	🤖 Agent	#513
Which repository forms run?	Only github.com/<owner>/<name>, the form posthog-node and the skill store; a bare owner/name is refused as unsupported	🤖 Agent	#513
Which status does each refusal get?	403 for immutable and posthog_code_not_available (including used-up credits and an unverifiable access check), 400 for the rest; keeps the spec's seven codes	🤖 Agent	#513
Where does the page learn that PostHog Code is available?	desktop/access/, the decision the route checks; credits are left to the route's refusal	🤖 Agent	#514
How does the page check GitHub?	Any GitHub integration on the project; whether it reaches this repository stays the route's refusal	🤖 Agent	#514
Where do refusals show?	In the popover, next to the sentence, with the server's detail and fix; a toast only when the popover is closed	🤖 Agent	#514, #517
How does the skill tell it runs inside PostHog Code?	By POSTHOG_TASK_RUN_ID, which every sandbox sets	🤖 Agent	#515
How does the uploaded source reach PostHog AI?	One hidden untrusted item; the trusted instruction names it and quotes no uploaded value	🤖 Agent	#515
Which workflow access does a person need?	Editor, the platform's mapping of a :write scope; the button names it before the click. Replaced the spec's "read access"; the alternative is a custom per-action permission at viewer	🤖 Agent	#517
Does a token need template read too?	Yes: hog_flow:read next to task:write, as the spec says	🤖 Agent	#517
Where does the GitHub repository rule live?	Only in the route; the page checks that a repository is recorded. Two tickets had built the rule twice; the alternative was a new read-only field	🤖 Agent	#517
How does the eval get the sidebar context?	From a fixture the page's jest test renders with the real builders; two tickets had built the text twice	🤖 Agent	#517
May an uploaded source pick the task's repository?	Yes, as the spec sets: the page shows it before the click, and the task only opens a pull request. Upload validation stays out of scope	🤖 Agent	#517
Does the MCP tool need the tasks product's agent-run flag?	No: the spec opens it to MCP clients behind its own flag, as experiments' cleanup PR tool does	🤖 Agent	#517
How does closing a chip on one template keep other pages' context?	Each template page gets its own dismiss group, and its skill chip is not dismissible, like the new-template page's; closing it dismissed the shared skill key everywhere. Alternative: group-scoped dismissal in the shared context store	🤖 Agent	#517
Which refusals count?	Every one: message template change refused {template_id, surface, code}, including ownership and instruction refusals	🤖 Agent	#517
What is the merge risk?	High: a billable task that writes to the customer's repository; shadow diff run	🤖 Agent	#517

Involvement: no grilling rounds for this stack; 1 prototype on the map (4 rounds); QA swarm with 2 panel rounds and 1 final read (GPT-6.1 Sol high: correctness, risk, design, systems, author perspective; Opus 5.5 high: fresh eyes, UI, agent documents); live run and real-app UI run captured; shadow diff run.

Closes Silthus#513
Closes Silthus#514
Closes Silthus#515
Closes Silthus#502

🤖 Generated with Claude Code

[trunk-io]

Merging to master in this repository is managed by Trunk.

• To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here