Skip to content

docs(automations): fresh clone per run, one comment per PR, cancel the previous run, PR gatekeeper recipe #174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sre-helmcode merged 3 commits into from
Oct 12, 2026
Merged

docs(automations): fresh clone per run, one comment per PR, cancel the previous run, PR gatekeeper recipe #174

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ce57964
docs(automations): fresh clone per run, one comment per PR, cancel th…
barckcode Oct 12, 2026
2ce5f21
docs(automations): review fixes: head-moved wording, recipe discovera…
barckcode Oct 12, 2026
85f7f95
docs(automations): say which run succeeded when the comment is missing
barckcode Oct 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 184 additions & 6 deletions src/content/docs-es/automations.mdx
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion src/content/docs-es/webhooks.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,8 @@ Por defecto, cualquier evento con una firma válida lanza un run. En un reposito

Puedes cambiarlo después con **Change** en el trigger.

Una automatización con **Fresh clone per run** también ignora, por defecto, las pull requests de forks y de autores sin permiso de escritura, diga lo que diga este filtro: su código se ejecutaría en tu workspace. Consulta [Pull requests de forks y de colaboradores externos](/es/docs/automations#pull-requests-de-forks-y-de-colaboradores-externos).

## Qué recibe el agente

El evento le llega al agente como datos, después de tus instrucciones y claramente marcado como algo que viene de fuera. Nunca se toma como instrucciones.
Expand Down Expand Up @@ -232,7 +234,7 @@ La pestaña **Deliveries** lista los últimos 50 webhooks que ha recibido la aut
| Resultado | Significado |
|---|---|
| **Run started** | Se ha creado un run. |
| **Ignored by a filter** | El filtro de eventos o de autores no ha coincidido. |
| **Ignored by a filter** | El filtro de eventos o de autores no ha coincidido o, con **Fresh clone per run**, la pull request venía de un fork o de un autor sin permiso de escritura. |
| **Stopped: caused by one of its own runs** | Se ha parado un [bucle](#bucles). |
| **Skipped: a run for the same subject is active** | Consulta [Cuando ya hay un run activo](/es/docs/automations#cuando-ya-hay-un-run-activo). |
| **Not accepted: your membership does not include runs** | No se ha podido crear el run: tu plan no tiene inferencia, o el workspace está parado o borrado, su agente no está instalado o no tiene clave de inferencia. El emisor no lo reintenta, así que el evento se pierde: mantén el workspace encendido. |
Expand Down
190 changes: 184 additions & 6 deletions src/content/docs/automations.mdx
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion src/content/docs/webhooks.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,8 @@ By default, any event with a valid signature starts a run. On a public repositor

You can change it later with **Change** on the trigger.

An automation with **Fresh clone per run** also ignores, by default, pull requests from forks and from authors without write access, whatever this filter says: their code would run in your workspace. See [Pull requests from forks and outside contributors](/docs/automations#pull-requests-from-forks-and-outside-contributors).

## What the agent receives

The event reaches the agent as data, after your instructions and clearly marked as coming from outside. It is never taken as instructions.
Expand Down Expand Up @@ -232,7 +234,7 @@ The **Deliveries** tab lists the last 50 webhooks the automation received in the
| Outcome | Meaning |
|---|---|
| **Run started** | A run was created. |
| **Ignored by a filter** | The event or actor filter did not match. |
| **Ignored by a filter** | The event or actor filter did not match, or, with **Fresh clone per run**, the pull request came from a fork or an author without write access. |
| **Stopped: caused by one of its own runs** | A [loop](#loops) was stopped. |
| **Skipped: a run for the same subject is active** | See [When a run is already active](/docs/automations#when-a-run-is-already-active). |
| **Not accepted: your membership does not include runs** | The run could not be created: your plan has no inference, or the workspace is stopped or deleted, its agent is not installed or it has no inference key. The sender does not retry it, so the event is lost: keep the workspace running. |
Expand Down
128 changes: 120 additions & 8 deletions src/data/openapi.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2566,7 +2566,10 @@
"enabled": true,
"disabled_reason": null,
"created_at": "2026-10-11T09:00:00Z",
"updated_at": "2026-10-11T09:00:00Z"
"updated_at": "2026-10-11T09:00:00Z",
"checkout": "workspace",
"sticky_pr_comment": false,
"checkout_untrusted_heads": false
}
]
}
Expand Down Expand Up @@ -2837,7 +2840,10 @@
"enabled": true,
"disabled_reason": null,
"created_at": "2026-10-11T09:00:00Z",
"updated_at": "2026-10-11T09:00:00Z"
"updated_at": "2026-10-11T09:00:00Z",
"checkout": "workspace",
"sticky_pr_comment": false,
"checkout_untrusted_heads": false
}
}
}
Expand Down Expand Up @@ -3233,7 +3239,7 @@
"platformToken": []
}
],
"description": "Starts a run of the automation now, with its saved instructions, secrets and approval settings: the same run a webhook or a schedule would start. Its agent proposes actions, which apply automatically or wait for your approval as the automation says. Follow it like any run (`GET /v1/runs/{id}/events`). Runs you start this way are never skipped by `skip_if_active`, but they count towards the automation's `max_runs_per_hour` and the limits of every run.\n\nAuthentication: a member API key (`sk-...`) or a platform token (`nan_pat_...`) with the `automations:run` scope. Runs need a plan with inference.",
"description": "Starts a run of the automation now, with its saved instructions, secrets and approval settings: the same run a webhook or a schedule would start. Its agent proposes actions, which apply automatically or wait for your approval as the automation says. Follow it like any run (`GET /v1/runs/{id}/events`). Runs you start this way are never skipped by `skip_if_active` or cancelled by `supersede`, but they count towards the automation's `max_runs_per_hour` and the limits of every run.\n\nAuthentication: a member API key (`sk-...`) or a platform token (`nan_pat_...`) with the `automations:run` scope. Runs need a plan with inference.",
"parameters": [
{
"name": "id",
Expand Down Expand Up @@ -3333,7 +3339,10 @@
"exposure": "agent"
}
],
"agent_finished_at": null
"agent_finished_at": null,
"checkout": null,
"superseded_by": null,
"sticky_pr_comment": false
}
}
}
Expand Down Expand Up @@ -3398,7 +3407,10 @@
"exposure": "agent"
}
],
"agent_finished_at": null
"agent_finished_at": null,
"checkout": null,
"superseded_by": null,
"sticky_pr_comment": false
}
}
}
Expand Down Expand Up @@ -3974,7 +3986,10 @@
"exposure": "agent"
}
],
"agent_finished_at": "2026-10-11T09:03:41Z"
"agent_finished_at": "2026-10-11T09:03:41Z",
"checkout": null,
"superseded_by": null,
"sticky_pr_comment": false
}
}
}
Expand Down Expand Up @@ -5805,6 +5820,29 @@
],
"format": "date-time",
"description": "When the agent of an automation run finished, before any approval."
},
"checkout": {
"oneOf": [
{
"$ref": "#/components/schemas/RunCheckout"
},
{
"type": "null"
}
],
"description": "On an automation run (`phase: \"propose\"`) whose automation uses `checkout: \"fresh\"`: the commit it works on. `null` for every other run, apply runs included."
},
"superseded_by": {
"type": [
"string",
"null"
],
"format": "uuid",
"description": "When a newer run for the same subject cancelled this one (`concurrency_policy: \"supersede\"`): that run's id. The run ends `cancelled` with `error_code: \"cancelled\"`."
},
"sticky_pr_comment": {
"type": "boolean",
"description": "On an automation run (`phase: \"propose\"`): whether its pull request comments edit the automation's one comment per pull request (`sticky_pr_comment` of the automation when the run started). `false` on other runs."
}
}
},
Expand Down Expand Up @@ -5873,6 +5911,22 @@
},
"unit": {
"type": "string"
},
"checkout": {
"type": "object",
"description": "Only on runs with a fresh checkout: the commit checked out.",
"properties": {
"mode": {
"type": "string",
"enum": [
"fresh"
]
},
"sha": {
"type": "string",
"pattern": "^[0-9a-f]{40}$"
}
}
}
}
},
Expand Down Expand Up @@ -6844,9 +6898,10 @@
"type": "string",
"enum": [
"queue",
"skip_if_active"
"skip_if_active",
"supersede"
],
"description": "`skip_if_active`: a new event for the same pull request, issue, workflow or schedule is skipped while a run for it is active, including one awaiting approval."
"description": "What happens when a new event arrives for the same pull request, issue, branch or schedule while a run for it is active, including one awaiting approval. `queue`: a new run is queued. `skip_if_active`: the new event is skipped. `supersede`: the new event starts a run and cancels the active runs of this automation for the same pull request, issue, branch or Stripe object (they end `cancelled`, with `superseded_by` set); events without such a subject (schedules, manual runs, NaN senders) behave as `queue`."
},
"max_trigger_depth": {
"type": "integer",
Expand Down Expand Up @@ -6875,6 +6930,25 @@
"updated_at": {
"type": "string",
"format": "date-time"
},
"checkout": {
"type": "string",
"enum": [
"workspace",
"fresh"
],
"default": "workspace",
"description": "Where the agent works. `workspace`: in `cwd`, in your workspace, as any run. `fresh`: NaN clones `github_repo` for each run into a folder of its own, at the event's exact commit (the pull request's head commit for a `pull_request` event on that repository, the run's head commit for a `workflow_run` event on that repository, the latest commit of the default branch otherwise), and deletes it when the run ends, whatever the outcome; `cwd` is not used and `git_isolation` must be `null` or `false`. A `git_push` is refused in `fresh` mode. Needs `github_repo`."
},
"sticky_pr_comment": {
"type": "boolean",
"default": false,
"description": "`true`: the automation keeps one comment per pull request. Each run's `github_pr_comment` edits the comment NaN posted before for this automation instead of adding a new one. NaN only edits a comment it posted itself and marked for this automation; issue comments are never edited."
},
"checkout_untrusted_heads": {
"type": "boolean",
"default": false,
"description": "Only with `checkout: \"fresh\"`. `false` (the default): pull requests from forks, pull requests whose author has no write access to the repository (not an owner, member or collaborator) and `workflow_run` events from forks are ignored (delivery outcome `ignored`), because the agent and the tests it runs would execute that code with the automation's agent secrets in your workspace. `true`: they run too. Your decision."
}
}
},
Expand Down Expand Up @@ -7320,6 +7394,44 @@
"description": "Plain text, masked."
}
}
},
"RunCheckout": {
"type": "object",
"description": "The fresh checkout of a run whose automation uses `checkout: \"fresh\"`.",
"properties": {
"mode": {
"type": "string",
"enum": [
"fresh"
]
},
"sha": {
"type": [
"string",
"null"
],
"pattern": "^[0-9a-f]{40}$",
"description": "The commit checked out. `null` until the run has resolved the default branch."
},
"pr_number": {
"type": [
"integer",
"null"
],
"description": "The pull request, when the event was about one."
},
"base_ref": {
"type": [
"string",
"null"
],
"description": "The pull request's base branch, when there is one."
},
"label": {
"type": "string",
"description": "For display, for example `fresh @ 0123456789ab` or `fresh @ default branch`."
}
}
}
},
"responses": {
Expand Down
44 changes: 43 additions & 1 deletion src/lib/openapiAutomations.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,8 @@ describe('openapi.json: the Automations contract (cloud-api)', () => {
'git_isolation', 'timeout_seconds', 'approval_kinds', 'approval_ttl_seconds', 'inference', 'github_repo',
'secrets', 'max_runs_per_hour', 'concurrency_policy', 'max_trigger_depth', 'enabled', 'disabled_reason',
'created_at', 'updated_at',
// Migration 110 (PR gatekeeper), same order as automationJSON.
'checkout', 'sticky_pr_comment', 'checkout_untrusted_heads',
]);
expect(sc.Automation.properties.template.enum).toEqual(['custom', 'pr_review', 'issue_triage', 'ci_failure', 'generic_webhook']);
expect(sc.Automation.properties.approval_kinds.items.enum).toEqual(ACTION_KINDS);
Expand All @@ -120,6 +122,26 @@ describe('openapi.json: the Automations contract (cloud-api)', () => {
expect(sc.Automation.description).toMatch(/Never carries a secret value/);
});

it('the PR gatekeeper settings (cloud-api PR #343): checkout, sticky comment, untrusted heads, supersede', () => {
const p = sc.Automation.properties;
expect(p.checkout).toMatchObject({ type: 'string', enum: ['workspace', 'fresh'], default: 'workspace' });
expect(p.checkout.description).toMatch(/clones `github_repo` for each run/);
expect(p.checkout.description).toMatch(/deletes it when the run ends, whatever the outcome/);
expect(p.checkout.description).toMatch(/`git_push` is refused/);
expect(p.checkout.description).toMatch(/`git_isolation` must be `null` or `false`/);
expect(op('/automations/{id}/runs', 'post').description).toContain('or cancelled by `supersede`');
expect(p.sticky_pr_comment).toMatchObject({ type: 'boolean', default: false });
expect(p.sticky_pr_comment.description).toMatch(/only edits a comment it posted itself/);
expect(p.checkout_untrusted_heads).toMatchObject({ type: 'boolean', default: false });
expect(p.checkout_untrusted_heads.description).toMatch(/Only with `checkout: "fresh"`/);
expect(p.checkout_untrusted_heads.description).toMatch(/forks/);
expect(p.checkout_untrusted_heads.description).toContain('`ignored`');
expect(p.concurrency_policy.enum).toEqual(['queue', 'skip_if_active', 'supersede']);
expect(p.concurrency_policy.description).toMatch(/`supersede`[^.]*cancels the active runs/);
expect(p.concurrency_policy.description).toContain('`superseded_by`');
expect(p.concurrency_policy.description).toMatch(/behave as `queue`/);
});

it('Trigger, template and delivery objects match the handlers', () => {
expect(Object.keys(sc.Trigger.properties)).toEqual([
'id', 'object', 'automation_id', 'kind', 'enabled', 'token_prefix', 'signature_scheme', 'event_filter',
Expand Down Expand Up @@ -271,13 +293,17 @@ describe('cloud-api golden responses validate against the published schemas', ()
'approvals_list.json', 'automation_deliveries.json', 'automation_get.json', 'automation_templates.json',
'automation_triggers.json', 'automations_list.json', 'run_get_adhoc.json', 'run_get_propose.json',
'run_list_propose.json', 'trigger_create.json',
]);
// cloud-api PR #343: a fresh-checkout, sticky, supersede automation and a superseded run.
'automation_get_gatekeeper.json', 'run_get_gatekeeper.json',
].sort());
});

const ref = (name: string): Schema => ({ $ref: `#/components/schemas/${name}` });
const cases: Array<[string, unknown, Schema]> = [
['automations_list', golden('automations_list'), ref('AutomationList')],
['automation_get', golden('automation_get'), ref('Automation')],
['automation_get_gatekeeper', golden('automation_get_gatekeeper'), ref('Automation')],
['run_get_gatekeeper', golden('run_get_gatekeeper'), ref('Run')],
['automation_templates', golden('automation_templates'), op('/automations/templates', 'get').responses['200'].content['application/json'].schema],
['automation_triggers', golden('automation_triggers'), ref('TriggerList')],
['trigger_create .trigger', golden('trigger_create').trigger, ref('Trigger')],
Expand All @@ -292,6 +318,22 @@ describe('cloud-api golden responses validate against the published schemas', ()
expect(validate(value, schema)).toEqual([]);
});

it('the gatekeeper goldens carry the new settings and the run checkout', () => {
expect(golden('automation_get_gatekeeper')).toMatchObject({ checkout: 'fresh', sticky_pr_comment: true, concurrency_policy: 'supersede', checkout_untrusted_heads: false });
expect(golden('run_get_gatekeeper')).toMatchObject({ state: 'cancelled', error_code: 'cancelled', sticky_pr_comment: true });
expect(golden('run_get_gatekeeper').superseded_by).toMatch(/^[0-9a-f-]{36}$/);
expect(Object.keys(golden('run_get_gatekeeper').checkout).sort()).toEqual(Object.keys(s.components.schemas.RunCheckout.properties).sort());
for (const g of ['automation_get', 'automations_list']) expect(JSON.stringify(golden(g))).toContain('"checkout":"workspace"');
for (const g of ['run_get_propose', 'run_get_adhoc']) expect(golden(g)).toMatchObject({ checkout: null, superseded_by: null, sticky_pr_comment: false });
});

it('the validator rejects a checkout mode or a sha the schema does not allow', () => {
expect(validate({ ...golden('automation_get_gatekeeper'), checkout: 'worktree' }, ref('Automation'))).not.toEqual([]);
expect(validate({ ...golden('automation_get_gatekeeper'), concurrency_policy: 'cancel' }, ref('Automation'))).not.toEqual([]);
const run = golden('run_get_gatekeeper');
expect(validate({ ...run, checkout: { ...run.checkout, sha: 'main' } }, ref('Run'))).not.toEqual([]);
});

it('the validator still rejects a field the schema does not declare', () => {
expect(validate({ ...golden('automation_get'), secret_value: 'x' }, ref('Automation'))).not.toEqual([]);
});
Expand Down
Loading
Loading