Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions migration/yaml-parity-v1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,16 @@ from the harness's step bindings, not a separate operation inventory.

Capture API declarations are independent of client/server SDK type:

- `capture_ai_v0`: AI capture delivery, including the ordinary-capture no-reroute
case. That case accepts either legacy batch or analytics-v1 ordinary traffic.
- `capture_ai_v0`: AI capture delivery on the legacy `/i/v0/ai/batch/` route, including
the ordinary-capture no-reroute case. That case accepts either legacy batch or
analytics-v1 ordinary traffic.
- `capture_ai_v1`: AI capture delivery on the capture v1 `/i/v1/ai/events` route; its
no-reroute case expects analytics-v1 ordinary traffic. `capture-ai-v1-delivery.feature`
adds misroute, lane isolation, ceiling and batching cases under the same capability.
- `capture_ai_returns_uuid`: the AI method returns the admitted event's UUID
(posthog-python, posthog-node, posthog-ios, posthog-android; not posthog-go or
posthog-rs). `event_uuid_override`: the capture and AI methods accept a
caller-supplied `uuid` (server SDKs; not posthog-ios or posthog-android).
- `capture_v1`: analytics-v1 delivery through ordinary capture.
- `capture_v0`: legacy capture; `capture_v0_batch` additionally selects the batch
format checks, while `capture_v0_event` selects the event format checks.
Expand Down
53 changes: 53 additions & 0 deletions migration/yaml-parity-v1/capture-ai-v1-delivery.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
@public @acceptance @capture-ai @api_capture_ai_v1 @requires:capture_ai_v1
Feature: AI capture v1 delivery
Misroute reporting, lane isolation, the per-event ceiling and byte batching on the capture v1 AI lane.
The step phrases are the capture-ai spec's; the harness binds them with its capture v1 suite update.

Background:
Given an isolated SDK with empty persistent storage
And the mock PostHog server is reset

Scenario: A non-AI event on the AI method is sent and its misroute is reported
Given the SDK is initialized with token "phc_test_key" and flush threshold 1
And the mock serves these ordered AI responses:
| status | body |
| 200 | {"results":{"*":{"result":"drop","details":"misrouted_event"}}} |
When capture_ai is called with JSON arguments:
"""application/json
{"distinct_id":"test_user","event":"button_clicked"}
"""
And pending captures are flushed
Then exactly 1 capture request should have been received
And every capture request path should be one of:
| path |
| /i/v1/ai/events |
And the SDK should report the event "button_clicked" as dropped with details "misrouted_event"

Scenario: A failing AI lane does not delay analytics delivery
Given the SDK is initialized with token "phc_test_key" and flush threshold 1
And the mock delays every request to "/i/v1/ai/events" until released
When capture_ai is called with distinct id "test_user" and event "$ai_generation"
And capture is called with distinct id "test_user" and event "button_clicked"
And 2000 milliseconds elapse without a public SDK call
Then the analytics batch endpoint should have received "button_clicked"
When the delayed AI request is released and succeeds
And pending captures are flushed
Then the AI batch endpoint should have received "$ai_generation"

Scenario: An event at the ceiling is sent and one over it is dropped with a content-free log
Given the SDK is initialized with token "phc_test_key" and flush threshold 2
When capture_ai is called with event "$ai_generation" named "at-ceiling" whose properties serialize to exactly 8388608 bytes
And capture_ai is called with event "$ai_generation" named "over-ceiling" whose properties serialize to 9437184 bytes and contain the marker "SECRET-MARKER"
And pending captures are flushed
Then the AI batch endpoint should receive exactly 1 event
And the received event should be "at-ceiling"
And the SDK should log the event name and byte size of "over-ceiling"
And the log output should not contain "SECRET-MARKER"

Scenario: AI batches close before the byte target and keep order
Given the SDK is initialized with token "phc_test_key" and flush threshold 10
When capture_ai is called sequentially 3 times with an event named "gen-{index}" whose properties serialize to about 3 MiB
And pending captures are flushed
Then at least 2 capture request should have been received on the AI batch endpoint
And no received request body should exceed the SDK's byte target plus envelope overhead
And the events "gen-0", "gen-1" and "gen-2" should have been received in that order across the requests
85 changes: 85 additions & 0 deletions migration/yaml-parity-v1/capture-ai-v1.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
@public @acceptance @capture-ai @api_capture_ai_v1 @requires:capture_ai_v1
Feature: AI capture v1 YAML parity
AI capture on capture v1 is declared independently of runtime and identity model.

Background:
Given an isolated SDK with empty persistent storage
And the mock PostHog server is reset
And the SDK is initialized with token "phc_test_key" and flush threshold 1

Scenario: AI capture uses the capture v1 AI endpoint
Comment thread
eli-r-ph marked this conversation as resolved.
Resolved
When capture_ai is called with JSON arguments:
"""application/json
{"distinct_id":"test_user","event":"$ai_generation","properties":{"$ai_model":"gpt"}}
"""
And pending captures are flushed
Then exactly 1 capture request should have been received
And every capture request path should be one of:
| path |
| /i/v1/ai/events |
And the first received event field "event" should equal "$ai_generation"
And the first request should authenticate with bearer token "phc_test_key"

Scenario: AI capture avoids the legacy AI endpoint
When capture_ai is called with JSON arguments:
"""application/json
{"distinct_id":"test_user","event":"$ai_generation"}
"""
And pending captures are flushed
Then exactly 1 capture request should have been received
And no capture request should use "/i/v0/ai/batch/"
And no capture request should use "/batch"

Scenario: Ordinary capture does not reroute AI-named events
When capture is called with JSON arguments:
"""application/json
{"distinct_id":"test_user","event":"$ai_generation"}
"""
And pending captures are flushed
Then exactly 1 capture request should have been received
And every capture request path should be one of:
| path |
| /i/v1/analytics/events |
And no capture request should use "/i/v1/ai/events"

@requires:capture_ai_returns_uuid
Scenario: AI capture returns the generated wire UUID
When capture_ai is called with JSON arguments:
"""application/json
{"distinct_id":"test_user","event":"$ai_generation"}
"""
Then AI capture should return an admitted event UUID
Comment thread
eli-r-ph marked this conversation as resolved.
Resolved
When pending captures are flushed
Then the first received event UUID should be valid
And the AI capture return should equal the first received event UUID

@requires:capture_ai_returns_uuid @requires:event_uuid_override
Scenario: AI capture preserves a supplied UUID in its return and on the wire
Comment thread
eli-r-ph marked this conversation as resolved.
Resolved
When capture_ai is called with JSON arguments:
"""application/json
{"distinct_id":"test_user","event":"$ai_generation","uuid":"0198c0de-0000-7000-8000-000000000abc"}
"""
Then AI capture should return an admitted event UUID
And the AI capture return should equal "0198c0de-0000-7000-8000-000000000abc"
When pending captures are flushed
Then the first received event field "uuid" should equal "0198c0de-0000-7000-8000-000000000abc"
And the AI capture return should equal the first received event UUID

Scenario: AI timestamps normalize the instant without rewriting property strings
When capture_ai is called with JSON arguments:
"""application/json
{"distinct_id":"test_user","event":"$ai_generation","timestamp":"2025-01-02T08:34:05+05:30","properties":{"timestamp_like":"2025-01-02T08:34:05+05:30"}}
"""
And pending captures are flushed
Then every event in the first capture request should have UTC timestamp "2025-01-02T03:04:05Z"
And the first received event property "timestamp_like" should equal "2025-01-02T08:34:05+05:30"

Scenario: AI capture sends options and strips the legacy option properties
When capture_ai is called with JSON arguments:
"""application/json
{"distinct_id":"test_user","event":"$ai_generation","properties":{"$process_person_profile":false},"options":{"custom_flag":true}}
"""
And pending captures are flushed
Then the first received event option "custom_flag" should equal JSON true
And the first received event option "process_person_profile" should equal JSON false
And the first received event property "$process_person_profile" should be absent
Comment thread
eli-r-ph marked this conversation as resolved.
Resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-10-25
50 changes: 50 additions & 0 deletions openspec/changes/archive/2026-10-25-add-capture-ai-v1/design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
## Context

Capture's AI lane is a separate deployment that serves `POST /i/v1/ai/events` with the same request and response contract as `/i/v1/analytics/events` (`rust/capture/src/v1/analytics/router.rs` `ai_routes` in PostHog/posthog). It accepts larger bodies, applies an 8 MiB ceiling to each event's `properties`, drops a non-`$ai_` event with `misrouted_event`, and shares the AI byte budget and quota with the analytics endpoint. The analytics endpoint still accepts `$ai_*` events and forwards them to the AI topic; PostHog plans to stop that once every shipped SDK has an AI method.

The four server SDKs implement one AI lane on their next-major branches:

- posthog-python `v1`: `capture_ai` and `capture_ai_immediate`; `capture_ai_compression`, `capture_ai_max_queue_size`, `capture_ai_timeout`, `capture_ai_max_event_bytes`
- posthog-go `v1`: `EnqueueAI` / `EnqueueAIWithContext`; `CaptureAICompression`, `CaptureAIMaxQueueSize`, `CaptureAIBatchUploadTimeout`
- posthog-rs `v1`: `capture_ai`, `capture_ai_batch` and their immediate variants; `capture_ai_compression`, `capture_ai_max_queue_size`
- posthog-node capture v1 series: `captureAi` / `captureAiImmediate`; `captureAiCompression`, `captureAiMaxQueueSize`, `captureAiRequestTimeoutMs`

The current spec predates all four: it names only Python and Node, ties wrapper routing to `enable_full_ai_capture`, and lists `/i/v0/ai/batch/` as the transport.

## Goals / Non-Goals

**Goals:**
- State the capture v1 AI lane so the next SDK adds AI capture the same way.
- Fix the wrapper contract: the AI method is the only route for PostHog's wrappers, with no fallback to `capture`.
- Give the harness a `capture_ai_v1` capability and cases.

**Non-Goals:**
- The analytics wire, headers, retries and layer order, which `add-capture-v1` covers and this change references.
- The OpenTelemetry AI bridge requirement, unchanged.
- The client-SDK persistence model beyond the "Client SDKs:" notes (durable queue, disk cap); posthog-ios 4.0, posthog-android 4.0 and posthog-flutter 6.0 are in scope through those notes.

## Decisions

**The AI method is the only route for wrappers, whatever the flag says.** On legacy capture the wrappers sent through `capture` unless `enable_full_ai_capture` was set, and the spec made that the contract. The SDK owners reversed it during the ports: an AI event can be megabytes, and the analytics endpoint will stop accepting `$ai_*` events once every SDK has an AI method, so a wrapper that still falls back to `capture` would start losing events on that day. The flag now governs truncation and media redaction only. Alternative rejected: keep the fallback for a client object that lacks the AI method. Only a user's stand-in or mock can lack it on a v1 client, and the fallback would route those events to the lane that is going away; the wrapper logs one error at construction and turns its own capture off instead.

**Lane configuration is required; the per-event cap stays optional and lower-only.** Compression, queue capacity and request timeout are public on all four SDKs with the same defaults (none, 1000, 30 s), so they are a SHALL. Only Python exposes the per-event cap, and only to lower it, so that stays a MAY. Draft #118 also required rejecting invalid values at construction; posthog-go substitutes the default for a non-positive AI queue size, so this change asks for validation of the effective values and leaves the failure mode to the SDK.

**Node's gzip default is recorded, not overruled.** posthog-node 5.x gzips every request by default and the series keeps that for both lanes; the other three default to no compression. The requirement states the default as the SDK's choice and names both.

**Size is measured where the server measures it, or with headroom.** Capture measures `properties` only. posthog-rs and posthog-node measure `properties` exactly; posthog-python and posthog-go measure the whole event and add 64 KiB of headroom. The requirement fixes the observable rule (never refuse an event capture accepts; never send one it refuses) and allows either mechanism.

**The harness feature mirrors the v0 one.** `capture-ai-v1.feature` reuses the registered step phrases of `capture-ai.feature` with the v1 path; its one new phrase, property absence, is bound in the harness with the capture v1 suite update. It is selected by `@requires:capture_ai_v1`; the v0 feature keeps `capture_ai_v0`.

## Risks / Trade-offs

- [The analytics endpoint's acceptance of `$ai_*` events ends on a date PostHog has not set] → the requirement says "temporary" and that SDKs never rely on it, as the public docs do.
- [The posthog-node series is unmerged] → every rule here is also in Python, Go or Rust; Node appears only in option names and the compression variation.
- [Draft #118 and this change both touch the Configuration section] → #118 is superseded; its author is the PostHog bot, so closing it is a maintainer call.

## Migration Plan

Spec-only. Archive on this branch after review. SDK adapters declare `capture_ai_v1` when they move to the v1 AI endpoint.

## Open Questions

- Resolved 2026-10-27: capture v1 SDKs retry `404` on the AI endpoint (a deployment without the AI lane) the same way as on analytics, per `retry-queue`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
## Why

The `capture-ai` spec still describes the AI route as a Python and Node beta on `/i/v0/ai/batch/`, with routing of PostHog's own AI wrappers tied to the `enable_full_ai_capture` flag. Four server SDKs now ship AI capture on capture v1 (`/i/v1/ai/events`) in their next majors, posthog-go and posthog-rs among them, and the wrappers always use the AI method. The harness has no `capture_ai_v1` capability, so the AI parity cases cannot select a capture v1 SDK at all.

## What Changes

- Extend applicability to posthog-go v2 (`EnqueueAI`) and posthog-rs 1.0 (`capture_ai`), next to posthog-python 8.0 and posthog-node 6.0.
- Add the capture v1 AI lane: `POST /i/v1/ai/events`, the same body, headers and per-event results as analytics capture, `misrouted_event` for a non-AI event on that endpoint, and the analytics endpoint's temporary acceptance of `$ai_*` events.
- **BREAKING** for the wrapper contract: PostHog's AI wrapper libraries always send through the SDK's AI method; `enable_full_ai_capture` controls only truncation and media redaction. A wrapper given a client without the AI method turns its own capture off instead of falling back to `capture`.
- Add the AI lane's configuration surface (compression, queue capacity, request timeout, an optional lower-only per-event cap) with its defaults and the rule that analytics settings never reach the AI lane. This supersedes draft #118.
- Add the AI size rules: an 8 MiB ceiling on `properties`, a 5 MiB request target, size-aware batching that keeps order, and the oversize log line that names only the event and its size.
- Replace the transport section's v0 values with capture v1.
- Add `migration/yaml-parity-v1/capture-ai-v1.feature` under a new `capture_ai_v1` capability, mirroring the v0 cases on the v1 endpoint, and document the capability in the suite README.

## Capabilities

### New Capabilities

_None._

### Modified Capabilities

- `capture-ai`: capture v1 AI lane, wrapper routing, lane configuration, size rules, extended applicability.

## Impact

- posthog-python `v1`, posthog-go `v1`, posthog-rs `v1` and the posthog-node capture v1 series conform, with one recorded variation: posthog-node 6 defaults the AI lane's compression to gzip, the other three to none.
- Draft #118 covers the lane configuration; this change absorbs it and drops its "rejected at construction" scenario, which posthog-go does not satisfy (a non-positive AI queue size takes the default).
- SDKs without AI capture (Ruby, PHP, .NET, Elixir, Java, the client SDKs) are unaffected until they add it.
- The harness adapters for the four SDKs can declare `capture_ai_v1`; the existing `capture_ai_v0` feature stays for the legacy route.
Loading
Loading