chore(compliance): bump the SDK test harness to 1.26.0 by turnipdabeets · Pull Request #988 · PostHog/posthog-ios

turnipdabeets · 2026-10-28T23:09:49Z

💡 Motivation and Context

Update: moved to 1.26.0, the version the server SDK PRs use (e.g. PostHog/posthog-rs#317). 1.26.0 renames event_size.batches_close_before_byte_target_and_keep_order to event_size.batches_close_before_byte_target (it no longer checks order across requests) and runs reported_drops, property_layers and fail_open on the AI lane too. The local run below was on 1.25.0; CI on this PR runs 1.26.0.

Moves the capture v1 branch's compliance adapter to the latest SDK test harness, as a harness maintainer asked in PostHog/posthog-android#881 (comment). Stacked on #938.

sdk_compliance_adapter/docker-compose.yml: default image sdk-test-harness:1.14.0 → 1.26.0. The workflow takes the image from this file and pins no version itself.
Read the 1.15.0–1.25.0 release notes. 1.15–1.22 only touch the feature flag rules v2 contract and the acceptance suite, which this adapter doesn't run. 1.23 adds flags_response_v4: not declared, so the harness keeps checking ?v=2, which the adapter sends. 1.24 and 1.25 add the capture capabilities below.

Capabilities in /health:

Capability	Declared	Why
`disable_geoip`	Yes	`/init` already maps `disable_geoip` to `config.disableGeoIp`, and the SDK sets `$geoip_disable` on events.
`generated_payloads`	Yes	`/capture_ai` now pads `properties` to `pad_properties_to_bytes` with a `$test_padding` string that starts with `marker`. The SDK already enforces the 8 MiB AI ceiling and 5 MiB byte-aware batches.
`property_layers`	Yes	`/init` now registers `super_properties` with `register()` and turns `before_send.set_properties` into a `setBeforeSend` hook.
`fail_open`	Yes	`/init` no longer rejects an empty token; it passes the token as given. `setup()` stays disabled for an empty or `phx_` token, so nothing is sent.
`bounded_retries`	No	The SDK retries capture without a limit (the retry counter only ramps the delay), so `max_retries_respected` doesn't apply.
`historical_migration`	No	The SDK has no historical migration setting.
`reported_drops`	No	The SDK reports per-event drops only through debug logging. It has no callback or error list the adapter could read.
`log_capture`	No	`hedgeLog` prints straight to stdout, with no log sink to hook. Reading the SDK's lines would mean redirecting the adapter process's stdout.
`event_uuid_override`	No	`capture`/`captureAi` in 4.0 take no caller uuid. This keeps `capture_ai_keeps_supplied_uuid` and the other uuid-override tests skipped.

Adapter/CI only, with no SDK change, so there's no changeset. Earlier adapter-only PRs (e.g. #622) didn't add one either.

💚 How did you test it?

Ran the suite locally the way the workflow does: adapter built natively (swift build -Xswiftc -DTESTING), harness ghcr.io/posthog/sdk-test-harness:1.25.0 in Docker (colima). 229/229 passed, 0 failed (capture_v1, capture_ai_v1, feature_flags), on the final commit.

The 1.24 behaviour changes pass against the SDK: retries_on_404, retries_on_429 and retries_on_502 (both lanes), implements_backoff, the warning partial-batch tests, and failing_ai_lane_does_not_delay_analytics. The new gated tests also pass: property_layers.*, fail_open.*, event_size.event_over_ceiling_is_not_sent, event_size.batches_close_before_byte_target_and_keep_order, and geoip_disable_injected_into_properties (both lanes).

swiftformat --lint and swiftlint are clean on the adapter sources.

📝 Checklist

All commits are signed (see Commit signing)
I reviewed the submitted code.
I added tests to verify the changes.
I updated the docs if needed.
No breaking change or entry added to the changelog.

If releasing new changes

Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Claude Code (Opus) read the harness release notes and the 1.25.0 contracts (ADAPTER_GUIDE.md, capture_analytics_v1_tests.yaml, capture_ai_v1_tests.yaml), checked each capability against the SDK source, and ran the suite locally. It declared a capability only where the SDK already has the behaviour and the adapter change is a thin mapping. It skipped log_capture rather than capture the process's stdout, and reported_drops because the SDK has no drop callback or error list to read. It also left the existing max_retries → pushNotificationSubscriptionMaxRetries mapping alone, since that's out of scope here.

🤖 Generated with Claude Code

Declare disable_geoip, generated_payloads, property_layers and fail_open, and add the adapter fields they need. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

posthog

PostHog Review · Standard · reviewed c49ae6a: 1 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

github-actions · 2026-10-28T23:52:27Z

posthog-ios Compliance Report

Date: 2026-10-29T02:16:14.646975+00:00
Duration: 973132ms

✅ All Tests Passed!

233/233 tests passed

Capture_V1 Tests

✅ 106/106 tests passed

View Details

Test	Status	Duration
Endpoint And Method.Targets V1 Endpoint	✅	2854ms
Endpoint And Method.Does Not Use Legacy Endpoints	✅	2747ms
Required Headers.Has Authorization Bearer Header	✅	2669ms
Required Headers.Has Content Type Json	✅	2734ms
Required Headers.Has Posthog Sdk Info Format	✅	2789ms
Required Headers.Has Posthog Attempt Header	✅	2840ms
Required Headers.Has Posthog Request Id	✅	2679ms
Required Headers.Has Posthog Request Timestamp	✅	2688ms
Required Headers.Has User Agent	✅	2698ms
Body Format.Body Has Created At And Batch	✅	2819ms
Body Format.No Api Key In Body	✅	2773ms
Body Format.No Sent At In Body	✅	2740ms
Event Format.Event Has Required Root Fields	✅	2701ms
Event Format.Event Uuid Is Valid	✅	284ms
Event Format.Event Timestamp Is Rfc3339	✅	2738ms
Event Format.Non Utc Event Timestamp Is Converted To Utc	✅	286ms
Event Format.Distinct Id Is String	✅	2657ms
Event Format.Distinct Id At Root Not Properties	✅	2783ms
Event Format.Custom Properties Preserved	✅	2821ms
Event Format.Set Properties Preserved	✅	2802ms
Event Format.Set Once Properties Preserved	✅	2762ms
Event Format.Groups Properties Preserved	✅	364ms
Event Format.Sdk Generates Uuid If Not Provided	✅	2747ms
Event Format.Event Has Required Root Fields Batch	✅	2820ms
Event Format.Event Uuid Is Valid Batch	✅	2766ms
Event Format.Event Timestamp Is Rfc3339 Batch	✅	2787ms
Event Format.Distinct Id Is String Batch	✅	2812ms
Event Format.Distinct Id At Root Not Properties Batch	✅	2835ms
Event Format.Custom Properties Preserved Batch	✅	2728ms
Event Format.Set Properties Preserved Batch	✅	2831ms
Event Format.Set Once Properties Preserved Batch	✅	2819ms
Event Format.Groups Properties Preserved Batch	✅	2849ms
Event Format.Sdk Generates Uuid If Not Provided Batch	✅	2805ms
Batch Behavior.Multiple Events In Single Batch	✅	2844ms
Batch Behavior.Batch Envelope Smoke	✅	2856ms
Batch Behavior.Flush With No Events Sends Nothing	✅	372ms
Batch Behavior.Flush At Triggers Batch	✅	1173ms
Batch Behavior.Created At Reflects Batch Creation Time	✅	2664ms
Deduplication.Generates Unique Uuids	✅	2786ms
Deduplication.Different Events Same Content Different Uuids	✅	2704ms
Deduplication.Preserves Uuid On Retry	✅	9280ms
Deduplication.Preserves Timestamp On Retry	✅	9504ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	7889ms
Deduplication.No Duplicate Events In Batch	✅	2910ms
Header Behavior On Retry.Attempt Header Starts At One	✅	3035ms
Header Behavior On Retry.Attempt Header Increments On Retry	✅	10325ms
Header Behavior On Retry.Request Id Preserved On Retry	✅	8910ms
Header Behavior On Retry.Different Requests Have Different Request Ids	✅	4981ms
Header Behavior On Retry.Request Timestamp Changes On Retry	✅	9204ms
Response Format Validation.Success Response Has Uuid Keyed Results	✅	2793ms
Response Format Validation.Success Response Has Ok For Each Event	✅	2728ms
Response Format Validation.Success No Retry After When All Ok	✅	2772ms
Response Format Validation.Success Retry After Present When Retry Events	✅	2736ms
Response Format Validation.Success No Retry After When Drop Only	✅	2920ms
Response Format Validation.Response Echoes Request Id	✅	2877ms
Retry Behavior.Retries On 408	✅	9311ms
Retry Behavior.Retries On 500	✅	9250ms
Retry Behavior.Retries On 503	✅	11151ms
Retry Behavior.Retries On 504	✅	9107ms
Retry Behavior.Retries On 404	✅	8715ms
Retry Behavior.Retries On 429	✅	9152ms
Retry Behavior.Retries On 502	✅	9248ms
Retry Behavior.Retryable Errors Have Retry After	✅	2401ms
Retry Behavior.Respects Retry After On Retryable Error	✅	10852ms
Retry Behavior.Does Not Retry On 400	✅	4799ms
Retry Behavior.Does Not Retry On 401	✅	4659ms
Retry Behavior.Does Not Retry On 402	✅	2355ms
Retry Behavior.Does Not Retry On 413	✅	4669ms
Retry Behavior.Does Not Retry On 415	✅	4889ms
Retry Behavior.Non Retryable Errors Have No Retry After	✅	4690ms
Retry Behavior.Implements Backoff	✅	17631ms
Partial Batch Handling.Handles 200 Full Success	✅	4707ms
Partial Batch Handling.Handles 200 With All Ok	✅	5850ms
Partial Batch Handling.Does Not Retry Dropped Events	✅	5805ms
Partial Batch Handling.Does Not Retry Warning Events	✅	5863ms
Partial Batch Handling.Prunes Ok Events On Partial Retry	✅	7885ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry	✅	7953ms
Partial Batch Handling.Retries Only Retry Events From Partial	✅	7763ms
Partial Batch Handling.Partial Retry Preserves Uuids	✅	7830ms
Partial Batch Handling.Partial Retry Attempt Header Increments	✅	7751ms
Partial Batch Handling.Partial Retry Request Id Preserved	✅	7736ms
Partial Batch Handling.Respects Retry After On Partial	✅	7752ms
Partial Batch Handling.Unknown Result Treated As Terminal	✅	5893ms
Partial Batch Handling.Mixed Ok Drop Warning No Retry	✅	5934ms
Compression.Sends Gzip Content Encoding	✅	2823ms
Compression.No Content Encoding When Disabled	✅	2685ms
Compression.Compressed Body Is Decompressible	✅	415ms
Error Handling.Does Not Retry On Unknown 4Xx	✅	2313ms
Event Options.Cookieless Mode Override	✅	2860ms
Event Options.Disable Skew Correction Override	✅	2800ms
Event Options.Process Person Profile Override	✅	2812ms
Event Options.Product Tour Id Override	✅	2686ms
Event Options.Unset Options Omitted	✅	2832ms
Event Options.Options Override In Batch	✅	2886ms
Event Options.Unknown Option Passes Through	✅	2711ms
Event Options.Option Value Is Not Converted	✅	2749ms
Event Options.Lenient Bool Option Values Are Not Converted	✅	2826ms
Event Options.Option Wins Over Legacy Property	✅	2750ms
Event Options.Legacy Property Fills Unset Option	✅	2852ms
Event Options.Null Option Falls Back To Legacy Property	✅	2820ms
Property Layers.Caller Property Beats Registered Property	✅	2805ms
Property Layers.Before Send Runs Last	✅	2827ms
Fail Open.Empty Api Key Sends Nothing	✅	1375ms
Fail Open.Personal Api Key Sends Nothing	✅	1247ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties	✅	2693ms
Geoip And Historical Migration.Historical Migration Absent By Default	✅	2655ms

Capture_Ai_V1 Tests

✅ 110/110 tests passed

View Details

Test	Status	Duration
Endpoint And Method.Targets V1 Endpoint	✅	301ms
Endpoint And Method.Does Not Use Legacy Endpoints	✅	2731ms
Endpoint And Method.Capture Does Not Reroute Ai Named Events	✅	2668ms
Endpoint And Method.Capture Ai Does Not Reroute Non Ai Events	✅	2814ms
Lane Isolation.Failing Ai Lane Does Not Delay Analytics	✅	11200ms
Required Headers.Has Authorization Bearer Header	✅	2703ms
Required Headers.Has Content Type Json	✅	3132ms
Required Headers.Has Posthog Sdk Info Format	✅	2788ms
Required Headers.Has Posthog Attempt Header	✅	2806ms
Required Headers.Has Posthog Request Id	✅	2774ms
Required Headers.Has Posthog Request Timestamp	✅	2864ms
Required Headers.Has User Agent	✅	413ms
Body Format.Body Has Created At And Batch	✅	2825ms
Body Format.No Api Key In Body	✅	373ms
Body Format.No Sent At In Body	✅	2851ms
Event Format.Event Has Required Root Fields	✅	2801ms
Event Format.Event Uuid Is Valid	✅	470ms
Event Format.Event Timestamp Is Rfc3339	✅	2839ms
Event Format.Non Utc Event Timestamp Is Converted To Utc	✅	2928ms
Event Format.Distinct Id Is String	✅	2833ms
Event Format.Distinct Id At Root Not Properties	✅	2775ms
Event Format.Custom Properties Preserved	✅	418ms
Event Format.Set Properties Preserved	✅	2831ms
Event Format.Set Once Properties Preserved	✅	2754ms
Event Format.Groups Properties Preserved	✅	2781ms
Event Format.Sdk Generates Uuid If Not Provided	✅	2872ms
Event Format.Event Has Required Root Fields Batch	✅	2848ms
Event Format.Event Uuid Is Valid Batch	✅	2887ms
Event Format.Event Timestamp Is Rfc3339 Batch	✅	2891ms
Event Format.Distinct Id Is String Batch	✅	2845ms
Event Format.Distinct Id At Root Not Properties Batch	✅	2799ms
Event Format.Custom Properties Preserved Batch	✅	2896ms
Event Format.Set Properties Preserved Batch	✅	2812ms
Event Format.Set Once Properties Preserved Batch	✅	2863ms
Event Format.Groups Properties Preserved Batch	✅	2927ms
Event Format.Sdk Generates Uuid If Not Provided Batch	✅	2967ms
Batch Behavior.Multiple Events In Single Batch	✅	3025ms
Batch Behavior.Batch Envelope Smoke	✅	2935ms
Batch Behavior.Flush At Triggers Batch	✅	1192ms
Batch Behavior.Created At Reflects Batch Creation Time	✅	2748ms
Deduplication.Generates Unique Uuids	✅	3092ms
Deduplication.Different Events Same Content Different Uuids	✅	2871ms
Deduplication.Preserves Uuid On Retry	✅	9309ms
Deduplication.Preserves Timestamp On Retry	✅	8871ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	7927ms
Deduplication.No Duplicate Events In Batch	✅	3029ms
Header Behavior On Retry.Attempt Header Starts At One	✅	2884ms
Header Behavior On Retry.Attempt Header Increments On Retry	✅	10329ms
Header Behavior On Retry.Request Id Preserved On Retry	✅	5402ms
Header Behavior On Retry.Different Requests Have Different Request Ids	✅	5142ms
Header Behavior On Retry.Request Timestamp Changes On Retry	✅	9207ms
Response Format Validation.Success Response Has Uuid Keyed Results	✅	2777ms
Response Format Validation.Success Response Has Ok For Each Event	✅	2928ms
Response Format Validation.Success No Retry After When All Ok	✅	2898ms
Response Format Validation.Success Retry After Present When Retry Events	✅	2869ms
Response Format Validation.Success No Retry After When Drop Only	✅	2826ms
Response Format Validation.Response Echoes Request Id	✅	2806ms
Retry Behavior.Retries On 408	✅	9295ms
Retry Behavior.Retries On 500	✅	9377ms
Retry Behavior.Retries On 503	✅	11746ms
Retry Behavior.Retries On 504	✅	9217ms
Retry Behavior.Retries On 404	✅	9281ms
Retry Behavior.Retries On 429	✅	8764ms
Retry Behavior.Retries On 502	✅	8718ms
Retry Behavior.Retryable Errors Have Retry After	✅	6200ms
Retry Behavior.Respects Retry After On Retryable Error	✅	10841ms
Retry Behavior.Does Not Retry On 400	✅	4799ms
Retry Behavior.Does Not Retry On 401	✅	4756ms
Retry Behavior.Does Not Retry On 402	✅	4799ms
Retry Behavior.Does Not Retry On 413	✅	2319ms
Retry Behavior.Does Not Retry On 415	✅	2343ms
Retry Behavior.Non Retryable Errors Have No Retry After	✅	4800ms
Retry Behavior.Implements Backoff	✅	18282ms
Partial Batch Handling.Handles 200 Full Success	✅	4745ms
Partial Batch Handling.Handles 200 With All Ok	✅	6002ms
Partial Batch Handling.Does Not Retry Dropped Events	✅	5742ms
Partial Batch Handling.Does Not Retry Warning Events	✅	5900ms
Partial Batch Handling.Prunes Ok Events On Partial Retry	✅	7805ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry	✅	7830ms
Partial Batch Handling.Retries Only Retry Events From Partial	✅	7867ms
Partial Batch Handling.Partial Retry Preserves Uuids	✅	7901ms
Partial Batch Handling.Partial Retry Attempt Header Increments	✅	7839ms
Partial Batch Handling.Partial Retry Request Id Preserved	✅	7833ms
Partial Batch Handling.Respects Retry After On Partial	✅	7781ms
Partial Batch Handling.Unknown Result Treated As Terminal	✅	5852ms
Partial Batch Handling.Mixed Ok Drop Warning No Retry	✅	5929ms
Compression.Sends Gzip Content Encoding	✅	2872ms
Compression.No Content Encoding When Disabled	✅	2771ms
Compression.Compressed Body Is Decompressible	✅	2712ms
Error Handling.Does Not Retry On Unknown 4Xx	✅	4835ms
Event Options.Cookieless Mode Override	✅	2693ms
Event Options.Disable Skew Correction Override	✅	2709ms
Event Options.Process Person Profile Override	✅	2757ms
Event Options.Product Tour Id Override	✅	2901ms
Event Options.Unset Options Omitted	✅	2672ms
Event Options.Options Override In Batch	✅	2836ms
Event Options.Unknown Option Passes Through	✅	332ms
Event Options.Option Value Is Not Converted	✅	2806ms
Event Options.Lenient Bool Option Values Are Not Converted	✅	2764ms
Event Options.Option Wins Over Legacy Property	✅	280ms
Event Options.Legacy Property Fills Unset Option	✅	2719ms
Event Options.Null Option Falls Back To Legacy Property	✅	332ms
Event Size.Event Over Ceiling Is Not Sent	✅	7071ms
Event Size.Batches Close Before Byte Target	✅	8314ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties	✅	2714ms
Geoip And Historical Migration.Historical Migration Absent By Default	✅	296ms
Property Layers.Caller Property Beats Registered Property	✅	2671ms
Property Layers.Before Send Runs Last	✅	2911ms
Fail Open.Empty Api Key Sends Nothing	✅	1353ms
Fail Open.Personal Api Key Sends Nothing	✅	1342ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	2759ms
Request Payload.Flags Request Uses V2 Query Param	✅	2822ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	2784ms
Request Payload.Flags Request Omits Authorization Header	✅	2811ms
Request Payload.Token In Flags Body Matches Init	✅	2911ms
Request Payload.Groups Round Trip	✅	2890ms
Request Payload.Groups Default To Empty Object	✅	2839ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	2824ms
Request Payload.Disable Geoip Omitted Defaults To False	✅	2787ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	2798ms
Request Lifecycle.No Flags Request On Init Alone	✅	90ms
Request Lifecycle.No Flags Request On Normal Capture	✅	2842ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	5534ms
Request Lifecycle.Mock Response Value Is Returned To Caller	✅	2757ms
Retry Behavior.Retries Flags On 502	✅	2852ms
Retry Behavior.Retries Flags On 504	✅	2858ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	✅	2936ms

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

chore(compliance): bump the test harness to 1.25.0

c49ae6a

Declare disable_geoip, generated_payloads, property_layers and fail_open, and add the adapter fields they need. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

turnipdabeets self-assigned this Oct 28, 2026

turnipdabeets requested a review from a team as a code owner October 28, 2026 23:09

posthog Bot reviewed Oct 28, 2026

View reviewed changes

Comment thread sdk_compliance_adapter/Sources/main.swift Resolved

chore(compliance): cap generated payload size before allocating padding

ecaeeab

chore(compliance): bump the test harness to 1.26.0

cb36de2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

turnipdabeets changed the title ~~chore(compliance): bump the SDK test harness to 1.25.0~~ chore(compliance): bump the SDK test harness to 1.26.0 Oct 29, 2026

turnipdabeets merged commit f5e139d into feat/v4-capture-v1 Oct 29, 2026
44 of 45 checks passed

turnipdabeets deleted the chore/v4-harness-1.25 branch October 29, 2026 15:46

eli-r-ph mentioned this pull request Oct 30, 2026

fix: print the empty and personal API key setup errors without debug #996

Open

6 tasks done

Navigation Menu

Repository navigation

chore(compliance): bump the SDK test harness to 1.26.0 - #988

chore(compliance): bump the SDK test harness to 1.26.0#988
turnipdabeets merged 3 commits into
feat/v4-capture-v1from
chore/v4-harness-1.25

turnipdabeets commented Oct 28, 2026 •

edited

Loading

Uh oh!

posthog Bot left a comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

turnipdabeets commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💡 Motivation and Context

💚 How did you test it?

📝 Checklist

If releasing new changes

🤖 Agent context

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

posthog-ios Compliance Report

✅ All Tests Passed!

Capture_V1 Tests

Capture_Ai_V1 Tests

Feature_Flags Tests

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

turnipdabeets commented Oct 28, 2026 •

edited

Loading

github-actions Bot commented Oct 28, 2026 •

edited

Loading