chore(compliance): run the 1.26.0 test harness against the adapter by turnipdabeets · Pull Request #902 · PostHog/posthog-android

turnipdabeets · 2026-10-28T23:15:16Z

💡 Motivation and Context

Update: moved to 1.26.0, the version the server SDK PRs use (e.g. PostHog/posthog-rs#317). 1.26.0 renames event_size.batches_close_before_byte_target_and_keep_order to event_size.batches_close_before_byte_target (it no longer checks order across requests) and runs reported_drops, property_layers and fail_open on the AI lane too. The local run below was on 1.25.0; CI on this PR runs 1.26.0.

Stacked on #881. A harness maintainer asked that the capture v1 branches move to the latest harness. This PR moves the adapter to posthog-sdk-test-harness 1.25.0 and declares the new capabilities the Android SDK actually supports.

CI: test-harness-version goes from 1.20.0 to 1.26.0. The reusable workflow SHA is unchanged; dependabot keeps bumping it.
Local docker-compose.yml: the image goes from 0.10.0 to 1.26.0.
Adapter: new capabilities generated_payloads, log_capture, property_layers and fail_open.

Capabilities

Capability	Declared	Why
`generated_payloads`	Yes	`/capture_ai` honors `pad_properties_to_bytes` and `marker` by adding a `$test_padding` string, so `properties` serializes (compact JSON, UTF-8) to the requested size. This runs the 8 MiB ceiling and the ~5 MiB batch tests against `captureAi`.
`log_capture`	Yes	The adapter installs a recording `PostHogLogger` that still prints each line. `/state` returns `log_lines`. The SDK logs an over-ceiling AI event by name and size, never by content.
`property_layers`	Yes	`/init` registers `super_properties` with `register()` and installs a `before_send` hook for `set_properties`.
`fail_open`	Yes	`setup()` already disables the SDK for an empty or `phx_` token. The adapter passes `api_key` through as given, so it needed no change beyond declaring the capability.
`event_uuid_override`	No	Mobile `capture`/`captureAi` take no caller uuid. In 1.25.0, `capture_ai_keeps_supplied_uuid` is server-only and requires this capability, so it is skipped.
`bounded_retries`	No	Capture v1 retries until it succeeds, with backoff capped at 30 s. There is no capture retry limit to map `max_retries` to. The harness guide says SDKs that retry without a limit leave it out.
`disable_geoip`	No	The SDK has no config option that adds `$geoip_disable` to events.
`historical_migration`	No	The SDK has no historical migration setting.
`reported_drops`	No	The SDK has no public error surface for dropped events. It only logs per-event `drop` results. The duplicate-copy test also requires `event_uuid_override`.

Behaviour changes in 1.24

The SDK under test passes all of them unchanged: 404, 429 and 502 are retried, implements_backoff passes, the partial-batch warning tests pass, and failing_ai_lane_does_not_delay_analytics and capture_ai_does_not_reroute_non_ai_events pass. No SDK change is needed.

No spec divergence: this PR changes only the test adapter and CI, not SDK behaviour.

💚 How did you test it?

I ran the full 1.25.0 suite locally: 228 passed, 0 failed across capture_v1, capture_ai_v1 and feature_flags, with --sdk-type server. The new event_size.*, property_layers.* and fail_open.* tests all ran and passed.

The docker compose up path didn't finish on my machine: the emulated amd64 Gradle build ran out of memory in the Colima VM. So I built the adapter with installDist on the host, ran it in an eclipse-temurin:17-jre container, and ran ghcr.io/posthog/sdk-test-harness:1.25.0 against it on the same Docker network. CI on this PR runs the regular Dockerfile path.

./gradlew spotlessKotlinCheck passes.

📝 Checklist

All commits are signed (see Commit signing)
I reviewed the submitted code.
I added tests to verify the changes.
I updated the docs if needed.
No breaking change or entry added to the changelog.

If releasing new changes

Ran pnpm changeset to generate a changeset file

No changeset: this changes only the compliance adapter and CI, and no published package changes. That matches the earlier chore(compliance) commits.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Claude Code (Opus 5.5) read the 1.21.0 to 1.25.0 release notes and the 1.25.0 contracts and ADAPTER_GUIDE.md, checked the SDK source for each new capability, and declared only the ones the SDK supports and the adapter now implements. Anna Garcia directed the work, including keeping event_uuid_override undeclared because mobile capture takes no caller uuid.

🤖 Generated with Claude Code

Bump the harness from 1.20.0 (CI) and 0.10.0 (local docker-compose) to 1.25.0, and declare the 1.25 capabilities the SDK supports: generated_payloads, log_capture, property_layers and fail_open. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

posthog

PostHog Review · Standard · reviewed 84d3f39: 1 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

…I properties

github-actions · 2026-10-28T23:27:52Z

posthog-android Compliance Report

Date: 2026-10-29T02:02:00.452104+00:00
Duration: 475291ms

✅ All Tests Passed!

232/232 tests passed

Capture_V1 Tests

✅ 105/105 tests passed

View Details

Test	Status	Duration
Endpoint And Method.Targets V1 Endpoint	✅	310ms
Endpoint And Method.Does Not Use Legacy Endpoints	✅	23ms
Required Headers.Has Authorization Bearer Header	✅	19ms
Required Headers.Has Content Type Json	✅	20ms
Required Headers.Has Posthog Sdk Info Format	✅	20ms
Required Headers.Has Posthog Attempt Header	✅	23ms
Required Headers.Has Posthog Request Id	✅	18ms
Required Headers.Has Posthog Request Timestamp	✅	20ms
Required Headers.Has User Agent	✅	18ms
Body Format.Body Has Created At And Batch	✅	20ms
Body Format.No Api Key In Body	✅	17ms
Body Format.No Sent At In Body	✅	16ms
Event Format.Event Has Required Root Fields	✅	18ms
Event Format.Event Uuid Is Valid	✅	18ms
Event Format.Event Timestamp Is Rfc3339	✅	22ms
Event Format.Non Utc Event Timestamp Is Converted To Utc	✅	18ms
Event Format.Distinct Id Is String	✅	16ms
Event Format.Distinct Id At Root Not Properties	✅	16ms
Event Format.Custom Properties Preserved	✅	17ms
Event Format.Set Properties Preserved	✅	19ms
Event Format.Set Once Properties Preserved	✅	20ms
Event Format.Groups Properties Preserved	✅	16ms
Event Format.Sdk Generates Uuid If Not Provided	✅	19ms
Event Format.Event Has Required Root Fields Batch	✅	24ms
Event Format.Event Uuid Is Valid Batch	✅	22ms
Event Format.Event Timestamp Is Rfc3339 Batch	✅	20ms
Event Format.Distinct Id Is String Batch	✅	22ms
Event Format.Distinct Id At Root Not Properties Batch	✅	21ms
Event Format.Custom Properties Preserved Batch	✅	22ms
Event Format.Set Properties Preserved Batch	✅	21ms
Event Format.Set Once Properties Preserved Batch	✅	20ms
Event Format.Groups Properties Preserved Batch	✅	19ms
Event Format.Sdk Generates Uuid If Not Provided Batch	✅	34ms
Batch Behavior.Multiple Events In Single Batch	✅	25ms
Batch Behavior.Batch Envelope Smoke	✅	23ms
Batch Behavior.Flush With No Events Sends Nothing	✅	13ms
Batch Behavior.Flush At Triggers Batch	✅	1026ms
Batch Behavior.Created At Reflects Batch Creation Time	✅	15ms
Deduplication.Generates Unique Uuids	✅	28ms
Deduplication.Different Events Same Content Different Uuids	✅	17ms
Deduplication.Preserves Uuid On Retry	✅	7014ms
Deduplication.Preserves Timestamp On Retry	✅	7017ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	7023ms
Deduplication.No Duplicate Events In Batch	✅	24ms
Header Behavior On Retry.Attempt Header Starts At One	✅	12ms
Header Behavior On Retry.Attempt Header Increments On Retry	✅	12015ms
Header Behavior On Retry.Request Id Preserved On Retry	✅	7015ms
Header Behavior On Retry.Different Requests Have Different Request Ids	✅	2025ms
Header Behavior On Retry.Request Timestamp Changes On Retry	✅	7014ms
Response Format Validation.Success Response Has Uuid Keyed Results	✅	14ms
Response Format Validation.Success Response Has Ok For Each Event	✅	15ms
Response Format Validation.Success No Retry After When All Ok	✅	15ms
Response Format Validation.Success Retry After Present When Retry Events	✅	2008ms
Response Format Validation.Success No Retry After When Drop Only	✅	15ms
Response Format Validation.Response Echoes Request Id	✅	11ms
Retry Behavior.Retries On 408	✅	7011ms
Retry Behavior.Retries On 500	✅	7016ms
Retry Behavior.Retries On 503	✅	7019ms
Retry Behavior.Retries On 504	✅	7014ms
Retry Behavior.Retries On 404	✅	7015ms
Retry Behavior.Retries On 429	✅	6015ms
Retry Behavior.Retries On 502	✅	6015ms
Retry Behavior.Retryable Errors Have Retry After	✅	4011ms
Retry Behavior.Respects Retry After On Retryable Error	✅	10022ms
Retry Behavior.Does Not Retry On 400	✅	4016ms
Retry Behavior.Does Not Retry On 401	✅	4013ms
Retry Behavior.Does Not Retry On 402	✅	4016ms
Retry Behavior.Does Not Retry On 413	✅	4015ms
Retry Behavior.Does Not Retry On 415	✅	4019ms
Retry Behavior.Non Retryable Errors Have No Retry After	✅	4014ms
Retry Behavior.Implements Backoff	✅	14018ms
Partial Batch Handling.Handles 200 Full Success	✅	2014ms
Partial Batch Handling.Handles 200 With All Ok	✅	3022ms
Partial Batch Handling.Does Not Retry Dropped Events	✅	3020ms
Partial Batch Handling.Does Not Retry Warning Events	✅	3018ms
Partial Batch Handling.Prunes Ok Events On Partial Retry	✅	7016ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry	✅	7013ms
Partial Batch Handling.Retries Only Retry Events From Partial	✅	7013ms
Partial Batch Handling.Partial Retry Preserves Uuids	✅	7014ms
Partial Batch Handling.Partial Retry Attempt Header Increments	✅	7013ms
Partial Batch Handling.Partial Retry Request Id Preserved	✅	7014ms
Partial Batch Handling.Respects Retry After On Partial	✅	7021ms
Partial Batch Handling.Unknown Result Treated As Terminal	✅	3018ms
Partial Batch Handling.Mixed Ok Drop Warning No Retry	✅	3022ms
Compression.Sends Gzip Content Encoding	✅	12ms
Compression.No Content Encoding When Disabled	✅	14ms
Compression.Compressed Body Is Decompressible	✅	15ms
Error Handling.Does Not Retry On Unknown 4Xx	✅	4017ms
Event Options.Cookieless Mode Override	✅	12ms
Event Options.Disable Skew Correction Override	✅	9ms
Event Options.Process Person Profile Override	✅	9ms
Event Options.Product Tour Id Override	✅	10ms
Event Options.Unset Options Omitted	✅	10ms
Event Options.Options Override In Batch	✅	16ms
Event Options.Unknown Option Passes Through	✅	11ms
Event Options.Option Value Is Not Converted	✅	12ms
Event Options.Lenient Bool Option Values Are Not Converted	✅	13ms
Event Options.Option Wins Over Legacy Property	✅	10ms
Event Options.Legacy Property Fills Unset Option	✅	12ms
Event Options.Null Option Falls Back To Legacy Property	✅	14ms
Property Layers.Caller Property Beats Registered Property	✅	16ms
Property Layers.Before Send Runs Last	✅	17ms
Fail Open.Empty Api Key Sends Nothing	✅	3011ms
Fail Open.Personal Api Key Sends Nothing	✅	3010ms
Geoip And Historical Migration.Historical Migration Absent By Default	✅	12ms

Capture_Ai_V1 Tests

✅ 110/110 tests passed

View Details

Test	Status	Duration
Endpoint And Method.Targets V1 Endpoint	✅	11ms
Endpoint And Method.Does Not Use Legacy Endpoints	✅	11ms
Endpoint And Method.Capture Does Not Reroute Ai Named Events	✅	12ms
Endpoint And Method.Capture Ai Does Not Reroute Non Ai Events	✅	10ms
Lane Isolation.Failing Ai Lane Does Not Delay Analytics	✅	7017ms
Required Headers.Has Authorization Bearer Header	✅	13ms
Required Headers.Has Content Type Json	✅	12ms
Required Headers.Has Posthog Sdk Info Format	✅	16ms
Required Headers.Has Posthog Attempt Header	✅	13ms
Required Headers.Has Posthog Request Id	✅	12ms
Required Headers.Has Posthog Request Timestamp	✅	12ms
Required Headers.Has User Agent	✅	11ms
Body Format.Body Has Created At And Batch	✅	10ms
Body Format.No Api Key In Body	✅	9ms
Body Format.No Sent At In Body	✅	10ms
Event Format.Event Has Required Root Fields	✅	9ms
Event Format.Event Uuid Is Valid	✅	12ms
Event Format.Event Timestamp Is Rfc3339	✅	13ms
Event Format.Non Utc Event Timestamp Is Converted To Utc	✅	12ms
Event Format.Distinct Id Is String	✅	11ms
Event Format.Distinct Id At Root Not Properties	✅	10ms
Event Format.Custom Properties Preserved	✅	11ms
Event Format.Set Properties Preserved	✅	12ms
Event Format.Set Once Properties Preserved	✅	11ms
Event Format.Groups Properties Preserved	✅	15ms
Event Format.Sdk Generates Uuid If Not Provided	✅	15ms
Event Format.Event Has Required Root Fields Batch	✅	15ms
Event Format.Event Uuid Is Valid Batch	✅	15ms
Event Format.Event Timestamp Is Rfc3339 Batch	✅	14ms
Event Format.Distinct Id Is String Batch	✅	15ms
Event Format.Distinct Id At Root Not Properties Batch	✅	16ms
Event Format.Custom Properties Preserved Batch	✅	17ms
Event Format.Set Properties Preserved Batch	✅	17ms
Event Format.Set Once Properties Preserved Batch	✅	15ms
Event Format.Groups Properties Preserved Batch	✅	16ms
Event Format.Sdk Generates Uuid If Not Provided Batch	✅	15ms
Batch Behavior.Multiple Events In Single Batch	✅	21ms
Batch Behavior.Batch Envelope Smoke	✅	18ms
Batch Behavior.Flush At Triggers Batch	✅	1014ms
Batch Behavior.Created At Reflects Batch Creation Time	✅	13ms
Deduplication.Generates Unique Uuids	✅	22ms
Deduplication.Different Events Same Content Different Uuids	✅	13ms
Deduplication.Preserves Uuid On Retry	✅	7013ms
Deduplication.Preserves Timestamp On Retry	✅	7014ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	7015ms
Deduplication.No Duplicate Events In Batch	✅	23ms
Header Behavior On Retry.Attempt Header Starts At One	✅	11ms
Header Behavior On Retry.Attempt Header Increments On Retry	✅	12016ms
Header Behavior On Retry.Request Id Preserved On Retry	✅	7014ms
Header Behavior On Retry.Different Requests Have Different Request Ids	✅	2020ms
Header Behavior On Retry.Request Timestamp Changes On Retry	✅	7014ms
Response Format Validation.Success Response Has Uuid Keyed Results	✅	12ms
Response Format Validation.Success Response Has Ok For Each Event	✅	14ms
Response Format Validation.Success No Retry After When All Ok	✅	13ms
Response Format Validation.Success Retry After Present When Retry Events	✅	2007ms
Response Format Validation.Success No Retry After When Drop Only	✅	13ms
Response Format Validation.Response Echoes Request Id	✅	11ms
Retry Behavior.Retries On 408	✅	7012ms
Retry Behavior.Retries On 500	✅	7014ms
Retry Behavior.Retries On 503	✅	7018ms
Retry Behavior.Retries On 504	✅	7014ms
Retry Behavior.Retries On 404	✅	7013ms
Retry Behavior.Retries On 429	✅	7010ms
Retry Behavior.Retries On 502	✅	7013ms
Retry Behavior.Retryable Errors Have Retry After	✅	4010ms
Retry Behavior.Respects Retry After On Retryable Error	✅	10019ms
Retry Behavior.Does Not Retry On 400	✅	4014ms
Retry Behavior.Does Not Retry On 401	✅	4013ms
Retry Behavior.Does Not Retry On 402	✅	4014ms
Retry Behavior.Does Not Retry On 413	✅	4014ms
Retry Behavior.Does Not Retry On 415	✅	4014ms
Retry Behavior.Non Retryable Errors Have No Retry After	✅	4013ms
Retry Behavior.Implements Backoff	✅	14015ms
Partial Batch Handling.Handles 200 Full Success	✅	2013ms
Partial Batch Handling.Handles 200 With All Ok	✅	3019ms
Partial Batch Handling.Does Not Retry Dropped Events	✅	3017ms
Partial Batch Handling.Does Not Retry Warning Events	✅	3018ms
Partial Batch Handling.Prunes Ok Events On Partial Retry	✅	7014ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry	✅	7015ms
Partial Batch Handling.Retries Only Retry Events From Partial	✅	7013ms
Partial Batch Handling.Partial Retry Preserves Uuids	✅	7013ms
Partial Batch Handling.Partial Retry Attempt Header Increments	✅	7014ms
Partial Batch Handling.Partial Retry Request Id Preserved	✅	7014ms
Partial Batch Handling.Respects Retry After On Partial	✅	7017ms
Partial Batch Handling.Unknown Result Treated As Terminal	✅	3016ms
Partial Batch Handling.Mixed Ok Drop Warning No Retry	✅	3020ms
Compression.Sends Gzip Content Encoding	✅	14ms
Compression.No Content Encoding When Disabled	✅	12ms
Compression.Compressed Body Is Decompressible	✅	14ms
Error Handling.Does Not Retry On Unknown 4Xx	✅	4013ms
Event Options.Cookieless Mode Override	✅	10ms
Event Options.Disable Skew Correction Override	✅	10ms
Event Options.Process Person Profile Override	✅	9ms
Event Options.Product Tour Id Override	✅	11ms
Event Options.Unset Options Omitted	✅	9ms
Event Options.Options Override In Batch	✅	13ms
Event Options.Unknown Option Passes Through	✅	11ms
Event Options.Option Value Is Not Converted	✅	14ms
Event Options.Lenient Bool Option Values Are Not Converted	✅	14ms
Event Options.Option Wins Over Legacy Property	✅	10ms
Event Options.Legacy Property Fills Unset Option	✅	11ms
Event Options.Null Option Falls Back To Legacy Property	✅	12ms
Event Size.Event Over Ceiling Is Not Sent	✅	3352ms
Event Size.Over Ceiling Drop Is Reported Or Logged Without Content	✅	2040ms
Event Size.Batches Close Before Byte Target	✅	6073ms
Geoip And Historical Migration.Historical Migration Absent By Default	✅	12ms
Property Layers.Caller Property Beats Registered Property	✅	9ms
Property Layers.Before Send Runs Last	✅	10ms
Fail Open.Empty Api Key Sends Nothing	✅	1007ms
Fail Open.Personal Api Key Sends Nothing	✅	1009ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	23ms
Request Payload.Flags Request Uses V2 Query Param	✅	15ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	13ms
Request Payload.Flags Request Omits Authorization Header	✅	13ms
Request Payload.Token In Flags Body Matches Init	✅	12ms
Request Payload.Groups Round Trip	✅	14ms
Request Payload.Groups Default To Empty Object	✅	17ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	13ms
Request Payload.Disable Geoip Omitted Defaults To False	✅	14ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	13ms
Request Lifecycle.No Flags Request On Init Alone	✅	5ms
Request Lifecycle.No Flags Request On Normal Capture	✅	13ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	17ms
Request Lifecycle.Mock Response Value Is Returned To Caller	✅	18ms
Retry Behavior.Retries Flags On 502	✅	316ms
Retry Behavior.Retries Flags On 504	✅	315ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	✅	12ms

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

turnipdabeets self-assigned this Oct 28, 2026

turnipdabeets requested a review from a team as a code owner October 28, 2026 23:15

posthog Bot reviewed Oct 28, 2026

View reviewed changes

Comment thread sdk_compliance_adapter/src/main/kotlin/com/posthog/compliance/ComplianceAdapter.kt Resolved

chore(compliance): count the marker's serialized bytes when padding A…

5e15556

…I properties

chore(compliance): bump the test harness to 1.26.0

c4ec4ea

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

turnipdabeets changed the title ~~chore(compliance): run the 1.25.0 test harness against the adapter~~ chore(compliance): run the 1.26.0 test harness against the adapter Oct 29, 2026

turnipdabeets merged commit 50c19d2 into feat/v4-capture-v1 Oct 29, 2026
13 of 14 checks passed

turnipdabeets deleted the chore/v4-harness-1.25 branch October 29, 2026 15:47

Navigation Menu

Repository navigation

chore(compliance): run the 1.26.0 test harness against the adapter - #902

chore(compliance): run the 1.26.0 test harness against the adapter#902
turnipdabeets merged 3 commits into
feat/v4-capture-v1from
chore/v4-harness-1.25

turnipdabeets commented Oct 28, 2026 •

edited

Loading

Uh oh!

posthog Bot left a comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

turnipdabeets commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💡 Motivation and Context

Capabilities

Behaviour changes in 1.24

💚 How did you test it?

📝 Checklist

If releasing new changes

🤖 Agent context

Uh oh!

posthog Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

posthog-android Compliance Report

✅ All Tests Passed!

Capture_V1 Tests

Capture_Ai_V1 Tests

Feature_Flags Tests

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

turnipdabeets commented Oct 28, 2026 •

edited

Loading

github-actions Bot commented Oct 28, 2026 •

edited

Loading