Skip to content

fix: retry desktop capture v1 requests that get 404 or 429 - #678

Merged
turnipdabeets merged 10 commits into
feat/v6-desktop-capture-v1from
fix/v6-desktop-retry-404-429
Oct 28, 2026
Merged

turnipdabeets merged 10 commits into
feat/v6-desktop-capture-v1from
fix/v6-desktop-retry-404-429

Conversation

@turnipdabeets

@turnipdabeets turnipdabeets commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Stacked on #673. We agreed that SDKs should drop events only when they can never be delivered (event too large, duplicate uuid) or when the buffer is full. A 404 usually means a misconfigured proxy, and a 429 now comes only from an overloaded Envoy. Both can recover, so dropping the batch loses data for no reason.

This ports PostHog/posthog-go#395 to the Dart desktop transport (Windows/Linux): 404 and 429 join 408/500/502/503/504 as retried statuses. Redirects (other than same-origin 307/308) are still terminal. Flutter on iOS and Android gets the same fix through the native SDKs.

It also brings desktop retry, timeout and flush behavior in line with posthog-android 4.0 (PostHog/posthog-android#897) and posthog-ios 4.0 (PostHog/posthog-ios#979), so capture v1 behaves the same on every 6.0 platform:

  • Backoff (ed36592). A transient failure (network error or a retried status), or a 2xx whose per-event results ask to retry, no longer resends inside the flush (it used to try 3 times, 3 s apart, then pause 6/12/24/30 s). Each one now pauses the queue: 1 s, doubling up to 30 s, with ±25% jitter. Retry-After is a floor applied after the jitter, capped at 30 s. It no longer replaces the backoff, so Retry-After: 1 doesn't make the SDK resend every 3 s. A delivered or dropped batch resets the backoff. The jitter source is injectable for tests.
  • Request timeout (1ae278f). The 10 s whole-request timeout could fail a large batch on a slow uplink on every attempt and keep it stuck at the head of the queue. A request now fails only after 10 s without progress: connecting (HttpClient.connectionTimeout), the socket taking a 64 KiB chunk of the body (addStream respects socket backpressure), the response headers, or a chunk of the response body. This is as close to OkHttp's and URLSession's idle timeouts as dart:io allows. Like those, it can't see bytes still in the kernel send buffer, so the wait for headers starts once the last chunk is handed to the socket.
  • 413 (7c25cac). Halves flushAt along with the batch size, keeping flushAt ≤ the batch size, and ends the flush instead of resending at once. The smaller batches go out with the next flush (the next capture over flushAt, or the periodic flush). The natives differ here: Android pauses with backoff after a 413, while iOS ends the flush without one. Desktop follows iOS and adds no backoff.
  • Byte flush trigger (b7cd99c). The queue now also flushes once 5 MiB of serialized events is queued (the captureV1MaxBatchBytes from feat!: capture v1 on every platform, with per-event options and captureAi #673), not only at flushAt. The file queue tracks each event's size: measured when the event is added, and read from the file sizes when a later process loads the queue.

Divergences from native: no backoff after a 413 (matches iOS, not Android), and the timeout limitation above.

💚 How did you test it?

  • 404/429: moved from the terminal-status loop to the retried loop in queue_flush_test.dart, and 429 added to the Retry-After pause test. The capture v1 404 test checks that the batch stays queued and nothing falls back to /batch.
  • Backoff: captureRetryDelay unit tests (doubling and cap, jitter bounds, an out-of-range jitter unit, Retry-After as a floor and its 30 s cap). Client tests with fixed jitter cover the 1/2/4/8/16/30 s schedule, ±25% jitter (first retry at 750 ms and 1250 ms), Retry-After 5 over a 1 s backoff and Retry-After: 1 under a 16 s backoff, and a partial-retry 2xx advancing the backoff while a full delivery resets it. testClient fixes the jitter at the midpoint, so the existing timing tests stay deterministic.
  • Timeout: a fake request covers an upload that takes a chunk every 6 s (finishes past 10 s, no abort), an upload that stalls (aborted at exactly 10 s, events kept), and a response body that trickles in past 10 s. The existing connect, body-timeout and cancel-before-retry tests still pass. A local experiment confirmed that HttpClientRequest.addStream stops pulling chunks once a peer stops reading.
  • 413: the batch size and flushAt both halve, each 413 ends its flush, the next capture over the new flushAt flushes, and the periodic flush sends the halved batches.
  • Byte trigger: two 2.5 MiB events flush right away while far below flushAt, and FileEventQueue.byteCount is checked across add, remove and a reload from disk.
  • flutter test in posthog_flutter (1324 passed), plus dart analyze . (no issues) and dart format --set-exit-if-changed . (no changes) from the repo root.

📝 Checklist

  • All commits are signed (see Commit signing)
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

No new changeset is needed because capture v1 on desktop hasn't shipped yet. The existing v6-desktop-capture-v1 changeset from #673 now says that a 404 or 429 is retried, that redirects still drop the batch, and that the queue backs off like iOS and Android.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Claude Code made the 404/429 change from a brief that named posthog-go#395 as the reference. A later brief asked it to match the posthog-android 4.0 and posthog-ios 4.0 retry, timeout and flush behavior, and it checked each gap against the native queue code. It left alone property order, options, process_person_profile, the AI endpoint and the per-path retry identity, which are being done separately. /flags keeps its own retry (502/504 only, once, after 300 ms).

🤖 Generated with Claude Code

A 404 usually means a misconfigured proxy and a 429 an overloaded Envoy; both are worth retrying with the same backoff, Retry-After handling and retry budget. Redirects stay terminal.
@turnipdabeets turnipdabeets self-assigned this Oct 27, 2026
@turnipdabeets
turnipdabeets requested a review from a team as a code owner October 27, 2026 18:23
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

posthog-flutter Compliance Report

Date: 2026-10-28 20:19:59 UTC
Duration: 96508ms

✅ All Tests Passed!

45/45 tests passed


Capture Tests

✅ 29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 127ms
Format Validation.Event Has Uuid ✅ 112ms
Format Validation.Event Has Lib Properties ✅ 111ms
Format Validation.Distinct Id Is String ✅ 110ms
Format Validation.Token Is Present ✅ 110ms
Format Validation.Custom Properties Preserved ✅ 112ms
Format Validation.Event Has Timestamp ✅ 112ms
Retry Behavior.Retries On 503 ✅ 5324ms
Retry Behavior.Does Not Retry On 400 ✅ 2113ms
Retry Behavior.Does Not Retry On 401 ✅ 2113ms
Retry Behavior.Respects Retry After Header ✅ 8120ms
Retry Behavior.Implements Backoff ✅ 15436ms
Retry Behavior.Retries On 500 ✅ 5221ms
Retry Behavior.Retries On 502 ✅ 5223ms
Retry Behavior.Retries On 504 ✅ 5223ms
Retry Behavior.Max Retries Respected ✅ 15442ms
Deduplication.Generates Unique Uuids ✅ 120ms
Deduplication.Preserves Uuid On Retry ✅ 5221ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10330ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5226ms
Deduplication.No Duplicate Events In Batch ✅ 116ms
Deduplication.Different Events Have Different Uuids ✅ 112ms
Compression.Sends Gzip When Enabled ✅ 113ms
Batch Format.Uses Proper Batch Structure ✅ 110ms
Batch Format.Flush With No Events Sends Nothing ✅ 107ms
Batch Format.Multiple Events Batched Together ✅ 117ms
Error Handling.Does Not Retry On 403 ✅ 2113ms
Error Handling.Does Not Retry On 413 ✅ 2114ms
Error Handling.Retries On 408 ✅ 5223ms

Feature_Flags Tests

✅ 16/16 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 13ms
Request Payload.Flags Request Uses V2 Query Param ✅ 8ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 7ms
Request Payload.Flags Request Omits Authorization Header ✅ 7ms
Request Payload.Token In Flags Body Matches Init ✅ 7ms
Request Payload.Groups Round Trip ✅ 7ms
Request Payload.Groups Default To Empty Object ✅ 6ms
Request Payload.Person Properties Distinct Id Auto Populated When Caller Omits It ✅ 7ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 6ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 6ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 6ms
Request Lifecycle.No Flags Request On Init Alone ✅ 3ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 109ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 14ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 7ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 110ms

…e 404

After a batch fails its retries, later flushes make one attempt each and
pause 6s doubling to 30s until a batch is delivered. Before, every flush,
including the one each capture starts at flushAt, sent 4 requests, so a
missing capture route kept resending the batch.

A 404 prints one warning per client even without debug, naming the URL
and the path a proxy or self-hosted instance must serve.

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed fef8375: 1 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Comment thread posthog_flutter/lib/src/core/posthog_core_stateless.dart Outdated
A host with proxy credentials in its userinfo would print them to app logs.
… fix/v6-desktop-retry-404-429

# Conflicts:
#	.changeset/v6-desktop-capture-v1.md

@marandaneto marandaneto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good; no qualifying findings. Verdict: correct.

turnipdabeets and others added 5 commits October 28, 2026 11:52
… fix/v6-desktop-retry-404-429

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…g-android

A transient failure or a 2xx whose per-event results ask to retry no longer
resends within the flush (3 tries, 3 s apart). Each one pauses the queue for
1 s doubling to 30 s, with ±25% jitter from an injectable source. A
Retry-After is a floor applied after the jitter, capped at 30 s, instead of
replacing the backoff, so `Retry-After: 1` no longer resends every 3 s.
A delivered or dropped batch resets the backoff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uration

The 10 s whole-call timeout could fail a large batch on a slow uplink on
every attempt. A request now fails only after 10 s without progress:
connecting (HttpClient.connectionTimeout), the socket taking a 64 KiB body
chunk, the response headers, or a response body chunk. This is the closest
dart:io gets to the per-read/per-write idle timeouts of OkHttp and
URLSession: like them, it can't see the bytes still in the kernel send
buffer, so the wait for headers starts once the last chunk is handed over.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 413 halved only the batch size and resent at once in the same flush. Like
posthog-ios and posthog-android, it now also halves flushAt, kept within
the batch size, and the smaller batches go out with the next flush (the
next capture over flushAt, or the periodic flush). No backoff is added:
posthog-ios ends the flush without one; posthog-android pauses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like posthog-ios and posthog-android, a full capture v1 request's worth of
queued bytes (captureV1MaxBatchBytes) now flushes as flushAt does, so a few
large events go out before the queue starts dropping them. The file queue
keeps each event's serialized size: measured when it is added, and read
from the file sizes when a later process loads the queue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed b7cd99c: 1 Should fix.

Comment thread posthog_flutter/lib/src/core/posthog_core_stateless.dart
turnipdabeets added a commit that referenced this pull request Oct 28, 2026
Brings in #678's backoff, idle-progress timeout, 413 handling and 5 MiB
queued-bytes flush trigger, applied per capture lane so the analytics and
AI endpoints each keep their own backoff, flushAt and batch size.

- FileEventQueue: one size tracker, a running total (byteCount) kept on
  add, remove and load, used by both the 5 MiB flush trigger and the AI
  50 MiB cap. add() still takes pre-encoded json/bytes to skip a second
  encode.
- _CaptureLane gains flushAt; failedFlushes now drives captureRetryDelay
  per lane, and a 413 halves the lane's batch size and flushAt and ends
  its flush.
- captureAiTimeout is the AI lane's idle-progress timeout, like
  analytics' 10 s, instead of a whole-request bound.
- Test: the AI and analytics endpoints back off independently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A body that trickles in more often than the 10 s idle timeout reset it
forever, holding the flush. The body now also fails after 30 s in total.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@turnipdabeets
turnipdabeets merged commit 22d3e0e into feat/v6-desktop-capture-v1 Oct 28, 2026
24 of 25 checks passed
@turnipdabeets
turnipdabeets deleted the fix/v6-desktop-retry-404-429 branch October 28, 2026 20:27
turnipdabeets added a commit that referenced this pull request Oct 28, 2026
#673's head already matches #678's branch, merged in the previous commit;
this records #673's squashed #678 as merged so #679 can target #673.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants