Skip to content

feat(flags): report unresolved local flags on evaluate_flags snapshots - #319

Draft
gustavohstrassburger wants to merge 2 commits into
mainfrom
posthog/unresolved-local-flags
Draft

gustavohstrassburger wants to merge 2 commits into
mainfrom
posthog/unresolved-local-flags

Conversation

@gustavohstrassburger

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Why: sdk-specs #85 defines how a server SDK reports flags that local evaluation could not resolve. Today such a flag is absent from the evaluate_flags snapshot, so it looks the same as a flag that does not exist, and $feature_flag_called reports flag_missing for both. With only_evaluate_locally: true this is permanent. posthog-node already implements this (PostHog/posthog-js#5234); posthog-python (#1133) and posthog-server (#896) are in review.

Changes:

  • FeatureFlagEvaluations#unresolved_flags maps each unresolved flag to a PostHog::UnresolvedFlagReason (experience_continuity, unsupported_definition, unresolved_dependency, missing_context). When several causes apply, the spec's fixed precedence picks one. only / only_accessed snapshots have no entries, and reading the hash is not an access.
  • A snapshot read of an unresolved flag reports local_evaluation_inconclusive instead of flag_missing.
  • InconclusiveMatchError and RequiresServerEvaluation take an optional reason:, set at every raise site. A bad person value is missing_context; a bad filter value is unsupported_definition.
  • When definitions load, the SDK logs a WARN line for flags local evaluation can never resolve. Each flag warns once until its definition changes; 304 and repeated shared-cache loads do not repeat it.

Behavior changes (all in the changeset):

  • A condition or cohort property group no longer stops at the first filter it cannot check. If another filter does not match, the flag can resolve locally. A static cohort still forces the fallback only when it is the first filter in its condition that cannot be checked, as in posthog-node.
  • An inactive flag with experience continuity resolves to false locally.
  • A flag that depends on a flag with experience continuity no longer resolves locally.
  • A flag that depends on a flag that needs the server becomes inconclusive (unresolved_dependency) at that condition instead of forcing the fallback, as in posthog-node. A later condition can now resolve it.
  • When local evaluation left a flag unresolved and the remote fallback marks it as failed, the snapshot no longer holds it as false; it is absent and listed as unresolved. A failed flag with no local definition keeps its false value.

Divergences from the spec or posthog-node:

  • The SDK has no device-id bucketing, so the device-id acceptance row does not apply. A missing group key still resolves to false, as in posthog-node.
  • A regex that only the server can evaluate reports unsupported_definition, even when the person value causes the deferral. Patterns that always need the server are in the load-time warning; posthog-node does not warn about regexes.

💚 How did you test it?

New RSpec examples mirror every scenario tagged @unresolved_flags_capable and @unresolvable_flag_warning_capable in acceptance/public/evaluate-flags.feature, plus precedence, the condition loop, dependencies, warning dedup, failed fallback and unexpected errors. Run locally:

  • bundle exec rspec (1988 examples, 0 failures, 40 pending for the optional OpenTelemetry bundle)
  • bundle exec rubocop
  • bundle exec rake public_api:check

No manual testing in an app.

📝 Checklist

  • All commits are signed
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

  • Written with Claude Code in PostHog Desktop, from a cross-SDK plan to apply sdk-specs Add Action workflow for publishing Ruby gem #85 to all server SDKs. It follows posthog-node and the posthog-python and posthog-server PRs from the same plan.
  • PostHog::Utils.convert_to_datetime (@api private) now takes a required reason: so date parsing can tell a bad person value from a bad filter value.

Created with PostHog Desktop

🤖 Generated with Claude Code

Implements the "Snapshot unresolved flag reporting" and "Warning for flags local evaluation never resolves" requirements of the evaluate-flags spec (PostHog/sdk-specs#85).

- Add `FeatureFlagEvaluations#unresolved_flags`, with a `PostHog::UnresolvedFlagReason` for each flag and the spec's fixed precedence.
- Report `local_evaluation_inconclusive` instead of `flag_missing` when a snapshot read hits an unresolved flag.
- Do not stop a condition at the first inconclusive filter, so a filter that does not match resolves the condition locally.
- Check `active` before experience continuity, also for dependencies.
- Log one warning per flag that local evaluation can never resolve, repeated only when its definition changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: 39793b31-809e-4689-a5d8-b4359cbb22c1
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

posthog-ruby-sync Compliance Report

Date: 2026-10-28T03:57:10.598571+00:00
Duration: 56051ms

⚠️ Some Tests Failed

39/47 tests passed, 8 failed


Capture Tests

⚠️ 23/30 tests passed, 7 failed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 10ms
Format Validation.Event Has Uuid ✅ 7ms
Format Validation.Event Has Lib Properties ✅ 9ms
Format Validation.Distinct Id Is String ✅ 7ms
Format Validation.Token Is Present ✅ 7ms
Format Validation.Custom Properties Preserved ✅ 7ms
Format Validation.Event Has Timestamp ✅ 10ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 7ms
Retry Behavior.Retries On 503 ✅ 5322ms
Retry Behavior.Does Not Retry On 400 ❌ 8ms
Retry Behavior.Does Not Retry On 401 ❌ 6ms
Retry Behavior.Respects Retry After Header ✅ 8015ms
Retry Behavior.Implements Backoff ❌ 339ms
Retry Behavior.Retries On 500 ✅ 5116ms
Retry Behavior.Retries On 502 ✅ 5115ms
Retry Behavior.Retries On 504 ✅ 5116ms
Retry Behavior.Max Retries Respected ❌ 427ms
Deduplication.Generates Unique Uuids ✅ 17ms
Deduplication.Preserves Uuid On Retry ✅ 5115ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10319ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5150ms
Deduplication.No Duplicate Events In Batch ✅ 20ms
Deduplication.Different Events Have Different Uuids ✅ 10ms
Compression.Sends Gzip When Enabled ✅ 7ms
Batch Format.Uses Proper Batch Structure ✅ 7ms
Batch Format.Flush With No Events Sends Nothing ✅ 4ms
Batch Format.Multiple Events Batched Together ❌ 19ms
Error Handling.Does Not Retry On 403 ❌ 6ms
Error Handling.Does Not Retry On 413 ❌ 5ms
Error Handling.Retries On 408 ✅ 5139ms

Failures

retry_behavior.does_not_retry_on_400

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

retry_behavior.does_not_retry_on_401

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

retry_behavior.implements_backoff

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

retry_behavior.max_retries_respected

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

batch_format.multiple_events_batched_together

Expected 1 requests, got 5

error_handling.does_not_retry_on_403

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

error_handling.does_not_retry_on_413

500, message='Internal Server Error', url='http://sdk-adapter:8080/capture'

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 9ms
Request Payload.Flags Request Uses V2 Query Param ✅ 8ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 9ms
Request Payload.Flags Request Omits Authorization Header ✅ 10ms
Request Payload.Token In Flags Body Matches Init ✅ 11ms
Request Payload.Groups Round Trip ✅ 7ms
Request Payload.Groups Default To Empty Object ✅ 8ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 7ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 8ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 8ms
Request Lifecycle.No Flags Request On Init Alone ✅ 3ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 7ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 10ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 8ms
Retry Behavior.Retries Flags On 502 ✅ 112ms
Retry Behavior.Retries Flags On 504 ✅ 151ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 9ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'groups', 'person_properties', 'group_properties', 'flag_keys_to_evaluate', 'token']

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

posthog-ruby-async Compliance Report

Date: 2026-10-28T03:57:47.336313+00:00
Duration: 98515ms

⚠️ Some Tests Failed

46/47 tests passed, 1 failed


Capture Tests

✅ 30/30 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 11ms
Format Validation.Event Has Uuid ✅ 106ms
Format Validation.Event Has Lib Properties ✅ 110ms
Format Validation.Distinct Id Is String ✅ 107ms
Format Validation.Token Is Present ✅ 107ms
Format Validation.Custom Properties Preserved ✅ 112ms
Format Validation.Event Has Timestamp ✅ 106ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 8ms
Retry Behavior.Retries On 503 ✅ 5312ms
Retry Behavior.Does Not Retry On 400 ✅ 2109ms
Retry Behavior.Does Not Retry On 401 ✅ 2108ms
Retry Behavior.Respects Retry After Header ✅ 8117ms
Retry Behavior.Implements Backoff ✅ 15818ms
Retry Behavior.Retries On 500 ✅ 5213ms
Retry Behavior.Retries On 502 ✅ 5212ms
Retry Behavior.Retries On 504 ✅ 5208ms
Retry Behavior.Max Retries Respected ✅ 15622ms
Deduplication.Generates Unique Uuids ✅ 112ms
Deduplication.Preserves Uuid On Retry ✅ 5212ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 10318ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5216ms
Deduplication.No Duplicate Events In Batch ✅ 112ms
Deduplication.Different Events Have Different Uuids ✅ 108ms
Compression.Sends Gzip When Enabled ✅ 106ms
Batch Format.Uses Proper Batch Structure ✅ 107ms
Batch Format.Flush With No Events Sends Nothing ✅ 5ms
Batch Format.Multiple Events Batched Together ✅ 111ms
Error Handling.Does Not Retry On 403 ✅ 2107ms
Error Handling.Does Not Retry On 413 ✅ 2109ms
Error Handling.Retries On 408 ✅ 5212ms

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 108ms
Request Payload.Flags Request Uses V2 Query Param ✅ 107ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 108ms
Request Payload.Flags Request Omits Authorization Header ✅ 109ms
Request Payload.Token In Flags Body Matches Init ✅ 110ms
Request Payload.Groups Round Trip ✅ 107ms
Request Payload.Groups Default To Empty Object ✅ 107ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 108ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 107ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 108ms
Request Lifecycle.No Flags Request On Init Alone ✅ 3ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 106ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 111ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 108ms
Retry Behavior.Retries Flags On 502 ✅ 210ms
Retry Behavior.Retries Flags On 504 ✅ 243ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 109ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Field 'geoip_disable' not found in /flags request body at path 'geoip_disable'. Available keys: ['distinct_id', 'groups', 'person_properties', 'group_properties', 'flag_keys_to_evaluate', 'token']

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant