Skip to content

test: Run the FDv2 contract tests - #194

Merged
beekld merged 10 commits into
mainfrom
bklimt/SDK-2889/fdv2-contract-tests
Sep 4, 2026
Merged

test: Run the FDv2 contract tests#194
beekld merged 10 commits into
mainfrom
bklimt/SDK-2889/fdv2-contract-tests

Conversation

@beekld

@beekld beekld commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Runs the cross-SDK FDv2 contract test suite against the SDK to exercise the data system end to end. The contract test service now accepts the harness's data system configuration and drives it through the public DataSystemBuilder. CI runs both the FDv1 and FDv2 suites.

The service advertises the fdv1-fallback capability so the fallback-directive scenarios run. There is no FDv2 capability flag, so the suite is selected by the harness version.

A suppression file skips scenarios that do not apply to this SDK:

  • The payload-filter URL scenarios, since the SDK exposes no payload filter.
  • The terminal FDv1-fallback scenario, since the SDK follows the directive's TTL rather than treating it as terminal.

Note

Overview
Adds cross-SDK FDv2 contract test coverage by extending the contract-test service and running both harness versions in CI.

The harness can now pass dataSystem configuration (initializers, synchronizers, FDv1 fallback). The service maps that to DataSystemBuilder with FDv2 streaming/polling builders and advertises the fdv1-fallback capability. Client setup builds a fresh transport per SDK component (shared connector only), matching normal SDK behavior.

CI/Makefile: bumps the contract-tests action to v1.3.0, runs FDv1 then FDv2 (version: v3) against one background service (stop_service: false on the first run), and adds local contract-tests-fdv2 / run-contract-tests-fdv2 with a dedicated suppressions file for the v3 harness (includes existing FDv1 skips).

Dependency: launchdarkly-server-sdk-evaluation 2.2.0 → 2.2.1 (patch).

Reviewed by Cursor Bugbot for commit 9edc163. Bugbot is set up for automated code reviews on this repo. Configure here.

@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from 6944725 to ab1746f Compare August 10, 2026 18:41
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch 2 times, most recently from 68bf950 to 5c6a460 Compare August 10, 2026 22:49
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from 5c6a460 to 1b8fc83 Compare August 12, 2026 21:45
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from 1b8fc83 to d28128d Compare August 12, 2026 22:10
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch 2 times, most recently from 3a85ea8 to 8f35a44 Compare August 14, 2026 20:53
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from 8f35a44 to 651dae5 Compare August 14, 2026 22:21
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch 2 times, most recently from 482a877 to a93bce7 Compare August 17, 2026 06:33
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch 2 times, most recently from c85c28e to d4a90b4 Compare August 24, 2026 18:02
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from d4a90b4 to b78d364 Compare August 24, 2026 18:42
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from b78d364 to 4753222 Compare August 25, 2026 22:33
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from 4753222 to 0a4ce17 Compare August 25, 2026 22:54
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from 0a4ce17 to b156124 Compare August 27, 2026 21:09
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from 2847233 to 4fe30d3 Compare August 28, 2026 17:24
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch 4 times, most recently from 21f13be to 7848a93 Compare September 3, 2026 19:08
@beekld
beekld marked this pull request as ready for review September 3, 2026 19:16
@beekld
beekld requested a review from a team as a code owner September 3, 2026 19:16

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c7e15f6. Configure here.

Comment thread contract-tests/testharness-suppressions-fdv2.txt Outdated
Comment thread contract-tests/src/client_entity.rs Outdated
.or_else(|| derive_fallback_base_url(&synchronizers)),
fallback.poll_interval_ms,
)),
None => select_fallback_synchronizer(&synchronizers).map(|sync| {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you should have to do all of this. If there isn't a fallback provided by the test harness, you should assume we don't want one set and just leave it as optional.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@beekld
beekld requested a review from keelerm84 September 3, 2026 21:08
Base automatically changed from bklimt/SDK-2873/fdv2-config-api to main September 3, 2026 22:44
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from 410d95e to b62264c Compare September 3, 2026 22:44

# Rust honors the TTL-based FDv1 fallback directive (SDK-2527).
# This scenario tests the terminal semantics instead.
streaming/fdv2/FDv1 fallback directive/directive without FDv1 fallback configured halts the data system

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you still need this suppression once we changed the FDv1 configuration stuff?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, good catch! I had thought that was failing because of the changes to our FDv1 fallback logic, but you're right. It was the bug with how we configured for contract tests.

@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from b62264c to f220bce Compare September 4, 2026 17:34
@beekld
beekld force-pushed the bklimt/SDK-2889/fdv2-contract-tests branch from f220bce to 9edc163 Compare September 4, 2026 17:36
@beekld
beekld merged commit 65b53cc into main Sep 4, 2026
14 checks passed
@beekld
beekld deleted the bklimt/SDK-2889/fdv2-contract-tests branch September 4, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants