Skip to content

fix(backend): stop logging an error when no trial offer is configured - #14802

Merged
Pwuts merged 1 commit into
devfrom
pwuts/trial-offer-off-variation-guard
Sep 23, 2026
Merged

Pwuts merged 1 commit into
devfrom
pwuts/trial-offer-off-variation-guard

Conversation

@Pwuts

@Pwuts Pwuts commented Sep 22, 2026

Copy link
Copy Markdown
Member

Why / What / How

Why: get_trial_offer logs Invalid card-required-trial-offer; refusing trial enrollment at ERROR on every eligibility check in production — roughly four lines per 20 minutes into platform_server_errors and Sentry since the v0.8.0 rollout. Nothing is broken: the card-required-trial-offer flag simply has two variations, and the one production serves is {"enabled": false}. That object is not null, so it reached TrialOffer.model_validate, failed extra="forbid", and took the error path. Dev serves the real offer, so the noise never showed there.

What: a flag payload that never claims to be an offer — no version key, or not an object at all — is now read as "no offer configured" and logged at DEBUG. A payload that carries version and still fails validation keeps its ERROR: that one is a broken offer someone has to fix.

How: version is the discriminator because TrialOffer requires it with no default, so every offer that can validate carries it and no disabled variation does. The guard sits before model_validate, matching how the two sibling LaunchDarkly JSON-object consumers (_fetch_cost_limits_flag, get_tier_stripe_price) narrow a raw flag value.

Return values are unchanged for every possible flag value — None in, None out, exactly as before, for the disabled variation and for a broken offer alike. Only the log level moves, so no caller behaviour changes.

Changes 🏗️

  • subscription_trial_config.py: get_trial_offer returns None at DEBUG for a non-offer flag value instead of falling through to the ERROR handler.
  • subscription_trial_config_test.py: the disabled variation and the other non-offer shapes produce no record above DEBUG; a version-carrying offer that fails validation still produces exactly one ERROR.

The LaunchDarkly side is unchanged and is not part of this PR.

Verified

I ran backend/data/subscription_trial_config_test.py (43 passed), backend/util/architecture_test.py and backend/blocks/test/test_block.py (1755 passed, 84 skipped), and ruff check on both changed files. I read production's flag configuration through the LaunchDarkly REST API to confirm the premise rather than inferring it. I mutated the fix back to the original if raw is None shape and confirmed the new tests fail — 5 failed / 38 passed. Evidence in a comment below.

The two consumers of get_trial_offer (subscription_trial_routes.py, subscription_trial_checkout.py) both branch on is None; I reasoned about them from the call sites rather than executing their suites, on the grounds that the return value is provably unchanged.

Agents and large language models used

Claude Code with Claude Opus 5

Checklist 📋

For code changes:

  • I have clearly listed my changes in the PR description
  • I have made a test plan
  • I have tested my changes according to the test plan:
    • The disabled variation {"enabled": false} returns None and logs nothing above DEBUG
    • None, False, {} and {"enabled": true} behave the same way
    • An offer carrying version that fails validation still logs exactly one ERROR and returns None
    • A valid offer still parses
    • Reverting the guard turns the new tests red

The card-required-trial-offer flag's disabled variation is an object,
{"enabled": false}, not null. get_trial_offer treated only None as "no
offer", so the disabled variation reached TrialOffer.model_validate,
failed extra="forbid" and logged an ERROR on every eligibility check.

Treat a payload that never claims to be an offer -- no version key, or
not an object at all -- as absence, at DEBUG. A payload that carries
version and fails validation is still an ERROR: that one is a broken
offer someone needs to fix.

Return values are unchanged for every possible flag value; only the log
level moves.

Co-Authored-By: Claude Opus 5 (Claude Code) <noreply@anthropic.com>
@github-project-automation github-project-automation Bot moved this to 🆕 Needs initial review in AutoGPT development kanban Sep 22, 2026
@github-actions github-actions Bot added platform/backend AutoGPT Platform - Back end cla: pending CLA not yet signed by all contributors size/m cla: signed CLA signed by all contributors labels Sep 22, 2026
@Pwuts

Pwuts commented Sep 22, 2026

Copy link
Copy Markdown
Member Author

🤖 Evidence

The premise, read from LaunchDarkly

card-required-trial-offer in production, via the REST API (values elided where they are product terms rather than evidence):

variations:
  0 {"enabled": false}
  1 {"version": "pro-equivalent-v1", ...the real offer...}
on: True
offVariation: 0
fallthrough: {"variation": 0}
rules: 0
targets: []

The flag is on with no rules and no targets, so its fallthrough serves variation 0 to every user. {"enabled": false} is not null, so before this PR it reached TrialOffer.model_validate and took the ERROR path on every eligibility check. Variation 1 is the only one carrying version, which is what the new guard keys on.

Mutation

Restoring the original if raw is None: return None and re-running the module:

FAILED test_flag_value_that_is_not_an_offer_is_silent[flag_value0] - assert [<LogRecord: ... enrollment">] == []
FAILED test_flag_value_that_is_not_an_offer_is_silent[flag_value1] - assert [<LogRecord: ... enrollment">] == []
FAILED test_flag_value_that_is_not_an_offer_is_silent[flag_value2] - assert [<LogRecord: ... enrollment">] == []
FAILED test_flag_value_that_is_not_an_offer_is_silent[None]        - AssertionError: assert [] == ['No card-req...r configured']
FAILED test_flag_value_that_is_not_an_offer_is_silent[False]       - assert [<LogRecord: ... enrollment">] == []
=================== 5 failed, 38 passed in 21.69s ===================

The four object/bool cases fail because the unpatched code emits the ERROR the PR exists to remove; the None case fails because it emits no record at all. test_offer_that_fails_validation_is_still_an_error stays green under the mutation, which is correct — the fix does not touch that path.

The log assertions attach a plain logging.Handler to the module logger rather than using caplog, which captures nothing for backend.* loggers under this app's logging config.

Test runs

backend/data/subscription_trial_config_test.py ....... 43 passed in 26.81s
+ backend/util/architecture_test.py .................. 46 passed in 59.38s
backend/blocks/test/test_block.py ......... 1755 passed, 84 skipped in 38.50s
ruff check <both changed files> ............ All checks passed!

Note for whoever merges second

#14799 also touches subscription_trial_config.py and its test module — it adds fields to TrialOffer and appends tests, and does not change get_trial_offer's body. The two should merge cleanly; if they do conflict it will be at the end of the test file, and both sides are keeps.

@github-actions github-actions Bot removed the cla: pending CLA not yet signed by all contributors label Sep 22, 2026
@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 2596dd7d-0cba-474a-83d7-e8c172d34254

📥 Commits

Reviewing files that changed from the base of the PR and between de282b8 and 0ab9a24.

📒 Files selected for processing (2)
  • autogpt_platform/backend/backend/data/subscription_trial_config.py
  • autogpt_platform/backend/backend/data/subscription_trial_config_test.py

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.

📜 Recent review details
🔇 Additional comments (2)
autogpt_platform/backend/backend/data/subscription_trial_config.py (1)

73-76: LGTM!

autogpt_platform/backend/backend/data/subscription_trial_config_test.py (1)

1-2: LGTM!

Also applies to: 12-30, 187-204, 207-220


Walkthrough

get_trial_offer now treats flag values without an offer version as unconfigured and returns None. Tests cover non-offer values and confirm that invalid offer data still produces an error log.

Changes

Trial offer flag handling

Layer / File(s) Summary
Handle unconfigured and invalid offers
autogpt_platform/backend/backend/data/subscription_trial_config.py, autogpt_platform/backend/backend/data/subscription_trial_config_test.py
The guard returns None for non-dictionary flag values and dictionaries without a version key. Tests check non-offer values and their logs, and confirm that invalid offer data still produces an error log.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Suggested reviewers: ntindle

Merge Risk: ⚪ Minimal · up to 0ab9a

Disabled or non-offer flag values now avoid erroneous validation logs without changing handling of malformed configured offers. The change is ready to merge.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 28.57% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: preventing an error log when no trial offer is configured.
Description check ✅ Passed The description explains the cause, code change, expected behavior, and tests. It is directly related to the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 PR Overlap Detection

This check compares your PR against all other open PRs targeting the same branch to detect potential merge conflicts early.

🔴 Merge Conflicts Detected

The following PRs have been tested and will have merge conflicts if merged after this PR. Consider coordinating with the authors.

Summary: 1 conflict(s), 0 medium risk, 0 low risk (out of 1 PRs with file overlap)


Auto-generated on push. Ignores: openapi.json, lock files.

@codecov

codecov Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.65%. Comparing base (de282b8) to head (0ab9a24).
⚠️ Report is 1 commits behind head on dev.

Additional details and impacted files
@@           Coverage Diff           @@
##              dev   #14802   +/-   ##
=======================================
  Coverage   83.65%   83.65%           
=======================================
  Files        4054     4054           
  Lines      305117   305146   +29     
  Branches    27672    27670    -2     
=======================================
+ Hits       255241   255270   +29     
  Misses      45208    45208           
  Partials     4668     4668           
Flag Coverage Δ
platform-backend 88.03% <100.00%> (+<0.01%) ⬆️
platform-frontend-e2e 27.47% <ø> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Platform Backend 88.05% <100.00%> (+<0.01%) ⬆️
Platform Frontend 66.18% <ø> (-0.01%) ⬇️
AutoGPT Libs ∅ <ø> (∅)
Classic AutoGPT 28.43% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Pwuts
Pwuts marked this pull request as ready for review September 22, 2026 23:26
@Pwuts
Pwuts requested a review from a team as a code owner September 22, 2026 23:26
@Pwuts
Pwuts requested review from Bentlybro and ntindle and removed request for a team September 22, 2026 23:26
@Pwuts
Pwuts enabled auto-merge September 22, 2026 23:27
@Pwuts

Pwuts commented Sep 22, 2026

Copy link
Copy Markdown
Member Author

/review

@autogpt-pr-reviewer

autogpt-pr-reviewer Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

🤖 Review of 0ab9a24 posted: #14802 (review)

@autogpt-pr-reviewer autogpt-pr-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Verdict: Approved — No findings require changes before approval; this review counts as an approval.

A well-scoped log-severity fix that downgrades a false-positive ERROR to DEBUG when the trial-offer flag serves its disabled variation, while preserving ERROR alerts for genuinely broken offers. Return values are provably unchanged for all inputs, both callers are unaffected, and targeted tests cover both branches.

Risk level: low | Human review: not required | Duration: 189s | Reviewed: 0ab9a247

GitHub CI on reviewed head: success

Validation and specialist details

Specialist Reports

Specialist Status Summary
security ✅ PASS Logging-level-only change in get_trial_offer that narrows the pre-validation guard; validation is not weakened, return values and enrollment gating are provably unchanged, and no user-controlled data or secrets are exposed.
architect ✅ PASS Minimal, well-scoped log-severity fix that narrows a raw LaunchDarkly flag value before validation, matching an existing sibling pattern with return behavior provably unchanged and good test coverage.
performance ✅ PASS Two-line guard on a hot path that skips Pydantic validation for the disabled flag variation and cuts steady-state error-pipeline load; no performance regressions found.
testing ✅ PASS Small, well-targeted log-level fix with strong, regression-catching tests that pin both the new DEBUG path and the preserved ERROR path; no blocking defects.
quality ✅ PASS Clean, well-tested log-level fix that is provably behavior-preserving and follows the established raw-flag-narrowing convention; no defects found.
product ✅ PASS Correct, well-scoped logging-level fix that silences false-positive ERRORs for the flag's disabled variation while preserving ERROR alerts for genuinely broken offers, with return values provably unchanged for all inputs.
discussion ✅ PASS CI is green on head 0ab9a24 and there are no open human-reviewer requests.
ui-reviewer (local) ✅ PASS Correct, well-tested backend logging fix: a non-offer flag value is now DEBUG instead of ERROR while return values and caller behavior are provably unchanged.
ui-reviewer (hosted) ✅ PASS Correct, minimal backend fix that downgrades a benign non-offer flag payload from ERROR to DEBUG while preserving ERROR for genuinely broken offers; return values are unchanged for all flag values, no caller behavior changes, and the 43 targeted tests pass.

@Pwuts
Pwuts added this pull request to the merge queue Sep 22, 2026
@github-project-automation github-project-automation Bot moved this from 🆕 Needs initial review to 👍🏼 Mergeable in AutoGPT development kanban Sep 22, 2026
Merged via the queue into dev with commit 58fb04c Sep 23, 2026
56 checks passed
@Pwuts
Pwuts deleted the pwuts/trial-offer-off-variation-guard branch September 23, 2026 00:12
@github-project-automation github-project-automation Bot moved this from 👍🏼 Mergeable to ✅ Done in AutoGPT development kanban Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: signed CLA signed by all contributors platform/backend AutoGPT Platform - Back end size/m

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

1 participant