Skip to content

fix: stop emitting thinking blocks with null signatures from chat-completions responses - #12

Open
Michael J. Jabbour (michaeljabbour) wants to merge 1 commit into
mainfrom
fix/drop-fabricated-thinking-blocks
Open

fix: stop emitting thinking blocks with null signatures from chat-completions responses#12
Michael J. Jabbour (michaeljabbour) wants to merge 1 commit into
mainfrom
fix/drop-fabricated-thinking-blocks

Conversation

@michaeljabbour

Copy link
Copy Markdown

Summary

provider-chat-completions fabricated thinking content blocks from OpenAI-compatible reasoning_content fields, then persisted them into session history with signature: null. Anthropic's API strict-validates every thinking block's signature as a non-empty string on replay; a single null-signature block anywhere in history 400s the entire next request (messages.N.content.0.thinking.signature.str: Input should be a valid string) and bricks the session the moment a user switches providers mid-conversation.

Root cause

OpenAI-style chat-completions APIs have no signed extended thinking — the concept doesn't exist on that side of the wire. This module never set signature anywhere, so the ThinkingBlocks it fabricated always fell back to the field's None default. That's fine as long as the session stays on chat-completions, but the moment history is replayed to provider-anthropic (e.g. after a provider swap), those null-signature blocks get rejected outright.

What this PR does

Implements the issue's preferred fix (option 1): stop emitting persisted thinking content blocks from this module.

  • Deletes the two persisted-append sites:
    • non-streaming _build_response() (previously amplifier_module_provider_chat_completions/__init__.py:650)
    • streaming _complete_streaming() (previously :992)
  • Keeps the ephemeral live-thinking UI stream fully intact: ThinkingContent event blocks and the llm:stream_block_* hook events are unchanged, since they're render-only and never replayed to a provider.
  • Persisted assistant content is now [TextBlock] (+ ToolCallBlocks when tools are called) for chat-completions responses; text / content_blocks are unaffected.

This is consistent with the module's own outbound wire path, which already silently drops ThinkingBlock when re-serializing history (_convert_messages_to_wire, :554-555) — a persisted content list without thinking blocks is already a fully-supported round-trip shape for this module.

Explicitly out of scope: provider-openai is untouched. Its Responses-API reasoning artifacts are functional for OpenAI-side replay; cross-provider safety for those is handled by the sibling consumer-side PR in provider-anthropic.

Tests

TDD: both new tests were written first and confirmed failing against the old behavior before the fix landed.

  • Inverted tests/test_provider.py::test_reasoning_content_becomes_thinking_blocktest_reasoning_content_does_not_become_thinking_block: asserts _build_response() never returns a ThinkingBlock while the TextBlock still carries the response text.
  • Added tests/test_streaming.py::test_reasoning_content_does_not_become_persisted_thinking_block: asserts await provider.complete(...) never returns a ThinkingBlock in resp.content, while confirming the ephemeral llm:stream_block_delta (block_type="thinking") events still fire unchanged.
  • All pre-existing ephemeral-UI streaming tests (test_streaming.py reasoning/thinking section) pass unchanged, proving the live-thinking UI stream was not touched.
  • Full suite: 149 passed, 5 skipped, 1 pre-existing failure unrelated to this change (TestConfigParsing::test_default_base_url, confirmed failing identically on origin/main prior to this diff — a stale test/code mismatch around a hardcoded default base_url, out of scope for this surgical fix).

Offline (no live API) verification: constructed faked OpenAI SDK objects carrying reasoning_content, called _build_response(...) and await provider.complete(...) directly, and confirmed the actual persisted response objects (result.content, resp.content) contain zero ThinkingBlocks while TextBlocks survive intact.

Related

Part of the cross-provider resume hardening set tracked on microsoft-amplifier/amplifier-support#208. Sibling PRs in provider-anthropic: consumer-side thinking-block sanitization for #207, and an effort-clamp fix for #289.

Fixes microsoft-amplifier/amplifier-support#206

…pletions responses

OpenAI-style chat-completions APIs have no signed extended thinking, so the
two persisted-append sites in this module fabricated ThinkingBlock content
with signature=None. Anthropic strict-validates every thinking block's
signature as a non-empty string on replay; a single null-signature block in
history 400s the entire next request and bricks the session after a
provider switch.

Deletes the two persisted ThinkingBlock appends (non-streaming
_build_response, streaming _complete_streaming) while keeping the ephemeral
live-thinking UI stream (ThinkingContent event blocks / llm:stream_block_*
hook events) intact -- those are render-only and never replayed to a
provider.

Fixes microsoft-amplifier/amplifier-support#206
@michaeljabbour

Copy link
Copy Markdown
Author

Cross-provider resume hardening set — complete PR index

This PR is the producer-side fix in the set tracked on microsoft-amplifier/amplifier-support#208. Full set for reviewers:

PR Repo Role
provider-anthropic#72 provider-anthropic Option B — consumer-side sanitization of invalid thinking blocks — Fixes support#207
provider-chat-completions#12 (this PR) provider-chat-completions Producer fix — stop fabricating signature: null thinking blocks — Fixes support#206
provider-anthropic#71 provider-anthropic Effort clamp (max→highest supported tier) — Addresses support#289
amplifier-app-cli#232 amplifier-app-cli Option A — resume-time provider/model mismatch warning + confirm, plus provider persisted in session metadata — Addresses support#208

Belt-and-suspenders note: even with this producer fix merged, provider-anthropic#72's consumer-side filter still protects against other producers (e.g. provider-openai Responses artifacts) and legacy transcripts written before this fix.

Design rationale and deferred items: support#208 comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant