Skip to content

🪨 fix: Strip Assistant Prefill for Claude 4.6+ on Bedrock - #622

Merged
berry-13 merged 2 commits into
LibreChat-AI:mainfrom
berry-13:berry-13/fix-bedrock-prefill
Oct 17, 2026
Merged

berry-13 merged 2 commits into
LibreChat-AI:mainfrom
berry-13:berry-13/fix-bedrock-prefill

Conversation

@berry-13

@berry-13 berry-13 commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Claude 4.6 and every later model reject a trailing assistant message (prefill) with a 400. #621 widened the gate to every 4.6+ and 5.x id, but only CustomAnthropic applies it; CustomChatBedrockConverse sends the converted messages as they are, so a Bedrock run on anthropic.claude-sonnet-4-6, global.anthropic.claude-haiku-5-5 or any other 4.6+ id that ends on an assistant turn still fails.

The Bedrock stream and the local non-streaming converter now drop trailing assistant turns for those models after conversion, using the same modelDisallowsAssistantPrefill gate. A cache point that rode a stripped turn is re-anchored on the new tail when no surviving message carries one, as the Anthropic path does. Non-streaming calls for those models go through the local converter the stream already uses, since the inherited converter cannot strip and system turns are only lifted out during conversion. Claude 3.x through 4.5 and non-Anthropic Bedrock models (Nova, Llama, Mistral) are unchanged. The index.ts diff includes formatting the pre-commit Prettier hook applied to that file.

Tracked in berry-13/LibreChat#398.

Change Type

  • Bug fix (non-breaking change which fixes an issue)

Testing

  • npx jest src/llm/bedrock/prefill.test.ts: 25 passed, covering Bedrock, inference-profile and ARN ids for 4.6+, 5.x and Fable as stripped, 3.x through 4.5, dated 4.0, Nova, Llama, Mistral and missing ids as unchanged, cache point re-anchoring, and the invoke and stream requests for Haiku 5.5 (stripped) and Haiku 4.5 (kept), and a prefill followed by a system message. Without the index.ts wiring the two Haiku 5.5 request tests fail.
  • npx jest src/llm/bedrock: 15 suites, 224 passed.
  • npx tsc --noEmit and ESLint on the touched files: clean (only pre-existing warnings).

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective
  • New and existing unit tests pass locally with my changes

@berry-13
berry-13 marked this pull request as ready for review October 17, 2026 16:33
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T16:44:45.953156Z f52beae New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 94da8559cb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/llm/bedrock/index.ts Outdated
@berry-13
berry-13 merged commit aea8711 into LibreChat-AI:main Oct 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant