feat: add anthropic claude llm2 extension - #2249
Open
leepokai wants to merge 8 commits into
Open
Conversation
leepokai
force-pushed
the
feat/anthropic-llm2-extension
branch
from
July 27, 2026 09:52
449f210 to
8a64e81
Compare
Add a native Anthropic Messages API extension. No extension previously targeted Claude. bedrock_llm_python is an Amazon Nova extension that happens to use Bedrock's model-agnostic Converse API, so overriding its model property could reach Claude, but that path is undocumented, defaults to Nova, and cannot express adaptive thinking or effort. The extension conforms to the existing llm-interface.json contract, so no change to ten_ai_base is required. Translation rules between the two contracts: - the system prompt is a top-level parameter, not a message - tool arguments are a JSON string on one side, a parsed object on the other - parallel tool_use and tool_result blocks are merged into single turns, since Anthropic expects one turn per group rather than one per call - image URLs carry both remote URLs and inline base64 data URIs - native thinking blocks map onto the existing reasoning events, so no think-tag parsing is needed Capability gating keeps the extension usable across the whole served model family, whose feature windows do not line up: - adaptive thinking and effort are omitted for older models that reject them, including Haiku 4.5 - the fallbacks parameter is sent only to models that accept it; Sonnet 5 and Opus 4.8 reject it outright - effort xhigh is clamped to high on Opus 4.6 and Sonnet 4.6, so a misconfigured graph degrades instead of failing every turn Sampling parameters are deliberately absent, as current models reject temperature, top_p and top_k; effort replaces them.
leepokai
force-pushed
the
feat/anthropic-llm2-extension
branch
from
July 27, 2026 09:56
8a64e81 to
5e62f89
Compare
When `base_url` is unset the SDK falls back to the ANTHROPIC_BASE_URL environment variable, and a variable that is set but empty is not None -- so the base URL became "" and every request failed with a bare "Connection error" from httpx, with nothing pointing at the cause. An untouched copy of .env.example produced exactly that, since it shipped `ANTHROPIC_BASE_URL=` with no value. Give it the real default, matching OPENAI_API_BASE and GROK_API_BASE, and pass the default explicitly so the extension no longer depends on the variable being absent rather than empty.
…letion The runtime can deliver a cmd before on_start has built the client. Timing from a graph run shows chat_completion handled ~1ms before on_start logged its first line, which made on_call_chat_completion raise "AnthropicLLM is not initialized" and fail a request that was merely early. Gate the generator on an event set in on_start's finally block, so a request that races startup waits for it instead. The failure paths still report their real reason rather than timing out, and a client that never initializes gives up after ten seconds.
Drives the extension through the TEN runtime rather than calling the adapter directly, so it covers what a unit test cannot reach: addon registration, the llm-interface.json import, property injection, and the chat_completion cmd dispatched by AsyncLLM2BaseExtension. Results are consumed exactly as main_cascade_python consumes them. test_extension_loads_and_dispatches needs no credentials -- it uses an invalid key and asserts the failure comes back from the API, which can only happen if the whole path worked. The rest call Anthropic for real and skip unless ANTHROPIC_API_KEY is set, covering streaming, reasoning events, the thinking_display toggle, a two-turn tool call round trip, and the legacy-model path. Both fixes in the preceding commits were found by running these.
Without this the extension cannot be selected from the playground, and a graph has to be hand-edited to try it. Follows the pattern the file already uses for the other seven variants: copy voice_assistant, swap one node, leave the existing graphs untouched.
`main_python` sends `parameters={"temperature": 0.7}` on every turn, and
forwarding it verbatim returns
400 `temperature` may only be set to 1 when thinking is enabled or in
adaptive mode
so every turn of a real voice graph failed. `top_p` and `top_k` are
rejected outright on the same models.
Drop those three on the thinking path and keep them on the legacy path,
where they are still valid. The graph cannot be the place to fix this --
the value is hardcoded in main_python rather than read from property.json,
and that extension is shared with the other LLM addons.
Found by driving asr_result through main_control into this extension with
the transport, STT and TTS mocked out; both directions are covered by new
tests.
- lock the new path dependency. manifest.json listed anthropic_llm2_python but manifest-lock.json had no entry, so an install driven from the lock would not resolve the addon. Added the single entry rather than regenerating, to keep the other 50 packages at their pinned versions. - survive a bad property. An exception escaping on_start reaches the runtime's _exit_on_exception, which calls os._exit -- so one invalid effort level took down the whole worker instead of this node. - clamp `max` as well as `xhigh` on Opus 4.6 and Sonnet 4.6. `max` ranks above `xhigh`, so a model that predates one cannot accept the other. - ignore a caller-supplied `LLMRequest.model`, matching openai_llm2_python. vision_analyze_tool_python hardcodes "gpt-4o", which would have gone to Anthropic as a 404. - only use the beta messages resource when refusal fallback is actually attached, so `refusal_fallback: false` stays on /v1/messages as the README promises. - close the Anthropic and httpx clients on stop; a worker cycles graphs and was leaking a connection pool each time. - drop the dead claude-opus-4-0 and claude-sonnet-4-0 aliases: both are past end-of-life and return not_found whether gated or not. - hold a reference to the test watchdog task so it cannot be collected mid-sleep, and cancel it once the request completes.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What
Adds
anthropic_llm2_python, a native Anthropic Messages API extension.No extension currently targets Claude.
bedrock_llm_pythonis an Amazon Nova extension that happens to use Bedrock's model-agnostic Converse API, so overriding itsmodelproperty could reach Claude — but that path is undocumented, defaults to Nova, and cannot express adaptive thinking or effort.The extension conforms to the existing
llm-interface.jsoncontract, so no change toten_ai_baseis required — the whole change lands in this repo.Shape of the change
13 new files (the extension, its
tests/, and a graph variant), plus aone-line fix to
ai_agents/.env.exampleand one line in the code map. Noexisting logic is modified: the seven graphs already in the
voice-assistant
property.jsonare untouched, and the newvoice_assistant_anthropicis appended alongside them following the samecopy-one-graph-swap-one-node pattern they use.
Translation rules
The two contracts disagree in a few places, and the adapter absorbs all of them:
tool_useandtool_resultblocks are merged into single turns, since Anthropic expects one turn per group rather than one message per callImageURL.urlcarries both remote URLs and inline base64 data URIs, so the source type is chosen per valuethinkingblocks map onto the existing reasoning events, so no think-tag parsing is neededModel compatibility
Three capability gates are applied, because the feature windows do not line up:
effortclaude-opus-5(default),claude-fable-5low–maxclaude-sonnet-5,claude-opus-4-8,claude-opus-4-7low–maxclaude-opus-4-6,claude-sonnet-4-6xhigh→highclaude-haiku-4-5,claude-sonnet-4-5,claude-opus-4-5effortare omitted for older served models that reject them, notably Haiku 4.5.fallbacksis sent only where accepted — Sonnet 5 and Opus 4.8 return'<model>' does not support the 'fallbacks' parameter.effort: "xhigh"is clamped tohighon Opus 4.6 and Sonnet 4.6 so a misconfigured graph degrades instead of failing every turn.A model newer than this extension takes the current-generation path by default, so new releases work without a code change.
Defaults chosen for voice graphs
effort: "low"— these graphs are realtime voice assistants where turn latency dominates. Configurable.thinking_display: "summarized"— the API default isomitted, which streams thinking blocks with empty text, leaving a reasoning pane blank with no error.max_tokens: 2048— on current modelsmax_tokenscaps thinking and response text together, so 512 would truncate mid-answer.refusal_message.Sampling parameters are deliberately absent: current models reject
temperature,top_pandtop_k.effortreplaces them.Verification
Shipped tests — inside the TEN runtime, against the real API
tests/runs the extension through the runtime rather than calling theadapter directly, so it covers addon registration, the
llm-interface.jsonimport, property injection, and thechat_completioncmd dispatched byAsyncLLM2BaseExtension. Results areconsumed exactly as
main_cascade_pythonconsumes them.These are the only checks that exercise a request going in and a response
coming out: streamed deltas accumulating into the final text,
thinkingblocks arriving as reasoning events and disappearing when
thinking_displayisomitted, a two-turn tool call whose mergedtool_resultturn the real API accepts, and a legacy model taking thedegraded path without a 400.
test_extension_loads_and_dispatchesneeds no credentials — it uses aninvalid key and asserts the failure comes back from the API, which can
only happen if the whole path worked. The rest skip unless
ANTHROPIC_API_KEYis set.Through the agent loop, with the transport, STT and TTS mocked
A second harness builds a graph of
ten:test_extension -> main_control -> llm -> message_collectorusingset_test_mode_graph, injects anasr_result, and reads back the chunksmessage_collectorwould have put on the wire to the browser. Only theAnthropic call is real; Agora, Deepgram and ElevenLabs are not involved
and no credentials for them are needed.
Three bugs these two harnesses found, all fixed here
None were reachable by testing the adapter in isolation.
main_pythonsendsparameters={"temperature": 0.7}on each request; forwarding it is a400 temperature may only be set to 1 when thinking is enabled. Theadapter now drops
temperature,top_pandtop_kon the thinkingpath and keeps them on the legacy path, where they remain valid. The
graph cannot fix this — the value is hardcoded in
main_python, notread from
property.json, and that extension is shared with the otherLLM addons.
deliver
chat_completionbeforeon_starthas built the client; timingfrom a graph run shows the cmd handled ~1 ms before
on_startloggedits first line, so the extension raised
AnthropicLLM is not initializedon a request that was merely early..env.examplebroke every request. It shippedANTHROPIC_BASE_URL=with no value; the SDK falls back to that variablewhen
base_urlis unset, and a variable that is set but empty is notNone, so the base URL became""and requests failed with a bareConnection error.Checked during development, not shipped
the real Anthropic SDK behind
httpx.MockTransportand asserting theexact HTTP body while parsing genuine SSE into real event types.
tool-call round trip each, with
effort: "xhigh"set to exercise theclamp. The
fallbacksandxhighgates were written in response toreal 400s from that run, not from the docs.
black --line-length 80clean; pylint 10.00/10 againsttools/pylint/.pylintrc.End to end through the playground, with real speech
The full voice graph, driven by Chrome's fake microphone fed a WAV of
spoken English. Microphone -> Agora RTC -> Deepgram ->
main_control->this extension -> Anthropic ->
main_control->message_collector->Agora data stream -> the browser's own rendering. No mocks anywhere in
that path.
What the dashboard showed:
and the sentence split the graph performed on the stream:
That the segmentation is clean is itself the check on delta accumulation:
main_python.parse_sentenceswould produce duplicated or truncatedfragments if the deltas carried increments rather than the text so far.
This run covers one model, one question, no tools. Everything harder —
the merged
tool_resultturn, thinking blocks, the capability gates, thesampling-parameter drop — is covered by the eight runtime tests above,
which need only one credential and can run in CI. The two are
complementary; this one proves the pipeline joins up on real data.
Unrelated nit, not changed here
docs/ai/L1/05_workflows.mdlistsopenai_llm2_pythonagainstAsyncLLMBaseExtension; it actually usesAsyncLLM2BaseExtension. Left alone to keep this PR focused — happy to fix separately.