Skip to content

feat: add anthropic claude llm2 extension - #2249

Open
leepokai wants to merge 8 commits into
TEN-framework:mainfrom
leepokai:feat/anthropic-llm2-extension
Open

feat: add anthropic claude llm2 extension#2249
leepokai wants to merge 8 commits into
TEN-framework:mainfrom
leepokai:feat/anthropic-llm2-extension

Conversation

@leepokai

@leepokai leepokai commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

What

Adds anthropic_llm2_python, a native Anthropic Messages API extension.

No extension currently targets Claude. bedrock_llm_python is an Amazon Nova extension that happens to use Bedrock's model-agnostic Converse API, so overriding its model property could reach Claude — but that path is undocumented, defaults to Nova, and cannot express adaptive thinking or effort.

The extension conforms to the existing llm-interface.json contract, so no change to ten_ai_base is required — the whole change lands in this repo.

Shape of the change

13 new files (the extension, its tests/, and a graph variant), plus a
one-line fix to ai_agents/.env.example and one line in the code map. No
existing logic is modified: the seven graphs already in the
voice-assistant property.json are untouched, and the new
voice_assistant_anthropic is appended alongside them following the same
copy-one-graph-swap-one-node pattern they use.

Translation rules

The two contracts disagree in a few places, and the adapter absorbs all of them:

  • the system prompt is a top-level parameter, not a message
  • tool arguments are a JSON string on the TEN side, a parsed object on the Anthropic side
  • parallel tool_use and tool_result blocks are merged into single turns, since Anthropic expects one turn per group rather than one message per call
  • ImageURL.url carries both remote URLs and inline base64 data URIs, so the source type is chosen per value
  • native thinking blocks map onto the existing reasoning events, so no think-tag parsing is needed

Model compatibility

Three capability gates are applied, because the feature windows do not line up:

Model Reasoning effort Refusal fallback
claude-opus-5 (default), claude-fable-5 yes lowmax yes
claude-sonnet-5, claude-opus-4-8, claude-opus-4-7 yes lowmax no
claude-opus-4-6, claude-sonnet-4-6 yes xhighhigh no
claude-haiku-4-5, claude-sonnet-4-5, claude-opus-4-5 no ignored no
  • Adaptive thinking and effort are omitted for older served models that reject them, notably Haiku 4.5.
  • fallbacks is sent only where accepted — Sonnet 5 and Opus 4.8 return '<model>' does not support the 'fallbacks' parameter.
  • effort: "xhigh" is clamped to high on Opus 4.6 and Sonnet 4.6 so a misconfigured graph degrades instead of failing every turn.

A model newer than this extension takes the current-generation path by default, so new releases work without a code change.

Defaults chosen for voice graphs

  • effort: "low" — these graphs are realtime voice assistants where turn latency dominates. Configurable.
  • thinking_display: "summarized" — the API default is omitted, which streams thinking blocks with empty text, leaving a reasoning pane blank with no error.
  • max_tokens: 2048 — on current models max_tokens caps thinking and response text together, so 512 would truncate mid-answer.
  • Refusals return HTTP 200 with empty content. For a voice agent that is silence, indistinguishable from a network fault, so the request falls back server-side where supported and otherwise speaks refusal_message.

Sampling parameters are deliberately absent: current models reject temperature, top_p and top_k. effort replaces them.

Verification

Shipped tests — inside the TEN runtime, against the real API

tests/ runs the extension through the runtime rather than calling the
adapter directly, so it covers addon registration, the
llm-interface.json import, property injection, and the
chat_completion cmd dispatched by AsyncLLM2BaseExtension. Results are
consumed exactly as main_cascade_python consumes them.

$ task test-extension \
    EXTENSION=agents/ten_packages/extension/anthropic_llm2_python

collected 8 items
test_extension_loads_and_dispatches                PASSED
test_streaming_text                                PASSED
test_graph_supplied_sampling_parameters            PASSED
test_legacy_model_keeps_sampling_parameters        PASSED
test_reasoning_reaches_the_graph                   PASSED
test_thinking_display_omitted_suppresses_reasoning PASSED
test_tool_call_round_trip                          PASSED
test_legacy_model_path                             PASSED

====================== 8 passed in 27.06s ======================

These are the only checks that exercise a request going in and a response
coming out: streamed deltas accumulating into the final text, thinking
blocks arriving as reasoning events and disappearing when
thinking_display is omitted, a two-turn tool call whose merged
tool_result turn the real API accepts, and a legacy model taking the
degraded path without a 400.

test_extension_loads_and_dispatches needs no credentials — it uses an
invalid key and asserts the failure comes back from the API, which can
only happen if the whole path worked. The rest skip unless
ANTHROPIC_API_KEY is set.

Through the agent loop, with the transport, STT and TTS mocked

A second harness builds a graph of
ten:test_extension -> main_control -> llm -> message_collector using
set_test_mode_graph, injects an asr_result, and reads back the chunks
message_collector would have put on the wire to the browser. Only the
Anthropic call is real; Agora, Deepgram and ElevenLabs are not involved
and no credentials for them are needed.

>>> asr_result injected: "In one short sentence, what is the capital of Japan?"

UI <- {"data_type":"transcribe","role":"user",      "text":"In one short sentence, what is the capital of Japan?","is_final":true}
UI <- {"data_type":"transcribe","role":"assistant", "text":"The",                           "is_final":false}
UI <- {"data_type":"transcribe","role":"assistant", "text":"The capital of Japan is Tokyo.","is_final":false}
UI <- {"data_type":"transcribe","role":"assistant", "text":"The capital of Japan is Tokyo.","is_final":true}

Three bugs these two harnesses found, all fixed here

None were reachable by testing the adapter in isolation.

  • Every turn of a real voice graph failed. main_python sends
    parameters={"temperature": 0.7} on each request; forwarding it is a
    400 temperature may only be set to 1 when thinking is enabled. The
    adapter now drops temperature, top_p and top_k on the thinking
    path and keeps them on the legacy path, where they remain valid. The
    graph cannot fix this — the value is hardcoded in main_python, not
    read from property.json, and that extension is shared with the other
    LLM addons.
  • A request arriving during startup was rejected. The runtime can
    deliver chat_completion before on_start has built the client; timing
    from a graph run shows the cmd handled ~1 ms before on_start logged
    its first line, so the extension raised
    AnthropicLLM is not initialized on a request that was merely early.
  • A copied .env.example broke every request. It shipped
    ANTHROPIC_BASE_URL= with no value; the SDK falls back to that variable
    when base_url is unset, and a variable that is set but empty is not
    None, so the base URL became "" and requests failed with a bare
    Connection error.

Checked during development, not shipped

  • 109 offline checks — one suite driving the adapter directly, one running
    the real Anthropic SDK behind httpx.MockTransport and asserting the
    exact HTTP body while parsing genuine SSE into real event types.
  • A live matrix over all 10 served models above: a plain turn plus a
    tool-call round trip each, with effort: "xhigh" set to exercise the
    clamp. The fallbacks and xhigh gates were written in response to
    real 400s from that run, not from the docs.
  • black --line-length 80 clean; pylint 10.00/10 against
    tools/pylint/.pylintrc.

End to end through the playground, with real speech

The full voice graph, driven by Chrome's fake microphone fed a WAV of
spoken English. Microphone -> Agora RTC -> Deepgram -> main_control ->
this extension -> Anthropic -> main_control -> message_collector ->
Agora data stream -> the browser's own rendering. No mocks anywhere in
that path.

What the dashboard showed:

TEN Agent connected. How can I help you today?     (greeting)
Hello.                                              (spoken, transcribed)
In two short sentences,
explain what the ten framework is used for.
The ten frame (or "ten framework") is a maths teaching tool - a grid of
ten spaces, usually two rows of five - used to help learners see and
build numbers up to ten. It supports understanding of number bonds,
subitising, place value, and early addition and subtraction strategies
like "making ten."

and the sentence split the graph performed on the stream:

Sent to TTS: is_final=False, text=The ten frame ... a grid of ten spaces,
Sent to TTS: is_final=False, text= usually two rows of five ... up to ten.
Sent to TTS: is_final=False, text= It supports understanding of number bonds,

That the segmentation is clean is itself the check on delta accumulation:
main_python.parse_sentences would produce duplicated or truncated
fragments if the deltas carried increments rather than the text so far.

This run covers one model, one question, no tools. Everything harder —
the merged tool_result turn, thinking blocks, the capability gates, the
sampling-parameter drop — is covered by the eight runtime tests above,
which need only one credential and can run in CI. The two are
complementary; this one proves the pipeline joins up on real data.

Unrelated nit, not changed here

docs/ai/L1/05_workflows.md lists openai_llm2_python against AsyncLLMBaseExtension; it actually uses AsyncLLM2BaseExtension. Left alone to keep this PR focused — happy to fix separately.

@leepokai
leepokai force-pushed the feat/anthropic-llm2-extension branch from 449f210 to 8a64e81 Compare July 27, 2026 09:52
leepokai added 2 commits July 27, 2026 17:55
Add a native Anthropic Messages API extension.

No extension previously targeted Claude. bedrock_llm_python is an Amazon
Nova extension that happens to use Bedrock's model-agnostic Converse API,
so overriding its model property could reach Claude, but that path is
undocumented, defaults to Nova, and cannot express adaptive thinking or
effort.

The extension conforms to the existing llm-interface.json contract, so no
change to ten_ai_base is required.

Translation rules between the two contracts:

- the system prompt is a top-level parameter, not a message
- tool arguments are a JSON string on one side, a parsed object on the other
- parallel tool_use and tool_result blocks are merged into single turns,
  since Anthropic expects one turn per group rather than one per call
- image URLs carry both remote URLs and inline base64 data URIs
- native thinking blocks map onto the existing reasoning events, so no
  think-tag parsing is needed

Capability gating keeps the extension usable across the whole served model
family, whose feature windows do not line up:

- adaptive thinking and effort are omitted for older models that reject
  them, including Haiku 4.5
- the fallbacks parameter is sent only to models that accept it; Sonnet 5
  and Opus 4.8 reject it outright
- effort xhigh is clamped to high on Opus 4.6 and Sonnet 4.6, so a
  misconfigured graph degrades instead of failing every turn

Sampling parameters are deliberately absent, as current models reject
temperature, top_p and top_k; effort replaces them.
@leepokai
leepokai force-pushed the feat/anthropic-llm2-extension branch from 8a64e81 to 5e62f89 Compare July 27, 2026 09:56
leepokai added 6 commits July 28, 2026 15:41
When `base_url` is unset the SDK falls back to the ANTHROPIC_BASE_URL
environment variable, and a variable that is set but empty is not None --
so the base URL became "" and every request failed with a bare
"Connection error" from httpx, with nothing pointing at the cause.

An untouched copy of .env.example produced exactly that, since it shipped
`ANTHROPIC_BASE_URL=` with no value. Give it the real default, matching
OPENAI_API_BASE and GROK_API_BASE, and pass the default explicitly so the
extension no longer depends on the variable being absent rather than empty.
…letion

The runtime can deliver a cmd before on_start has built the client. Timing
from a graph run shows chat_completion handled ~1ms before on_start logged
its first line, which made on_call_chat_completion raise
"AnthropicLLM is not initialized" and fail a request that was merely early.

Gate the generator on an event set in on_start's finally block, so a
request that races startup waits for it instead. The failure paths still
report their real reason rather than timing out, and a client that never
initializes gives up after ten seconds.
Drives the extension through the TEN runtime rather than calling the
adapter directly, so it covers what a unit test cannot reach: addon
registration, the llm-interface.json import, property injection, and the
chat_completion cmd dispatched by AsyncLLM2BaseExtension. Results are
consumed exactly as main_cascade_python consumes them.

test_extension_loads_and_dispatches needs no credentials -- it uses an
invalid key and asserts the failure comes back from the API, which can
only happen if the whole path worked. The rest call Anthropic for real and
skip unless ANTHROPIC_API_KEY is set, covering streaming, reasoning
events, the thinking_display toggle, a two-turn tool call round trip, and
the legacy-model path.

Both fixes in the preceding commits were found by running these.
Without this the extension cannot be selected from the playground, and
a graph has to be hand-edited to try it. Follows the pattern the file
already uses for the other seven variants: copy voice_assistant, swap one
node, leave the existing graphs untouched.
`main_python` sends `parameters={"temperature": 0.7}` on every turn, and
forwarding it verbatim returns

    400 `temperature` may only be set to 1 when thinking is enabled or in
    adaptive mode

so every turn of a real voice graph failed. `top_p` and `top_k` are
rejected outright on the same models.

Drop those three on the thinking path and keep them on the legacy path,
where they are still valid. The graph cannot be the place to fix this --
the value is hardcoded in main_python rather than read from property.json,
and that extension is shared with the other LLM addons.

Found by driving asr_result through main_control into this extension with
the transport, STT and TTS mocked out; both directions are covered by new
tests.
- lock the new path dependency. manifest.json listed
  anthropic_llm2_python but manifest-lock.json had no entry, so an install
  driven from the lock would not resolve the addon. Added the single entry
  rather than regenerating, to keep the other 50 packages at their pinned
  versions.
- survive a bad property. An exception escaping on_start reaches the
  runtime's _exit_on_exception, which calls os._exit -- so one invalid
  effort level took down the whole worker instead of this node.
- clamp `max` as well as `xhigh` on Opus 4.6 and Sonnet 4.6. `max` ranks
  above `xhigh`, so a model that predates one cannot accept the other.
- ignore a caller-supplied `LLMRequest.model`, matching
  openai_llm2_python. vision_analyze_tool_python hardcodes "gpt-4o", which
  would have gone to Anthropic as a 404.
- only use the beta messages resource when refusal fallback is actually
  attached, so `refusal_fallback: false` stays on /v1/messages as the
  README promises.
- close the Anthropic and httpx clients on stop; a worker cycles graphs
  and was leaking a connection pool each time.
- drop the dead claude-opus-4-0 and claude-sonnet-4-0 aliases: both are
  past end-of-life and return not_found whether gated or not.
- hold a reference to the test watchdog task so it cannot be collected
  mid-sleep, and cancel it once the request completes.
@leepokai

Copy link
Copy Markdown
Contributor Author
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant