Skip to content

feat(core): enable OpenRouter Claude prompt caching - #159

Open
dexhunter wants to merge 1 commit into
magnitudedev:mainfrom
dexhunter:perf/openrouter-claude-prompt-caching
Open

feat(core): enable OpenRouter Claude prompt caching#159
dexhunter wants to merge 1 commit into
magnitudedev:mainfrom
dexhunter:perf/openrouter-claude-prompt-caching

Conversation

@dexhunter

Copy link
Copy Markdown

Summary

  • enable prompt caching by default for Claude models on the official OpenRouter API endpoint, with promptCaching: false as an opt-out
  • forward BAML cache_control role metadata without changing prompt content or the existing four-breakpoint limit
  • report OpenRouter uncached, cache-read, and cache-write input tokens while preserving native Anthropic accounting and cumulative fallbacks

Fixes #80.

Performance

A deterministic five-turn prompt-cache harness reduced modeled uncached prompt tokens from 11,250 to 4,000 (64.4%). It verifies identical prompt content and order, cache-breakpoint limits, non-Claude and non-OpenRouter controls, endpoint spoofing controls, malformed usage clamping, native Anthropic behavior, and multi-model opt-out ordering before reporting the metric.

The direction was found by running autoresearch with Weco, then rebuilt as a focused patch and validated independently: public trajectory.

Validation

  • bun test packages/magnitude-core/src --timeout 15000 (46 passed)
  • cd packages/magnitude-core && bun run check
  • bun run build
  • strict evaluator: prompt_cache_uncached_input_tokens=4000
  • git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable automatic prompt caching for Anthropic models on OpenRouter

1 participant