Feat: Add prompt caching to bedrock models

[sQu4rks]

Problem

In the current implementation of the Agent block in bb-agent, the block does not support using prompt caching for bedrock-based models.

Issue #, if available: #269

Changes

Adds an optional cacheConfig parameter that mirrors the configuration from strands SDK.

const agent = new Agent(scope, 'support', {
	model: {
		deployed: { ...BedrockModels.BALANCED, cacheConfig: { strategy: 'auto' } },
		local: { provider: 'canned' },
	},
	systemPrompt: // ...
});

Validation

Added two unit tests to verify passing behavior.

Additionally, tested by writing a simple chat application that uses the below snippet

const agent = new Agent(scope, 'support', {
  model: {
    deployed: { ...BedrockModels.BALANCED, cacheConfig: { strategy: 'auto' } },
    local: { provider: 'canned' }, // no AWS/LLM needed for local dev
  },
  systemPrompt: // very long system prompt
  tools: // tools definition
});

to create a chat agent. Enabled invocation logs on Amazon Bedrock and the logs show that the first turn writes to the cache

"usage": {
  "inputTokens": 3,
  "cacheReadInputTokens": 0,
  "cacheWriteInputTokens": 3317,
  "outputTokens": 377,
  "totalTokens": 3697
}

and the subsequent turns read from it

"usage": {
  "inputTokens": 3,
  "cacheReadInputTokens": 3317,
  "cacheWriteInputTokens": 406,
  "outputTokens": 530,
  "totalTokens": 4256
}

Checklist

• PR description included
• Tests are changed or added
• Relevant documentation is changed or added (and PR referenced)

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[changeset-bot]

🦋 Changeset detected

Latest commit: 972da1b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@aws-blocks/bb-agent	Minor
@aws-blocks/blocks	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[Simone319]

What happen if the strategy is 'anthropic' but the model id is not anthropic? Shall we place a check on modelId for this strategy?

[sQu4rks]

Thanks for the comments @Simone319

I believe the strategy: anthropic is specifically designed for cases where - from the ARN - it's unclear that the underlying model is an anthropic model. The ARN could look something like this arn:aws:bedrock:{region}:{account-number}:inference-profile/my-team-profile (docs) so we'd need another API call (and accompanying permissions) to retrieve the underlying model (via GetInferenceProfile (api docs) to then parse the models.

I also checked the case quickly via a script (see below). Since cacheConfig is passed to the Strands BedrockModel it's behavior is entirely strands.

import { Agent } from '@strands-agents/sdk';
import { createStrandsModel } from '../../dist/model-factory.js';

process.env.AWS_REGION ??= 'us-east-1';
const MODEL_ID = process.env.MODEL_ID ?? 'openai.gpt-oss-120b-1:0'; // a non-Anthropic model

// Long, stable prefix so we clear the ~1,024-token cache minimum.
const SYSTEM_PROMPT = [
  'You are "Atlas", a meticulous senior support engineer for the ACME Cloud platform.',
  'Follow these operating rules on every single response, without exception:',
  ...Array.from({ length: 40 }, (_, i) =>
    `Rule ${i + 1}: Be precise, cite the relevant ACME Cloud service by name, prefer the ` +
    'least-privilege remediation, never invent API names, and always end with a one-line summary.'),
].join('\n');

const TURNS = [
  'A user reports 503s from EdgeCDN after a deploy. First triage step?',
  'Now they also see elevated latency on ObjectStore. What next?',
];

const usageOf = (r) => {
  const u = r.metrics?.accumulatedUsage ?? {};
  return { input: u.inputTokens ?? 0, output: u.outputTokens ?? 0, cacheWrite: u.cacheWriteInputTokens ?? 0, cacheRead: u.cacheReadInputTokens ?? 0 };
};

async function runScenario(label, cacheConfig) {
  console.log(`\n--- ${label} ---`);
  try {
    const model = await createStrandsModel({ provider: 'bedrock', modelId: MODEL_ID, cacheConfig });
    const agent = new Agent({ model, systemPrompt: SYSTEM_PROMPT, printer: false });
    for (let i = 0; i < TURNS.length; i++) {
      const u = usageOf(await agent.invoke(TURNS[i]));
      console.log(`  turn ${i + 1}: input=${u.input} output=${u.output} cacheWrite=${u.cacheWrite} cacheRead=${u.cacheRead}`);
    }
  } catch (err) {
    console.log(`  ERROR: ${err?.name ?? 'Error'}: ${err?.message ?? err}`);
  }
}

// 'auto'      → Strands warns and disables caching for a non-Anthropic model
await runScenario("strategy: 'auto'", { strategy: 'auto' });
// 'anthropic' → forces Anthropic-style cache points onto a non-Anthropic model
await runScenario("strategy: 'anthropic'", { strategy: 'anthropic' });

With strategy: 'anthropic' on a non-Anthropic model, Strands forwards the Anthropic-style cache points and Bedrock rejects the request with a clear ModelError ("You invoked an unsupported model or your request did not allow prompt caching").

--- strategy: 'auto' ---
model_id=<openai.gpt-oss-120b-1:0> | cache_config is enabled but this model does not support automatic caching
  turn 1: input=1679 output=533 cacheWrite=0 cacheRead=0
  turn 2: input=3375 output=1558 cacheWrite=0 cacheRead=0

--- strategy: 'anthropic' ---
  ERROR: ModelError: You invoked an unsupported model or your request did not allow prompt caching. See the documentation for more information.

So my suggestion would be to handle the warnings/errors but let Bedrock do the verification.