How the instrument and trace bedrock guardrail?

[ruqianq]

Hi I have a question about how to use openinference to parse the trace on bedrock guardrail? could you provide an example? thansk!

[dosubot[bot]]

To instrument and trace Bedrock Guardrail with OpenInference, you’ll want to use the openinference-instrumentation-bedrock package, connect to a Phoenix instance, and enable tracing on your Bedrock agent calls. Here’s a concise example:

1. Install dependencies:

   pip install openinference-instrumentation-bedrock arize-phoenix boto3

2. Connect to Phoenix and instrument boto3:

   from phoenix.otel import register
   from openinference.instrumentation.bedrock import BedrockInstrumentor
   import boto3
   import time
   
   # Register Phoenix tracer
   tracer_provider = register(project_name="my-llm-app", auto_instrument=True)
   
   # Instrument boto3 before creating the client
   BedrockInstrumentor().instrument()
   session = boto3.session.Session()
   client = session.client("bedrock-agent-runtime")

3. Invoke your Bedrock agent with tracing enabled:

   AGENT_ID = "your-agent-id"
   AGENT_ALIAS_ID = "your-agent-alias-id"
   input_text = "When is a good time to visit the Taj Mahal?"
   session_id = f"default-session1_{int(time.time())}"
   
   attributes = dict(
       inputText=input_text,
       agentId=AGENT_ID,
       agentAliasId=AGENT_ALIAS_ID,
       sessionId=session_id,
       enableTrace=True,
   )
   response = client.invoke_agent(**attributes)

4. View traces in Phoenix: All calls will be streamed to your Phoenix instance for observability and evaluation. You can analyze execution paths, tool calls, and more in the Phoenix UI
   [docs].

5. Parse tool calls from traces: To extract tool calls from trace outputs, use:

   import json
   
   def extract_tool_calls(output_value):
       tool_calls = []
       try:
           o = json.loads(output_value)
           if "content" in o and isinstance(o["content"], list):
               for item in o["content"]:
                   if isinstance(item, dict) and item.get("type") == "tool_use":
                       tool_name = item.get("name")
                       if tool_name:
                           tool_calls.append(tool_name)
       except (json.JSONDecodeError, TypeError, AttributeError):
           pass
       return tool_calls
   
   # Example: apply to a DataFrame column of outputs
   # trace_df["tool_call"] = trace_df["outputs"].apply(lambda x: extract_tool_calls(x) if isinstance(x, str) else [])

For a full notebook example, see the Amazon Bedrock Agents Tracing and Evals tutorial.

Let me know if you need more details on parsing or visualizing specific trace data!

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[Jgilhuly]

Hi @ruqianq 👋

Based on my understanding, we don't automatically trace guardrail invocations from Bedrock. I'll test and see whether we get the necessary info back from bedrock, in which case these can be added by a manual instrumentation call. Should have something for you this coming week

[mhoelzl-dt]

Any update on this? There still seems like there is no automatic tracing of guardrail invocations from Bedrock