test(e2e): enable policy guardrail suite with two-deploy form-policy flow#1582
Merged
Conversation
…flow Re-enable the policy engine e2e suite (previously gated off via SUITE_DISABLED while the AWS::BedrockAgentCore::Policy CFN type was pre-GA) and restructure it to the validated guardrail flow. The suite is now gated only by prereqs + AWS creds via .skipIf(!canRun), matching every other e2e suite. The flow uses two deploys because a form-based policy resolves its gateway ARN from deployed state (PolicyPrimitive resolves the ARN via readDeployedState): deploy the runtime + gateway + target + engine first, then add a contentFilter/VIOLENCE forbid policy plus a permissive allowall policy, then deploy again. Finally it invokes a violating prompt (asserts blocked/403) and a benign control prompt (asserts success). Constraint: form-based policies need the gateway ARN from deployed state, forcing the add-policy step after the first deploy Rejected: single inline forbid --statement policy | does not exercise the form-category path or prove benign traffic still passes Confidence: high Scope-risk: narrow Directive: keep the two-deploy ordering — adding the form policy before deploy #1 leaves the gateway ARN unresolved Not-tested: full live e2e run against AWS (requires the Policy CFN type live in-region; suite verified to compile, collect all 9 steps, and lint clean)
jesseturner21
had a problem deploying
to
e2e-testing
GitHub Actions
Failure
agentcore-devx-automation
Bot
added
the
claude-security-reviewing
github-actions
Bot
added
agentcore-harness-reviewing
Contributor
|
Claude Security Review: no high-confidence findings. (run) |
agentcore-devx-automation
Bot
removed
the
claude-security-reviewing
Contributor
Package TarballHow to installgh release download pr-1582-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.20.1.tgz |
Contributor
Coverage Report
|
The previous block assertion matched /denied|policy|403/i, which the CI
IAM authorization 403 ("not authorized to perform
bedrock-agentcore:InvokeGateway") satisfied — a false positive where the
invoke never reached the policy engine. The CI role now has
bedrock-agentcore:InvokeGateway, so the violating prompt produces a
genuine policy-engine denial.
Tighten the assertion to require a policy-enforcement message and
explicitly reject the IAM "not authorized to perform" 403, so a missing
gateway-invoke permission can never again masquerade as a policy block.
Constraint: CI role must hold bedrock-agentcore:InvokeGateway (added to the e2e-github-actions AgentInvocation statement) or no gateway invoke can reach the policy engine
Rejected: broaden regex to also accept 403 | that is exactly the false positive being removed
Confidence: high
Scope-risk: narrow
Directive: do not re-add bare 403/"not authorized" to the block regex — it masks IAM failures as policy denials
Not-tested: full live e2e (validated in CI re-run after this push)
jesseturner21
temporarily deployed
to
e2e-testing
GitHub Actions
Inactive
agentcore-devx-automation
Bot
added
the
claude-security-reviewing
Contributor
|
Claude Security Review: no high-confidence findings. (run) |
agentcore-devx-automation
Bot
removed
the
claude-security-reviewing
avi-alpert
approved these changes
Jun 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Re-enables the policy engine end-to-end suite (
e2e-tests/guardrail-block.test.ts) and restructures it to match the validated guardrail flow.Previously the suite was gated off via a
SUITE_DISABLED = trueflag while theAWS::BedrockAgentCore::PolicyCFN resource type was pre-GA. This PR removes that flag so the suite is gated only by prereqs + AWS creds (.skipIf(!canRun)), exactly like every other e2e suite.Why the flow changed
A form-based policy (
--form-category contentFilter ...) resolves its gateway ARN from deployed state —PolicyPrimitivereads the ARN viareadDeployedState(). So the gateway must be deployed before the policy is added. The suite now uses a two-deploy ordering:contentFilter/VIOLENCEforbid policy scoped to the deployed gateway/targetpermit (principal, action, resource is AgentCore::Gateway);)"i will kill you") → assert blocked (success=false, error matches/denied|policy|403|blockviolence/i)"hello") → assert successThe benign control case is new — it proves the engine blocks only violating content, not all traffic.
Verification
vitest list --project e2e)tsc --noEmit) passed