RAGContract4j

Contract testing for Java RAG systems.

RAGContract4j turns retrieval, authorization, citation, refusal, and robustness requirements into deterministic Maven and GitHub Actions quality gates.

中文定位：面向 Java RAG 系统的契约测试框架，将召回、权限、引用、拒答和鲁棒性要求接入 Maven 与 GitHub Actions。

Project status

The first v0.1 source and scope gate is implemented. A YAML contract and JSON trace produce structured violations, JSON/Markdown reports and a JUnit assertion. Required/forbidden sources, selected-context rank and authorization scopes are checked without a model, vector store or network access.

Spring AI trace capture, citation/claim evaluation and refusal assertions remain planned. Answer and citation data can be recorded, but is not evaluated yet. The current 0.1.0-SNAPSHOT is consumed from this source checkout; no public release is claimed.

Try the gate

Run the valid Team A sample from the repository root:

./mvnw -B -ntp verify

Then run a trace that retrieved a Team B document before filtering it out:

./mvnw -B -ntp -o -pl samples/developer-workspace -am -Drag.trace=team-a-leak.json test

The second command intentionally exits non-zero and reports:

RAG contract failed: team-a-production-db-runbook
UNAUTHORIZED_SCOPE at $.stages[0].documents[1]: document team-b/private-runbook with scope team-b is outside the contract identity

Both runs write report.json and report.md under samples/developer-workspace/target/rag-contract-report/ before the assertion. The latest run replaces these reports. Run the first command again to restore a passing sample. On Windows use .\mvnw.cmd in place of ./mvnw.

The first build may download Maven dependencies; subsequent fixture execution can run offline. The sample is a reactor module, not proof of public Maven distribution. See the sample and fixture contract.

Initial scope

rag-contract-core        trace and contract model
rag-contract-evaluator   explainable deterministic assertions
rag-contract-spring-ai   first framework adapter
rag-contract-testkit     JUnit 5 and fixture support
samples/developer-workspace
                         public/team/admin document contracts

Core values and evaluators have no parsing, model, vector-store or framework dependencies. Testkit owns YAML/JSON parsing and reports. The Spring AI module is still a placeholder; applications currently supply traces explicitly.

What a contract can express

• Implemented: required and forbidden document IDs, maximum one-based rank in selected context, allowed identity scopes and forbidden document scopes.
• Planned: answer claims, answer/refusal mode, citation support and query variants.

Unknown fields, including the planned answer expectations, fail parsing instead of being silently ignored. Scope checks inspect every recorded retrieval stage and selected context. Identities, scope labels and trace completeness come from trusted host/test-fixture code; this library does not enforce runtime access or verify the truth of supplied labels.

See docs/architecture.md and TODO.md.

Build

Requirements: Java 21. No global Maven installation is required.

./mvnw verify

On Windows:

.\mvnw.cmd verify

License

Apache License 2.0.