Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 103 additions & 2 deletions doppelganger-api-detector/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,35 @@ doppelgangerApiDetector {
// excludePaths.add('/actuator/health')
// excludeFiles.from('doppelganger-exclusions.yaml')
// excludeWellKnown.add('spring-boot-actuator')

// Configuration for the separate `scanContracts` task (see "Contract Response Coverage"
// below) - reuses controllerDirs, testDirs, rootDocument, contractsDir, the useXxx source
// flags, and the exclude* properties above.

// Whether scanContracts additionally computes, for every declared response code, how many
// contract tests cover it. The more expensive of the two statistics it can report.
// Default: false
includeResponseCoverage = false

// Name of scanContracts' own generated AsciiDoc report file, written to the same reportDir.
// Default: 'contract-coverage.adoc'
// scanContractsReportFileName = 'contract-coverage.adoc'

// Persists a per-(endpoint, response code) history of contract test coverage over time.
// Only meaningful together with includeResponseCoverage - see "Contract Response Coverage".
// Default: false
trackResponseCoverageHistory = false

// NDJSON file the response coverage history is read from/written to. Defaults into the
// project directory (not build/), since it's meant to be committed. A separate file from
// contractHistoryFile - response coverage is a Doppelganger-only concern.
// Default: file('doppelganger-api-detector-response-coverage-history.ndjson')
// responseCoverageHistoryFile = file('doppelganger-api-detector-response-coverage-history.ndjson')

// When true, responseCoverageHistoryFile is written back to disk. Only consulted when
// trackResponseCoverageHistory is true; the file is always read either way.
// Default: same as trackResponseCoverageHistory
// updateResponseCoverageHistory = true
}
----

Expand All @@ -290,6 +319,12 @@ doppelgangerApiDetector {
// excludePaths.add("/actuator/health")
// excludeFiles.from("doppelganger-exclusions.yaml")
// excludeWellKnown.add("spring-boot-actuator")

includeResponseCoverage.set(false)
// scanContractsReportFileName.set("contract-coverage.adoc")
trackResponseCoverageHistory.set(false)
// responseCoverageHistoryFile.set(file("doppelganger-api-detector-response-coverage-history.ndjson"))
// updateResponseCoverageHistory.set(true)
}
----

Expand All @@ -301,15 +336,17 @@ doppelgangerApiDetector {
|===
| Task name | Group | Description
| `detectDoppelgangerApis` | verification | Scans OpenAPI documentation, `@RestController` implementations, and test-level verification evidence, and reports endpoints that are declared and implemented but never verified against their contract.
| `scanContracts` | verification | Scans the same candidate endpoints, but reports response-code and contract-test *coverage* rather than a pass/fail verdict - see "Contract Response Coverage" below. Never fails the build on its own initiative.
|===

The task is **not** wired into `check` or `build` automatically. Teams generating their OpenAPI documentation from code, or migrating their tests onto one of the supported verification mechanisms gradually, would otherwise see every build fail immediately.
Neither task is wired into `check` or `build` automatically. Teams generating their OpenAPI documentation from code, or migrating their tests onto one of the supported verification mechanisms gradually, would otherwise see every build fail immediately.

Running the report explicitly:
Running the reports explicitly:

[source,shell]
----
./gradlew detectDoppelgangerApis
./gradlew scanContracts
----

To make the task part of every build, wire it into `check` (or `build`) yourself once you're ready to enforce it:
Expand Down Expand Up @@ -398,6 +435,70 @@ If a run is skipped for some other reason and you need to force it regardless (e
./gradlew detectDoppelgangerApis --rerun
----

== Contract Response Coverage (`scanContracts`)

NOTE: For a complete field-by-field reference to `responseCoverageHistoryFile` - its exact JSON schema, fingerprint algorithm, and lifecycle semantics - see link:docs/response-coverage-history-file-format.adoc[Response Coverage History File Format].

`detectDoppelgangerApis` answers a binary question per endpoint: does it have *at least one* contract test at all? That collapses two more useful, more granular signals into a single yes/no. `scanContracts` separates them out, for every endpoint both declared in the OpenAPI documentation and implemented by a `@RestController` method - the same candidate set `detectDoppelgangerApis` computes:

* *Declared response codes* - how many distinct response codes (`200`, `400`, `401`, `403`, `404`, `422`, `500`, `503`, ...) the OpenAPI operation actually documents.
* *Contract test count* - how many distinct contract tests exist for the endpoint, across every enabled verification source (Spring RestDocs, the Atlassian OpenAPI request validator, Spring Cloud Contract) - reusing the same three sources `detectDoppelgangerApis` uses.

Setting `includeResponseCoverage = true` adds a third, more expensive statistic: for every declared response code, how many of those contract tests were detected to actually assert it - including a response code declared but covered by *zero* tests, so a coverage gap is visible at a glance. This is not merely hidden when disabled; it is never computed, since detecting each test's asserted status code costs more than simply counting tests.

=== A worked example

An endpoint `GET /v1/foobars` declares two response codes, `200` and `404`. Two contract tests assert a `200` response and one asserts a `404`. With `includeResponseCoverage = true`, `scanContracts` reports: `200` covered by 2 test(s), `404` covered by 1 test(s).

[source,groovy]
----
doppelgangerApiDetector {
rootDocument = file('src/main/resources/openapi/openapi.yaml')
includeResponseCoverage = true
}
----

[source,shell]
----
./gradlew scanContracts
----

=== How a test's status code is detected

A test's asserted status code is detected best-effort from the same call-chain shapes each verification source already recognises:

[cols="1,3",options="header"]
|===
| Source | Status code shapes recognised
| Spring RestDocs | MockMvc `status().isOk()` / `status().isNotFound()` / ... (the well-known `StatusResultMatchers` method names) or the numeric `status().is(404)`; WebTestClient `expectStatus().isOk()` or the numeric `expectStatus().isEqualTo(404)`.
| Atlassian OpenAPI request validator | REST Assured `.then().statusCode(404)`.
| Spring Cloud Contract | The contract's own `response { status 200 }` (Groovy) or `response: status: 200` (YAML) block.
|===

A test whose status code can't be detected this way - asserted through a variable, a helper method, or a custom matcher - still counts towards the endpoint's overall contract test count; it simply contributes no evidence to any specific response code's count, and is called out in the report so the data is never silently lossy.

[NOTE]
====
A detected status code is matched against a declared response code by *exact string equality only*: a test asserting `404` counts towards a declared `"404"` response, never towards a declared `"4XX"` range wildcard or a `"default"` response - even though either might, in the OpenAPI specification's own semantics, legitimately cover that same test. This is the same class of accepted heuristic imprecision documented elsewhere in this plugin family (e.g. how a Spring Cloud Contract example path is matched against a template).
====

=== Tracking response coverage history

Setting `trackResponseCoverageHistory = true` persists, across builds, a history of contract test coverage per endpoint and response code - keyed by a fingerprint of the endpoint's verb, path, and response code, tracking a live test-count gauge rather than one-time milestone timestamps. It is only meaningful together with `includeResponseCoverage`: `scanContracts` fails eagerly if `trackResponseCoverageHistory` is `true` while `includeResponseCoverage` is `false`, since there would be no per-response-code data to persist.

Like `contractHistoryFile`, the plugin itself has no dependency on git or any other version control system - branch-based control over when to advance the history is entirely a CI concern, expressed through `updateResponseCoverageHistory` (overridable for the whole build via the `-PdoppelgangerApiDetector.updateResponseCoverageHistory` project property, independently of `-PdoppelgangerApiDetector.updateContractHistory`).

`responseCoverageHistoryFile` is a separate NDJSON file from `contractHistoryFile` - deliberately not folded into the shared cross-plugin schema Shadow and Mirage API Detector also read, since response coverage is a Doppelganger-only concern with its own record shape:

[source,json]
----
{"schemaVersion":1}
{"fingerprint":"a1f3c9d0e21b7f44-200","verb":"GET","path":"/v1/foobars","responseCode":"200","testCount":2,"firstDeclaredAt":"2026-01-14T09:02:11Z","firstCoveredAt":"2026-01-20T11:15:44Z","lastSeenAt":"2026-08-12T07:00:00Z","removedAt":null}
{"fingerprint":"a1f3c9d0e21b7f44-404","verb":"GET","path":"/v1/foobars","responseCode":"404","testCount":1,"firstDeclaredAt":"2026-01-14T09:02:11Z","firstCoveredAt":"2026-02-01T08:30:00Z","lastSeenAt":"2026-08-12T07:00:00Z","removedAt":null}
----

The generated report reflects the loaded (and possibly just-advanced) history as a `== Response Coverage Over Time` section, once `trackResponseCoverageHistory` finds at least one record.

== System Under Test Version

Every generated report includes a line just below the title identifying the version of the system under test that was scanned:
Expand Down
2 changes: 2 additions & 0 deletions doppelganger-api-detector/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ jacocoTestReport {
'com/arc_e_tect/gradle/doppelganger/DoppelgangerApiDetectorExtension.class',
'com/arc_e_tect/gradle/doppelganger/DoppelgangerApiDetectorPlugin.class',
'com/arc_e_tect/gradle/doppelganger/DetectDoppelgangerApisTask.class',
'com/arc_e_tect/gradle/doppelganger/ScanContractsTask.class',
])
}))
}
Expand All @@ -110,6 +111,7 @@ jacocoTestCoverageVerification {
'com/arc_e_tect/gradle/doppelganger/DoppelgangerApiDetectorExtension.class',
'com/arc_e_tect/gradle/doppelganger/DoppelgangerApiDetectorPlugin.class',
'com/arc_e_tect/gradle/doppelganger/DetectDoppelgangerApisTask.class',
'com/arc_e_tect/gradle/doppelganger/ScanContractsTask.class',
])
}))
}
Expand Down
Loading
Loading