Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Pinned multi-architecture base manifests; update through reviewed dependency maintenance.
FROM eclipse-temurin:21-jdk-alpine@sha256:cd87715a8d45cfaa42419207c64680234f62785c49055cccd20437b5c9018380 AS build
FROM eclipse-temurin:21-jdk-alpine@sha256:fd52084c35bbac78890d95d57707d31ffc9d7a3324199e89c06be2a9dc71bfa0 AS build
WORKDIR /app
COPY .mvn .mvn
COPY mvnw pom.xml ./
Expand All @@ -8,8 +8,9 @@ COPY src ./src
# CI runs the full suite (including PostgreSQL) before building the image.
RUN --mount=type=cache,target=/root/.m2 ./mvnw -B -DskipTests package

FROM eclipse-temurin:21-jre-alpine@sha256:1a29e1fe337eb28b5bec30f0ee8ed29f0ff80ab6f75dcf9313efe82911065a52 AS runtime
FROM eclipse-temurin:21-jre-alpine@sha256:b48e02e0ea88d7bfb305335df94923bdef394905f4147dccd52e873cf17b1088 AS runtime
WORKDIR /app
RUN apk add --no-cache --upgrade 'brotli-libs=1.2.1-r0'
RUN addgroup -g 10001 gateway && adduser -D -H -u 10001 -G gateway gateway
USER 10001:10001
EXPOSE 8080 9090
Expand Down
2 changes: 1 addition & 1 deletion Documentation/Architecture.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AI Inference Gateway architecture

Updated 2026-09-21. Sections explicitly marked **current** describe implemented Phase 6 code. All other sections describe the **target design**, to be implemented through [Phases.md](Phases.md). [Phase6.md](Phase6.md) records verification and open gates. [PRD.md](PRD.md) owns product scope; [API.md](API.md), [Security.md](Security.md), and [Operations.md](Operations.md) expand the contracts.
Updated 2026-10-26. Sections explicitly marked **current** describe accepted Phase 6 code. All other sections describe the **target design**, to be implemented through [Phases.md](Phases.md). [Phase6.md](Phase6.md) records acceptance evidence, re-verification and operating limits. [PRD.md](PRD.md) owns product scope; [API.md](API.md), [Security.md](Security.md), and [Operations.md](Operations.md) expand the contracts.

The product owner authorizes revising existing architecture and implementation when correctness, maintainability, sound design or future complexity warrants it. Decisions below are current design choices, not immutable constraints. Document significant replacements and their evidence using [Development.md](Development.md); keep dependent contracts, migrations and release gates aligned.

Expand Down
2 changes: 1 addition & 1 deletion Documentation/Operations.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Deployment and production readiness plan

Updated 2026-09-21. Local Compose, a non-root Dockerfile, Maven wrapper and CI build/test/scan workflow are implemented. [Phase6.md](Phase6.md) records observed verification and open gates. Helm, systemd, Podman, signed releases, HA and recovery procedures below remain **planned deliverables**. Use [README.md](../README.md) for current setup.
Updated 2026-10-26. Local Compose, a non-root Dockerfile, Maven wrapper and CI build/test/scan workflow are implemented. [Phase6.md](Phase6.md) records Phase 6 acceptance evidence, re-verification and remaining operating limits. Helm, systemd, Podman, signed releases, HA and recovery procedures below remain **planned deliverables**. Use [README.md](../README.md) for current setup.

The MVP upgrade path below is the current continuity strategy. Phase 6 may replace it with a documented transition if design/consumer/data assessment justifies a simpler architecture. Adjust the release evidence accordingly; preserve real data and supported release recovery requirements. A demo origin is not permission to reset an existing environment.

Expand Down
2 changes: 1 addition & 1 deletion Documentation/PRD.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AI Inference Gateway product requirements

Updated 2026-09-21. Status: product plan with Phase 6 foundation implemented; validation evidence and remaining gates are in [Phase6.md](Phase6.md). This document replaces the MVP-only scope. Production readiness is a release outcome to demonstrate, not a claim about the current code.
Updated 2026-10-26. Status: product plan with the Phase 6 foundation accepted and re-verified; evidence and remaining operating limits are in [Phase6.md](Phase6.md). This document replaces the MVP-only scope. Production readiness is a release outcome to demonstrate, not a claim about the current code.

## Product direction

Expand Down
29 changes: 21 additions & 8 deletions Documentation/Phase6.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Phase 6 implementation evidence

Updated 2026-09-21. **Implementation delivered; acceptance remains open for the full image dependency scan/SBOM and a clean-checkout remote CI run.** Local regression, PostgreSQL, source scan, container OS scan, packaging and startup checks passed. This is foundation evidence, not production readiness.
Updated 2026-10-26. **Implementation and Phase 6 acceptance are complete.** Local regression and PostgreSQL verification were repeated successfully on 2026-10-26. Current Trivy scans then identified five HIGH Jackson findings in the stale 3.1.5 dependency baseline and three HIGH findings in the pinned Alpine runtime. The Jackson BOM was raised to 3.1.8, the Temurin Alpine manifests were refreshed, the remaining fixed `brotli-libs` package was pinned, and the relevant verification was repeated. The original local source scan, container OS scan, packaging and startup checks passed, followed by clean-checkout GitHub Actions verification on the merged `main` revision, including the full container dependency scan and SBOM. This is foundation evidence, not production readiness.

## Baseline and design assessment

Expand All @@ -27,7 +27,7 @@ The implementation keeps the legacy request/success/error shapes; deliberate com

## Observed verification

Source base revision: `cb95950a2d78431656cb6ca4756a0b056c20bdc0`, **dirty working tree with these Phase 6 changes**. No new commit or remote workflow run was created. Logs and detailed Maven reports are local ignored build outputs; durable benchmark data is in [evidence/phase6-baseline.json](evidence/phase6-baseline.json).
The original local evidence used source base revision `cb95950a2d78431656cb6ca4756a0b056c20bdc0` with the Phase 6 changes still in a dirty working tree. Logs and detailed Maven reports are local ignored build outputs; durable benchmark data is in [evidence/phase6-baseline.json](evidence/phase6-baseline.json). The changes were subsequently committed as `775bdfe4d1627708044017cbbb3749ef61cfbb18`, merged by [pull request #1](https://github.com/616b73/AI-Inference-Gateway/pull/1), and verified from the clean merge revision `80944edb37f439dd0f171dcf70da4ef0252b5738`.

Environment: Windows 11, Java 21.0.7, Maven wrapper 3.9.11, Docker Desktop 4.90.0 / engine 29.7.2, PostgreSQL 16.15 (`postgres:16-alpine` pinned digest in CI). Runtime image: Temurin 21.0.12+8, Alpine 3.24.2, UID/GID 10001. Tests used a disposable PostgreSQL container on loopback port 15432 and random `p6_…` schemas; fixtures dropped only their own schemas. No existing database was reset.

Expand All @@ -42,11 +42,24 @@ Environment: Windows 11, Java 21.0.7, Maven wrapper 3.9.11, Docker Desktop 4.90.
| Source security scan | Trivy 0.74.0: zero high/critical dependency, secret or configuration findings after Tomcat patch |
| Runtime OS scan | Trivy 0.74.0: 73 Alpine packages checked; zero high/critical findings |
| Full runtime dependency scan / SBOM | **Not verified locally**: Java index downloads from both official registries ended with unexpected EOF |
| GitHub Actions clean-checkout verification | **Not run remotely**; workflow configured to fail on test failures or high/critical findings and retain reports |
| GitHub Actions clean-checkout verification | [Run 35695280770](https://github.com/616b73/AI-Inference-Gateway/actions/runs/35695280770) passed on merged `main` revision `80944ed`: PostgreSQL build/tests, artifact manifest, container build, dependency/secret/configuration scan, full container vulnerability scan, container SBOM and evidence preservation |

### Local re-verification (2026-10-26)

Revision `cf469207ffe4874df0a0eaa567dd1465499a15f9` plus the pending documentation and Jackson security refresh was re-verified on Windows 11 with Java 21.0.12.1, Maven 3.9.11, Docker Engine 29.8.2 and the pinned PostgreSQL 16.15 image. Maven used the existing project cache; this constrained runner supplied Mockito as a Java agent and allowed loopback networking explicitly.

- `clean test`: 89 discovered, 88 passed, one opt-in benchmark skipped, zero failures/errors.
- `-Ppostgres-it clean verify`: the same regular suite passed and all four PostgreSQL integration tests passed, covering a fresh V1–V6 install, populated V5→V6 upgrade, production startup and active-demo-credential rejection.
- The opt-in `BaselineBenchmarkTest` passed again after the dependency refresh with 320 successful synthetic loopback calls including warm-up and rewrote the ignored `target/phase6-baseline.json` result.
- The Jackson BOM override was raised from 3.1.5 to 3.1.8, clearing the five HIGH denial-of-service advisories reported by the failed GitHub source scan: CVE-2026-68497, CVE-2026-89407, CVE-2026-89425, CVE-2026-91776 and CVE-2026-91777.
- A Trivy 0.74.0 offline source scan with a freshly downloaded vulnerability database reported zero HIGH/CRITICAL vulnerabilities, secrets or misconfigurations.
- The refreshed online source gate passed. Its next container scan exposed fixed Alpine advisories CVE-2026-111497, CVE-2026-77214 and CVE-2026-93990; refreshing both Temurin Alpine manifest pins and pinning `brotli-libs` 1.2.1-r0 cleared all three in a local Trivy OS-package scan of the rebuilt image.

The offline source scan warned that some parent-managed child dependency versions could not be resolved. A complementary executable-JAR scan could not initialize Trivy's Java advisory database because the registry download ended with an unexpected EOF. Consequently, the replacement commit's online GitHub Actions run remains the authoritative full dependency and image check; the full container dependency scan and SBOM were not regenerated locally. Release qualification must repeat all security gates against then-current advisories.

Final JAR SHA-256: `ae60015186ed6b34b557ca1d0d23fc05567293efc1d719aad26459f399505aa8` (64,614,319 bytes). Final packaged image ID: `sha256:c0017a487d066808cb90721745876856d319d40ce857aa7156ac01163c6ed1fd`. These identify a local uncommitted build, not a signed release. `scripts/release_manifest.py` writes source revision/dirty state, artifact hash/size and toolchain target to `target/release-manifest.json`.

The source scan used a complete local Maven repository mounted at `/root/.m2/repository`, `--offline-scan`, a vulnerability DB updated 2026-09-21T07:13Z, and Trivy's embedded configuration checks after a policy download failed. It excluded `.git`, `.build-cache`, and generated `target` output. The independent OS scan explicitly excluded JARs; it is not a substitute for the pending full image scan. CI retains the full source/JAR/image scans and has no vulnerability suppression or `continue-on-error` bypass.
The source scan used a complete local Maven repository mounted at `/root/.m2/repository`, `--offline-scan`, a vulnerability DB updated 2026-09-21T07:13Z, and Trivy's embedded configuration checks after a policy download failed. It excluded `.git`, `.build-cache`, and generated `target` output. The independent local OS scan explicitly excluded JARs and was not treated as a substitute for the full image scan; the later clean-checkout CI run completed the full container scan and SBOM. CI retains the source/JAR/image scan evidence and has no vulnerability suppression or `continue-on-error` bypass.

### Reproduction

Expand All @@ -66,11 +79,11 @@ The benchmark used loopback HTTP, synthetic Ollama output, a 1,024-character pro

End-of-run used Java heap was 72,000,456 bytes, a snapshot rather than a peak-memory or leak measurement. Sequential gateway/direct p95 difference was approximately 59.11 ms; subtracting percentiles is only a comparison, not a paired per-request overhead distribution. The sequential gateway sample lasted approximately 5.3 seconds. This short closed workload does not validate the PRD's 100 requests/s, 200-stream, one-hour target or its ≤50 ms overhead goal. BCrypt verification and synchronous diagnostics are included; this experiment did not profile their individual contribution. CPU utilization, RSS, peak buffers and production PostgreSQL latency were not measured here.

## Remaining acceptance and operating limits
## Acceptance closure and operating limits

The two external Phase 6 gates left open by the local run were closed by [GitHub Actions run 35695280770](https://github.com/616b73/AI-Inference-Gateway/actions/runs/35695280770) on 2026-09-22. The workflow checked out merge revision `80944edb37f439dd0f171dcf70da4ef0252b5738` and passed the PostgreSQL build/test profile, dependency/secret/configuration scan, full container vulnerability scan, container SBOM and evidence-preservation steps without a bypass. Phase 6 is accepted on that evidence.

1. Run the committed workflow from a clean checkout; retain its exact revision, test reports and scan artifacts. No remote CI result is implied by local success.
2. Complete the full image dependency scan and container SBOM once the Java advisory database can be downloaded. Do not accept a partial scan or disable that gate to publish a release.
3. Recheck dependency/base-image advisories at release time. Pins provide repeatability, not permanent security; maintain them.
Dependency and base-image advisories must still be rechecked at release time. Pins provide repeatability, not permanent security; maintain them.

Immediate downstream disconnect cancellation, bounded stream writes, deployment-isolated pools, indexed identities, tenancy, durable usage/budgets, configuration refresh, recovery drills and production load/soak tests remain future phases. The current total deadline actively aborts upstream I/O but does not hard-limit all synchronous database work or downstream delivery. Local diagnostics can be dropped; they are not audit or billing records. No real Ollama/cloud-provider smoke test, full Compose model download, external consumer migration or production deployment was performed.

Expand Down
8 changes: 4 additions & 4 deletions Documentation/Phases.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AI Inference Gateway delivery phases

Updated 2026-09-21. This is the forward implementation plan, not a completion log. **Phase 6 implementation is present with local test evidence; its remaining build/security/remote-CI gates are tracked in [Phase6.md](Phase6.md). Phases 7–15 have not started.** [Roadmap.md](Roadmap.md) preserves actual history. Requirement IDs refer to [PRD.md](PRD.md); technical decisions refer to [Architecture.md](Architecture.md).
Updated 2026-10-26. This is the forward implementation plan, not a completion log. **Phase 6 is accepted and was re-verified locally on 2026-10-26; evidence is recorded in [Phase6.md](Phase6.md). Phases 7–15 have not started.** [Roadmap.md](Roadmap.md) preserves actual history. Requirement IDs refer to [PRD.md](PRD.md); technical decisions refer to [Architecture.md](Architecture.md).

## Working approach

Expand All @@ -21,7 +21,7 @@ Use small vertical slices with an owner, acceptance evidence, compatibility/migr
| 4 | Provider/log visibility endpoints | Recorded complete |
| 5 | Log filters, expanded tests and onboarding docs | Recorded complete |

The original 62-test baseline was reproduced before Phase 6 changes. The expanded suite and PostgreSQL checks now pass; [Phase6.md](Phase6.md) records counts, environment, benchmark and remaining gates. None is production certification.
The original 62-test baseline was reproduced before Phase 6 changes. The expanded suite, PostgreSQL checks, clean-checkout build, security scans and container SBOM passed; [Phase6.md](Phase6.md) records counts, environment, benchmark and CI evidence. None is production certification.

## Dependencies and releases

Expand Down Expand Up @@ -55,7 +55,7 @@ The graph describes **completion dependencies**. Observability, CI, operations,

## Phase 6 Foundation and baseline repair

**Status:** Implementation delivered and locally tested; acceptance remains open until every recorded build/scan gate and the clean-checkout CI workflow pass. This is not production readiness. See [Phase6.md](Phase6.md) for reproducible evidence and limitations.
**Status:** Accepted. Implementation passed the recorded local checks and the clean-checkout GitHub Actions build, PostgreSQL tests, security scans and container SBOM on merged revision `80944ed`. This is not production readiness. See [Phase6.md](Phase6.md) for reproducible evidence and limitations.

**Goal:** Establish safe, reproducible development and remove known MVP hazards. **Requirements:** F01, foundation of F07/F12. **Dependencies:** existing MVP.

Expand Down Expand Up @@ -267,7 +267,7 @@ Exit evidence includes attempts to bypass tool authorization, SSRF, credential f

## First implementation backlog

G001–G006 are implemented. G002–G006 have passing local tests; G001's full build/security/remote-CI acceptance follows the open gates in Phase6. G007–G012 are not started. These are issue-sized starting points, not an additional rigid sequence.
G001–G006 are implemented and accepted with local and clean-checkout CI evidence. G007–G012 are not started. These are issue-sized starting points, not an additional rigid sequence.

| ID | Deliverable | Dependency | Acceptance |
|---|---|---|---|
Expand Down
Loading
Loading