Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Documentation/Architecture.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AI Inference Gateway architecture

Updated 2026-09-21. Sections explicitly marked **current** describe implemented Phase 6 code. All other sections describe the **target design**, to be implemented through [Phases.md](Phases.md). [Phase6.md](Phase6.md) records verification and open gates. [PRD.md](PRD.md) owns product scope; [API.md](API.md), [Security.md](Security.md), and [Operations.md](Operations.md) expand the contracts.
Updated 2026-10-26. Sections explicitly marked **current** describe accepted Phase 6 code. All other sections describe the **target design**, to be implemented through [Phases.md](Phases.md). [Phase6.md](Phase6.md) records acceptance evidence, re-verification and operating limits. [PRD.md](PRD.md) owns product scope; [API.md](API.md), [Security.md](Security.md), and [Operations.md](Operations.md) expand the contracts.

The product owner authorizes revising existing architecture and implementation when correctness, maintainability, sound design or future complexity warrants it. Decisions below are current design choices, not immutable constraints. Document significant replacements and their evidence using [Development.md](Development.md); keep dependent contracts, migrations and release gates aligned.

Expand Down
2 changes: 1 addition & 1 deletion Documentation/Operations.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Deployment and production readiness plan

Updated 2026-09-21. Local Compose, a non-root Dockerfile, Maven wrapper and CI build/test/scan workflow are implemented. [Phase6.md](Phase6.md) records observed verification and open gates. Helm, systemd, Podman, signed releases, HA and recovery procedures below remain **planned deliverables**. Use [README.md](../README.md) for current setup.
Updated 2026-10-26. Local Compose, a non-root Dockerfile, Maven wrapper and CI build/test/scan workflow are implemented. [Phase6.md](Phase6.md) records Phase 6 acceptance evidence, re-verification and remaining operating limits. Helm, systemd, Podman, signed releases, HA and recovery procedures below remain **planned deliverables**. Use [README.md](../README.md) for current setup.

The MVP upgrade path below is the current continuity strategy. Phase 6 may replace it with a documented transition if design/consumer/data assessment justifies a simpler architecture. Adjust the release evidence accordingly; preserve real data and supported release recovery requirements. A demo origin is not permission to reset an existing environment.

Expand Down
2 changes: 1 addition & 1 deletion Documentation/PRD.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AI Inference Gateway product requirements

Updated 2026-09-21. Status: product plan with Phase 6 foundation implemented; validation evidence and remaining gates are in [Phase6.md](Phase6.md). This document replaces the MVP-only scope. Production readiness is a release outcome to demonstrate, not a claim about the current code.
Updated 2026-10-26. Status: product plan with the Phase 6 foundation accepted and re-verified; evidence and remaining operating limits are in [Phase6.md](Phase6.md). This document replaces the MVP-only scope. Production readiness is a release outcome to demonstrate, not a claim about the current code.

## Product direction

Expand Down
26 changes: 18 additions & 8 deletions Documentation/Phase6.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Phase 6 implementation evidence

Updated 2026-09-21. **Implementation delivered; acceptance remains open for the full image dependency scan/SBOM and a clean-checkout remote CI run.** Local regression, PostgreSQL, source scan, container OS scan, packaging and startup checks passed. This is foundation evidence, not production readiness.
Updated 2026-10-26. **Implementation and Phase 6 acceptance are complete.** Local regression and PostgreSQL verification were repeated successfully on 2026-10-26. The original local source scan, container OS scan, packaging and startup checks passed, followed by clean-checkout GitHub Actions verification on the merged `main` revision, including the full container dependency scan and SBOM. This is foundation evidence, not production readiness.

## Baseline and design assessment

Expand All @@ -27,7 +27,7 @@ The implementation keeps the legacy request/success/error shapes; deliberate com

## Observed verification

Source base revision: `cb95950a2d78431656cb6ca4756a0b056c20bdc0`, **dirty working tree with these Phase 6 changes**. No new commit or remote workflow run was created. Logs and detailed Maven reports are local ignored build outputs; durable benchmark data is in [evidence/phase6-baseline.json](evidence/phase6-baseline.json).
The original local evidence used source base revision `cb95950a2d78431656cb6ca4756a0b056c20bdc0` with the Phase 6 changes still in a dirty working tree. Logs and detailed Maven reports are local ignored build outputs; durable benchmark data is in [evidence/phase6-baseline.json](evidence/phase6-baseline.json). The changes were subsequently committed as `775bdfe4d1627708044017cbbb3749ef61cfbb18`, merged by [pull request #1](https://github.com/616b73/AI-Inference-Gateway/pull/1), and verified from the clean merge revision `80944edb37f439dd0f171dcf70da4ef0252b5738`.

Environment: Windows 11, Java 21.0.7, Maven wrapper 3.9.11, Docker Desktop 4.90.0 / engine 29.7.2, PostgreSQL 16.15 (`postgres:16-alpine` pinned digest in CI). Runtime image: Temurin 21.0.12+8, Alpine 3.24.2, UID/GID 10001. Tests used a disposable PostgreSQL container on loopback port 15432 and random `p6_…` schemas; fixtures dropped only their own schemas. No existing database was reset.

Expand All @@ -42,11 +42,21 @@ Environment: Windows 11, Java 21.0.7, Maven wrapper 3.9.11, Docker Desktop 4.90.
| Source security scan | Trivy 0.74.0: zero high/critical dependency, secret or configuration findings after Tomcat patch |
| Runtime OS scan | Trivy 0.74.0: 73 Alpine packages checked; zero high/critical findings |
| Full runtime dependency scan / SBOM | **Not verified locally**: Java index downloads from both official registries ended with unexpected EOF |
| GitHub Actions clean-checkout verification | **Not run remotely**; workflow configured to fail on test failures or high/critical findings and retain reports |
| GitHub Actions clean-checkout verification | [Run 35695280770](https://github.com/616b73/AI-Inference-Gateway/actions/runs/35695280770) passed on merged `main` revision `80944ed`: PostgreSQL build/tests, artifact manifest, container build, dependency/secret/configuration scan, full container vulnerability scan, container SBOM and evidence preservation |

### Local re-verification (2026-10-26)

Revision `cf469207ffe4874df0a0eaa567dd1465499a15f9` was re-verified on Windows 11 with Java 21.0.12.1, Maven 3.9.11, Docker Engine 29.8.2 and the pinned PostgreSQL 16.15 image. The working tree was clean before documentation cleanup. Maven ran offline from the existing project cache; this constrained runner supplied Mockito as a Java agent and allowed loopback networking explicitly.

- `clean test`: 89 discovered, 88 passed, one opt-in benchmark skipped, zero failures/errors.
- `-Ppostgres-it clean verify`: the same regular suite passed and all four PostgreSQL integration tests passed, covering a fresh V1–V6 install, populated V5→V6 upgrade, production startup and active-demo-credential rejection.
- The opt-in `BaselineBenchmarkTest` passed separately with 320 successful synthetic loopback calls including warm-up and rewrote the ignored `target/phase6-baseline.json` result.

This re-verification did not repeat dependency/container scans or regenerate an SBOM. Those acceptance gates remain supported by the clean-checkout GitHub Actions evidence above and must be rerun at release time against then-current advisories.

Final JAR SHA-256: `ae60015186ed6b34b557ca1d0d23fc05567293efc1d719aad26459f399505aa8` (64,614,319 bytes). Final packaged image ID: `sha256:c0017a487d066808cb90721745876856d319d40ce857aa7156ac01163c6ed1fd`. These identify a local uncommitted build, not a signed release. `scripts/release_manifest.py` writes source revision/dirty state, artifact hash/size and toolchain target to `target/release-manifest.json`.

The source scan used a complete local Maven repository mounted at `/root/.m2/repository`, `--offline-scan`, a vulnerability DB updated 2026-09-21T07:13Z, and Trivy's embedded configuration checks after a policy download failed. It excluded `.git`, `.build-cache`, and generated `target` output. The independent OS scan explicitly excluded JARs; it is not a substitute for the pending full image scan. CI retains the full source/JAR/image scans and has no vulnerability suppression or `continue-on-error` bypass.
The source scan used a complete local Maven repository mounted at `/root/.m2/repository`, `--offline-scan`, a vulnerability DB updated 2026-09-21T07:13Z, and Trivy's embedded configuration checks after a policy download failed. It excluded `.git`, `.build-cache`, and generated `target` output. The independent local OS scan explicitly excluded JARs and was not treated as a substitute for the full image scan; the later clean-checkout CI run completed the full container scan and SBOM. CI retains the source/JAR/image scan evidence and has no vulnerability suppression or `continue-on-error` bypass.

### Reproduction

Expand All @@ -66,11 +76,11 @@ The benchmark used loopback HTTP, synthetic Ollama output, a 1,024-character pro

End-of-run used Java heap was 72,000,456 bytes, a snapshot rather than a peak-memory or leak measurement. Sequential gateway/direct p95 difference was approximately 59.11 ms; subtracting percentiles is only a comparison, not a paired per-request overhead distribution. The sequential gateway sample lasted approximately 5.3 seconds. This short closed workload does not validate the PRD's 100 requests/s, 200-stream, one-hour target or its ≤50 ms overhead goal. BCrypt verification and synchronous diagnostics are included; this experiment did not profile their individual contribution. CPU utilization, RSS, peak buffers and production PostgreSQL latency were not measured here.

## Remaining acceptance and operating limits
## Acceptance closure and operating limits

The two external Phase 6 gates left open by the local run were closed by [GitHub Actions run 35695280770](https://github.com/616b73/AI-Inference-Gateway/actions/runs/35695280770) on 2026-09-22. The workflow checked out merge revision `80944edb37f439dd0f171dcf70da4ef0252b5738` and passed the PostgreSQL build/test profile, dependency/secret/configuration scan, full container vulnerability scan, container SBOM and evidence-preservation steps without a bypass. Phase 6 is accepted on that evidence.

1. Run the committed workflow from a clean checkout; retain its exact revision, test reports and scan artifacts. No remote CI result is implied by local success.
2. Complete the full image dependency scan and container SBOM once the Java advisory database can be downloaded. Do not accept a partial scan or disable that gate to publish a release.
3. Recheck dependency/base-image advisories at release time. Pins provide repeatability, not permanent security; maintain them.
Dependency and base-image advisories must still be rechecked at release time. Pins provide repeatability, not permanent security; maintain them.

Immediate downstream disconnect cancellation, bounded stream writes, deployment-isolated pools, indexed identities, tenancy, durable usage/budgets, configuration refresh, recovery drills and production load/soak tests remain future phases. The current total deadline actively aborts upstream I/O but does not hard-limit all synchronous database work or downstream delivery. Local diagnostics can be dropped; they are not audit or billing records. No real Ollama/cloud-provider smoke test, full Compose model download, external consumer migration or production deployment was performed.

Expand Down
8 changes: 4 additions & 4 deletions Documentation/Phases.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AI Inference Gateway delivery phases

Updated 2026-09-21. This is the forward implementation plan, not a completion log. **Phase 6 implementation is present with local test evidence; its remaining build/security/remote-CI gates are tracked in [Phase6.md](Phase6.md). Phases 7–15 have not started.** [Roadmap.md](Roadmap.md) preserves actual history. Requirement IDs refer to [PRD.md](PRD.md); technical decisions refer to [Architecture.md](Architecture.md).
Updated 2026-10-26. This is the forward implementation plan, not a completion log. **Phase 6 is accepted and was re-verified locally on 2026-10-26; evidence is recorded in [Phase6.md](Phase6.md). Phases 7–15 have not started.** [Roadmap.md](Roadmap.md) preserves actual history. Requirement IDs refer to [PRD.md](PRD.md); technical decisions refer to [Architecture.md](Architecture.md).

## Working approach

Expand All @@ -21,7 +21,7 @@ Use small vertical slices with an owner, acceptance evidence, compatibility/migr
| 4 | Provider/log visibility endpoints | Recorded complete |
| 5 | Log filters, expanded tests and onboarding docs | Recorded complete |

The original 62-test baseline was reproduced before Phase 6 changes. The expanded suite and PostgreSQL checks now pass; [Phase6.md](Phase6.md) records counts, environment, benchmark and remaining gates. None is production certification.
The original 62-test baseline was reproduced before Phase 6 changes. The expanded suite, PostgreSQL checks, clean-checkout build, security scans and container SBOM passed; [Phase6.md](Phase6.md) records counts, environment, benchmark and CI evidence. None is production certification.

## Dependencies and releases

Expand Down Expand Up @@ -55,7 +55,7 @@ The graph describes **completion dependencies**. Observability, CI, operations,

## Phase 6 Foundation and baseline repair

**Status:** Implementation delivered and locally tested; acceptance remains open until every recorded build/scan gate and the clean-checkout CI workflow pass. This is not production readiness. See [Phase6.md](Phase6.md) for reproducible evidence and limitations.
**Status:** Accepted. Implementation passed the recorded local checks and the clean-checkout GitHub Actions build, PostgreSQL tests, security scans and container SBOM on merged revision `80944ed`. This is not production readiness. See [Phase6.md](Phase6.md) for reproducible evidence and limitations.

**Goal:** Establish safe, reproducible development and remove known MVP hazards. **Requirements:** F01, foundation of F07/F12. **Dependencies:** existing MVP.

Expand Down Expand Up @@ -267,7 +267,7 @@ Exit evidence includes attempts to bypass tool authorization, SSRF, credential f

## First implementation backlog

G001–G006 are implemented. G002–G006 have passing local tests; G001's full build/security/remote-CI acceptance follows the open gates in Phase6. G007–G012 are not started. These are issue-sized starting points, not an additional rigid sequence.
G001–G006 are implemented and accepted with local and clean-checkout CI evidence. G007–G012 are not started. These are issue-sized starting points, not an additional rigid sequence.

| ID | Deliverable | Dependency | Acceptance |
|---|---|---|---|
Expand Down
31 changes: 29 additions & 2 deletions Documentation/Roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -393,11 +393,38 @@ Each entry records what was built, why certain decisions were made, and what was
- Phase 6 implementation did not establish production readiness. Remaining acceptance gates are explicitly tracked in Phase6; production qualification remains Phase 13.
- Generic identity, globally visible metadata, all-key BCrypt scans, startup registry refresh, synchronous diagnostics and absent streaming/tenant/budget controls remain visible limitations for Phases 7–12.

### Milestone 11: Phase 6 external verification closure (2026-09-22)

**Goal:** Close the clean-checkout build, security-scan and artifact gates that remained open after the local Phase 6 implementation run.

**What was done:**
- Committed the Phase 6 implementation as `775bdfe4d1627708044017cbbb3749ef61cfbb18`, merged it through [pull request #1](https://github.com/616b73/AI-Inference-Gateway/pull/1), and established merge revision `80944edb37f439dd0f171dcf70da4ef0252b5738` on `main`.
- Ran the `Foundation verification` workflow from a clean GitHub checkout. [Run 35695280770](https://github.com/616b73/AI-Inference-Gateway/actions/runs/35695280770) passed the PostgreSQL build/test profile, artifact manifest, container build, dependency/secret/configuration scan, full container vulnerability scan, container SBOM and evidence-preservation steps.
- Updated the active status documents to mark Phase 6 accepted while retaining its operating limits and the distinction between foundation acceptance and production qualification.

**Key decisions:**
- Accepted Phase 6 using the merged `main` workflow evidence rather than treating the earlier partial local image scan as sufficient.
- Kept production readiness explicitly open: provider compatibility, streaming, tenant isolation, durable accounting, recovery and production load evidence remain later-phase work.

### Milestone 12: Phase 6 re-verification and status cleanup (2026-10-26)

**Goal:** Confirm the accepted foundation still passes before starting Phase 7 and remove stale active-document references to open Phase 6 gates.

**What was done:**
- Re-ran the clean regular suite on revision `cf469207ffe4874df0a0eaa567dd1465499a15f9`: 89 tests discovered, 88 passed, one opt-in benchmark skipped and zero failures/errors.
- Re-ran `-Ppostgres-it clean verify` against a disposable pinned PostgreSQL 16.15 container: the regular suite and all four fresh-install, upgrade and production-startup integration tests passed.
- Ran the opt-in loopback benchmark separately: one benchmark test passed with 320 successful synthetic calls including warm-up.
- Aligned the active architecture, product, security, operations and phase documents with the already-closed Phase 6 acceptance gates.

**Limits:**
- The re-verification did not repeat dependency/container scans or regenerate an SBOM; the accepted clean-checkout GitHub Actions run remains the recorded evidence for those gates. Release qualification must refresh them against current advisories.
- Phase 7 implementation remains unstarted. G007 canonical contracts and G008 streaming transport evidence are next.

## Next Steps / Future Enhancements

The original six MVP phases (0–5) are recorded complete. Phase 6 implementation and local verification are recorded in Milestone 10 and [Phase6.md](Phase6.md); acceptance remains open for the unresolved build/security/clean-checkout CI gates listed there. Phases 7–15 have not started.
The original six MVP phases (0–5) and the Phase 6 foundation are accepted. Phase 6 implementation and local verification are recorded in Milestone 10; its later clean-checkout build, security and SBOM closure is recorded in Milestone 11 and [Phase6.md](Phase6.md). Phases 7–15 have not started.

Close Phase 6's remaining external verification gates, then start G007 provider/context contracts and G008 streaming transport experiment in [Phases.md](Phases.md). Protocol/provider and identity work can progress concurrently once contracts are agreed. First production GA requires F01–F13 and the evidence checklist in [Operations.md](Operations.md). Broader enterprise protocols and agent/serving extensions are later phases.
Start G007 provider/context contracts and G008 streaming transport experiment in [Phases.md](Phases.md). Protocol/provider and identity work can progress concurrently once contracts are agreed. First production GA requires F01–F13 and the evidence checklist in [Operations.md](Operations.md). Broader enterprise protocols and agent/serving extensions are later phases.

[PRD.md](PRD.md) owns scope, [Architecture.md](Architecture.md) owns design, and [Research.md](Research.md) explains the evidence and adaptations. Record completed work here as it occurs.

Loading
Loading