Repository navigation
[arch] Design discussion: shared pluggable service for cross-agent state (feedback, sessions, traces) #112
Description
Background
The v0.12.0 enterprise feature track added several stateful surfaces to BaseAgent's server layer:
POST/GET /v1/sessions— conversation persistenceGET /v1/traces— trace inspectionPOST/GET/PATCH /v1/feedback— user feedback collectionGET /metrics— Prometheus
All four follow the same shape: BaseAgent owns a pluggable store (Null / SQLite / Postgres), the server layer exposes REST endpoints, and the gateway-template proxies through. This was the right call when most deployments had one or two agents.
In multi-agent deployments (eg 10 agents fronted by a single gateway and UI) the per-agent ownership starts to chafe:
- Duplication — 10 Postgres pools, 10 schema migrations, 10 housekeeping loops, all writing the same tables
- Fan-out for cross-agent queries — "show me all thumbs-down feedback this week" requires the dashboard to hit 10 endpoints and merge client-side, OR query the shared Postgres directly out-of-band (the schema becomes a de-facto API)
- Schema becomes a contract — once N agents write the same table, schema changes need coordinated rollouts
- No auth boundary between agents sharing storage
- Sessions are conceptually cross-agent — a user talks to "the system," not to agent base-agent: Implement prompt loader (prompts.py) #4. Today there's no clean way to follow a conversation that gets routed to different agents
What to design
Open question: what does the shape of a shared 'agent platform' service look like, and which surfaces move there?
Initial options to discuss (not a decision, a starting point):
-
Status quo + documentation. Document that multi-agent deployments should point all agents at the same Postgres and treat the shared schema as a stable join point. Cheapest, but the rough edges remain.
-
Full extraction. A new FastAPI service (working name: `fipsagents-platform`) owns sessions + traces + feedback. BaseAgent becomes a thin client. Gateway routes `/v1/sessions`, `/v1/traces`, `/v1/feedback` to the platform service rather than fanning out to per-agent endpoints. One Postgres pool, one REST surface, one dashboard backend.
-
Partial extraction. Move feedback + sessions (genuinely cross-agent) but leave traces in BaseAgent shipping to an Otel collector (industry-standard answer, already partially done via `OTELTraceStore`). Less moving parts, addresses the highest-value duplication.
-
Something else. Maybe BaseAgent keeps everything but grows a 'remote store' adapter for each — same code, configurable backend (in-process vs HTTP). Lets a deployer choose per-feature without forcing a topology.
Things to think about during the discussion
- Migration story. The longer we wait, the more deployments depend on the per-agent endpoints. Cheap to do now while there's effectively one production user; observable migration later
- Memory is intentionally NOT in this list — `self.memory` is per-agent by design, and MemoryHub already provides the centralized option
- Metrics is also separate — Prometheus scrape targets are inherently per-pod, that's fine
- Auth — if multiple agents share a backend, who's allowed to write what? Today there's no model for this
- Deployment friction — every service we extract is another Helm chart, another readiness probe, another thing for ops to think about. Worth it iff the cross-agent benefits land
- Pluggability shape — same `FeedbackStore`/`SessionStore`/`TraceStore` ABCs we have today, just running in a different process? Or a different abstraction entirely?
Out of scope for this issue
This is a design discussion issue, not an implementation. The goal is to come out with a written architecture decision (in `docs/architecture.md` or similar) that we can point at when implementing.
Captured during the v0.12.0 feedback feature track. Conversation context: the per-agent ownership felt fine for one or two agents but the smell got louder once we considered the 10-agent case.
Activity
rdwj commentedon Apr 27, 2026
Decision: Option 4 — remote-store adapter, with a sibling platform service
Written into
docs/architecture.md(new section: "Cross-Agent Platform Service"). Summary below.Shape
We extend the existing pluggable-store pattern with HTTP implementations rather than changing where state ownership lives.
HttpFeedbackStore,HttpSessionStore,HttpTraceStoreadded tofipsagents.serveralongside the Null/SQLite/Postgres implementations. Same ABCs, HTTP transport.fips-agents/fipsagents-platform— new sibling repo. FastAPI service, one Postgres pool, one schema, one migration loop, re-exposes the store semantics as REST. Implemented on top of the SQLite/Postgres ABCs (not a parallel implementation).gateway-templategets a routing mode. When the platform service is deployed, gateway routes/v1/sessions/*,/v1/feedback/*,/v1/traces/*directly to it rather than fanning out to per-agent endpoints.HttpXStore, those endpoints become pass-throughs (or are turned off via config). Existing v0.12.0 deployments do not break.Why Option 4 over Options 2/3
TraceStoreship to OTEL internally as a later iteration without changing topology.Why this is right for Cost Tracking (#104)
#104 explicitly states "usage data attaches to session records — no new storage backend needed." That means token-usage data flows through
SessionStore.update(). Whether the session store isSqliteSessionStoreorHttpSessionStore, BaseAgent-side code is identical — the data lands wherever sessions land. No re-architecture needed when the platform service exists.Tradeoffs flagged in the architecture doc
Out of scope for the initial extraction
OTELTraceStore; can adopt later without a topology change.Next step
Implementation will be tracked in follow-up issues across
fipsagents-platform(new repo bootstrap),agent-template(HttpXStore implementations), andgateway-template(routing mode). To be filed at the start of the next session.docs: Record Cross-Agent Platform Service architecture decision
rdwj commentedon Apr 27, 2026
Implementation issues filed (next-session plan from
NEXT_SESSION.md):HttpFeedbackStore/HttpSessionStore/HttpTraceStore(this repo)/v1/sessions/*/v1/feedback/*/v1/traces/*The new sibling repo: https://github.com/fips-agents/fipsagents-platform
Bootstrap commit ships:
fipsagents.server.feedback.FeedbackStoreover SQLite/Postgres rather than reimplementing persistence.Sessions and traces follow once the pattern proves out. Tracking continues at fipsagents-platform#1.
docs: Mention fipsagents-platform sibling repo and architecture decision
feat: Platform routing for /v1/{feedback,sessions,traces}/* — closes #30
rdwj commentedon Apr 28, 2026
Closing — decision implemented end-to-end
The architecture decision (option 2: full extraction) was recorded in
docs/architecture.md§ "Cross-Agent Platform Service" and has now landed across all three repos.What shipped
fipsagents-platform(new sibling repo)GET /v1/sessions/{id}/cost_dataread companion (fipsagents-platform#5)fipsagents(agent-side BaseAgent + server)HttpSessionStore/HttpTraceStore/HttpFeedbackStoreand per-store backend override (feat: HTTP store implementations for Cross-Agent Platform Service (#112 impl) #114, feat(server): HTTP-backed FeedbackStore/SessionStore/TraceStore #115)SessionStore.update(), per-turn cost accumulator, cumulative HTTP read path (feat(server): SessionStore.update() + per-turn cost tracking #117, fix(llm): stream_options.include_usage so cost tracking actually fires #119, feat(server): HttpSessionStore.get_cost_data reads from platform #120)PricingConfig,BudgetEnforcer,tenant_id/session_idPrometheus labels, OTEL GenAI semantic conventions (feat(server): PricingConfig + GET /v1/sessions/{id}/usage #121, feat(server): tenant_id + session_id labels on agent_tokens_total #122, feat(server): BudgetEnforcer with per-session + per-tenant limits #123, feat(server): OTEL GenAI semantic conventions on trace spans #124, release in release: fipsagents 0.15.0 — Cost Tracking v2 #125)gateway-templateplatform.urlrouting mode for/v1/feedback*,/v1/sessions/*,/v1/traces/*with per-prefix opt-out, agent carve-out forGET /v1/sessions/{id}/usage(gateway-template#31, tagv0.5.0)Verified end-to-end
Cluster-smoked against
fipsagents-platform 0.2.1+ an agent onfipsagents 0.15.0+RedHatAI/gpt-oss-20bon RHPDS in namespacefipsagents-cost-smoke. Cumulativecost_dataaccumulation,/v1/sessions/{id}/usagedollar aggregation (cost_usd=$0.1207from a 3-turn convo, math exact),agent_tokens_total{tenant_id="cost-smoke"}Prometheus labels, and BudgetEnforcer soft-warnings at both per-session and per-tenant scopes all confirmed against real vLLM traffic.Not covered here
Two follow-ups deliberately outside this issue's scope, tracked separately:
BudgetEnforcerhas "this agent process's view" semantics; a cross-agent aggregator is a future capability onfipsagents-platform.Closing as implemented.