Repository navigation
Description
Part of #1437 (assessment: laptop data plane on Praxis) · prep work, useful to Cortex on its own
Status: there is no shipped Cortex–Praxis integration today (the paused
cmd/cortex-praxisconverter is not one), and the interim path is not decided (LLM traffic to praxis-ai as an inference server, or all traffic through Praxis via #1491). Nothing here depends on that choice: each item is useful to Cortex now, and would make a possible move to Praxis easier whichever path is taken.
Problem
Pricing a request needs the whole plugin request context.
settle.Settle(pctx, rates)readspctx.Host,pctx.Extensions.Inferenceandpctx.ResponseHeaders(LiteLLM cost headers, andContent-TypeviaIsEventStream);pctx.Pathonly in log lines.Avoidedreadspctx.Extensions.Custom(tool-prune's record), the request body andBodyMutated().Publishwrites the result back intopctxunder two keys;Storekeeps it in pipeline state, which litellm-budget-track reads withLoadand rewrites withAmend.
Eight exported functions take *pipeline.Context: Settle, Avoided, IsEventStream, Comparable, Store, Load, Publish, Amend.
Settle is called only from inside the inference-parser plugin (cost.go:72-112), which also calls Store, Avoided and IsEventStream; litellm-budget-track calls Comparable, Load and Amend. So pricing can run only inside Cortex's Go pipeline: not from a recorded event, not in a test without building a pctx, not from another proxy.
Proposal
This issue owns settle's input. #1497 item 3 owns recording the cost-relevant headers on the event, and #1503 depends on this one.
Settle(in Input, rates) SettledwithInput{Host, Inference, ResponseHeaders}.IsEventStreamis derived fromResponseHeaders;Pathis only for logging and can be passed separately or dropped.Comparable(in Input)the same way.Avoidedkeeps apctxform for now: it needs tool-prune's record, the forwarded body andBodyMutated(), none of which belong in a pricing input.Store,Load,PublishandAmendare how plugins exchange the result; they stay onpctx(their serialization is Decouple plugins from Go pipeline internals: session facts and serializable records #1499 item 4).- Keep
Settle(pctx, rates)as a thin wrapper over the new form, so existing callers and tests don't change. No behaviour change. - Where the wrapper lives (the inference-parser plugin, or a separate cost step) is decided in Turn the MCP, A2A and inference parsers into libraries, with saved real-traffic test cases #1503.
Done when
SettleandComparablehave plain-value forms, and thepctxversions are wrappers over them.- A table test prices from plain
Inputvalues, including the LiteLLM cost-header cases and theComparablechecks on the discount and margin headers. (Gateway multipliers are on the rates side,pricing/multiplier.go, and are unaffected.) - Existing cost tests pass unchanged (about 43 test calls use
Settle(pctx, …)).
Evidence read at main @ e455a04 (2026-10-25); not built.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsNew/ToDo
Activity