Skip to content

Price a request from plain values instead of the plugin request context #1501

Open

Description

@huang195

Part of #1437 (assessment: laptop data plane on Praxis) · prep work, useful to Cortex on its own

Status: there is no shipped Cortex–Praxis integration today (the paused cmd/cortex-praxis converter is not one), and the interim path is not decided (LLM traffic to praxis-ai as an inference server, or all traffic through Praxis via #1491). Nothing here depends on that choice: each item is useful to Cortex now, and would make a possible move to Praxis easier whichever path is taken.

Problem

Pricing a request needs the whole plugin request context.

  • settle.Settle(pctx, rates) reads pctx.Host, pctx.Extensions.Inference and pctx.ResponseHeaders (LiteLLM cost headers, and Content-Type via IsEventStream); pctx.Path only in log lines.
  • Avoided reads pctx.Extensions.Custom (tool-prune's record), the request body and BodyMutated().
  • Publish writes the result back into pctx under two keys; Store keeps it in pipeline state, which litellm-budget-track reads with Load and rewrites with Amend.

Eight exported functions take *pipeline.Context: Settle, Avoided, IsEventStream, Comparable, Store, Load, Publish, Amend.

Settle is called only from inside the inference-parser plugin (cost.go:72-112), which also calls Store, Avoided and IsEventStream; litellm-budget-track calls Comparable, Load and Amend. So pricing can run only inside Cortex's Go pipeline: not from a recorded event, not in a test without building a pctx, not from another proxy.

Proposal

This issue owns settle's input. #1497 item 3 owns recording the cost-relevant headers on the event, and #1503 depends on this one.

Done when

  • Settle and Comparable have plain-value forms, and the pctx versions are wrappers over them.
  • A table test prices from plain Input values, including the LiteLLM cost-header cases and the Comparable checks on the discount and margin headers. (Gateway multipliers are on the rates side, pricing/multiplier.go, and are unaffected.)
  • Existing cost tests pass unchanged (about 43 test calls use Settle(pctx, …)).

Evidence read at main @ e455a04 (2026-10-25); not built.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestlaptopThe local (laptop) tool: install, abctl, the numbers

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions