Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
9d6bc21
fix: count every Deep Agents model call in token usage
mnajafian-nv Oct 19, 2026
5de4904
feat: add what_ran provenance to run results
mnajafian-nv Oct 18, 2026
4f16bff
docs: describe RunResult.what_ran in the Python SDK guide
mnajafian-nv Oct 18, 2026
26f06b0
docs: regenerate API references for what_ran
mnajafian-nv Oct 18, 2026
638cb2e
feat: bound what_ran size and entry counts
mnajafian-nv Oct 19, 2026
6dea00a
docs: describe what_ran bounds in the Python SDK guide
mnajafian-nv Oct 19, 2026
e3a2876
docs: regenerate API references for what_ran bounds
mnajafian-nv Oct 19, 2026
cdc0636
docs: name NeMo Fabric in the Deep Agents usage limits
mnajafian-nv Oct 20, 2026
f076c10
test: cover what_ran on a realistic config
mnajafian-nv Oct 20, 2026
d64832d
test: cover what_ran for each bundled descriptor and at the entry caps
mnajafian-nv Oct 20, 2026
8eba8b5
test: loosen the catalog count and rename the what_ran size test
mnajafian-nv Oct 20, 2026
4585ac7
fix: report Deep Agents cached input tokens
mnajafian-nv Oct 20, 2026
fee4c92
fix: report zero Deep Agents usage when no model call ran
mnajafian-nv Oct 20, 2026
2ee128d
fix: report unknown Deep Agents usage with an Agent Protocol subagent
mnajafian-nv Oct 20, 2026
06de4c0
docs: say a failed Deep Agents call may have been billed
mnajafian-nv Oct 20, 2026
e535051
style: format the Deep Agents remote subagent check
mnajafian-nv Oct 20, 2026
cb42bfe
test: keep zero observed Deep Agents calls unknown with a remote suba…
mnajafian-nv Oct 20, 2026
ce74415
fix: preserve empty OpenShell environment arguments
mnajafian-nv Oct 22, 2026
790f369
fix: avoid premature Relay gateway startup failures
mnajafian-nv Oct 22, 2026
5db4917
docs: use the NeMo Fabric product name
mnajafian-nv Oct 22, 2026
4bea319
feat: report complete and observed model usage
mnajafian-nv Oct 22, 2026
1802df2
feat: add core and plan facts to run results
mnajafian-nv Oct 22, 2026
be32c6e
docs: use the full product name in Relay guidance
mnajafian-nv Oct 22, 2026
563baac
docs: use the full product name in Relay guidance
mnajafian-nv Oct 22, 2026
06369c4
feat: estimate model cost with versioned pricing
mnajafian-nv Oct 22, 2026
cbb0699
fix: validate recovered usage and optional pricing rates
mnajafian-nv Oct 22, 2026
559ce0a
fix: preserve fractional cost precision across usage reducers
mnajafian-nv Oct 22, 2026
19a305f
fix: apply shared usage accounting corrections
mnajafian-nv Oct 22, 2026
a6318d4
chore: check packaged usage schema declarations
mnajafian-nv Oct 22, 2026
d35ebb3
fix: qualify child model pricing by provider endpoint
mnajafian-nv Oct 22, 2026
c198472
fix: return detached applied pricing rates in Python
mnajafian-nv Oct 22, 2026
06e2396
fix: preserve frozen catalog pricing for OpenShell results
mnajafian-nv Oct 22, 2026
630820f
fix: return detached applied pricing rates in Python
mnajafian-nv Oct 22, 2026
93b00ec
fix: expose omitted optional SDK usage attributes
mnajafian-nv Oct 23, 2026
f010d42
fix: keep optional SDK usage attributes nullable
mnajafian-nv Oct 23, 2026
101bce6
fix: reject Relay exit during successful health probe
mnajafian-nv Oct 23, 2026
2014cbe
fix: exclude unknown keys from request usage
mnajafian-nv Oct 23, 2026
616b824
chore: incorporate Relay readiness prerequisite
mnajafian-nv Oct 23, 2026
dc2b018
chore: incorporate Relay readiness prerequisite
mnajafian-nv Oct 23, 2026
3fd1566
fix: skip execution-host pricing during inspection
mnajafian-nv Oct 23, 2026
6efff4d
docs: use full product name in pricing references
mnajafian-nv Oct 23, 2026
fe6f17b
chore: incorporate accounting prerequisite
mnajafian-nv Oct 23, 2026
11f72a4
fix: enforce Relay gateway startup deadline
mnajafian-nv Oct 23, 2026
e3a2355
fix: classify empty usage coverage as unverified
mnajafian-nv Oct 23, 2026
c7b9589
chore: incorporate current main into OpenShell launcher fix
mnajafian-nv Oct 23, 2026
8cc5c23
chore: incorporate updated OpenShell prerequisite
mnajafian-nv Oct 23, 2026
a8f6fd8
test: account for unverified empty usage coverage
mnajafian-nv Oct 23, 2026
75d579a
chore: incorporate updated Relay prerequisite
mnajafian-nv Oct 23, 2026
e203f3d
chore: incorporate updated Relay prerequisite
mnajafian-nv Oct 23, 2026
793b4bb
chore: incorporate updated accounting prerequisite
mnajafian-nv Oct 23, 2026
46e7ccf
docs: clarify providerless pricing lookup order
mnajafian-nv Oct 23, 2026
05132d8
docs: regenerate pricing run-plan schema
mnajafian-nv Oct 23, 2026
85100cf
docs: clarify pricing references and usage summary guidance
mnajafian-nv Oct 23, 2026
83d8ada
docs: keep adapter usage guidance within current contract
mnajafian-nv Oct 23, 2026
2cdbab3
Merge corrected shared accounting guidance into pricing
mnajafian-nv Oct 23, 2026
ebcfe97
test: run OpenShell launcher checks with system Bash
mnajafian-nv Oct 23, 2026
4286931
chore: include system Bash test prerequisite
mnajafian-nv Oct 23, 2026
83b5e76
fix: reject Relay readiness after the startup deadline
mnajafian-nv Oct 23, 2026
5702ead
chore: sync Relay startup prerequisite
mnajafian-nv Oct 23, 2026
5390e81
chore: sync Relay startup prerequisite
mnajafian-nv Oct 23, 2026
5a8589a
chore: sync Relay startup prerequisite
mnajafian-nv Oct 23, 2026
2149bd1
fix: bound Pi Relay readiness checks
mnajafian-nv Oct 23, 2026
ba4567b
test: support alternative model schemas in catalog provenance checks
mnajafian-nv Oct 25, 2026
a1d5527
fix: reject positive usage with exhaustive empty models
mnajafian-nv Oct 26, 2026
54e073b
chore: integrate current shared accounting into pricing
mnajafian-nv Oct 26, 2026
f9b8f6a
chore: integrate shared accounting with execution provenance
mnajafian-nv Oct 26, 2026
5ef4add
chore: preserve task execution facts with host pricing
mnajafian-nv Oct 26, 2026
4c0dbc0
docs: qualify normalized model usage availability
mnajafian-nv Oct 26, 2026
3356f5a
chore: merge main into shared usage accounting
mnajafian-nv Oct 26, 2026
d7db56c
chore: merge main into execution facts
mnajafian-nv Oct 26, 2026
472a142
chore: merge main into model pricing
mnajafian-nv Oct 26, 2026
0363e25
fix: preserve complete costs and reject ambiguous catalog prices
mnajafian-nv Oct 26, 2026
b30b122
fix: enforce usage accounting parity across bindings
mnajafian-nv Oct 26, 2026
87568be
fix: enforce usage accounting parity across bindings
mnajafian-nv Oct 26, 2026
5217eb7
fix: enforce usage accounting parity across bindings
mnajafian-nv Oct 26, 2026
4e4ac3e
chore: reconcile usage accounting with current main
mnajafian-nv Oct 27, 2026
847d569
Merge upstream main and verify pricing catalog integrity
mnajafian-nv Oct 27, 2026
52203e0
Merge shared usage accounting into run facts
mnajafian-nv Oct 27, 2026
75abf04
Merge refreshed execution facts into pricing
mnajafian-nv Oct 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .agents/skills/contribute-adapter/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,11 @@ Keep descriptor claims, implementation, focused tests, public documentation,
catalog entries, and packaged metadata synchronized. Start with the narrowest
truthful capability set.

For usage changes, verify invocation-local accounting through the adapter contract and consumer SDK, including warm-session deltas, unknown counters, cache semantics, and unsuccessful results. Keep generated schemas and language bindings in parity. Preserve the closed `fabric.adapter/v1alpha2` `AgentUsage` shape; carry detailed cache-write, reasoning, peak request, and per-model data in the descriptor-declared `usage.extensions["nemo_fabric.detailed_token_usage.v1"]` object until a new adapter contract revision negotiates those fields. Consumers treat any detailed field for an input or output dimension as selecting the detailed counters for that dimension, so require complete, internally consistent counters instead of relying on the top-level view as a fallback.
For usage changes, verify invocation-local accounting through the adapter contract and consumer SDK, including warm-session deltas, omitted fields, cache semantics and unsuccessful results. Keep generated schemas and language bindings in parity. Preserve the closed `fabric.adapter/v1alpha2` `AgentUsage` shape.

For a collector using shared accounting, declare `usage.extensions["nemo_fabric.usage_accounting.v1"]` in the descriptor and use Python `usage_from_observations` or TypeScript `usageFromObservations`. Keep source parsing, deduplication and invocation ownership in the collector; core normalizes counts and applies qualified pricing. An omitted count stays unknown and a reported zero stays zero. Freeze observations at return; late usage cannot move to a later invocation. An execution failure alone does not erase settled usage.

The shared helper also supplies the legacy `nemo_fabric.detailed_token_usage.v1` projection for older consumers. Keep its descriptor declaration and inclusion flags consistent. Detailed legacy counters select their dimension without falling back to top-level counters. Do not replace incomplete shared accounting with legacy fields, logs or Relay totals, and do not promote client estimates to provider-reported cost.

For `instructions.system`, keep `config.system_instruction_modes`, planning
behavior, direct adapter validation, and target-native composition synchronized.
Expand Down
10 changes: 10 additions & 0 deletions .github/workflows/ci_python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,14 @@ jobs:
- name: Set up uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8

- name: Set up Node.js for reducer parity
if: matrix.platform == 'linux-amd64' && matrix.python-version == '3.12'
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6
with:
node-version: 24
cache: npm
cache-dependency-path: adapter-contract/typescript/package-lock.json

- name: Cache cargo build
uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
with:
Expand Down Expand Up @@ -115,6 +123,8 @@ jobs:
npm run build --prefix adapters/typescript --workspace nemo-fabric-adapters-pi

- name: Run pytest
env:
FABRIC_REQUIRE_TYPESCRIPT_REDUCER_PARITY: "1"
run: |
set -euo pipefail
just --set no_uv true test-python
Expand Down
3 changes: 2 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ chrono = { version = "0.4.45", default-features = false, features = ["std"] }
# Authenticate local health messages with the endpoint secret.
getrandom = "0.4.3"
hmac = "0.12.1"
sha2 = "0.10.9"

clap = { version = "4", default-features = false, features = ["derive", "std", "help", "usage", "error-context", "suggestions"] }
# Validate adapter-owned JSON Schema in Rust while preserving precise instance paths.
Expand All @@ -43,6 +42,8 @@ libc = "0.2.186"
schemars = { version = "1", features = ["derive"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# SHA-256 for execution-fact digests and reproducible price catalogs.
sha2 = { version = "0.10.9", default-features = false }
thiserror = "2"
windows-sys = { version = "0.61.2", features = ["Win32_Foundation", "Win32_Security", "Win32_System_Diagnostics_ToolHelp", "Win32_System_JobObjects", "Win32_System_Threading"] }
pyo3 = "0.28.2"
Expand Down
130 changes: 130 additions & 0 deletions adapter-contract/python/src/nemo_fabric_adapter_contract/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -601,6 +601,136 @@ def _validate(self) -> None:
)


class UsageDimension(StrEnum):
"""Raw dimensions in the declared usage-accounting extension."""

INPUT_TOKENS = "input_tokens"
CACHED_INPUT_TOKENS = "cached_input_tokens"
CACHE_WRITE_INPUT_TOKENS = "cache_write_input_tokens"
OUTPUT_TOKENS = "output_tokens"
REASONING_TOKENS = "reasoning_tokens"
TOTAL_TOKENS = "total_tokens"
PEAK_REQUEST_INPUT_TOKENS = "peak_request_input_tokens"
COST_USD = "cost_usd"


class GapReason(StrEnum):
"""Why complete accounting is unavailable."""

UNFINISHED_WORK = "unfinished_work"
MISSING_USAGE = "missing_usage"
SCOPE_UNVERIFIED = "scope_unverified"


@dataclass(slots=True, kw_only=True)
class ReportedUsage(ContractModel):
"""Raw provider/harness counts, retaining independent field presence."""

input_tokens: int | None = _optional()
cached_input_tokens: int | None = _optional()
cache_write_input_tokens: int | None = _optional()
input_tokens_include_cache: bool | None = _optional()
output_tokens: int | None = _optional()
reasoning_tokens: int | None = _optional()
output_tokens_include_reasoning: bool | None = _optional()
total_tokens: int | None = _optional()
peak_request_input_tokens: int | None = _optional()
cost_usd: float | None = _optional()

def _validate(self) -> None:
import math

for dimension in UsageDimension:
value = getattr(self, dimension.value)
if dimension is not UsageDimension.COST_USD:
_bounded_int(value, dimension.value, (1 << 64) - 1)
elif value is not None and (
not math.isfinite(value) or value < 0 or value * 1e9 >= 1 << 64
):
raise ContractValidationError(
"cost must fit finite nonnegative nano-USD"
)


@dataclass(slots=True, kw_only=True)
class UsageGap(ContractModel):
"""A nonempty set of incomplete raw dimensions."""

dimensions: list[UsageDimension]
reason: GapReason

def _validate(self) -> None:
if not self.dimensions or len(set(self.dimensions)) != len(self.dimensions):
raise ContractValidationError("gap dimensions must be nonempty and unique")


@dataclass(slots=True, kw_only=True)
class UsageRequest(ContractModel):
"""One deduplicated model-call snapshot, not an aggregate."""

usage: ReportedUsage
gap: GapReason | None = _optional()
tokens_aligned: bool = False


@dataclass(slots=True, kw_only=True)
class AccountedModel(ContractModel):
"""Complete and observed evidence for one literal model/provider identity."""

model: str
gaps: list[UsageGap]
provider: str | None = _optional()
reported: ReportedUsage = field(default_factory=ReportedUsage)
observed: ReportedUsage | None = _optional()

observed_tokens_aligned: bool = False
requests: list[UsageRequest] = _empty_list()
requests_exhaustive: bool = False

def _validate(self) -> None:
_nonblank(self.model, "model")
if self.requests_exhaustive and not self.requests:
raise ContractValidationError("exhaustive requests must be nonempty")
if self.provider is not None:
_nonblank(self.provider, "provider")
for gap in self.gaps:
if any(
getattr(self.reported, dimension.value) is not None
for dimension in gap.dimensions
):
raise ContractValidationError(
"complete value conflicts with explicit gap"
)


@dataclass(slots=True, kw_only=True)
class UsageAccounting(ContractModel):
"""Typed nemo_fabric.usage_accounting.v1 payload.

Declare its schema in extension_schemas.usage. This does not change the
closed AgentUsage contract. Per-model complete counts live in reported.
"""

models_exhaustive: bool
gaps: list[UsageGap]
observed: ReportedUsage | None = _optional()
models: list[AccountedModel] = _empty_list()

def _validate(self) -> None:
if self.models_exhaustive and not self.models and self.observed is not None:
for dimension in UsageDimension:
if dimension is UsageDimension.COST_USD:
continue
count = getattr(self.observed, dimension.value)
if count is not None and count > 0:
raise ContractValidationError(
"exhaustive empty model list conflicts with positive tokens"
)
identities = [(model.provider, model.model) for model in self.models]
if len(set(identities)) != len(identities):
raise ContractValidationError("duplicate model identity")


@dataclass(slots=True, kw_only=True)
class AgentRunResult(AgentContractBlock):
"""Southbound terminal result returned by an adapter target."""
Expand Down
Loading
Loading