add --report flag for HTML report generation of pipeline internals - #181
add --report flag for HTML report generation of pipeline internals#181nwaughachukwuma wants to merge 12 commits into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new --report flag to the mm cat command, which generates a self-contained HTML report of pipeline internals (such as encoder outputs, LLM messages, and responses) saved to mm_reports/. Feedback on these changes highlights a high-severity race condition in LlmBackend where self.last_messages is stored directly on the instance rather than in thread-local storage. Additionally, the reviewer suggests respecting the user-specified --output-dir option when writing HTML reports and using defensive dictionary access for llm_usage to prevent potential KeyError exceptions.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
- use a local price catalog for token cost computation. - add logic for keeping the catalog updated using a cron jon - wire token cost estimation into report (`--report` flag) generation
This branch adds a tokens-per-second (toks/s) `throughput` metric to mm cat's _verbose_ and _footer_ output for LLM-backed pipelines. `toks/s` is a CLI-level measure of end-to-end throughput — completion tokens over the full generate wall-clock — rather than a true decode number which is only knowable on the inference backend.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Chukwuma Nwaugha <nwaughac@gmail.com>
|
@spillai, please let me know what else is needed to get this merged. |
CI installs the dev extras with an unpinned 'ruff>=0.4', which now resolves to ruff 0.16.1 — its new/promoted rules flag 298 pre-existing errors and fail the test-python job. Pin to 0.15.4 (same fix as #181) so lint results are reproducible. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Cg3KtFq2ZWMck79bq6YvBV
1 participant
Uh oh!
There was an error while loading. Please reload this page.