Repository navigation
Conversation
…ndex data When the guidellm JSON contains requests.successful with turn_index fields (multi-turn benchmarks), the script now emits: - one aggregate row per concurrency level (turn_index = empty, unchanged) - one per-turn row per turn per concurrency level (turn_index = 0, 1, 2, …) Per-turn rows carry the same metadata as the aggregate row plus per-turn stats: TTFT/ITL/TPOT median+p95+p99+p1+p999+mean, request latency, output throughput, token counts. Benchmarks without turn_index data are unaffected. Add _percentile() helper and extract_per_turn_rows() function. Update README to clarify that per-turn rows are now emitted automatically. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
The turn_index key was added to the result dict but missing from the fieldnames list passed to combined_df[fieldnames], so pandas silently dropped it. CSV output now has 53 columns as the README states. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- _percentile(): use same k/f/c linear interpolation as Forge - Add _median(), _mean(), _sorted_field() helpers matching Forge's _turn_metric_values() helpers exactly - Add len(all_turns) <= 1 guard: skip per-turn extraction for single-turn runs (matches Forge PR #215 behavior) - Replace _percentile(50) calls with _median() for correctness Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This was referenced Oct 7, 2026
…ment-id flags When provided, these values are stamped on every output row and included in the fieldnames list so the dashboard's Filtered Data table renders a clickable MLflow artifact link (added to the schema in PR openshift-psap#83). Both flags are optional and default to empty string — no breaking change. Closes openshift-psap#107 Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Continuation of #104 for the
stagingbranch. Adds per-turn row generation to the manual import script — same changes as #105 targetingmain.PR #104 merged the basic
turn_indexcolumn addition; this PR brings the actual per-turn row generation that makes manual imports produce the same output as the Forge pipeline.Changes
import_manual_runs_json_v2.py_percentile(),_median(),_mean(),_sorted_field()helpers aligned with Forge PR #215extract_per_turn_rows()— emits per-turn rows fromrequests.successful[].turn_indexturn_indexadded tofieldnamesexport listREADME.md— per-turn row behaviour documentedTest results
Multi-turn (3 turns × 2 concurrencies): 2 aggregate + 6 per-turn rows ✅
Standard (no turn_index data): 3 aggregate rows only, unchanged behaviour ✅
🤖 Generated with Claude Code