Skip to content

Sharpen inference benchmark presentation - #1

Merged
devinnicholson merged 1 commit into
mainfrom
agent/sharpen-research-readme
Aug 7, 2026
Merged

Sharpen inference benchmark presentation#1
devinnicholson merged 1 commit into
mainfrom
agent/sharpen-research-readme

Conversation

@devinnicholson

Copy link
Copy Markdown
Owner

What changed

  • Replaced the 869-line chronological README with a concise research-facing overview.
  • Led with the canonical Qwen 1.5B/L4 KV-cache pressure experiment and its matched control.
  • Added the benchmark design, control methodology, system architecture, repository map, reproduction path, canonical artifacts, and explicit limitations.
  • Removed stale learning-log language, the Week 1 status, the TODO roadmap, and the long sequence of training notes from the landing page.
  • Updated the package description to reflect the current vLLM serving and scheduling work.

Why

The underlying experiments are technically strong, but the previous landing page buried the central result beneath procedural history and repetitive artifact links. The new presentation makes the systems contribution and evidentiary boundaries legible to research-engineering reviewers without overstating generality.

Validation

  • python3 -m unittest discover -s tests (70 tests passed)
  • git diff --check
  • Verified all relative links in the new README resolve locally.

@devinnicholson
devinnicholson marked this pull request as ready for review August 7, 2026 07:28
Copilot AI lite review requested due to automatic review settings August 7, 2026 07:28
@devinnicholson
devinnicholson merged commit 04d1263 into main Aug 7, 2026
1 check passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the repository’s landing-page presentation to be research-facing, highlighting the central KV-cache pressure result and providing a tighter map of the benchmark harness, methodology, and reproduction path.

Changes:

  • Replaced the long chronological README with a concise overview centered on the canonical prefix-cache pressure experiment and matched control.
  • Added benchmark controls, system design diagram, repository map, reproduction steps, canonical artifact links, and explicit scope/limitations.
  • Updated the package description to reflect the current vLLM KV-cache / scheduling / serving focus.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
README.md Rewrites the landing page to foreground the main result, controls, architecture, and reproducibility/artifact entry points.
pyproject.toml Updates the project description to match the repository’s current vLLM KV-cache/scheduling/serving benchmarking scope.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md

Latest extra-long follow-up:
[`results/prefix-cache-study-extra-long-merged-r8/key-results.md`](results/prefix-cache-study-extra-long-merged-r8/key-results.md)
The strongest server experiment uses `Qwen/Qwen2.5-1.5B-Instruct` on an NVIDIA L4 with 32 unique requests. The shared-prefix and matched unique-prefix workloads have comparable prompt shapes; the control contains no exact duplicate prompts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants