Skip to content

docs: the readme leads with agents building an engine per model, workload and gpu - #49

Merged
xiaguan merged 1 commit into
masterfrom
readme-agents
Oct 12, 2026
Merged

xiaguan merged 1 commit into
masterfrom
readme-agents

Conversation

@xiaguan

@xiaguan xiaguan commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

Rewrites the README around the new positioning.

Built by agents. Built for agents. kern lets agents change the whole inference stack, fast and safely, so every model, every workload and every GPU gets a high-performance engine of its own.

  • Opens with one night of the optimization loop: vLLM + kern served 17% more requests than stock vLLM; the same engine in SGLang served 10.5% more than SGLang's own model.
  • "Every model. Every workload. Every GPU.": the matrix moves out of the engine, each point is its own manifest.
  • "Agents change the whole stack": fast and safe, with the HF hub kernel diff kept as the example.
  • What agents have built: Qwen3.8 bring-up in one session, K3 kernel set by six agents, per-GPU manifests, DSv4.1 EP4 and full-size K3 PP8.
  • Try it: install, from source, inside vLLM, SGLang link.
  • Drops stale claims (under 3,000 lines, August qwen3-4b numbers); the named-constants reference moves to docs/manifest.md.

🤖 Generated with Claude Code

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T16:52:07.180451Z 3973323 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3973323241

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread README.md
Comment on lines +78 to +80
**Safe.** Zero trust: every kernel is pinned by SHA-256, every buffer it
touches is declared, and nothing ships without a verdict against the
reference. The engine stays the same, so a change can only break the

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid presenting arbitrary kernels as zero-trust

When users load a third-party module such as the preceding Hugging Face example, SHA-256 verifies its identity but does not constrain its memory accesses. The repository's own trust-boundary documentation says the verifier proves only declaration consistency and trusts a cubin that lies about its read/write ranges (docs/manifest.md:506-509), so the claim that every touched buffer is declared can lead users to execute untrusted GPU code under a false safety assumption; qualify this as integrity checking rather than kernel sandboxing.

Useful? React with 👍 / 👎.

Comment thread README.md
./target/release/kern run qwen3-4b --steps 320 # generate
./target/release/kern test qwen3-4b # verdict for a change: exit 0 PASS / 1 FAIL / 2 INCONCLUSIVE
./target/release/kern bench qwen3-4b --workload tools/profiles/quick.toml --out quick.json
./target/release/kern server qwen3-4b --port 8000 # OpenAI-compatible server

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Build kern-serve before invoking the server

In a clean source checkout, the preceding bare cargo build --release does not produce kern-serve: Cargo.toml deliberately excludes that package from default-members. Since kern server only searches for an already-built sibling or a kern-serve on PATH, this documented command fails for source builders; include cargo build --release -p kern-serve before this invocation or build both packages together.

Useful? React with 👍 / 👎.

@xiaguan
xiaguan force-pushed the readme-agents branch 16 times, most recently from 28a12f2 to 1557521 Compare October 12, 2026 06:00
…load and gpu

The old readme stopped in August: a 3,000-line runtime claim the code has
outgrown, qwen3-4b numbers, and none of the vLLM/SGLang hosting, the
overnight optimization loop or the multi-GPU work. The new one opens with
the loop's result (17% more requests than stock vLLM), frames each point
of the models x workloads x GPUs matrix as its own manifest, and lists
what agents have built on it. The named-constants reference moves into
docs/manifest.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: JinYan Su <751080330@qq.com>
@xiaguan
xiaguan merged commit b3557fb into master Oct 12, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant