Skip to content

CRUX verb parity: chat, serve and perf receipts against llama.cpp and ollama #5336

Closed as not planned

Description

@noahgift

Scope: The CRUX comparison slices: every apr verb mapped to its llama.cpp/ollama counterpart, the chat and serve slices through one client, per-cell speed receipts with bootstrap intervals, and a serve test that proves batched prefill is used.

Why it matters: Parity claims need the same inputs through all three tools; without it a gap cannot be named.

Done when: Each CRUX slice runs apr, llama.cpp and ollama on the same GGUF and prompts in CI and publishes a receipt per cell with n>=5 and a confidence interval.

Epic: #3999. Folded under triage rule 7b (2026-10-14): each row below was its own issue, closed as not planned and kept here as a sub-ticket. Reopen an issue when its work starts or its PR opens.

Sub-tickets

Activity

  1. added this to the 0.73.0 milestone on Oct 14, 2026
  2. noahgift commented on Oct 22, 2026

    @noahgift
    MemberAuthor

    Tracked in #3598 (triage 2026-10-22).

  3. noahgift commented on Oct 22, 2026

    @noahgift
    MemberAuthor

    tracked in #3999

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priorityenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions