Skip to content

Decisions serving for qwen27 (Ornith-1.5-122B): score/decide passes, JointHead scoring, a free decision slot, GPU class profiling #1190

Closed as not planned

Description

@grantoverton

PRs in this series

Kept up to date as each pull request is opened. The whole family stacks on #1125 (the qwen27 Metal family); every branch carries its parents' commits until they merge — review only each PR's own commits, named in its body.


The qwen27 Metal lane takes decision work beside text generation: an HTTP thread enqueues a score or decide task and the lane host's own engine thread runs the pass — first only at idle points (#1189), then beside live traffic on a free slot (#1192). One resident model serves both streams; there is no scheduler beside the engine.

#1191 adds a small trained side head (--joint-head) that scores each option on the post-final-norm hidden row at its own final-token position instead of label logits. The last member wires the existing gpu_profile.Trace per-class timers up on the 27B backend behind TF_Q27_PROFILE, off by default.

Activity

  1. ashhart commented on Oct 27, 2026

    @ashhart
    Owner

    Thank you, @grantoverton, for the decisions series. Our reviews on #1189 to #1215 (26 October) asked for the stack to build on main's decision queue, stay opt-in, and converge with the shared decision path in #1361, and we haven't heard back, so we're closing this tracker with the stack. A reworked PR against current main is very welcome, and you can reopen this issue with it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions