Qwen3-Embedding-8B (BF16) on native HIP: BF16 dense GEMM and a qwen3 embedding engine by laudney · Pull Request #1978 · ashhart/TensorFold

laudney · 2026-11-03T01:22:31Z

Part of #1977. Serves Qwen/Qwen3-Embedding-8B as Qwen publishes it (BF16) on the native HIP engine, through the existing /v1/embeddings route.

	`main`	this PR
`Qwen/Qwen3-Embedding-8B` on HIP	refused: "no backend for qwen3 checkpoints"	served on one Radeon AI PRO R9700 (gfx1201)
Model card example vs transformers fp32 (cosine distance, 4 inputs)	—	1.32e-4, 9.30e-5, 7.76e-5, 9.27e-5
Long inputs vs transformers fp32 (2,044, 4,043, 8,177, 16,383, 32,759 tokens, cut from pinned files)	—	1.16e-4, 5.24e-5, 7.00e-5, 9.90e-5, 9.22e-5
Full-width limit over the 9 pinned inputs	—	1.6e-4 = transformers BF16's own 1.572e-4, rounded up to two digits (the rule of #1981)
MRL widths 32, 1024, 4096: worst distance (limit at that width)	—	7.92e-5 (1.9e-4), 1.18e-4 (1.7e-4), 1.32e-4 (1.6e-4)
Card 2 x 2 scores	—	[[0.7479, 0.0759], [0.0886, 0.6324]] vs the card's [[0.7493, 0.0751], [0.0880, 0.6318]]
Batch vs alone vs token-ID inputs (all 9, the 32K one included)	—	identical fp32 bytes
BF16 dense weights on HIP	widened to fp32 (30 GiB for 8B) and run one row at a time	kept BF16 (14.1 GiB), dot2 GEMM tile with a BF16 weight decoder
MLX GEMM tiles (`affine_gemm_block`, `affine_gemm_kp`, `affine_wmma_gemm`)	—	identical disassembly on gfx1201 before and after
MRL widths	declared lists only	any width in a declared range (32 to 4096 here)
Embedding requests vs SGLang (28 probes)	whitespace-only text embedded; an over-long input refused as an over-full batch	refused as SGLang refuses them, with the input's token count
Load	—	4 s from the page cache, 32,768-token passes, 1.19 GiB scratch

Why

Serving the publisher's own checkpoint matters for benchmarking and adoption: these are the bytes SGLang and vLLM serve, so comparisons are like for like with no conversion step, and users run the checkpoint they already download.

What changes

quant.readStored keeps a dense BF16/FP16 tensor in its stored dtype; dense.Matrix carries the dtype. quant.read callers keep fp32 and their bits.
dense_gemm_block<DotBF16, 8>: the existing dot2 GEMM tile over a new Bf16Decoder (scale 1, no bias path; the tile leaves the x sums and bias folds out for a decoder without a bias, which is a compile-time branch the MLX decoder takes as before). project dispatches a dense weight by dtype; tf_embed_dense reads BF16 tables.
tf_heads_rows: the causal kernel's fp32 (heads, rows, d) to row-major activations, for attention without an output gate.
zig/src/families/qwen3/: config, BF16 model load, lastRows (independent inputs in one batch, positions from 0 each), a prompt-only engine sized to the HIP memory budget (the window halves to fit, down to 2,048), and a lane backend with the embedding head from the sentence-transformers folder (1_Pooling last token, Normalize).
native/hip.zig: the qwen3 registry entry; a family's Loaded can say it serves embeddings in its own window; a one-GPU family refuses --tp before any group forms.
lanes.embedding.Info.mrl_min: a range of MRL widths.
/v1/embeddings: whitespace-only inputs refused (Python's whitespace set, as SGLang strips), and an input past the context named with its token count.
Kernel contract: both new kernels registered (gates 1 to 4 pass), their gate-5 records from ROCm 10.1; a record can mark a chip not served (gfx1030's FP16 build keeps only the dense GEMM's trap body); controls stay strict.
tools/zig/oracle_qwen3_embedding.py: the model card's Transformers example, pinned (revision 1d8ad4ca, every snapshot file's SHA256, Transformers 5.22.1's package tree), checked before any model code runs, plus 5 long inputs of 2K to 32K tokens cut from pinned files; every input's text and token IDs are pinned by SHA256. check compares a server at full width and at MRL widths 32, 1024 and 4096.
docs/api.md (Embeddings) and the CHANGELOG: the shared /v1/embeddings refusals.
docs/recipes/qwen3-embedding-hip.md.

Gates (R9700, gfx1201, ROCm 10.1)

Gate	Result
`zig build test -Dhip=true`	518 of 518 steps pass (head `a56bda4e9` on `824e56281`)
`hip-kernel-test`	exit 0, the new dense GEMM tests included
`qwen3-hip-gpu-test` (pinned fp32 and BF16 references over the 9 inputs, files hashed against the loaded checkpoint)	exit 0, distances as in the table
`oracle_qwen3_embedding.py check` against the served model (9 inputs, widths 32, 1024, 4096 and full)	PASS
`hip-kernel-contract`	60 kernels pass gates 1 to 4, 5 controls red as planted; gate 5: the dense GEMM and `tf_heads_rows` current (their records in `hip_records_qwen3.zig`)
`hip-kernel-contract-limits`	the dense GEMM current on gfx1100, gfx1151 and gfx1201, not served on gfx1030; the only problems are main's `tf_sum_partials` and `tf_sum_partials_act` records (stale on ROCm 10.1; this PR touches neither the kernels nor their records)
`lean_check.py`	424 problems, the same as `upstream/main`

SGLang note: its server stops with a GPU fault (out-of-bounds index_select) on an out-of-vocabulary token id in /v1/embeddings; this route refuses it with a 400.

Overlaps

atc overlap finds textual conflicts with #1612, #1614, #1622, #1658, #1694 and #1877, all in the shared HIP lists every kernel PR appends to: zig/src/hip/kernel_tests.zig, zig/src/core/contract/hip_records.zig, zig/tests/hip/contract_main.zig, zig/src/hip/launches.zig, zig/src/hip/ops/ops.zig, zig/build/hip.zig and tiles/dot2_tiles.hip's instantiations. No shared logic; whichever lands second keeps both entries.

The reranker (PR B of #1977) is stacked on this branch.

ashhart · 2026-11-03T02:08:05Z

Thanks @laudney. It applies cleanly and holds: other HIP families and the MLX GEMM tiles are unchanged (BF16-as-stored only in qwen3, and the new tile branches compile out for the MLX decoder), BF16 weights stay BF16 with fp32 sums and one BF16 rounding, no split-K, batched = alone, and the worst cosine distance is 1.32e-4 against a 2.26e-4 bound on the card's inputs. Two small asks: (1) document the shared /v1/embeddings refusals (whitespace-only, over-long input) in docs/api.md and the CHANGELOG; (2) widen the pinned fidelity set to 2K-32K inputs and MRL widths 32, 1024 and 4096. It lands after #1979.

Review of ashhart#1978 asked for the request refusals this branch adds to every embedding family to be written down. A new Embeddings section in docs/api.md lists the whitespace-only text refusal, the over-long input that is named with its token count and the context_length_exceeded code before the batch limit, the batch limit itself, and how dimensions crops to an MRL width, and the CHANGELOG gains the Qwen3-Embedding-8B entry that points to it.

Review of ashhart#1978 asked for the pinned fidelity set to cover 2K to 32K inputs and MRL widths 32, 1024 and 4096. The oracle adds five long inputs, each the longest whole-line prefix of a pinned file within 2K to 32K tokens, pins every input's text and token IDs by SHA256, runs the long ones alone at full length, and checks a server in requests that fit its window, at full width and at each MRL width against the cropped fp32 vector; the GPU test covers the same nine inputs, and the recipe records the measured distances.

A checkpoint published in BF16 widened to fp32 by quant.read takes twice its memory and needs an fp32 kernel; Qwen3-Embedding-8B would not fit a 32 GiB card that way. quant.readStored reads like read but keeps a BF16 or FP16 dense tensor as stored, dense.Matrix carries the weight's dtype for the kernels, and dense.reference reads each element by its dtype. Callers of quant.read keep fp32 weights and their bits; the host tests cover both reads, views and the halves.

The repository's lean checker allows only single-line comments of at most 120 columns, and the stored-dtype change gave the dense format's module comment a second line. The comment now says the same in one line, so the checker adds no problem over main.

A BF16 checkpoint kept as stored needs a prompt GEMM that reads BF16 weights; HIP had only the fp32 one-row dense kernel. A BF16 weight decoder plugs into the existing dot2 GEMM tile with unit scales and no bias path, which the tile now leaves out for such a decoder; the MLX tiles compile to the same code. project dispatches a dense weight by its dtype, and the dense embedding gather reads BF16 tables. GPU tests check it against float64 on gfx1201.

The HIP qwen3 forward runs one shape of Qwen3 dense model, so its config reader takes the geometry and constants from config.json, defaults head_dim to 128 as Qwen3Config does, and refuses by name the features the forward does not run (projection biases, sliding window, RoPE scaling, head_dim above 256, a non-SiLU MLP, uneven GQA groups), with host tests on the 8B embedding and reranker configs.

Qwen3's attention has no output gate, so nothing turned the causal kernel's fp32 (heads, len, d) rows into o_proj's row-major input; tf_heads_rows does that transpose with one rounding to the output kind, reached through Ops.headsRows, with an exact kernel test, a kernel contract (copy and round, served Qwen3-8B shapes) and its compiled-usage record on all four RDNA chips.

The embedding and reranker checkpoints share one Qwen3 dense body, so the qwen3 family now uploads it as stored (BF16 projections, fp32 norms, an optional model. prefix), reads lm_head rows on the host for the reranker, and runs lastRows: inputs batched layer by layer, each attending only to itself from position 0, final-normed last rows out. It adds the module, host tests and an env-gated GPU check of both checkpoints against transformers.

Qwen3-Embedding-8B is trained for any output width from 32 to 4096, and the embedding route only took widths from a declared list, so a request for 384 or 2049 would be refused. The lane's embedding Info gains mrl_min: with it set, every width from that one to the full dimension crops; the list rule and the refusal without either stay. Tests cover both bounds, widths between them and the old rules.

Qwen3-Reranker-4B ties its output head to the embedding and stores no lm_head.weight, so readRows of the head failed for it; the model now records the stored name of its head rows (lm_head.weight, or the embedding as the checkpoint names it when tie_word_embeddings is set) and the GPU check reads that name, with host tests for both layouts.

The embedding route and lane contract had no device backend, and the HIP registry no qwen3 family. A prompt-only engine sizes its scratch to the memory budget, shrinking the window to fit, and a lane backend runs each batch's whole prompt in one step and returns the pooled last rows, unit-normed when modules.json lists Normalize; the head is read from the sentence-transformers folder. The registry gains qwen3, a family's Loaded can say it serves embeddings in its own window, and one-GPU families refuse --tp before any group forms.

The embedding gate compares the HIP engine with the model card's Transformers example, so the reference must come from known sources. The oracle pins the checkpoint revision, the SHA256 of every file in the snapshot, the Transformers 5.22.1 package tree and its Qwen3 files, checks them before any model code runs, writes them into each reference, and the server check refuses a reference made otherwise. It also checks distance, norms, batch and token-ID equality.

Loading Qwen3-Embedding-8B took about 400 s, warm cache or not. The loader released the checkpoint after every layer, and that release also drops each whole shard from the page cache, so every later layer's upload faulted its pages back in from disk. The loader now keeps the mapping until the checkpoint closes at the end of the load; the same server loads in 4 s and serves the same vectors.

Every kernel must pass the contract's gates before it lands, and the BF16 dense GEMM had only its unit test. It now runs gates 1 to 4 for fp32 and BF16 output at Qwen3-8B's projection shapes, on BF16 builds, and gate 5 records its compiled usage on gfx1100, gfx1151 and gfx1201. A record can mark a chip not served, as gfx1030's FP16 build keeps only the kernel's trap body; the audits skip such a chip for a launch, never for a planted control.

The embedding gate's limit comes from the BF16 bar's distance to the fp32 target, so a bar of other inputs or a non-finite vector could loosen it or make it pass on NaN. The oracle checker and the GPU test now refuse references that differ in dtype, texts, token ids or pins, or hold a ragged, zero or non-finite vector, and fail on any non-finite distance before comparing.

The unit test and the kernel contract bounded the BF16 dense GEMM's error with different rounding chains, and the contract's ignored the fold every 128 inputs, allowing about 28 times more error at K 12288. Both now take one chain from the launcher: a group's dot2s, then one add a group. Every contract shape still passes within it.

The GPU gate checked only the revision and the Transformers tree of its references, so a record without its checkpoint, weight or library hashes still passed where the oracle's checker refuses it. Both references must now carry the complete record, the same in each, and a refused reference fails without printing a diff.

Two references could carry the same wrong pin maps and still pass the GPU gate, which never looked at the checkpoint it loads. The test now hashes every pinned checkpoint and weight file in the directory it loads and requires every shard of the index to be pinned, before any GPU work; a host test refuses a wrong hash and an unpinned shard.

Probing SGLang's Qwen3-Embedding-8B server with the same requests as this one showed two gaps: whitespace-only text was embedded, and one input longer than the context was refused only as an over-full batch. Text that is empty or only Unicode whitespace, as Python's strip sees it, is now refused with its index in a list, and an input past the context is refused with its token count before the batch's own limit applies.

The BF16 dense GEMM runs only where the build has BF16 activations; on an FP16 build (RDNA2) its kernel keeps only a trap body, and the qwen3 engine failed late as a scratch count error. It now refuses before the checkpoint loads, naming the GPU it was built for.

The new qwen3 HIP family needs a recipe like the other native models: what it serves and refuses, how to build and serve it on gfx1201 with ROCm 10.1's hipcc, the request form the model card uses, and the pinned oracle and GPU gate with their results on a Radeon AI PRO R9700.

Review of ashhart#1978 asked for the request refusals this branch adds to every embedding family to be written down. A new Embeddings section in docs/api.md lists the whitespace-only text refusal, the over-long input that is named with its token count and the context_length_exceeded code before the batch limit, the batch limit itself, and how dimensions crops to an MRL width, and the CHANGELOG gains the Qwen3-Embedding-8B entry that points to it.

Review of ashhart#1978 asked for the pinned fidelity set to cover 2K to 32K inputs and MRL widths 32, 1024 and 4096. The oracle adds five long inputs, each the longest whole-line prefix of a pinned file within 2K to 32K tokens, pins every input's text and token IDs by SHA256, runs the long ones alone at full length, and checks a server in requests that fit its window, at full width and at each MRL width against the cropped fp32 vector; the GPU test covers the same nine inputs, and the recipe records the measured distances.

Merging main put hip_records.zig past lean_check's 600-line limit, so the tf_heads_rows record joins the BF16 dense GEMM record in hip_records_qwen3.zig, whose tests are now collected from the contract module's test block; the record values and chip order are unchanged. The oracle's new comments and one docstring become single lines, so lean_check reports the same problems as main.

laudney · 2026-11-03T09:22:54Z

Thanks @ashhart. Both asks are done, and the branch is rebased on 824e56281 (head a56bda4e9, 22 commits, no merge commit) so it lands after #1979 as you asked on #1981.

Ask	Before	Now
(1) shared `/v1/embeddings` refusals documented	not in `docs/api.md` or the CHANGELOG	new Embeddings section (whitespace-only text, an over-long input named with its token count and `context_length_exceeded` before the batch limit, the batch limit, MRL `dimensions`) and an Unreleased CHANGELOG entry
(2) pinned fidelity set	the card's 4 inputs (8 to 31 tokens), full width only	the card's 4 + 5 long inputs of 2,044, 4,043, 8,177, 16,383 and 32,759 tokens; full width and MRL widths 32, 1024, 4096

Results on one R9700 (gfx1201, ROCm 10.1), cosine distance to transformers fp32:

Width	Worst over 9 inputs	Limit = min(0.001, 2 x transformers BF16's worst at that width)
4096 (full)	1.32e-4 (card query 1); long inputs 1.16e-4, 5.24e-5, 7.00e-5, 9.90e-5, 9.22e-5	3.14e-4
1024	1.18e-4	3.25e-4
32	7.92e-5	3.77e-4

Every input, the 32,759-token one included, gives the same fp32 bytes alone, batched and from token IDs. Every vector has unit norm at every width. qwen3-hip-gpu-test passes the same 9 inputs on the GPU without a server.
The pins: each long input is the longest whole-line prefix, within 2K/4K/8K/16K/32K tokens, of a pinned file. The files are the card's LICENSE and README.md, and Transformers 5.22.1's tokenization_utils_base.py, generation/utils.py and modeling_utils.py (now also in LIBRARY). Every input's text and token IDs are pinned by SHA256 (INPUTS in the oracle, embed_inputs in the GPU test), and both refuse other inputs before the model loads. An MRL reference is the fp32 vector cropped and renormalized. The card's inputs still run as one padded batch, as the card runs them; each long input runs alone.
The commits: docs(api) for ask 1, test(qwen3) for ask 2. refactor(contract) keeps lean_check at main's 424: the merge with test(hip): re-derive every gate-5 record with ROCm 10.1 #1979 put hip_records.zig past 600 lines, so the tf_heads_rows record now sits with the dense GEMM's in hip_records_qwen3.zig, with values unchanged.
Gates on the new head: zig build test -Dhip=true 518/518 steps, hip-kernel-test passes, and hip-kernel-contract passes 60 kernels on gates 1 to 4, with the 5 controls red. The only limits problems are main's own tf_sum_partials / tf_sum_partials_act records, stale on ROCm 10.1. This PR touches neither those kernels nor their records.

Main has moved by 3 commits since the rebase. The only conflict is the CHANGELOG's first Unreleased line: keep both entries. I can rebase again whenever you want.

On the rebased ashhart#1978 base zig/build/cuda.zig reached 601 lines, past the lean checker's 600, with this PR's three lines for the rerank host step; the step moves unchanged (filter, side effects, name and description) to zig/build/rerank_host.zig, called with one line, as proposal_vocab_host.zig does.

ashhart · 2026-11-03T11:13:44Z

Thanks @laudney. An independent re-review on current main:
Applying it:

All 22 commits apply on main d8b80bd. Commit 20 (docs(api)) conflicts only in CHANGELOG.md's first Unreleased
line. The author says the same: keep both entries.
The lean check is 424 at the head. The author lines are @laudney's noreply.

Nothing was built or run here.

Verdict: CHANGES (one, small), then LAND before #1981. Both asks are met as we posted them. The one change
makes the oracle's bound follow the rule the lead accepted for #1981. The posted numbers already meet it.

Do the 22 commits equal the ones checked inside #1981?

Yes, all 22. By git patch-id --stable, commits 1-22 here equal commits 1-22 of #1981's series, one for one, subjects in the same order. That includes the three #1978 commits found new: the
recipe, docs(api), the wider fidelity set, and refactor(contract) moving the qwen3 HIP records into their own file
to keep lean at 424.

Ask 1: document the shared `/v1/embeddings` refusals. Met.

Commit 20 adds an Embeddings section to docs/api.md. It covers the request body (input forms, encoding_format,
dimensions) and the three shared refusals, each an HTTP 400 invalid_request_error:

whitespace-only text, by Python's whitespace set, with the input's index in a list;
one input past the model's context, with code context_length_exceeded, named with its token count before the batch
limit is checked;
inputs that together reach the batch limit.

It also says that dimensions crops and renormalizes, and gives Qwen3-Embedding-8B's MRL range (32 to 4,096).

The CHANGELOG entry says every family's /v1/embeddings now refuses a whitespace-only text. That's the cross-family
reach wanted named. It needs the rebase for its first line.

Ask 2: widen the fidelity set. Met as posted.

Commit 21 adds 5 long inputs of 2,044, 4,043, 8,177, 16,383 and 32,759 tokens. Each is the longest whole-line prefix
of a pinned file: the card's LICENSE and README.md, and three Transformers 5.22.1 sources.
Every input's text and token ids are pinned by SHA256, in both the oracle and the GPU test. Both refuse other
inputs before the model loads.
MRL widths 32, 1,024 and 4,096 are checked against the fp32 vector, cropped and renormalized.
The same 9 inputs give identical fp32 bytes alone, batched and from token ids, the 32K one included.

One fact for the lead: our earlier review's own text also asked for "at least 32 inputs from this repo's docs". Our posted comment
on the PR asked only for 2K-32K inputs and the three MRL widths. The set is 9 inputs (4 card plus 5 long), so it meets
the posted ask, not the longer list.

The bound, under the rule accepted for #1981

The oracle and the recipe gate each vector at min(0.001, 2 × transformers BF16's worst distance) from fp32
(oracle_qwen3_embedding.py:346-352). That's twice BF16's distance. The rule the lead accepted for #1981 is
transformers' own BF16 distance from FP32, rounded up. Measured against both (cosine distance, from the author's run on
one gfx1201):

Width	TensorFold worst over 9 inputs	Transformers BF16 worst	PR limit (2×)	Limit under the #1981 rule (1×)
4,096 (full)	1.32e-4	1.57e-4	3.14e-4	1.57e-4
1,024	1.18e-4	1.63e-4	3.25e-4	1.63e-4
32	7.92e-5	1.89e-4	3.77e-4	1.89e-4

TensorFold is inside BF16's own distance at every width. So the stricter rule passes on the posted numbers.

Change: set bar_factor to 1, with BF16's worst rounded up as for #1981, in the oracle, the GPU test and the
recipe. Then restate the limit as transformers BF16's own distance. No new run is needed if the posted distances are
re-gated, but the recipe's table must show the new limits.

What else changed since - The commits are rebased onto a newer main, so they land after #1979, as asked.

refactor(contract) moves tf_heads_rows's record into hip_records_qwen3.zig with its values unchanged, because
the merge with test(hip): re-derive every gate-5 record with ROCm 10.1 #1979 put hip_records.zig past 600 lines.
our earlier review's small points (the GEMM shape rule checked late, served = false as a gate-5 escape, the parallel-suite flake)
aren't addressed in the commit messages. They stay small.

Device runs

The author has posted all of these on one gfx1201 (ROCm 10.1). We'd re-run them only for our own receipt.

HIP, gfx1201 | Qwen3-Embedding-8B at the pinned revision | the oracle's 9 inputs at full width and MRL 32, 1,024 and
4,096, gated at transformers BF16's own distance: every vector inside, unit norm, alone = batched = token ids byte
for byte
HIP, gfx1201 | no model | hip-kernel-contract: 60 kernels pass gates 1-4, 5 controls red
HIP, gfx1201 | Qwen 3.5, Qwen 3.5 MoE, Flash Next, Nemotron | greedy token A/B against main: equal token_sha256
(our earlier review's run 1)

The review asked the embedding gate to follow the rule accepted for the reranker in ashhart#1981: transformers' own BF16 distance from fp32, rounded up, not twice it. The oracle and the GPU test now take the BF16 bar's worst cosine distance at each width, round it up to two significant digits with exact power-of-ten steps (never below it), and cap it at 0.001, so the limits are 1.6e-4, 1.7e-4 and 1.9e-4 at widths 4096, 1024 and 32; --bar-factor goes, and the recipe states the new limits beside the posted distances, which already meet them.

laudney · 2026-11-03T11:23:31Z

Thanks @ashhart. The bound now follows the #1981 rule, in one new commit on top (3b75ed4f4). Nothing was force-pushed.

Width	TensorFold worst over 9 inputs	Transformers BF16 worst	Old limit (2×)	New limit (1×, rounded up)
4,096 (full)	1.32e-4	1.572e-4	3.14e-4	1.6e-4
1,024	1.18e-4	1.624e-4	3.25e-4	1.7e-4
32	7.92e-5	1.884e-4	3.77e-4	1.9e-4

The oracle (limit_at) and the GPU test take transformers BF16's worst distance at each width and round it up to two significant digits. Then they cap it at 0.001. The --bar-factor flag is removed.
The round-up uses exact power-of-ten steps, so it never gives less than the BF16 distance, and an exact two-digit value stays the same. Python and Zig use the same steps, and both have the same tests at every two-digit value from 1.0e-9 to 0.99.
The recipe and the PR body show the new limits. The posted distances are gated again with no new run, as you said, and all of them are inside the new limits.
Checks: oracle unittest 11/11, qwen3-hip-gpu-test host tests pass, lean_check stays at 424.

The CHANGELOG conflict on the first Unreleased line is still the only conflict with main: keep both entries. #1981 is stacked on a56bda4e9, so it does not have this commit yet.

This was referenced Nov 3, 2026

Qwen3-8B embedding and reranker checkpoints (BF16) on native HIP #1977

Open

test(hip): re-derive every gate-5 record with ROCm 10.1 #1979

Merged

feat(qwen3): serve Qwen3-Reranker-8B on HIP behind an SGLang-compatible /v1/rerank #1981

Open

laudney added 22 commits November 3, 2026 08:35

laudney force-pushed the pr-qwen3-embedding-hip branch from ca6bf25 to a56bda4 Compare November 3, 2026 09:21

This was referenced Nov 3, 2026

HIP: run Qwen3-8B's BF16 projections on a gfx12 WMMA tile #2034

Open

Rerank: score a request's documents in one engine decision #2039

Open

Navigation Menu

Repository navigation

Qwen3-Embedding-8B (BF16) on native HIP: BF16 dense GEMM and a qwen3 embedding engine - #1978

Qwen3-Embedding-8B (BF16) on native HIP: BF16 dense GEMM and a qwen3 embedding engine#1978
laudney wants to merge 23 commits into
ashhart:mainfrom
laudney:pr-qwen3-embedding-hip

laudney commented Nov 3, 2026 •

edited

Loading

Uh oh!

ashhart commented Nov 3, 2026

Uh oh!

laudney commented Nov 3, 2026

Uh oh!

ashhart commented Nov 3, 2026

Uh oh!

laudney commented Nov 3, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

laudney commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changes

Gates (R9700, gfx1201, ROCm 10.1)

Overlaps

Uh oh!

ashhart commented Nov 3, 2026

Uh oh!

laudney commented Nov 3, 2026

Uh oh!

ashhart commented Nov 3, 2026

Do the 22 commits equal the ones checked inside #1981?

Ask 1: document the shared /v1/embeddings refusals. Met.

Ask 2: widen the fidelity set. Met as posted.

The bound, under the rule accepted for #1981

What else changed since - The commits are rebased onto a newer main, so they land after #1979, as asked.

Device runs

Uh oh!

laudney commented Nov 3, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

laudney commented Nov 3, 2026 •

edited

Loading

Ask 1: document the shared `/v1/embeddings` refusals. Met.