Repository navigation
feat: embedding 第二平台 Cambricon MLU590 接入 - #9
Merged
Merged
Conversation
按既有「三层实现 + 黄金数据」标准把 ops/embedding 从 P800 单平台扩展到 p800-kunlunxin + cambricon 双平台;p800_kunlunxin.py 逐字节未改动。 实现 - 新增 kernel/cambricon.py:前向委托原生 aten::index_select,反向委托 原生 aten::embedding_backward。MLU 原生 padding_idx + scale_grad_by_freq 完整(6/6 组合 err=0),因此不需要 P800/XPU 的 inverse-frequency 补偿 - sparse=True 前向接受、反向显式 NotImplementedError(torch_mlu 会返回 COO 稀疏梯度而不是报错,必须自己拦) - 新增 kernel/platform.py:按 tensor.device.type 路由的门面,附 dispatch_key_for / synchronize;测试、bench、A1 注册共用一条路径 - A1 key = AutogradPrivateUse1:两个 key 都命中,但 PrivateUse1 会覆盖 functorch 的 batch rule;Autograd key 同时与 p800 AutogradCUDA 同构 - 平台/profile/门禁命名统一 cambricon(别名 mlu/mlu590 归一) - register_a1 增加 MLU 侧注册守卫;_profile 支持 cambricon 与自动探测 验证(MLU590-M9 / mlu:0) - kernel 20/20 前向 + 6/6 反向 err 0;黄金 174/174 逐位 worst=0 - A1 op 拦截 5 次、注册=直调逐位;应用层拦截 8 次、logits/梯度 0 diff、 greedy 1.00、flag_gems gelu 生效;example.py 三层 PASS;20 单测通过 - perf gate:ops.embedding.cambricon.forward 0.060ms/142.8GB/s、 native.forward 0.045ms/187.9GB/s,FAIL 0 · WARN 0 · NEW 0(基线 33 case) 修复 - bench_dispatch.py a1 分支裸调 register_a1:返回的 torch.library.Library 无引用即被 GC 并反注册,历史 a1 数字(P800 A1−native ≈0.024ms)测的是 未注册状态,已作废并标注待复测;native/direct 与三层精度结论不受影响 - bench_perf.py 反向原生 scale 改为按平台 try/except,MLU 反向表补 native 对照 文档 - 新增 reports/cambricon.md(环境/实现选择/dispatch key/验证/性能/Library-GC 缺陷与复现);REPORT、README、四件套、ops/README、CHANGELOG 同步双平台口径, 并修正 ops/README 的黄金 117/117 → 174/174
# wt <wangt635@ustc.edu.cn>
TruNcat3
force-pushed
the
feat/embedding-cambricon-mlu
branch
from
October 7, 2026 02:46
63066db to
ea0f721
Compare
Collaborator
Author
|
📦 后续维护提示(rebase 合入后补记): 本 PR 合入后 ops/ 做了全库审计整改(#16),与你相关的变化:
另:rebase 后的合并版本在 MLU 机器上建议复跑一次 |
2 tasks done
zzy2039
pushed a commit
to zzy2039/FlagOS-OP
that referenced
this pull request
Oct 11, 2026
FLAG-OP#9 rebase 并列策略遗留: embedding 索引双行/平台合计注记双条 → 单行 三平台终态; sdpa_math 双平台已有 PLATFORM.md, 补齐 MERGE.md 七步。 # wt <wangt635@ustc.edu.cn>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
把
ops/embedding从 P800 单平台扩展为 P800/Kunlunxin + Cambricon MLU590 双平台,按仓库既有的「三层实现 + 黄金数据」标准交付。kernel/p800_kunlunxin.py逐字节未改动。平台命名沿用仓库既有约定
cambricon(与 13 个 MLU 算子、configs/devices/cambricon.yaml、perf/baselines/cambricon.json、flag_gems vendor 同名),别名mlu/mlu590/cambricon-mlu590归一。实现
ops/embedding/kernel/cambricon.py(新增)aten::index_select,反向委托原生aten::embedding_backwardops/embedding/kernel/platform.py(新增)tensor.device.type路由的门面 +dispatch_key_for/synchronize(复用ops/sdpa/kernel/triton_level.py的 facade 写法)ops/embedding/{register,_profile,example}.pyops.embedding.p800.forward,不改名)ops/embedding/test/*、script/*关键决策(均有实测依据,详见
ops/embedding/reports/cambricon.md):embedding_backward是完整的:padding_idx+scale_grad_by_freq与 CPU 参考 6/6 组合err=0,因此不做 P800/XPU 那套 inverse-frequency 补偿,MLU 后端是零补偿纯委托(原生缺口只在 XPU)。sparse=True前向接受、反向显式NotImplementedError:torch_mlu 对sparse=True反向会返回 COO 稀疏梯度而不是报错,必须自己拦;守卫用例覆盖。AutogradPrivateUse1:PrivateUse1与AutogradPrivateUse1实测都能命中,但PrivateUse1会覆盖 functorch 的 batch rule(BatchRulesModules.cpp报 Overriding warning);Autograd key 同时与 p800AutogradCUDA同构。p800_kunlunxin.py不动。验证(MLU590-M9,
MLU_VISIBLE_DEVICES=7,卡mlu:0)err=0;哨兵、守卫 2 项通过flag_gems.only_enable(['gelu'])生效python3 example.py cambricon三层 PASSpytest tests/unit -q20 passed性能
ops/embedding/reports/perf_fp16_cambricon.json(warmup=20 / iters=100,每次读回一个输出元素强制完成,speedup = native/ours):backward 16k×D128:dense 0.2651 / 0.2497ms(0.942x),
scale_grad_by_freq0.2944 / 0.2796ms(0.950x,原生分支仅 +11%,P800 手写补偿 +47%)。perf gate:
ops.embedding.cambricon.forward0.060ms / 142.8 GB/s、ops.embedding.native.forward0.045ms / 187.9 GB/s,FAIL 0 · WARN 0 · NEW 0;perf/baselines/cambricon.json增至 33 条 case(P800 基线不动)。顺带修的 bench 缺陷(影响历史数字)
bench_dispatch.py的 a1 分支原来裸调register_a1(...):返回的torch.library.Library没有引用时会被 GC,析构即反注册,于是 a1 子进程实际测的是「未注册状态」(F.embedding落回原生)。_a1_lib并断言;全仓扫描确认这是唯一裸调用点(sdpa 与各测试都持有lib)。A1−native ≈ 0.024ms、"A1 与 direct 同档/略快" 不可用,需在有torch_xmlir的环境复跑(本机无torch_xmlir,ModuleNotFoundError)。native / direct 两行不受影响。lib,拦截计数 5/8 均 > 0)。另修
bench_perf.py反向分支:原是 XPU 专用的if not scale:跳过,MLU 拿不到 native 对照 → 改为按平台 try/except。文档
ops/embedding/reports/cambricon.md(环境 · 实现选择 · 平台命名 · dispatch key · 验证 · 性能 · dispatch 与 Library-GC 缺陷 · 关键发现 · 复现命令)REPORT.md:硬件平台 1→2、双 key 路线、实现矩阵 +2 行、P800/cambricon 两套验证矩阵与关键数字、交付物 +4 项、遗留 +1 条README.md:标题 / 一页看全 / 快速开始(两平台)/ backward 与 Triton 章节 / 边界reports/{accuracy,test-report,performance,development}.md:补 cambricon 结果,标注作废的 0.024msops/README.md:embedding 行 1→2 平台,并修正既有错误「黄金 117/117」→ 174/174;库级汇总句同步CHANGELOG.md:Unreleased 新增 embedding 第二平台条目 +Fixed(Library GC、native scale 对照)校验
本地复现 CI 全部 5 步:Markdown 链接与锚点 0 错误 · Python
ast.parse通过 · YAML 通过 ·scripts/intake_validate.py --validate-only0 错误 · 性能基线结构检查通过;另pytest tests/unit -q20 passed。已知但非本 PR 引入:
scripts/check_env.py --device cambricon报 6 项不符,是d5c7a3d写锁的容器镜像(torch 2.12.1 / torch_mlu 1.34.1 / flag_gems 5.0.2 …)与本机 torch2.11 实测栈的既有漂移;--device mlu590为 OK。