Repository navigation
feat(musubi): Ideogram 4 LoRA 支持(前后端) - #425
Merged
Merged
Conversation
- families.py: per-family spec (scripts, network module, constraints, sampler/cache defaults) so adapter/launcher/preflight/run stay generic - manifest: register ideogram4-lora -> ideogram4; capabilities updated - adapter: family-aware constraints (FP8-only base, no turbo/weighting_scheme, blocks_to_swap<=33, asymmetric-CFG opt-in) and cache-only field filtering - launcher/process/run: family scripts, cache-stage extra flags, per-family tokenizer patching on launch - model_assets: ideogram4-lora assets (FP8 components + tokenizer) and the upstream rotary-embedding patch (pre-v0.3.6 snapshots) - config_import: detect ideogram4-lora configs and route them to their page
- new ideogram4-lora schema (FP8 components, ideogram4_shift sampling, caption validation, cache dtype/disk hints, sampler presets) - module table/i18n/router/catalog for model Ideogram 4 on musubi - MusubiGatePage takes the selected schema instead of hardcoding Krea 2; TrainingPage preflights every musubi schema - carry-over guard: vae/turbo/unconditional-DiT/fp8 flags no longer cross Krea 2 and Ideogram 4 pages
18 tasks
Owner
Author
|
验证补充:全量 pytest 已完整跑完(包含 5 个失败与前述一致(4 个 symlink 用例因 Windows 权限、1 个 README CLI 文档用例),均已在干净 |
- assets: add ModelScope mirrors for every Ideogram 4 component — the download dialog defaults to ModelScope, and a missing mirror failed the whole batch - tokenizer patch now follows (and reverts from) the local directory, and writes the installed source atomically - vae_dtype stays in the train TOML (it also drives the sampling VAE load); only text_cache_dtype is cache-only, and dry-run returns the filtered values - an unregistered family variant fails loudly instead of silently running Krea 2 - drop ideogram4_timestep_mu/std (ignored upstream) and annotate the inert preview CFG/steps fields - carry-over denies are scoped to model-family pages, so kohya keeps sharing an external VAE and its fp8 switches - tests: registry mapping, asset sources, patch compile guard, tokenizer re-point, dispatch guard, kohya carry-over
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
实现 #424:Musubi 引擎支持 Ideogram 4 LoRA(前后端)。
做了什么
后端
mikazuki/engines/musubi/families.py:把模型族(Krea 2 / Ideogram 4)的脚本名、LoRA 网络模块、参数约束、采样与缓存默认值集中登记;adapter/launcher/preflight/run/process全部改为按族分发,不再写死 krea2。manifest.TRAIN_TYPES增加ideogram4-lora → ideogram4,CAPABILITIES同步;/api/engines/musubi/status增加train_types。fp8_base/fp8_scaled)、turbo_dit仅 Krea 2、weighting_scheme只能none、blocks_to_swap ≤ 33、非对称 CFG 需显式开启use_unconditional_dit_for_lora_sampling;vae_dtype/text_cache_dtype作为缓存阶段参数传给缓存脚本、不写入训练 TOML。model_assets:ideogram4-lora资源清单(conditional/unconditional DiT、Qwen3-VL-8B 文本编码器、Flux2 VAE、tokenizer 目录)与两处上游补丁:load_ideogram4_text_encoder()在to_empty()后没有重建 rotary embedding,inv_freq未初始化会让文本缓存变成垃圾值(上游 v0.3.6 / PR #1139 修复)。这里不升级 pin,改为幂等源码补丁 + 写入前语法校验。ideogram4-lora(network_module + 标记字段 + 路径规则),跨页导入会提示跳转到/lora/ideogram4.html。前端
mikazuki/schema/ideogram4-lora.ts:FP8 组件路径、ideogram4_shift时间步采样、caption 结构校验开关、text_cache_dtype磁盘提示、blocks_to_swap上限 33、SageAttention 不可用说明、三个官方采样 preset。/lora/ideogram4.html)/ 引擎标签增加 Ideogram 4;MusubiGatePage不再写死 Krea 2,TrainingPage对全部 musubi schema 走预检。vae/turbo_dit/unconditional_dit/fp8_*不再在 Krea 2 ↔ Ideogram 4 之间串台。文档
docs/ideogram4-musubi.md:许可(非商用、仅发 FP8 量化权重、整合包不预置)、组件清单、缓存磁盘占用(bf16 约 13.6 MB/图)、参数要点、上游补丁说明与已知限制。验证
python -m pytest tests/test_musubi_ideogram4.py tests/test_musubi_backend.py tests/test_musubi_routes.py tests/test_engine_registry.py tests/test_model_assets.py tests/test_config_import.py tests/test_musubi_install.py→ 全绿(新增 28 项)。npm run typecheck、npm run lint(0 error)、npm test→ 61 文件 / 363 测试全绿(新增 schema、模块表与 Gate 页用例)。origin/dev检出上复现确认:4 个 symlink 用例因 Windows 权限失败、test_diffsynth_engine.py::test_missing_shard_fails_before_submission单条耗时约 104s(不是挂死)。还没做(需要 GPU 真机)
blocks_to_swap档位建议上游为 experimental(社区反馈过 TE 预缓存 OOM / resume 变慢),真机验收完成前不建议把这条能力写进发布说明。