Skip to content

Pin the default ASR, name it in the config, and add --bundle-asr - #164

Open
aviv1ron1 wants to merge 7 commits into
mainfrom
bugfix/155-158-asr-pinning-and-bundling
Open

Pin the default ASR, name it in the config, and add --bundle-asr#164
aviv1ron1 wants to merge 7 commits into
mainfrom
bugfix/155-158-asr-pinning-and-bundling

Conversation

@aviv1ron1

@aviv1ron1 aviv1ron1 commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Fixes #155
Fixes #158

Summary

ASR pinned and named in the config (#155)

  • With --enable-audio, compose always writes asr_model_id and a new asr_model_revision. The default model ibm-granite/granite-speech-5.0-470m-turboctc is pinned to its English-only revision 947f59a.
  • New --asr-revision flag. A custom --asr-model without it composes with a warning and loads unpinned. A revision in asr_pipeline_kwargs still overrides it.
  • Breaking: serving no longer falls back to a built-in ASR. An audio checkpoint without asr_model_id fails at server startup, with the two config.json lines that fix it.
  • DEFAULT_ASR_MODEL_ID moved to granite_switch.config. ASRTranscriber and get_transcriber now require model_id and accept a revision.
  • BUILD.md now lists the ASR model, its revision and whether it is bundled.

--bundle-asr (#158)

  • It's opt-in and turns on --enable-audio. It copies the ASR into <output>/asr/ (config, safetensors weights, tokenizer, feature extractor, README and license) and sets asr_bundled_path.
  • --asr-model can be a local directory, so the bundle can be built inside the airgapped network.
  • Compose fails unless the bundle's config, feature extractor and tokenizer load with local_files_only=True.
  • At serve time a bundled checkpoint loads its ASR from disk. If it's served by Hub repo id, only asr/ is fetched, which also works offline from the HF cache.
  • vLLM's loader only reads top-level weight files, so it doesn't load asr/*.safetensors as LLM weights.

Release

Version bumped to 0.2.0. CHANGELOG.md has a Breaking section. docs/AUDIO.md covers:

  • the new fields
  • an "Airgapped deployment" section
  • migrating checkpoints composed before 0.2.0
  • what an older library does with new checkpoints: it ignores the new fields, loads the ASR unpinned, and still goes online for a bundled checkpoint

Tests

  • test_asr_ctc_default_gpu.py no longer asserts transcript text, which is model quality (# pinned revision and still runs on every PR.
  • New unit tests cover the pinned revision being passed through, choosing between bundle and Hub, refusing a config with no model, and the new config fields.
  • tests/composer/test_asr_bundle.py covers the composer flags and bundling, offline, usingbuilt locally (tests/shared/tiny_asr.py).
  • test_multi_audio_compose_e2e.py now checks that the composed config names the pinned ASR.

@aviv1ron1
Pin the default ASR, require it in the config, and add --bundle-asr
429175f 
The default ASR model was loaded by name with no revision, so the Hub
repo's 2026-10-21 multilingual release silently changed every
transcript (now capitalized and punctuated) and broke the GPU tests
(#155). The default also lived only in the library, so a checkpoint's
config did not show which ASR it ran.

Compose with --enable-audio now always writes asr_model_id and a new
asr_model_revision; the default model is pinned to its English-only
revision 947f59a, and --asr-revision sets it for a custom model (left
unpinned with a warning otherwise). Serving has no fallback: an audio
checkpoint without asr_model_id fails at startup with the config fix.
This is breaking for checkpoints composed before 0.2.0, hence the
version bump.

For airgapped customers (#158), --bundle-asr copies the ASR into the
checkpoint's asr/ folder, sets asr_bundled_path, and checks at compose
time that it loads with no network. Serving loads a bundled ASR from
disk, or fetches only asr/ when served by Hub repo id. --asr-model may
be a local directory so the bundle can be built offline.

test_asr_ctc_default_gpu.py no longer asserts transcript text, which is
model quality and not ours to own; it now loads the pinned revision.

Fixes #155
Fixes #158

Signed-off-by: aviv ron <rona@il.ibm.com>
@codecov-commenter

codecov-commenter commented Nov 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@aviv1ron1

aviv1ron1 commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator Author

/gpu-test-multi

@github-actions

github-actions Bot commented Nov 3, 2026 •
edited
Loading

Copy link
Copy Markdown

✅ GPU tests passed — vllm30-multi

2975 passed, 69 skipped, 24 warnings in 16628.22s (4:37:08)

Commit: 2a5f1cea4779d2b53252d80093766f4fce541ca0
Full run & artifact log

Last 40 log lines 
tests/composer/test_compose_e2e.py:83
  /tmp/granite-switch/tests/composer/test_compose_e2e.py:83: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:124
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:124: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:130
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:130: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:230
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:230: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_upstream_files.py:218
  /tmp/granite-switch/tests/composer/test_upstream_files.py:218: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("upstream_build_e2e")

tests/unit/test_asr.py::TestMonoAndResample::test_resample_real_vllm_backend[8000-16000]
tests/unit/test_asr.py::TestMonoAndResample::test_resample_real_vllm_backend[44100-16000]
  /tmp/granite-switch/.venv/lib/python3.12/site-packages/torch/backends/mkldnn/__init__.py:96: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /__w/pytorch/pytorch/aten/src/ATen/Context.cpp:265.)
    torch._C._set_onednn_allow_tf32(_allow_tf32)

tests/vllm/test_probe_capture_safety.py::test_sr_pipeline_stack_partitions_post_attention_metadata
  /tmp/granite-switch/.venv/lib/python3.12/site-packages/flashinfer/cute_dsl/availability.py:135: DeprecationWarning: Arch from arch.py is deprecated. Use Arch from enums.py instead.
    from cutlass.base_dsl.arch import Arch

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========= 2975 passed, 69 skipped, 24 warnings in 16628.22s (4:37:08) ==========
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute
[rank0]:[W1104 02:29:35.539627397 ProcessGroupNCCL.cpp:1624] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
===== ALL GPU TESTS PASSED =====
[02:29:49] <job> Succeeded
[02:29:52] verified: found success sentinel in pod log
[02:29:52] cleanup: deleting <job> (exit=0)
<job> "<job>" deleted

Verdict: PASSED

@aviv1ron1
Accept any kwargs in the processor tests' get_transcriber stubs
2a5f1ce 
Two stubs in test_audio_processor.py still had get_transcriber's old
signature, so the processor's new revision= argument made them raise
TypeError: 13 tests failed on the vllm30 GPU leg (14 on vLLM < 0.29,
where test_uncached_path_reports_updates_not_applied also runs). The
stubs ignore their arguments, so they now take **kwargs and do not
break on the next signature change either.

Signed-off-by: aviv ron <rona@il.ibm.com>
@github-actions

github-actions Bot commented Nov 3, 2026 •
edited
Loading

Copy link
Copy Markdown

✅ GPU tests passed — vllm26-multi

2976 passed, 68 skipped, 21 warnings in 15616.31s (4:20:16)

Commit: 2a5f1cea4779d2b53252d80093766f4fce541ca0
Full run & artifact log

Last 40 log lines 
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

.venv/lib/python3.12/site-packages/torch/jit/_script.py:365: 14 warnings
  /tmp/granite-switch/.venv/lib/python3.12/site-packages/torch/jit/_script.py:365: DeprecationWarning: `torch.jit.script_method` is deprecated. Please switch to `torch.compile` or `torch.export`.
    warnings.warn(

tests/composer/test_compose_e2e.py:83
  /tmp/granite-switch/tests/composer/test_compose_e2e.py:83: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:124
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:124: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:130
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:130: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:230
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:230: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_upstream_files.py:218
  /tmp/granite-switch/tests/composer/test_upstream_files.py:218: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("upstream_build_e2e")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========= 2976 passed, 68 skipped, 21 warnings in 15616.31s (4:20:16) ==========
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute
[rank0]:[W1104 02:12:37.925283608 ProcessGroupNCCL.cpp:1575] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
===== ALL GPU TESTS PASSED =====
[02:12:56] <job> Succeeded
[02:12:59] verified: found success sentinel in pod log
[02:12:59] cleanup: deleting <job> (exit=0)
<job> "<job>" deleted

Verdict: PASSED

@aviv1ron1

aviv1ron1 commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator Author

/gpu-test-multi

@aviv1ron1
aviv1ron1 marked this pull request as ready for review November 4, 2026 08:11
@aviv1ron1
Clear the ASR bundle directory before bundling into it
481a432 
bundle_asr_model wrote into an existing <output>/asr/ with
dirs_exist_ok and never removed what was there. Re-composing into the
same --output with a different ASR model left the first model's shards,
tokenizer and feature-extractor files beside the new ones; the offline
validation could still pass, and serving then loaded a mix of two
models. The destination is now removed first, and a local --asr-model
that is the destination itself is refused rather than deleted.

Signed-off-by: aviv ron <rona@il.ibm.com>
@aviv1ron1
Filter a local ASR source's subdirectories like the Hub download
060ac40 
The copytree ignore for a local --asr-model only filtered files, so
every subdirectory was copied whole: a git-lfs clone brought its .git/
object store (about another 1 GB), and onnx/ or nested .bin weights
landed in the bundle although the patterns exist to keep them out. The
Hub path filters nested files through allow_patterns, so the two
sources gave different bundles. Local files are now matched by their
relative path with huggingface_hub's own filter_repo_objects, and
hidden directories are not walked.

Signed-off-by: aviv ron <rona@il.ibm.com>
@aviv1ron1
Bundle the ASR before saving the config that points at it
65e882a 
The ASR bundle was downloaded and validated only after save_pretrained
had written a config.json with asr_bundled_path set. A failed download,
or a model with no safetensors weights, raised with the weights and that
config already on disk: the output looked complete and failed only at
serve time. The bundle now runs first, so a failure stops the compose
before the config exists. It is also built in a staging directory and
swapped in only once it validates, so a failed re-compose leaves a
previous good bundle in place instead of an empty or partial asr/.

Signed-off-by: aviv ron <rona@il.ibm.com>
@aviv1ron1
Normalize asr_bundled_path and keep it inside the checkpoint
97032a6 
resolve_asr_source fetched a Hub checkpoint's bundle with the path
stripped of slashes but joined it to the root unstripped, so "/asr"
downloaded asr/* and then looked in the filesystem's /asr. Nothing
stopped a value such as "../../x", from a config that may come off the
Hub, pointing the ASR at any local directory. The path is now
normalized once, used for both the fetch pattern and the load path,
and refused when absolute, "." or reaching outside with "..".

Signed-off-by: aviv ron <rona@il.ibm.com>
@aviv1ron1
Refuse a local --asr-model without --bundle-asr; keep its path private
baed729 
A local directory as --asr-model was accepted with plain --enable-audio
and written verbatim into asr_model_id, so the checkpoint pointed at a
path that exists on the build machine only: it passed startup and then
failed on the first audio request once moved or uploaded. With
--bundle-asr the same full path, e.g. a home directory, went into the
published config and model card.

A local directory is now refused unless --bundle-asr is set, and is
then recorded by its directory name only; the bundle itself is built
from --asr-model. The ASR model is resolved before the model is built,
so a refused value fails the compose in seconds.

Signed-off-by: aviv ron <rona@il.ibm.com>
@aviv1ron1

aviv1ron1 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author

/gpu-test

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

✅ GPU tests passed — vllm26

2984 passed, 76 skipped, 21 warnings in 14732.71s (4:05:32)

Commit: baed729adece0ef1ae3b16f90b5e1654a127addf
Full run & artifact log

Last 40 log lines 
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute

<frozen importlib._bootstrap>:488
  <frozen importlib._bootstrap>:488: DeprecationWarning: builtin type SwigPyObject has no __module__ attribute

.venv/lib/python3.12/site-packages/torch/jit/_script.py:365: 14 warnings
  /tmp/granite-switch/.venv/lib/python3.12/site-packages/torch/jit/_script.py:365: DeprecationWarning: `torch.jit.script_method` is deprecated. Please switch to `torch.compile` or `torch.export`.
    warnings.warn(

tests/composer/test_compose_e2e.py:83
  /tmp/granite-switch/tests/composer/test_compose_e2e.py:83: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:124
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:124: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:130
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:130: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:230
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:230: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_upstream_files.py:218
  /tmp/granite-switch/tests/composer/test_upstream_files.py:218: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("upstream_build_e2e")

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========= 2984 passed, 76 skipped, 21 warnings in 14732.71s (4:05:32) ==========
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute
[rank0]:[W1104 15:13:40.342770859 ProcessGroupNCCL.cpp:1575] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
===== ALL GPU TESTS PASSED =====
[15:13:51] <job> Succeeded
[15:13:54] verified: found success sentinel in pod log
[15:13:54] cleanup: deleting <job> (exit=0)
<job> "<job>" deleted

Verdict: PASSED

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

✅ GPU tests passed — vllm30

2983 passed, 77 skipped, 24 warnings in 15646.23s (4:20:46)

Commit: baed729adece0ef1ae3b16f90b5e1654a127addf
Full run & artifact log

Last 40 log lines 
tests/composer/test_compose_e2e.py:83
  /tmp/granite-switch/tests/composer/test_compose_e2e.py:83: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:124
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:124: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:130
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:130: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_multi_audio_compose_e2e.py:230
  /tmp/granite-switch/tests/composer/test_multi_audio_compose_e2e.py:230: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("multi_audio_compose_e2e")

tests/composer/test_upstream_files.py:218
  /tmp/granite-switch/tests/composer/test_upstream_files.py:218: PytestUnknownMarkWarning: Unknown pytest.mark.xdist_group - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.xdist_group("upstream_build_e2e")

tests/unit/test_asr.py::TestMonoAndResample::test_resample_real_vllm_backend[8000-16000]
tests/unit/test_asr.py::TestMonoAndResample::test_resample_real_vllm_backend[44100-16000]
  /tmp/granite-switch/.venv/lib/python3.12/site-packages/torch/backends/mkldnn/__init__.py:96: UserWarning: TF32 acceleration on top of oneDNN is available for Intel GPUs. The current Torch version does not have Intel GPU Support. (Triggered internally at /__w/pytorch/pytorch/aten/src/ATen/Context.cpp:265.)
    torch._C._set_onednn_allow_tf32(_allow_tf32)

tests/vllm/test_probe_capture_safety.py::test_sr_pipeline_stack_partitions_post_attention_metadata
  /tmp/granite-switch/.venv/lib/python3.12/site-packages/flashinfer/cute_dsl/availability.py:135: DeprecationWarning: Arch from arch.py is deprecated. Use Arch from enums.py instead.
    from cutlass.base_dsl.arch import Arch

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========= 2983 passed, 77 skipped, 24 warnings in 15646.23s (4:20:46) ==========
sys:1: DeprecationWarning: builtin type swigvarlink has no __module__ attribute
[rank0]:[W1104 15:28:57.646037563 ProcessGroupNCCL.cpp:1624] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
===== ALL GPU TESTS PASSED =====
[15:29:09] <job> Succeeded
[15:29:12] verified: found success sentinel in pod log
[15:29:12] cleanup: deleting <job> (exit=0)
<job> "<job>" deleted

Verdict: PASSED

@aviv1ron1

aviv1ron1 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author

@antonpibm gpu tests passed. PR is ready for review and merging

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@antonpibm antonpibm Awaiting requested review from antonpibm antonpibm is a code owner
@freunda freunda Awaiting requested review from freunda freunda is a code owner
@yairallouche yairallouche Awaiting requested review from yairallouche yairallouche is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

audio: include local weights for ASR audio: unpinned default ASR model changed on the Hub, breaking test_asr_ctc_default_gpu.py

2 participants

@aviv1ron1 @codecov-commenter