How to switch decoding mode for UniASR model

[hnluo]

There are three decoding mode for UniASR model(fast、normal、offline)，for more model detailes, please refer to UniASR

If you want to decode in fast mode

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

inference_pipeline = pipeline(
    task=Tasks.auto_speech_recognition,
    model='damo/speech_UniASR_asr_2pass-minnan-16k-common-vocab3825',
    param_dict={"decoding_model": "fast"})

rec_result = inference_pipeline(audio_in='https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav')
print(rec_result)

If you want to decode in normal mode

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

inference_pipeline = pipeline(
    task=Tasks.auto_speech_recognition,
    model='damo/speech_UniASR_asr_2pass-minnan-16k-common-vocab3825',
    param_dict={"decoding_model": "normal"})

rec_result = inference_pipeline(audio_in='https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav')
print(rec_result)

If you want to decode in offline mode

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

inference_pipeline = pipeline(
    task=Tasks.auto_speech_recognition,
    model='damo/speech_UniASR_asr_2pass-minnan-16k-common-vocab3825',
    param_dict={"decoding_model": "offline"})

rec_result = inference_pipeline(audio_in='https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example_zh.wav')
print(rec_result)

[lixikun]

Great job! but I test the model's rtf is slower than the paramform model, Do we have the plan to update to paramform model?

[suyufan]

I ran the above code and got an error
KeyError: 'asr-inference is not in the pipelines registry group auto-speech-recognition. Please make sure the correct version of ModelScope library is used.'

Environment：
Ubuntu
Python 3.9.19
funasr 1.0.22
modelscope 1.9.4
tensorflow 2.4.1
torch 2.2.2
cuda 11.7

Can you help me? give me your environments information.

[LauraGPT]

The decoding_model parameter in the original example is correct, but that example belongs to the legacy ModelScope/FunASR stack used by this 2023 model card.

Choose one mode when constructing the pipeline:

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

mode = "normal"  # "fast", "normal", or "offline"

inference_pipeline = pipeline(
    task=Tasks.auto_speech_recognition,
    model="damo/speech_UniASR_asr_2pass-minnan-16k-common-vocab3825",
    param_dict={"decoding_model": mode},
)

result = inference_pipeline(audio_in="asr_example.wav")
print(result)

Create a new pipeline if you need to change the mode. The decoder/beam-search state is initialized for the selected mode.

The modes mean:

• fast: one-pass, low-latency streaming decoding.
• normal: two-pass decoding; the first pass emits low-latency text and the second pass refreshes it with the higher-accuracy result. This is the default.
• offline: one-pass, high-accuracy offline decoding.

The registry error reported above is a version mismatch, not an invalid param_dict value. asr-inference was supplied by the ModelScope audio pipeline used by FunASR v0.2.0 and ModelScope v1.3.0. Mixing FunASR 1.0.22 with ModelScope 1.9.4 does not provide that legacy pipeline registration.

I reproduced and verified the original API on CPU with this isolated environment:

Python       3.9.25
FunASR       0.2.0 (Git tag v0.2.0)
ModelScope   1.3.0
PyTorch      1.13.1+cpu
torchaudio   0.13.1+cpu
NumPy        1.23.5
SciPy        1.10.1
PyArrow      10.0.1
Transformers 4.26.1

Using the official model snapshot and its bundled sample, all three independently constructed pipelines returned:

欢迎大家来体验达摩院推出的语音识别模型

Use a separate Python 3.9 environment for this legacy checkpoint rather than downgrading a current FunASR environment. A simple current AutoModel(model=...) call is not yet a drop-in replacement for this checkpoint because its repository still uses legacy ModelScope metadata and component names. The current UniASR implementation still maps the three values as documented in the source.

Regarding RTF: UniASR combines streaming and non-streaming paths, and normal performs the two-pass workflow. It is therefore not expected to have the same throughput profile as Paraformer. For batch/offline file transcription where throughput is the priority, use a Paraformer model. Use UniASR when you need low-latency partial results followed by a higher-accuracy refresh. Also note that this particular checkpoint is the Minnan UniASR model, not a general all-language checkpoint.