diff --git a/README.md b/README.md index 6fb12cfcb..b07c26560 100644 --- a/README.md +++ b/README.md @@ -264,12 +264,19 @@ hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf .\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend cuda ``` +Use `funasr-llamacpp-linux-x64-vulkan.tar.gz` on Linux GPU systems with a +working Vulkan driver/ICD: + +```bash +./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav --backend vulkan +``` + The current Windows CUDA package targets CUDA architecture 86. RTX 50 / Blackwell GPUs report compute capability 12.0 (`sm_120`) and should use the CPU package or build from source with `-DCMAKE_CUDA_ARCHITECTURES=120` until a dedicated CUDA asset is published. -**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.7](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.7) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.7/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/) +**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.8](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.8) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.8/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.8/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/) [OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html) @@ -294,6 +301,7 @@ asset is published. ## What's new - 2026/07/22: **v1.3.23 on PyPI** — packaging and onboarding refresh for this week's community integrations: the PyPI long description now highlights the current OpenAI-compatible server path, llama.cpp/GGUF runtime notes, Windows CUDA architecture guidance, and browser quickstart links shipped in the repository docs. Runtime code is unchanged from v1.3.22. Install with `python -m pip install -U "funasr==1.3.23"`. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.3.23) +- 2026/07/22: **llama.cpp runtime v0.1.8** — adds `funasr-llamacpp-linux-x64-vulkan.tar.gz` for SenseVoiceSmall on Linux Vulkan GPUs. Run `llama-funasr-sensevoice ... --backend vulkan`; CPU, AVX2, macOS arm64, Windows CPU/AVX2, and Windows CUDA packages remain available. [Release ->](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.8) - 2026/07/19: **v1.3.22 on PyPI** — `funasr-server` now fills OpenAI-compatible `verbose_json.segments` for text-only SenseVoice/Paraformer fallback responses, so subtitle clients no longer see an empty `segments` array when `text` is populated. Install with `python -m pip install -U "funasr==1.3.22"`. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.3.22) - 2026/07/19: **v1.3.21 on PyPI** — fixes first-import onboarding in fresh environments where users install `funasr` before choosing a platform-specific PyTorch build. `import funasr` and `funasr.__version__` now work without torch; accessing `AutoModel` still requires PyTorch and raises a clear install hint. Install with `python -m pip install -U "funasr==1.3.21"`. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.3.21) - 2026/07/19: **v1.3.20 on PyPI** — PyPI metadata and install guidance now point at the current FunASR docs, community integrations, and quoted `python -m pip install -U "funasr>=1.3.19"` commands for Fun-ASR-Nano deployment paths. This is a documentation/packaging sync; runtime code remains unchanged from v1.3.19. Install with `python -m pip install -U "funasr==1.3.20"`. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.3.20) diff --git a/README_zh.md b/README_zh.md index 87ae5e85b..69c22c5a4 100644 --- a/README_zh.md +++ b/README_zh.md @@ -138,6 +138,7 @@ Whisper 是单个模型,**FunASR 是一个工具箱**——按场景挑模型 - 2026/07/19:**v1.3.19 已发布到 PyPI** — 实时 WebSocket 长会话排障文档已随包发布。启动服务时加上 `--enable-spk --log-session-stats-interval 30`,如果仍遇到断连或内存增长,请在 issue 中附上输出的 `Session stats:` 日志。安装命令:`python -m pip install -U "funasr==1.3.19"`。[长会话诊断 →](docs/vllm_guide_zh.md#长会话诊断) · [发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.3.19) - 2026/07/19:**v1.3.18 已发布到 PyPI** — CLI 的 SRT/TSV 字幕输出现在会请求句级时间戳,并在需要时加载标点模型;`funasr audio.wav --output-format srt --output-dir ./subs` 会输出分句字幕,不再退化成一个全文字幕块。安装命令:`python -m pip install -U "funasr==1.3.18"`。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.3.18) - 2026/07/18:**v1.3.16 已发布到 PyPI** — Fun-ASR-Nano 实时服务新增客户端分句模式。一个 WebSocket 会话可连续发送 PCM,并用 `COMMIT` 提交每个句子;无需加载服务端 VAD,短句可正常结束,多轮时间戳保持递增。执行 `pip install --upgrade funasr` 后,可用 `funasr-realtime-server --endpoint-mode client` 启动。[使用文档 →](examples/industrial_data_pretraining/fun_asr_nano/docs/realtime_demo.md) +- 2026/07/22:**llama.cpp runtime v0.1.8** — 新增 Linux Vulkan 预编译包 `funasr-llamacpp-linux-x64-vulkan.tar.gz`,可在支持 Vulkan driver/ICD 的 Linux GPU 上运行 `llama-funasr-sensevoice ... --backend vulkan`;CPU、AVX2、macOS arm64、Windows CPU/AVX2、Windows CUDA 包继续保留。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.8) - 2026/07/18:**llama.cpp runtime v0.1.7** — 新增 SenseVoiceSmall 的 Windows CUDA 预编译包 `funasr-llamacpp-windows-x64-cuda.zip`,并保留 Linux / macOS / Windows CPU 包。下载 GGUF 模型后,可在支持的 NVIDIA GPU 上运行 `llama-funasr-sensevoice ... --backend cuda`。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.7) - 2026/05/24:**vLLM 推理引擎** — Fun-ASR-Nano 解码加速 2-3 倍。支持流式 WebSocket 服务(VAD + 说话人分离 + 热词)。[文档 →](docs/vllm_guide_zh.md) · [实时 WS 调优 →](docs/vllm_guide_zh.md#67-生产并发与多进程部署) · [API 稳定性清单 →](docs/vllm_guide_zh.md#生产-api-稳定性清单) - 2026/05/24:**动态 VAD** — 自适应静音阈值(默认开启),短句不切碎、长句自动切分。[详情 →](docs/vllm_guide_zh.md#7-动态-vad) @@ -304,11 +305,18 @@ hf download FunAudioLLM/fsmn-vad-GGUF fsmn-vad.gguf --local-dir .\gguf .\llama-funasr-sensevoice.exe -m .\gguf\sensevoice-small-q8.gguf --vad .\gguf\fsmn-vad.gguf -a audio.wav --backend cuda ``` +Linux GPU 用户可下载 `funasr-llamacpp-linux-x64-vulkan.tar.gz`,在已安装可用 +Vulkan driver/ICD 的机器上运行: + +```bash +./llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav --backend vulkan +``` + 当前 Windows CUDA 包面向 CUDA architecture 86。RTX 50 / Blackwell GPU 会报告 compute capability 12.0(`sm_120`),在专用 CUDA 产物发布前,请使用 CPU 包,或从 源码构建并设置 `-DCMAKE_CUDA_ARCHITECTURES=120`。 -**预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.7](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.7) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.7/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/) +**预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.1.8](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.1.8) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.8/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.1.8/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始:** [funasr.com/llama-cpp](https://www.funasr.com/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/) [OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html) diff --git a/runtime/llama.cpp/tests/test_download_funasr_model.sh b/runtime/llama.cpp/tests/test_download_funasr_model.sh index 2bd9e8a6c..30dedb120 100755 --- a/runtime/llama.cpp/tests/test_download_funasr_model.sh +++ b/runtime/llama.cpp/tests/test_download_funasr_model.sh @@ -170,6 +170,18 @@ assert_readme_quickstart() { printf 'non-portable Releases link in %s\n' "$readme" >&2 exit 1 fi + if ! grep -F 'runtime-llamacpp-v0.1.8' <<<"$section" >/dev/null; then + printf 'missing current runtime v0.1.8 link in %s\n' "$readme" >&2 + exit 1 + fi + if ! grep -F 'funasr-llamacpp-linux-x64-vulkan' <<<"$section" >/dev/null; then + printf 'missing Linux Vulkan runtime asset in %s\n' "$readme" >&2 + exit 1 + fi + if ! grep -F -- '--backend vulkan' <<<"$section" >/dev/null; then + printf 'missing Vulkan backend command hint in %s\n' "$readme" >&2 + exit 1 + fi } assert_readme_quickstart "$REPO_ROOT/README.md" '### CPU / Edge'