When attempting to run Gemma4 GGUF from HuggingFace on the IQ9-EVK NPU, E4B-it-qat-q4_0-gguf works, but increasing the size to
26B-A4B-it-qat-q4_0-gguf does not work. It will load on CPU, however. Any ideas how to get the larger model working?
ubuntu@ubuntu:~$ geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf --compute npu
> What is the capital of France?
The capital of France is **Paris**.
— 9.5 tok/s • 8 tok • 0.2 s first token —
> /exit
ubuntu@ubuntu:~$ geniex infer google/gemma-4-26B-A4B-it-qat-q4_0-gguf --compute npu
⚠️ Oops. Model failed to load.
👉 Try these:
- Redownload the model.
- Verify your system meets the model's requirements.
- Check your NPU / GPU driver version and update it if it's out of date.
- See help in our discord or slack.
ubuntu@ubuntu:~$ geniex infer google/gemma-4-26B-A4B-it-qat-q4_0-gguf --compute cpu
> What is the capital of France?
The capital of France is **Paris**.
— 13.0 tok/s • 8 tok • 0.5 s first token —
> /exit
Thanks!
When attempting to run Gemma4 GGUF from HuggingFace on the IQ9-EVK NPU,
E4B-it-qat-q4_0-ggufworks, but increasing the size to26B-A4B-it-qat-q4_0-ggufdoes not work. It will load on CPU, however. Any ideas how to get the larger model working?Thanks!