Gemma 4 template warning: 26B MoE performance drop and broken reasoning tags (1.117)

[perdimanoklboliyezeevich-sketch]

Hi everyone, I'm getting a recurring warning in the console when running Gemma 4 models (using the built-in Gemma 4 Instruct presets) on the recent KoboldCPP versions: "common_chat_try_specialized_template: detected an outdated gemma4 chat template, applying compatibility workarounds. Consider updating to the official template." While the dense 31B model handles this workaround fine (chugging along stably at a heavy but consistent pace), my experience with the 26B MoE version degraded significantly:
Massive Performance Drop: The generation speed on my hardware (GTX 1660 Super with partial CPU offloading) dropped nearly by half — from a smooth ~17 t/s on older versions down to 8-10 t/s now. It feels like the compatibility workaround breaks the optimized dynamic MoE expert offloading, forcing the backend to push execution entirely to the CPU.
Broken Thinking Blocks: The model frequently fails to hide its reasoning process. The tags and internal English drafts either leak directly into the visible response or the reasoning phase gets completely skipped/disabled by the backend to save context.
It looks like the llama.cpp core under the hood expects the new official Gemma 4 token/channel formatting, but Kobold's built-in UI presets are lagging behind, triggering this fallback mode that hits MoE routing hard.
Could we get the built-in Gemma 4 template configurations updated to match the latest official layout natively?
Thanks for all your hard work on the project!

[perdimanoklboliyezeevich-sketch]

Update on the issue: I've done some deeper testing regarding why the vision capabilities and MoE performance degrade so hard under this workaround. Here is what's happening under the hood:
After deep-diving into the backend behavior, the issue is that older GGUF repacks (like early community fine-tunes/mixes) have the early pre-release Gemma 4 template baked directly into their GGUF metadata. Because llama.cpp core recently aligned with Google's final official specification, it now forces a strict compatibility fallback mode whenever it encounters these older files.
This emergency fallback completely destroys performance and functionality on mid-range hardware (like my GTX 1660 Super with partial CPU offloading):
MoE Layer Routing is Broken: In the 26B MoE version, execution gets pushed heavily to the CPU, tanking token speed from 17 t/s down to 8 t/s.
Vision/Multimodal is Broken: The text-and-image interleaving maps incorrectly. The models are literally blinded—they cannot read pixel text (OCR) anymore and spiral into heavy hallucinations, completely losing the visual context.

Since reverting llama.cpp core changes is not an option, could we get an option in KoboldCPP to manually override/force the chat template via the UI or CLI flags?
If we could manually force KoboldCPP to apply the updated official Gemma 4 template structure, bypassing the automated metadata check, it would save these community GGUF files from being completely unusable for Vision and MoE tasks.
Thanks for considering this workaround!

[LostRuins]

Yes, you can override the chat template easily. Simply pass your desired template when loading with --jinjatemplate or via the GUI

You can also just redownload from https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF which works well for me.

[perdimanoklboliyezeevich-sketch]

Thanks for the fast reply! I tried your suggestion and forced the updated official Gemma 4 Jinja template via the GUI.
Unfortunately, the issue goes deeper than just the chat template layout, and it seems to be an underlying issue with how the current llama.cpp core handles early/outdated Gemma 4 GGUF metadata fallback:

Vision is still blind: Even with the official template forced, the visual mapping (mmproj) is completely broken for these older July GGUF repacks. The model receives empty/garbage visual tokens and completely hallucinates the image contents (spinning into wild poetic/philosophical descriptions instead of performing actual OCR on the text).

Severe Performance Hit: Forcing the new official template explicitly triggers the hardware reasoning channels (<|think|>), which drops generation speed on mid-range setups (like my GTX 1660s with CPU offloading) even further - down to a painful 4 t/s (from 8-10 t/s on the fallback template) (for 12b).

It looks like early community repacks of Gemma 4 compiled before the final Google spec alignment are fundamentally incompatible with the new vision/reasoning architecture changes in the backend. Forcing the template fixes the formatting but breaks the weights execution.
I'll probably just have to look for a completely re-quantized post-August GGUF repack that natively bakes the official metadata in.

Thanks again for your help and for maintaining KoboldCPP!

[LostRuins]

okay, well so long as a fresh download with a fresh setup works that should be good