Is there a reason why...

[NoMansPC]

Model switching is still not friendly in Koboldcpp? Yes, I know that we can create those .kcpps files, but trying new models is still a pain in the ass. It frustrates me because this project is constantly up to date, but man, I really wish they would improve this one feature.

[LostRuins]

How can it be made easier? Do you know you can swap models at runtime using the admin endpoint? You can just throw all the gguf files into one directory and swap between them using a single baseconfig.

[NoMansPC]

How can it be made easier? Do you know you can swap models at runtime using the admin endpoint? You can just throw all the gguf files into one directory and swap between them using a single baseconfig.

There are other UIs that have made this easier. You can select a GGUF, context, and some other configs, and then click load. Koboldcpp seems to deliberately make it difficult to swap models, so much so you have to unlock the feature in the admin tab. I'm using it for now since Oobabooga hasn't been updated in months, but it's not been ideal.

[jabberjabberjabber]

@LostRuins develops this and offers it free, and is especially receptive to feature requests compared to almost every other dev running a large project. Are you allergic to asking for things nicely instead of acting like he is peeing in your oatmeal?

[henk717]

Its also just by design. By default KoboldCpp runs in a mode where we assume its a server with other users also using it. The whole security model is around this, the webserver is closely tied to the running model, etc.
If someone can't be bother to enable the admin mode and router mode theres not much we can do, theres no plans to change this to be a default.