Feature idea for ComfyUI integration: Add a way to paste or attach API workflow json

[Thoron77]

I'm running ComfyUI dedicated server in my home network and it would be nice to be able to use custom workflows instead of the default checkpoint one build in the integration.
Saving workflow as API one in ComfyUI basically looks like the one in klite.embd at generate_comfy_image function in genimg_payload variable. The pasted/loaded json with workflow would overwrite the setup fields and instead take all the parameters as is from the json.
But of course I may be completely wrong on how it works.

[Thoron77]

A little update, if you'll be considering it, its important to beside adding the json with API workflow, change in poll_image_db() hardcoded filename object structure to something to iterate trough outputs. I think it will be up to the user to make sure the last image in the structure will be the final one ;)
The line with problematic "9":
let finalfilename = resp2[key].outputs["9"].images[0].filename;
To something like:
let finalfilename = ""; let outputs = Object.keys(resp2[key].outputs); outputs.forEach((el) => { if(resp2[key].outputs[el].images[0].filename){ finalfilename=resp2[key].outputs[el].images[0].filename; } });
I know its far from optimal or perfect, but it works ;)
Also it was a pain to figure out the outputs is not an array :)

[LostRuins]

What is the default workflow currently lacking? Perhaps it can be added as a toggle/param instead of having to handle an entire dynamic workflow.

[Thoron77]

I'm by no means an expert, I'm just tinkering with your awesome solution.
I was thinking about it but it may not be possible. The way it is setup right now is with using checkpoints assuming those checkpoints have both clip and vae baked in. Which is really nice and elegant solution minimizing complexity of the workflow. But I wanted to use Flux models and those checkpoints usually lack both vae and clip and those need to be injected separately, which complicates it all.
Thanks to your awesome feature to unpack the executable and modify files I've transplanted my own workflow and it works but I understand its wonky at best.

I haven't been able to crack down on ComfyUI API to find a way for example to pull a list of vae or clips, can't even list other models than checkpoints - but thats probably just me ;)

All in all it would be nice to be able to use other models, add loras, controlnets, etc. But that is way beyond the scope of Kobold and probably such a niche thing its not worth your time. This is why I thought adding own workflow which directly corresponds with how custom ComfyUI server is setup would be flexible enough for it. It would be up for the user to make sure all components are setup properly.

Also thank you for taking your time to read my comment :)

[LostRuins]

Oh, that is possible! You just need to load the CLIP, VAE and whatever add-ons in KoboldCpp! It will be handled automatically. You can select individual Model, T5, Clip-L, Clip-G, VAE, and LoRAs in the launcher or command line.

[Thoron77]

I know but I thought it was for local image generation loaded alongside the LLM on the same machine. I have dedicated server for ComfyUI to maximize the vram for LLM on local machine :)