sd: support for CLIP and VAE on different devices

[wbruna]

Support for placing CLIP or VAE on separate devices (e.g. diffusion on Vulkan0, VAE on Vulkan1). It also enables keeping the diffusion model itself on CPU.

The first two commits adapt the C++ code: the interface receives device numbers instead of booleans, with -1 for "main device" and -2 for "CPU", and the backend includes a global config to choose which model gets which device. The last commit changes the sdclipgpu and sdvaecpu boolean parameters to accept "CPU", "main" or a device number.

Tested on Vulkan with my GPU and iGPU. Seems to work fine with command-line and config settings; however, I wasn't able to fully test the launcher, because there doesn't seem to be a way to select a discrete GPU and an iGPU through it (so I likely got its 1-based indexes wrong).

[LostRuins]

merged your other PR so now this conflicts

[wbruna]

Fixed.

[LostRuins]

So I looked through this PR and it does seem like quite a lot of changes + complexities for something that doesn't seem too useful in my opinion.

Especially with the ability to already use offload_cpu (runtime load/unload for each component) it doesn't really seem too useful compare to simply using the same GPU for all image gen components. Is there something I'm missing.

Also it does modify a bunch of extra upstream code too.

[wbruna]

So I looked through this PR and it does seem like quite a lot of changes + complexities for something that doesn't seem too useful in my opinion.

Especially with the ability to already use offload_cpu (runtime load/unload for each component) it doesn't really seem too useful compare to simply using the same GPU for all image gen components. Is there something I'm missing.

offload_cpu doesn't help situations which benefit from a second GPU for the same gen: a weaker card with more memory (like an iGPU) could e..g. run a video VAE which wouldn't fit on the main GPU. Also, its cost isn't trivial: it pins a lot of extra system RAM, and introduces extra latency for all generations.

We (and upstream) do get requests for this functionality from time to time.

Also it does modify a bunch of extra upstream code too.

Kind of? It's mostly the unavoidable device initialization for each component. I expect that code to change upstream when multi-device support gets implemented, but in that case dropping our changes would be simple enough.

Edit: rebased on top of #2204 to avoid conflicts.