sd: build each source file separately

[wbruna]

Use Makefile rules to build each sd .cpp source file separately, and create the corresponding object lists dynamically.

I've added an extra header to serve as a 'bridge' between sdtype_adapter.cpp and stable-diffusion.cpp internals, so the former now only depends on the public API and the extension functions. This way, all sd source files can be built without changes, and reused by the sdmain binary.

Additional sources will only need to be included in the appropriate source list (one for Koboldcpp, another for sdmain only). As an added benefit, the build should be faster now, since the individual sources are simpler than the bundle, and can be built in parallel.

This will likely conflict with #2184 , but rebasing should be fairly simple.

[LostRuins]

actually also requires 81f2b5c

[wbruna]

actually also requires 81f2b5c

Ah, the cmake thing.

I wonder if that could be adapted to help updating our static Makefile. Or maybe another Makefile generator that we could run ourselves whenever the source tree changes?

[LostRuins]

Agree the makefile is a little tedious but not easy to replace at the moment - cmake doesn't work too well for a multi target build since in the makefile we do quite a bit of mixing and matching (inherited from back in the day where all backend/intrinsic support was based on compile flags). The makefile also has the advantage of working right out of the box with no dependencies at all.

[wbruna]

Agree the makefile is a little tedious but not easy to replace at the moment - cmake doesn't work too well for a multi target build since in the makefile we do quite a bit of mixing and matching (inherited from back in the day where all backend/intrinsic support was based on compile flags). The makefile also has the advantage of working right out of the box with no dependencies at all.

I was thinking more about manually running a generator which could update the Makefile, then committing the result.

[LostRuins]

hmm..How would the generator know e.g. debug.cpp is only needed for sdmain, or that something.cpp isnt needed at all?

[wbruna]

hmm..How would the generator know e.g. debug.cpp is only needed for sdmain, or that something.cpp isnt needed at all?

Making it fully automatic would be very tricky: we have e.g. to include either sdtype_adapter.cpp or vocab.cpp, depending on the target. But what I miss the most are .cpp -> .h strict dependencies: those could in principle be fully automated.