Duplex Stream Support

[mitchmindtree]

This is a tracking issue for supporting duplex streams. Duplex streams are streams that have device-synchronised input and output, an essential requirement for many real-time and pro-audio applications.

This is a follow-up to #116 but focused specifically on duplex support.

[sletz]

CoreAudio has a notion of "aggregate" device that allows to combine several inputs/outputs in a single device, that will 1) deal with audio clock synchronization 2) provide a single full duplex callback to deal with.

This aggregate device can be built with an API (not only the system Audio Midi Setup tool).

Possible code to look at, Rust here at : https://github.com/padenot/cubeb-coreaudio-rs or C++ here from JACK project here :
https://github.com/jackaudio/jack2/blob/develop/macosx/coreaudio/JackCoreAudioDriver.h and https://github.com/jackaudio/jack2/blob/develop/macosx/coreaudio/JackCoreAudioDriver.mm

[xasopheno]

+1 for this feature. It would allow me to move from portaudio and to cpal.

[ErikNatanael]

In JACK duplex streams are the generally the default. Or rather, all ports opened by a single client will be handled by a single callback regardless of their type:

• A client is created
• Input and output ports are opened for the client
• The ports are transferred to a ProcessHandler containing the single audio callback. The ports give direct access to read/write their buffers in the callback.
• When activating the client it is consumed together with the ProcessHandler yielding an AsyncClient.

Hope we can find an abstraction that works well for all Hosts that support duplex streams!

[baadc0de]

Hey guys, I'm considering the complexity of this feature and whether it's something I could feasibly tackle. Is it possible for someone more experienced with the code to make a rundown of what would need to be coded/refactored for this to land? Anyone started to work on it that I could join / support?

[baadc0de]

I've made a stab at it for ASIO. It compiles and runs without crashing, but I'm remotely controlling my test windows machine so I don't know if it actually works (reads and outputs samples).

The code is in this branch:
https://github.com/audiocloud/cpal/tree/baadc0de/duplex-streams

There is a new method DeviceTrait::build_full_duplex_stream.

[ErikNatanael]

Thanks for making progress on this @baadc0de! If there is agreement on the interface for creating/using duplex streams I can do the JACK implementation (very small changes needed). Are we following the structure in #116?

Maybe a first implementation could just have all hosts that don't support duplex streams (yet) not return any default device/valid formats for duplex and we can then implement them one by one instead of one huge PR for every host to keep it from being too great of an undertaking?

[baadc0de]

Oops I did not check #116 for the proposed API. It seems good, much better than what I added. I'll see if I can rework my end to conform. It also seems better to return empty iterators for duplex formats / devices when duplex is not supported. My current implementation has a blanket unimplemented! in the trait instead, which is not ideal.

[ollpu]

I took a stab at an end-to-end design for the API: see #553.

@baadc0de I'd be happy to incorporate your work on ASIO into this once there is some consensus on the details.

[kawogi]

Does a duplex device always have one input and one output or can there be more of them (like merger, mixer, splitter, etc.)?
How flexible are those in- and outputs configuration wise? I assume they all have the same sample rate. But what about sample format (i32, f32, …), endianness, number of channels or interleaving? What about buffer sizes? Are they guaranteed to be the same length across all in- and outputs per call? Do all those mixtures exist in the wild or are they more coupled?

Maybe I can help to develop and implement a proper API if I learn more about the use cases.