audio over UDP
ideas, opinions, inspirations
interested because
ORCA,Pilot,gull
https://www.instagram.com/p/B4-9QWyJESG/?igshid=1c8bb4e6va7q7
2 Likes
AVB is purpose built for this as it has the time concept built into the entire structure of the protocol. Itâs also a well accepted standard. UDP has no built in notion of time or bandwidth and requires many other layers on top to make it reliable and functional for such purposes.
2 Likes
Hereâs the AVB SDK:
Itâs not clear to me if this is ready for software workflows yet; they havenât put all parts of the stack into this SDK yet.
Another option is Dante, a similar stack that also offers a couple out-of-the-box apps for connecting software on the same box and across the network with virtual soundcards. The apps arenât free, unfortunately ($30-50) but your music apps donât need to have specific Dante support, they just connect to virtual soundcards, so you could in theory use it with Orca etc. immediately.
Another option for building network support directly into apps is NDI. Itâs designed for video but it supports audio too, their SDK is not open source but it is royalty-free. A lot of video apps have built-in support for NDI already.
https://ndi.tv/sdk/
I donât see any problems with using UDP for transporting audio.
Yes, AVB (and Dante) is a great standard for large scale systems in concert halls and sporting venues. But depending on your application it might be totally overkill to implement. Additionally AVB requires AVB-specific routers/switches to work.
Iâve written some protocols for transporting audio data over UDP and have been pretty satisfied. Mostly they existed in the form of JACK clients for Linux.
Itâs really important not to optimize before testing (what may or may not be considered a naive solution).
EDIT: Also, from what I remember, OpenAvnu absolutely requires an Intel I210 Ethernet controller
Also, to note: ORCA is a MIDI app (and OSC and UDP already, of course), and MIDI can also be transported over the network using an existing standard, RTP-MIDI.
Do not use UDP for audio without very good reasons and lots of engineering available.
The obvious issues of not guaranteed delivery, out of order reception, and duplicate reception, are just the start. Network intermediaries make many assumptions about UDP based protocols that can make things even worse. Implementations other than Berkeley Sockets (what you get on Linux) can be pretty iffy for UDP.
I have many years of network protocol design and implementation⌠include several years specifically on media applications.
Is there a reason you donât want to use TCP? Whatâs the context?
5 Likes
so cool, thank you
fantasticâŚ
context, art:
https://vimeo.com/373813078
no network needs, same machine, linux
(elementary Juno, MBP 13" mid 2012)
itâd be great to have an app for ORCA
that has 10 chan of synth a la pilot
and 6 chan of sample/slice a la gull
theyâre UDP
ability to export .wav file (pilot has this)
and hundredrabbits themes (skins, pilot has this)
probably an electron.js idea
2 Likes
Ah - okay - so several things to think about:
â When pilot and gull say they are a âUDP synthesizerâ and âUDP sound machineâ, they mean that they are controlled via UDP messages, not that they send audio that way.
UDP for this kind of control is okay⌠if you donât mind what happens when you lose a packet. And yes, you can lose a packet even if there is no physical network, and both applications are on the same machine (though it is rare if you keep the load downâŚ)
In these synts the worst that will happen is youâd lose a note or trigger, or miss a parameter change. In practice, with ORCA on the same machine, not likely an issue (well, unless it is a very underpowered machine).
â You could easily build an app that had the sound generation features you desire⌠and 16 channels of control is find for UDP - and in this situation, unlikely to be a concern.
What you wouldnât want is to run both something pilot-esque and something else gull-esque⌠And try to pipe audio between them (or mix them) via UDP. Instead, youâd push audio around via Jack (which if you needed a real network, has that stuff available).
â But - ORCA can send OSC just as easily as raw UDP⌠so why not just build the audio engine you want in SuperCollider, and send it OSC? SuperCollider has exactly the right architecture here: Control routing via OSC, and audio routing either internal to the scsynth engine, or externally via Jack.
SuperCollider can trivially record the final output to WAV or AIFF files - so its got that going for it, too.
3 Likes
thank you,
so cool to have your insights about this
1 Like
+1 on JACK, especially since it seems like you are looking for ways to share audio between applications on the same computer. It was built for that.
After years of using JACK on linux, I learned recently that it also works quite well on OSX on top of core audio if you donât mind using the jackd commandline interface. As a developer, this simplifies things quite a bit. Instead of having to target each platform backend, all I have to do is write something using the JACK api.
There are a few options for JACK over a network, though I havenât used any myself. JackTrip and netjack come to mind. It looks like JackTrip has more recent development activity than netjack.
Necro-bumping this with the question whether anyone ever tried to run VBAN on the Monome Norns.
I havenât heard of VBAN before but I have spent time trying to get the Merging RAVENNA driver working via aes67-linux-daemon. The only norns device I would expect much success with using either solution would be an rpi4 based shield. The rpi4 has its ethernet port connected over PCI instead of USB which should yield lower CPU overhead and lower latency.
Unfortunately Iâve found the Merging RAVENNA driver for Linux very unreliable even when the Linux machine is very powerful (high end 32 core AMD workstation). Their macOS and Windows drivers are much more stable, it fact the macOS driver is quite solid driving 32 channels at 48k in/out of an old 2013 mac pro.
Was contemplating getting an updated Norns shield anyway, since mine is from 2019. So Iâd go for a Pi4 based one.
Digging a little further I found netjack, an extension for the jackd ecosystem. Maybe this would be a more promising route?