Network Music

Realtime online jamming is not very popular because it’s not a very good experience in the current state of technology. The name of the game here is latency.

Musical perception

As a rule of thumb, musicians playing live are able to ignore latency shorter than 5ms. This is easy to prove as most PA equipment in small venues will have latency around this mark.

However, latency above 10ms starts being frustrating, and once you get above 25ms you’ll be pretty much unable to play.

Video encoding

When you are recording video, the signal from your camera needs to be encoded. Like with audio, lossless encoding would take a ton of space so pretty much all consumer video formats are lossy.

If you’re recording to a file and you have a PC or a laptop, the encoder will likely put most priority on the quality of the file, not much priority in the size of the file. If you’re recording on a phone, the encoder will likely compress more to save on space, but even in this case size is not of paramount importance. This is because space efficient encoding requires a lot of CPU power, to the point where some devices might not be unable to record and encode in real time.

That’s part of the reason Netflix or Amazon Video look very good and enable you to stream HD even on relatively poor Internet connections while Skype and Zoom can look less stellar even on good connections on both ends of the call. Netflix is able to compress efficiently ahead of time while your Zoom call doesn’t have this luxury.

Also, video encoding adds some latency.

The Internet

The Internet as we know it was designed as a decentralized system able to withstand natural disasters, sabotage, and bombings of communication hubs. It’s pretty robust but that comes with some drawbacks, and unfortunately one of them is latency.

If you open a terminal on your computer and type “ping facebook.com” you’ll see how fast it takes for the simplest possible data packet to round-trip from you to the closest facebook.com server and back. If you’re really close to one of their datacenters, you might get something in the ballpark of 10-20ms. But typically you’ll see something closer to 50ms. And that’s pretty much the simplest case.

If you’re part of this industry you might object that there’s ways to cut this latency in half or more by using a specialized protocol. Yes, true, online gaming is proof that you can get pretty good results. However, games “cheat” by:

  • optimizing how much data is sent at any given point;
  • having a central server which is the point of reference and “the clock” for what is going on;
  • predicting players’ actions so that a lot of jitter is unnoticeable;
  • “fixing” small player errors that happen due to known latency.

Two-way video communication cannot use those optimizations easily.

Jittter, what’s that?

I mentioned jitter above which is the final nail to the coffin. Jitter is the variation of latency in time. You see, due to network congestion and network re-configuration, as well as hardware and software behavior, latency constantly changes. Imagine how hard it would be to play along if your click track constantly floats between 115 and 125 BPM. “On average” it might have been 120 BPM but you’d still be beyond frustrated trying to play along.

Specialized apps?

The current gold standard for regular VC calls is roundtrip latency of up to 300ms with jitter up to 30ms. That’s way beyond what’s acceptable for jamming.

You might have some success trying out various communication systems designed for gaming where latency is more important than in regular voice conference apps. However, even there you’ll find that the experience is probably bad enough that it’s hopeless. Especially now that so many people are sheltered-in-place so network and system congestion is high. Additionally, gaming systems usually sacrifice audio quality to achieve better performance. For us a relatively low-latency connection that sounds like AM radio would be a tough sell though.

Now that I gave you some technical background, you might be able to communicate the above to the people you want to jam with and try some specialized software like:

Your band needs to be relatively close geographically

Imagine there’s a fiber-optic wire between your computer and your band mate’s computer. If you are 1000 miles away from each other, the fastest roundtrip theoretically possible is 10ms: that’s because of the speed of light. But even a direct fiber-optic cable does not operate at 100% light speed. There will be many “hops”, which is network switching hardware, between you and your friend. In effect, you’re unlikely to see latency smaller than 30ms.

So, if you’re in the same city and the number of hops between you is relatively low, some of the gaming apps or the specialized apps might work. Good luck!

2 Likes

A friend keeps inviting me to Endlesss – I suppose I downloaded the app on Testflight (or what its called) since it has been open for Beta-testing. But I have so many machines calling for me that I haven’t found time to really dive into it.

They have some TV-channel on Twitch as well where you can see people jamming, I believe.

1 Like

In case you missed this:

After a couple years of jamming this way, I still love to do it.

3 Likes

FYI and quickly typed from phone: yesterday I did a short session where I had rtpmidi on windows, ableton as daw, externally synced via midi clock (via rtpmidi) over the internet, where the other end was my music buddy on osx (has rtpmidi inbuilt), where Cubase was sending midi clock. Latency was around 20ms which should be ok for syncing drum machines etc. He was on wifi, we will try tomorrow when both are cabled.
In parallel talking to each other with the help of whereby.com.
Obviously firewall ports need to be opened (5004 and 5005) for udp and port redirection done, so you should have some basic networking knowledge.
Also looking into jamulus, which can be self hosted for better latency (also open source)

2 Likes

Anybody doing any remote jams? What technology are you using? How are you doing it? Should we jam? Tips and tricks appreciated.

1 Like

I haven’t tried using JackTrip in a few years but that is the highest quality way to go I think. Michael Dessen just posted a great playlist introducing the system and how it can be used.

1 Like

I haven’t tried JackTrip and would like to.

Here’s an approach I’ve used a great deal and we’ve had a ton of fun with it:

3 Likes

Thanks I am going to give that a go

netpd is great and has just been updated! i’m trying to organise jams every monday round 23h00 utc+2 (time zone in France)

Unfortunately about 3 hours left in my work day at that hour.

argh, time zones make my brain hurt. if you can suggest a good time for where you are, on the weekend or something then let me know!

This is a handy tool

I’m usually up for a jam between 5pm and 3am UTC on weekend days, with a little advance warning.

1 Like

thanks!
is that the same as between 3pm and 5pm UTC? now i’m really confused!

Oops! Corrected my typo. 5pm-3am

Hi, I just wanted to bring people’s attention to a system that is really very well adapted to collaborative music making in these days of social distancing, long hours at home, etc - Netpd (www.netpd.org). It’s a system built in Pure Data that creates a connection between remote computers and syncs a shared set of instruments, effects, and sequencers.

It requires Pd Vanilla and a handful of externals (listed in the readme). Lots of help is available on the website.

If anyone is interested in getting organised to get a few people on it at the same time we can use this thread…

… or use the new dedicated netpd forum http://untalk.netpd.org/

3 Likes

Definitely interested in getting in on this. I’ve been interested such systems since the pandemic went down and would love to be a part of one that is already in such a mature state. I hadn’t heard of netpd until this post but it has certainly piqued my curiosity.

great! if you have any problems setting it up or using it please post about it here so that we can help anyone else doing the same…

I recommend getting the netpd+instruments bundle from the netpd downloads page.
pd vanilla from here: https://puredata.info/downloads
the externals can be added using Pd’s ‘Help > Find Externals’ function

  • binfile
  • else
  • iemnet
  • iemlib
  • osc
  • slip
  • zexy

I have it running. Just waiting for someone to host a server :wink:

netpd provides a server so nobody else has to (although the server setup can be downloaded from github if you do want to have a private server)!

if no-one else is logged on you can still play around with the instruments and whoever joins will then download those instruments when they ‘unpatch’

so, to give you the netpd basics…
main.pd launches netpd with a chat window
unpatch opens up the instrument panel.
the instruments:
‘master’ - start/stop and tempo
‘mx’ - the mixer (add effects ‘e-lib’, ‘rfxlib’, ‘dynlib’) (click aux for aux channels)
each synth (‘sine’, ‘lilacid’, etc) has an unstep for sequencing or is triggered by ‘qseq3’
live audio can be streamed with ‘evil’

this is all the ‘readymade’ stuff but of course as it is pd the idea is also that you could build your own instruments and so on. they would just have to be given the netpd wrapper in order to become shareable.

Because the software being used for remote choirs is LOW-latency, not NO-latency, perfect rhythmic unity is nearly impossible. Similarly, the unified choral sound typically asked for in traditional rep is also difficult to achieve. These two factors inspired C4 to curate, solicit, and create repertoire that embraces timbre, improvisation, and asynchronous performing, sometimes juxtaposing such sections with moments of metric unity in rhythmically simple and homophonic textures that don’t sound out of place with a little lag.

As I’ve written about previously, such aleatoric writing where individual performers have some creative control over their parts is not new to the music world, but it is uncommon in much of the choral world. Ensembles interested in shifting to a remote choir format, however, seem to show a surge of interest in pieces that allow for dense textures outside of the stereotypical four-part polyphony.

I think it’s fascinating that the technical constraints of network music are inspiring new forms of composition.

On another topic: did anybody get a chance to check out the Network Music Festival last weekend? I’m frustrated because my attention was elsewhere and so I missed it…

https://networkmusicfestival.org/