Norns.online

norns.online

connect your norns to others: https://norns.online

connect. norns.online a both a script and a website.

listen and visualize. after starting, norns.online will run in the background letting you run any other script. all norns audio+visuals will then be beamed to norns.online/<yourname>.

guide. the website has the same inputs as your norns to provide remote guidance and access.

collaborate. connect to other norns via “rooms.” anyone in a “room” shares live audio. the latency is ~4 seconds, so you can imagine that you all are playing together while being socially distanced by a distance of a quarter-mile.

note: the script requires ffmpeg and mpv, which are automatically installed if you use this program (~300 MB).

future directions:

  • fix all the :bug::bug::bug:
  • audio sharing sync (very hard)
  • audio sharing as input to softcut/engine
  • tape sharing?

Requirements

  • norns
  • internet connection

Documentation

  • K3 toggles internet
  • K2 changes name
  • K1+K2 updates
  • more params in global menu

quick start

online

  • press K3. open browser to norns.online/<yourname>. if this is the first time running, wait for the mpv and ffmpeg programs to be installed (~300 MB).
  • use norns normally, your norns will stay online in the background.

norns↔norns audio sharing

room_sharing

  • go to gloal parameters and make sure both “send audio” and “allow rooms” are set to “enabled”.
  • change the “room” to the room you want to share audio. make sure your norns partner uses the same room.
  • go to main screen and press K3 to go online. you should now be sharing audio with any other norns in that room.
  • adjust “room vol” to change the level of incoming audio.

uses

  • make an internet radio from your norns
  • collaborate between two norns
  • twitch plays norns
  • make demos (screen capture norns.online)
  • download screenshots (right-click image at norns.online to download)
  • tech support other people’s norns
  • !?!?!?

faq

how does the norns.online webpage work? norns runs a service that sends screenshot updates to norns.online/<yourname>. the website at norns.online/<yourname> sends inputs back to norns. norns listens to to inputs and runs the acceptable ones (adjustable with parameters). if enabled, norns will also stream packets of audio and send those to the website. the website will buffer them and play them so anyone with your address can hear your norns.
how does audio streaming work? a pre-compiled jack_capture periodically captures the norns output into 2-second flac files into a /dev/shm temp directory. each new flac packet is immediately sent out via websockets and then deleted. because of buffering, expect a lag of at least 4 seconds. when in a room, audio from other norns is piped into your norns via mpv. the incoming audio from other norns is added at the very end of the signal chain so (currently) it cannot be used as input to norns engines.
is this secure?if you are online, you have security through obscurity (weak security). that means that anyone with the url norns.online/<yourname> can access your norns so you can make <yourname> complicated to be more secure. code injection is not possible, as i took precautions to make sure the inputs are sanitized on the norns so that only enc() and key() and _menu.setmode() functions are available. but, even with these functions someone could reset your norns / make some havoc. if this concerns you, don't share <yourname> with anyone or avoid using this script entirely.
how much bandwidth does this use? if audio is enabled, a fair amount. the norns sends out screenshots periodically, but at the highest fps this is only ~18 kB/s. however, if audio is enabled - the norns sends flac packets periodically (~170 kB/s = ~616 MB/hr). if you are audio-sharing a room you will be receiving about that much for each norns in the room. i tried reducing bandwidth by using lossy audio (ogg) however the gapless audio playback only worked without pops when using flac or wav.
how much cpu does this use? not too much. on a raspberry pi 3b+ this uses about ~4% total CPU for capturing and sending audio data. screenshots also take cpu and higher fps takes more. the exact fps depends on the max fps (set in params) and how fast the screen changes (only updated screens are sent). at max it might take up to 30% of the cpu (15 fps!), but usually its 1-15%.

Download

v1.0.0 - https://github.com/schollz/norns.online/archive/v1.0.0.zip

34 Likes

Twenty ↶⟳⤾⤿⤺⤻⤸↺↷↻⟲⤹ of respect

3 Likes

incredible performance opportunities here

two questions:

is the name given permanent? if i wanna give temp access is that possible?

can you explain use of multiple norns simultaneously?

1 Like

the name can be changed at any time. while at the norns.online screen, press k2 and enter a new name. (or you can also edit the name in the /home/we/dust/code/norns.online/config.json file).

for temp access, you might just change the name to something random and then change it back to a more secret name.

you can assign the same name to multiple norns and the browser pulled up to that name will send the same inputs to all of them. one caveat here is that the browser will then receive the screen input from all the norns and show the screens interlaced with each other which could be hard to interpret…but it can be done.

more simply, you could have multiple norns and multiple names for each and then just have multiple browser windows to finagle their controls. i’ve been told this would be useful for a classroom setting in which norns is used as a teaching tool…

agree :slight_smile: i think this opens up opportunities for designing a script that is conducive to crowdsourced control… or audience participation somehow… maybe like a “twitch plays pokemon” there could be a script where people vote on the next chord change or something. my genuine hope is that this enables exploration into other realms of music!

5 Likes

Amazing!

For the radio idea, what about WebRTC?
I’ve just seen it in RPI for camera feeds, but I don’t see why wouldn’t work with the frame buffer.
The process would be:

  1. A media channel from RPI to web client sending the frame buffer (very small resolution) + Audio
  2. A data channel from web client to the RPI with the buttons/encoders.
    It’s a totally different way of doing it, but it can also be interesting checking.
3 Likes

exactly what i wanted to hear!

thanks

1 Like

this is a really good idea. better yet, its led me down a rabbit hole that i think will work. and its easy: essentially i can send bundles of wav samples via the websockets and then play them in the browser audio buffer.

however, there’s a bit of a snaggy situation with getting the audio from the norns. the only way i can think of is to use the tape to record to disk and then just chunk out the recording on disk while it is being written…

is there a better way to tap into the audio stream on norns?

1 Like

With Jack you can hook up to outputs and inputs as you please, here’s an example https://github.com/xmacex/connect-opz/blob/main/lib/connect-opz.sh.

I am not sure what needs to be in the other end of the jack connection… could you connect it directly to a WebRTC input? I’ve no idea.

1 Like

this seems perfect, but i’m not sure how to use it…i likely need to have some sort of ring buffer to pipe it into?

in the meantime i did make something that works. its reallllly hacky though. basically i start recording to tape and then use ffmpeg to take the last X seconds every X seconds as a mp3 and send that mp3 over to the browser which adds it to the buffer. its seemingly working really well, no audible blips. only downside is a long lag time (because ffmpeg takes a long time to trim a file). also it requires you to be recording everything you stream on the norns…

new release: v0.1.0, now with audio streaming! now your norns can instantly be an internet streaming radio…

currently offline check out my norns live: https://norns.online/infinitedigits (audio will start automatically after a few seconds)

thanks @xmacex your hint led me down a good path. i ended up using a jack_capture utility to periodically capture audio on a rolling file basis. files are stored in memory (/dev/shm) and sent out + deleted once they are finished writing so memory doesn’t build up.

the lag is only a few seconds, amazingly, and the audio is high quality 48k stereo flac. sometimes there are audible pops but i’m not getting them consistently.

9 Likes

This is amazing @infinitedigits ! The audio streaming function is something I’ve been trying to figure out in general with a webserver running JACKaudio. I’m not too familiar with GO and generally have more experience with Node.js. Can you point me to the right place in your code that basically takes care of the sending audio files to the browser via websockets?

2 Likes

This is really meta… I can’t quite understand anything I am reading. It’s a Norns for Norns? I mean, I guess it just had to be a matter of time…

2 Likes

sure! basically what i’m doing is running jack_capture from the Go program running on the norns. jack_capture does the hard part of streaming the audio from jack. i just run it in the background, using this command run from the temp dir (/dev/shm):

> jack_capture -f flac --port system:playback_1 --port system:playback_2 --recording-time 36000 -Rf 96000 -z 4

basically it captures all output and routes it to a file. the -Rf 96000 means that it will create a new file every 96000 samples (at 48k, that’s 2 seconds). the Go program senses the newly made file, reads it and then deletes it (important! so you don’t run out of memory). then the Go program takes the data and converts it to base64 and sends it through the websocket as a string (relevant lines of code):

// b = bytes from reading file
mp3data := base64.StdEncoding.EncodeToString(b)
n.ws.WriteJSON(Message{
    MP3: mp3data,
})

browsers are amazingly smart now so you can simply use this javascript function to play base64 encoded audio data:

    currentPlayingFinished  = 0;
    function playMP3(mp3data) {
        audioCtx.decodeAudioData(base64ToArrayBuffer(mp3data), function(buffer) {
            console.log(buffer);
            // Get an AudioBufferSourceNode.
            // This is the AudioNode to use when we want to play an AudioBuffer
            var source = audioCtx.createBufferSource();

            // set the buffer in the AudioBufferSourceNode
            source.buffer = buffer;

            // connect the AudioBufferSourceNode to the
            // destination so we can hear the sound
            source.connect(audioCtx.destination);

            // start the source playing
            console.log(currentPlayingFinished);
            if (currentPlayingFinished <= audioCtx.currentTime) {
                currentPlayingFinished = audioCtx.currentTime + 1
            }
            source.start(currentPlayingFinished);
            currentPlayingFinished = currentPlayingFinished + buffer.duration;
        })
    }

that code will automatically buffer by 1 second. the decodeAudioData is part of the web audio api and it automatically detects wav/mp3/flac!

incepted norns! basically norns.online extends the norns to the internet so you can play it / hear it from a browser. like teamviewer but for norns. of course, its not practical for replacing the physical interface. but the website-based interface can be useful for other things - like streaming to multiple computers (w/o twitch!), having someone remote control your norns for fun/education, or experimentation with crowdsourced music, or ?!??!

1 Like

The obvious next contraption is to have norms also receive audio from the interwebs! :smiley: The same process of just patching to jack might just work.

2 Likes

yeah this would be next level! norns<->norns collaboration in (not quite but close) realtime!

i was trying to do this but i can’t seem to figure out the jack controls for playing an audio file. i think i use jack_connect to connect some audio to the system:capture_1 and system:capture_2 but i’m lost in how to actually get a file to play…any suggestions?

Outside of norns, in the wider world of SuperCollider, this is a thing people have been doing, no?

1 Like

yes, it took me awhile but i got it! i found mpv the easiest to do this with.

can start a mpv server with:

mkfifo /dev/shm/mpvfifo #fifo file for sending commands
mpv --jack-port="crone:input_(1|2)" --input-file=/dev/shm/mpvfifo --idle

and then you can enqueue files with:

echo "loadfile some_audio_file.wav append-play" > /dev/shm/mpvfifo

if you add more loadfile commands they get enqueued and quickly changed to the next track without audible blips. the jack-port assigns the audio to crone inputs so they can be used for softcut stuff. this is optional too, can easily assign to system playback and bypass that stuff.

okay! with this piece down it should be piece of cake to do the rest…

1 Like

Haven’t loaded this yet and this may be a dumb question, but I can allow friends to play and listen to my norns on the web? I tell everyone that they need a norns but most demos are intimidating or confusing. I’m sure just being able to show friends the dumb way I use it will sell a bunch of norns.

1 Like

yes! just run it and then go play on your norns. it will stream to norns.online/<yourname>. here’s an example of streaming (my norns, possibly offline)

release v1.0.0

i think at this point norns.online has come into being functional and i’m very happy with the result. this version fixes a lot of bugs and also introduces live audio sharing.

what is it? norns.online a both a script and a website.

what does it do? when you run the script,it will stay in the background and beam your norns audio+visual data to norns.online/<yourname>.

what else does it do? when online, you can control your norns from the website at norns.online/<yourname>.

what else does it do? when online, it also allows you to connect to other norns via “rooms” where you can share live audio. the latency is ~4 seconds, so you can imagine that you both are playing together while being safely socially distanced by a distance of a quarter-mile.

thanks @Quixotic7 and @anxietymachine for being very patient while i solved bugs in the audio sharing. also thanks to @xmacex for pointing out a lot of JACK tricks. so many thanks to @zebra for providing continuous expertise and the shoulders to stand on, really everything i’ve ever made was made possible by all your work!

there are probably still :bug::butterfly: so lmk i’ll have my bug catcher ready.

9 Likes