Is it possible for other programs to be “plumbed into the UI device selection &c”? If so, how would that be done? I’m brand new to this and haven’t looked into it yet, but the device_midi.c file looks like MIDI support would be easy. So, I’m now wanting to find out how the UI on the screen is done, and how that might be done with C code. Thanks for any pointers!

…Tim… (author of KeyKit)

3 Likes

I’m scanning the previous development conversations here, it looks like a lot of my questions (including how the screen can be accessed) will be answered after I get through them all.

…Tim…

5 Likes

right now, not very directly. there is a program called matron which runs the lua interpreter and interfaces with screen, GPIO, and USB devices. its components aren’t really set up to be used by other C programs, either as libraries or as IPC endpoints.

@PaulBatchelor has made some C snippets that might be helpful if you just want to replace matron with something else.

the norns lua API provides “script” writers with high-level interfaces to MIDI and screen and everything. one thing that we don’t have (hasn’t come up yet) is any modular way to extend the lua API with components written in C. (such extensions have to be baked into matron.) but there is nothing stopping you from (say) launching an executable in a lua script and then talking to it with sockets or OSC or something.

the “audio backend” on norns consists of a mixer program called crone, alongside an instance of sclang+scsynth and a buffer-cutter called softcut. communication with these components is done via OSC.

norns v3 is under development and will likely alter and/or extend some of this stuff.

5 Likes

Thanks much for the pointer to the C snippets! Using OSC to talk to Lua from another executable should be pretty easy for most things that matron manages. …Tim…

1 Like

This would be super useful. A few days ago I “accidentally” recorded over an hour of me going through various scripts and noodling around (intended to record a couple of minutes but forgot to stop recording). There were a few bits that ended up being quite useful! Not a huge deal to pull it over to my computer and cut out loops, but it would be convenient to just load the audio into a script on norns, cut and save loops, and repeat.

paging @mlogger from this thread

as a demonstration i added an engine that compiles and executes arbitrary .scd code at runtime.

putting this in ‘development’ because, well, basically have been holding off on pushing this kind of thing until the 3.0 update, which is approaching.

3.0 update will overhaul, streamline and extend the supercollider interface for norns.
engine classes and environments are simpler (easier to develop off-norns),
lifecycle management is simpler (e.g. Main.stop in sclang will not mess anything up),
there are convenience methods for exposing Classes to lua (less boilerplate,)
and proxy/factory methods to connect interpreted code to lua (if you don’t want to use classes.)

and finally i am thinking about ways to make the OSC connections more flexible (for arbitrary RPC-like action.) though it would be nice if someone who enjoys that kind of thing would take it over. i don’t think the hardcoded lua <-> SC OSC system is quite powerful enough to be a general solution right now for talking to other OSC-enabled environments. but with a little TLC i think we could make a fine standard that could be easily ported to PD or whatever.

anyways, here is simple thing that works with current systems:


this engine just takes three commands:

  • script(file) runs a .scd file
  • chunk(text) compiles and executes a chunk of sclang code (or tries to)
  • clear() executes ~cleanup function and cleans the current Environment

it’s important that each script contains ~cleanup function which frees any persistent resources!

this lua script itself is very dumb and does nothing but re-run a couple different .scd files, with no interaction.

if you wanted to get values to and from the script (e.g. control stuff with encoders / param system), probably easiest to set up explicit matching OSC connections in lua osc.event and in the sclang code. (and then before you know it you’re adding more stuff to the wrapper engine and then we are reinventing the 3.0 stuff presently in the pipeline, so maybe hold off a bit.)

scripts should be able to use MIDI and HID devices but i haven’t tried it. (maybe norns will steal the MIDI ports, i dunno.)

graphics and window-driven UI stuff in sclang, will not work.

it would be a simple exercise to use the keycode libraries from one of the text-driven norns scripts, and make a little sclang repl here, with an OSC connection for feedback.


i think i want to put this here because it feels worthwhile to mention that norns lua environment is pretty fully featured with not a lot of restrictions. one can create shell processes, use thread-backed coroutines for asynchronous stuff, do file I/O, pipes, sockets, and OSC. and it’s very performant compared to, say, python or node.js.

so… norns lua can act as a poweful launch configuration manager for other arbitrary processes, including audio process. (and can manage JACK connections through a shell.) i actually use it this way pretty frequently in my day job as an audiology research scientist. (where, unfortunately, the apps and settings themselves are proprietary stuff.)

just throwing that out there. if someone wants run csound or automatism patches on norns, we can work together a little bit and make that a more integrated experience, if that’s attractive.

21 Likes

Wooah thankyou so much!

I think its been mentioned before, in the Supercollider thread, but Fredrik Olofsson’s posts Supercollider patches on his twitter account https://twitter.com/redFrik They all work great with your patch although obviously you can’t shut them down unless you add the ~cleanup function.

He’s posted some interesting stuff anyway, if people wanted to try the scrawp patch and replace the a, b and c.scd files in the lib folder.

4 Likes

Any idea why I can’t see alsa VirMIDI devices on norns ??

IIRC - i believe because norns uses rawmidi. Search way back up in this thread for more context Norns: development

Yes I’ve red about this but virmidi is rawmidi and alsa. It does create a midiC1D0 in /dev/snd so I don’t understand what’s missing to be recognized .

device management code only looks at the USB bus.

so, this has come up before and i just want to be super super clear:

device_monitor.c has a really specific purpose, which is dealing with hotplugged USB devices. we should have called it usb_monitor.

this doesn’t mean we shouldn’t add other interfaces for other kinds of midi device. so PRs for that are appreciated even if they are off of the critical path for the monome norns project. but i can’t see a reason for a virtual midi device, or a GPIO device, to have anything to do with the C module that watches and responds to changes in the /dev tree.

for virtual devices (software variants besides vanilla norns) or GPIO devices (HW variants,) it makes a lot more sense for me to just have functions which explicitly initialize them, and which can be plumbed into the lua layer where “user-side” customization is easy.

does that make sense?

in other words, just wanna be clear that we want to improve the USB detection stuff and make it more flexible for other kinds of USB devices (in particular, more transparent use of cloned grids - even though you can see how that’s not exactly on our critical path either) but hacking it to work with device types that aren’t hotpluggable just muddies the water.

did you see this in the same repo as Scrawp?

i took a stab at programatically scraping all of redfrik’s sctweets that are single synthdefs. doesn’t work 100% and i thought it would be a lot more fun with some parameterization, but put it aside before i got there…

5 Likes

I am not trying to hack device_monitor. I just finally managed to have rtpmidi working and wanted to understand why norns doesn’t see the loopback so it could make use of the rtp. I now understand this is not possible :slight_smile:

Sorry for the misunderstanding .

gotcha, and sorry - wasn’t directed at you, just pre-emptive as it has come up before. if anyone does want to hack in virtual midi support…

Do you have any suggestion on how to implement this ? Or we just don’t ?

I’m clear on the purpose of device_monitor.c, but want to pick a small nit here - bluetooth devices* (as one example) will show up in the /dev tree as they become available. They could be detected by device_monitor, but it would need to be extended to look at other subsystems (bluetooth in this case) - or as you suggest, perhaps a separate set of functions would be better for that use case.

I’ve attempted this a few times, but it’s a bit beyond my current skills. I’ve gotten to the point of detecting devices, but can’t figure out how to glue it together.

* applicable to the shield (pi has built in BT) and could also be done on OG norns with a BT nub.

1 Like

oh yeah, i forgot about bluetooth… thanks.

anyways we’re working on changes to the monitor layer that will just allow you to customize it without hacking matron.

i’ve used 2 bluetooth nubs which just make things show up in /dev as class-compliant devices. they adapt to specific BT profiles. one is a logitech thing that maps HCI devices to HID devices. the other one (which i can’t find at the moment) is similar but for ACM devices. i’ve used these with vanilla norns without problems.

i guess other BT adapters exist that just make things show up in rfcomm and call it a day? (can anyone show me an example of what this looks like?)

1 Like

That’s a great idea. No I haven’t tried those other ones.

What i did try though was Mads Kjeldgaard’s Akkorder supercollider album patch, which is fairly audio intensive. I contacted him and asked if I could try getting his patch to work on Norns with your script. It’s a generatative algorithmic driven album and it said…“If you do not like the included music on the album, you can just generate some new material to replace or expand it with the patch” :slight_smile:

I tried it on a vanilla Norns which ran it at 50% CPU for about a minute then eventually it glitches out into feedback. I then tried it on Fates with a Pi4 and it plays all the way through to 7 mins with the occasion click with CPU max @20%. Each track self generates and streams directly to the tape folder as it’s being played. Interestingly though, even though the regular norns becomes feedback. If you leave the patch running to the end (it auto switches off after 7 minutes) there are no glitches at all on resulting wav file. The same with fates also.