Soundplane for Norns

Branching off from the Approaching: norns thread where it was asked if the Soundplane could communicate directly with norns. No one has done it yet, but it seems very doable—if anyone wants to take it on, I’m here to answer questions.

I just made things easier, hopefully, by splitting the low-level Soundplane code into its own repository: https://github.com/madronalabs/soundplanelib . This comes with clear directions for getting started on MacOS.

A Linux version will require libusb. Various people have gotten it to work in a few distributions and what’s in the repo now should not be too far off the mark. I’m happy to incorporate pull requests and eventually to maintain a working norns version when I take the plunge.

14 Likes

Would make this Soundplane & Norns user very very happy!

Very cool! Now all I need is a Soundplane… I wonder which will come first, Crow or Model B?..

Perhaps we can broaden this effort to SoM in general.
( given the small number of soundplane users, and not all are going to want to spend $800 on a Norns., once the underlying ‘tech’ is working, then writing the Matron bit is easy)

I’ve been ‘playing’ in this area for quite a while, it started in 2015, when I was getting the Eigenharps to work on the rPI, and found a bug in the isosynchronous usb handling in the rpi kernel,
you can see that report here , this is relevant since it affected the soundplane as well.

fortunately, at some point it got fixed :partying_face:
but… ever since then, Ive found it can at times be a bit ‘fragile’, with slight odd issues.
e.g. randomly, some kernel releases , the Eigenharp Pico needs a usb hub, but next release and it doesn’t, repeat and rinse!


anyway… here is where I am at the moment, having tested the soundplane on a wide variety of boards!

PI based solutions

  • a CM3 based solution 4.14.93
    works perfectly
  • Norns Clone - rPI 3b+ (setup as Norns)- 4.14.62 (note: norns uses 4.14.52)
    did not appear to work, no data received BUT caveat!
  • Norns Clone - rPI 3b+ , upgrade kernel to latest 4.19.27
    this APPEARED not to work, and accidentally, I found that after 30-45 seconds, it starts to work - so initialisation (calibration?) was taking a very long time (its ‘instant’ on other board)
    (its possible the the 4.14.62 may have worked, but just taken a long time to initialise - though I thought id waited a while on it too !)
  • rPI ‘standard’ stretch, minimal install 4.14.48
    same slow initialisation.

Non rPI solutions

  • Organelle - 3.x kernel, arch linux
    does not work, submit_urb_error -28, I think its simply a bandwidth issue, though oddly it works with eigenharps - but its on a really old 3.x kernel - I need to re-test with a a 4.x kernel
  • Beaglebone Black w/ Bela include Bela Salt (Eurorack) - 4.4.113 xenomai
    always worked perfectly :slight_smile:
  • ASUS Tinkerboard 4.4.32
    appears to get NO data, but doesn’t matter how long you leave it, it doesn’t initialise,
    again an old kernel, so might work with a 4.6+ kernel
    note: as you’ll see in the bug report above, the fixed appeared to come around 4.4.48!

(macOS also works fine with my same code)

Summary

on the rPI platform, there seems to be some kind of minor issue with it taking a long time to initialise… lets, assume the ‘failed’ one is a testing issue, even though it seems unlikely.
the CM3 platform is interesting, its theoretically same setup as other rPI, but doesn’t have the issue… and the CM3 hardware should not matter, it uses the same BCM2835 as the rPI3.

anyway, I guess the Norns will work… perhaps with slow initialisation, and perhaps the kernel may need reviewing - but we will only know once its tested on an actual norns.

on my side, for SOM, I do want to test the Organelle and Tinkerboard on later kernels.
and I guess, I’ll break out usbmon and see if i can find out what’s going on with the initialisation on the rPI boards.

Im pretty excited in this area, as Ive got a couple things ‘coming’ , that are going to be awesome with the Soundplane (and Eigenharps)


enough words… getting rid of the computer, can be really liberating :slight_smile:

here is the Soundplane working without computer, supplying CV directly to an AE modular. (I also do the same for Eurorack with Bela Salt , but no video yet :wink: )

and, very similar tech (usb iso) , and same goal, ditching the computer!
… here is the Eigenharp pico with an Organelle,

5 Likes

An update on the Soundplane software: I’ve regrouped all of the source code to make it as clear as possible how to get started and to make compiling the Madrona DSP library and small Soundplane example much quicker. The full Soundplane app now has three components, each in its own repository.

madronalib is my C++ library for writing DSP applications. It contains a lot of useful DSP code as well as app support code for cross-platform, real-time apps. All of my software relies on this code so it gets frequently tested on Mac and Windows and soon, one hopes, Linux.

soundplanelib is the library for supporting the Soundplane hardware over USB. It does not require madronalib. It includes an example app, HelloSoundplane, that displays the Soundplane pressure as ASCII art in the terminal. If you do use the Soundplane on a Linux-based platform, please contribute any fixes to this repository via pull requests.

soundplane is the GUI-based app for working with the Soundplane on Mac OS: it supports options for touch detection as well as configurable zones that can send out notes and controls via OSC and MIDI. It requires both madronalib and soundplanelib.

@TheTechnobear if calibration is happening slowly, my first guess is that the driver is dropping most of the frames. The Mac driver keeps stats about this—you could add similar code to the libusb one. Also I note that the libusb driver has a mutex, which could possibly be slowing things down based on the platform—just a guess.

2 Likes

An update: Greg Wuller has added libusb support to soundplanelib. He has tested this on Debian and Raspbian with libusb-1.0. I’ve incorporated that change and also updated the soundplane app code to use the latest madronalib.

Hopefully this paves the way for a canonical soundplane+norns integration.

4 Likes

Oh that is great news, it will help get things going again… now if I could find my notes :grinning:

UPDATE: I’ve coaxed current madronalib into compiling on norns so that leaves factoring out a headless soundplane as the next step.

UPDATE2: A bit of a progress report. I’ve extracted a portion of the soundplane code into a separate repository with the intention of creating both a GUI-less standalone client with the latest touch tracker and a library suitable for wrapping into a Lua module for norns.

Initial experimentation on a stock rpi4 + buster + 5.10 kernel has unfortunately uncovered that the usb transfers become unstable after about 30 seconds, frames with zeroed out values are being passed into the anomaly filter every 20-30 frames. The unpacker is consistently dropping transfers from one endpoint or the other at a rate of ~2-5 times per second throughout. When the zeroed out frames start it appears to be the result of the unpacker consistently dropping transfers from one endpoint or the other. The libusb based driver does not exhibit the same behavior on macOS so the hunt now becomes determining if this is a kernel related, rpi hardware related, or soundplanelib related…

UPDATE3: The linux instability at the 30 second mark appears consistent on both rpi3 + raspbian stretch + 4.19.x and nanopi neo2 + armbian buster + 5.10.x kernel. The neo2 case is interesting since it is an entirely different chipset and usb controller from the rpi family.

UPDATE4: Found the source of the Linux problems, further investigation regarding the number of seemingly empty packets included in transfers is probably warranted but this change prevents the Unpacker and anomaly filter from exploding: Discard unmatched packets with seqNum == 0 by ngwese · Pull Request #4 · madronalabs/soundplanelib · GitHub

UPDATE5: …and the touch tracker looks to be working (on norns and other platforms). Now the fun begins.

7 Likes

If there’s any support I can lend to making this happen let me know. I’m not a very experienced programmer, but I know enough to be dangerous (and potentially helpful as an alpha/beta tester).

This is something I’ve wanted to get going for a long time but just ran into the limits of my knowledge of linux / C hacking and time for learning vs. making sound.

1 Like

Small progress update, the most minimal path is working - continuous touch information is fed directly into the norns lua layer (no sound):

Next steps include some performance tuning, checking for memory leaks, and determining the root cause of the empty USB packets (my current theory is it might be due to the weaker memory ordering on ARM vs x86_64).

Most of the work right now is fairly low level and I haven’t had the cycles to figure out if it can be split up. As soon as something is in an alpha/testable state I’ll post more details.

UPDATE:

for the intrepid tester and member of the tiny cross section of folks in possession of a soundplane and norns there is now something to experiment with.

specifically:

  • update the latest norns release 210927
  • run ;install https://github.com/ngwese/spn/releases/download/v0.3/spn.zip in the maiden repl
  • navigate to SYSTEM > MODS and enable the SPN mod by turning E3 until it displays +
  • invoke SYSTEM > RESTART to load the associated mod and engine for supercollider
  • select SPN from the script menu

when initially connected and/or the script starts up, the soundplane will go through a calibration routine which takes ~5 seconds. wait for ~5 seconds before playing or monitor the calibration progress from the maiden repl.

at this time there is zero documentation. once documentation is ready and some additional voicing is complete i’ll start a dedicated thread.

for the testers please feel free to open GH issues if you are comfortable with that.

13 Likes

Not sure how I missed this update earlier, but this is so great, thank you for posting.

I just gave this a shot and I don’t think it’s working as intended on my norns.

Actions taken/device state

  • stock Norns
  • ‘fresh install’ from disk image
  • update to 210927
  • install spn mod per instructions
  • restart

Observations:

  • I get an audible ‘pop’ when loading the ‘spn’ script, but no other feedback aside from the output in maiden matron/supercollider (pasted below).
  • loading the ‘vis’ and ‘ctl’ scripts similarly produce no discernible feedback.
  • led on the Soundplane is not illuminated
  • no obvious ‘calibration’ output in maiden when connecting device or starting script

After a restart, here’s the output from maiden of matron and supercollider upon loading the ‘spn’ script:

matron:
======

# script clear
calling: spn_cleanup
# script load: /home/we/dust/code/spn/spn.lua
calling: spn_init
# script run
loading engine: Assemblage
>> reading PMAP /home/we/dust/data/spn/spn.pmap
m.read: /home/we/dust/data/spn/spn.pmap not read.
Engine.register_commands; count: 4
___ engine commands ___
stop	 	i
stop_all	 	
touch	 	ifff
voices	 	i
___ polls ___
amp_in_l
amp_in_r
amp_out_l
amp_out_r
cpu_avg
cpu_peak
pitch_in_l
pitch_in_r
# script init
supercollider:
=============

a CroneAudioContext
Assemblage.init: server = localhost, xg = Group(1001), out = Bus(audio, 0, 2, localhost)
Assemblage.init: sending defs...
 + assemblage_sine
 + assemblage_haze
 + assemblage_place
[ CroneEngine adding command, touch, ifff, a Function ]
[ CroneEngine adding command, stop, i, a Function ]
[ CroneEngine adding command, stop_all, , a Function ]
[ CroneEngine adding command, voices, i, a Function ]
-----------------------
-- crone: done loading engine, starting reports
--------
engine: an Engine_Assemblage
command entry: [ 0, touch, ifff ]
command entry: [ 1, stop, i ]
command entry: [ 2, stop_all,  ]
command entry: [ 3, voices, i ]
cpu_peak
cpu_avg
amp_in_l
amp_in_r
amp_out_l
amp_out_r
pitch_in_l
pitch_in_r
Assemblage.init: server synced
Assemblage.init: setting default configuration...
Assemblage.init: allocating voices...
AssemblageVoice.init(localhost, Group(1013), Bus(audio, 0, 
2, localhost), assemblage_haze)
AssemblageVoice.init(localhost, Group(1013), Bus(audio, 0, 2, localhost), assemblage_haze)
AssemblageVoice.init(localhost, Group(1013), Bus(audio, 0, 2, localhost), assemblage_haze)
AssemblageVoice.init(
localhost, Group(1013), Bus(audio, 0, 2, localhost), assemblage_haze)
AssemblageVoice.init(localhost, Group(1013), Bus(audio, 0
, 2, localhost)
, assemblage_haze)
AssemblageVoice.init(localhost, Group(1013), Bus(audio, 0, 2, localhost), assemblage_haze)
AssemblageVoice.init(localhost, Group(1013), Bus(audio, 0, 2, localhost), assemblage_haze)
AssemblageVoice.init(localhost, Group(1013), Bus(audio,
 0, 2, localhost), assemblage_haze
)
Assemblage.init: done

Sorry - my bad, I think I know the problem. I’m away from my device at the moment but if you aren’t afraid of ssh then try the following:

If the above is too much rest assured I’ll get this handled in the mod in the next day or two.

The behavior described; the soundplane light not going on, is a strong indication that it wasn’t being detected. All the other output looks good. The spn script has some parameters to adjust the touch response like the normal client app.

2 Likes

This worked. I don’t have time to dig in now, but will definitely later today.

This is so amazing thank you!!

3 Likes

i’ve pushed a new version which installs the udev rules on first run and updated the testing instructions. for @NiC or anyone else who has the previous version installed you’ll need to re-run the ;install https://github.com/ngwese/spn/releases/download/v0.3/spn.zip command in order to pick up the latest version (since the URL changed) and restart.

as a small bonus there is a new sound called chime which has been added.

documentation remains on the todo list.

1 Like

Wait, so all technicalities aside does this mean Soundplane is now fully recognized by Norns or is it still a bit dodgy? This would actually be quite some big (niche) news.

1 Like

essentially yes, api polish aside, this is as tight and direct an integration within the norns environment as is possible. touch events are being delivered directly to the lua scripting layer using the same touch tracker code which is in the macOS desktop app. all the major touch tracking parameters can be tweaked and multi zone layouts can be specified via lua and changed under script control (something which is not possible in the main soundplane client at the moment)

with the mod installed/enabled soundplane input can be integrated into other scripts (not just the spn script)

in practice i’ve found it sufficiently solid to continue to use it and intend to maintain it.

8 Likes