Lower-level audio programming

I’ve finally dived into writing software for performance in the deeper realm of computing. Currently using the rtaudio-library for C++, a new world of audio-related segfaults and unexpected, uncomfortable and tinnitus-inducing audio-glitches has opened itself unto me (not recommending wearing headphones while getting a buffer overflow in your output stream).

But the documentation for audio programming in C/C++ is sparse, — or rather, more difficult to come by. So does anyone of you have any finds and tips and tricks and ideas and specifically documentation or other resources for audio related programming closer to the metal?

10 Likes

This subject is easily the most fun I’ve had on a computer.
What’s your platform? Which audio service layer are you using? (JACK/ALSA/ASIO/CoreAudio/etc)

I find documentation in this area either highly-generalized/abstracted or very platform-dependent. But for me the larger picture slowly came into focus as I spent more time with the materials and experimenting with my preferred platform/service-layer.

EDIT: Might be worth it to mention that I first started working with JUCE and rtaudio as I thought it would be more straightforward. Actually the opposite became true and it was much easier for me to understand how all the pieces fit together without them being abstracted for me by some huge platform-independent library.

For now, here’s some generalized stuff which helped me…

Here’s a fairly good article on dealing with audio streams, though a little heavy-handed on the mutex-hate.
Real-time audio programming 101: time waits for nothing

Great article on implementing a short-time fourier transform with fftw3 for creating phase vocoders (frequency/phase shifters) and the like
Short Time Fourier Transform with FFTW3

And of course the venerable musicdsp.org

Lastly something I found on Facebook today as motivation to not give up:

25 Likes

Those are great shares, thanks! Really need to get my hands dirty with fft when I get the time.

I’m on Arch Linux (stock kernel) using Jack on Alsa. I’m coming from puredata and openframeworks, and are well versed in both, but have the last year or so been more and more obsessed with removing the abstractions and going deeper. It makes so much sense for so many reasons (several of which Kim Cascone mentions).

Speaking of, I’ve tried not to jump so much between C++ and C (in general, not only audio-related stuff) because I started writing C-like in C++ and (to some degree) vice versa, so I’ve focused on C++ mainly because of vectors, less pointers and a writing style I find easier to parse as a human. But I see you’re using C in Candor — may I ask why?

these are great resources, thank you!

i’m really excited about gentoo. my hope is to strip it down to almost nothing to get as close as possible to the feeling of programming direct to chip firmware rather than a desktop app.

C is also my preferred language-- i’m hoping not to touch C++ even for audio. is this a realistic goal?

1 Like

I don’t think the choice of distro really makes that much of a difference in most cases, at least that’s been my experience. I’ve used Debian, Gentoo, CentOS and Ubuntu all for quite a number of years in many different contexts. I find it’s usually easiest to drop the GUI entirely and just work with the console, then the main differences are usually just package management and the init system.

As far as doing the math / audio part of programming there’s going to be little difference in doing it in C vs. C++. I think it’s totally realistic to just write everything in C, it’s not that unusual…

However, I find the big benefit of C++ is being able to organize code using namespaces and classes, and occasionally use templates for a few things. C++11 has come a long way from when I was programming C++ even a decade ago, there’s lots of conveniences (foreach iteration, auto keyword, constant expressions) that make programming much more expressive without compromising efficiency (in most cases). Also C++ has stricter type safety than C so the compiler can catch a lot of less obvious mistakes for you

5 Likes

What kind of documentation are you looking for? How to program C/C++ in general? How to use certain libraries? Algorithms? All of the above?

Well, it basically started with what seemingly was rtaudio continuing to output the old buffer even after memsetting to 0, and I realized that I didn’t know where to go to figure out what I did wrong — I realized I didn’t even know basics like whether the stream should open and close between playbacks, or how I should treat the timing and threading.

But instead of asking about such specifics (which I am sure are answered somewhere out there already), I thought I could rather check whether this topic had any interest on this forum, and open a thread for us to share resources related to purist/minimalist/non-consumerist software creation in less abstracted languages.

So yeah, specifics about libraries or general audiomath is interesting. So is discussions about languages and techniques (audio in assembly anyone?), or more philosophical/idealistic/political reasons for spending months creating tools instead of using what’s already out there. Interesting code you’ve written or found, or links to other fora related to the same topic is also cool.

If you want something more specific, I’m looking for something like [this book] (http://www.pd-tutorial.com/english/index.html), just for the C-family instead, and also more stuff like this.

3 Likes

musicdsp.org is a brilliant resource. the kvraudio dsp forums are where a lot of really good discussions and research happen these days (couple high-level people from NI post there, urs from u-he, andy simper from cytomic…).

be careful what of c++ you use. most audio software is written in c++, but ross bencina’s “time waits for nothing” is required reading if you’re doing dsp work.

personally, i use c for everything. i have an entire vst written in c. no problems.

-w

5 Likes

I came across a pretty good discussion on C vs. C++ in the ChibiOS forums, as it pertains to embedded programming:

http://forum.chibios.org/phpbb/viewtopic.php?f=24&t=2702

Check out “Audio Programming” on MIT Press for the C side. Also check out “Electronic Music and Sound Design” from MIT as well that goes into the theory using MAX/MSP. Both massive resources.

8 Likes

Coming to this thread pretty late! I just wanted to say thanks for posting the Kim Cascone video – his point at the end about the importance of being slow during the creative process is so important. My composition teacher used to talk about living with a text you’re setting; needing that ephemeral research time to live & allow yourself to experiment, and just let life throw entropy at you – which can sometimes be the key that unlocks that moment when things begin to click… I have many producer friends who obsess over getting their workflows to a point where they can pump out new things really fast, and I feel are missing the opportunity for reflection that comes with slowing down your workflow, at least sometimes…

9 Likes

we are heavily off topic from low level audio programming :slight_smile:

I am a big fan of his - was on his micro sound list years ago and some of my work ended up on compilations from that list.

I like some of his views about making software - I think though there is something to be said for “instant” composition - I look to people like AMM , Evan Parker etc or the likes of John Zorn (or the cloud of people around him) who can sit with an instrument and just play - I think one of my goals is to be able to sit with electronic instruments and just play. It may well be that that process involves writing software (one reason I’m so keen on Norns - it feels like a big step in the right direction for my ambitions. The Organelle was somewhat that but not quite there for me)

5 Likes

on the topic of C/C++ - having now returned to coding as a hobby -(it was my career until relatively recently but I don’t do it in my day job at all now) it’s nice to be writing code for fun.

I was a C/C++ engineer during my 20’s (I worked on telemetry systems for Williams F1 for a while! proper real time ;-)) and I find I switch between them freely - C++ changed a lot in the intervening time but I’ve caught up a bit. A lot of C ends up with pseudo object orientation anyway (& I’m old enough to remember when C++ was basically a pre-processor for C) - value in both

2 Likes

Well I fully appreciate you taking the time to express that even after this time :slight_smile: I fully agree and to expand on what you’ve said about workflows–I have a hunch that many (including my) magic/extraordinary moments occur after a deep level of comfortability has been developed over time with a creative process and movements in that workflow become more refined, subtle, less dramatic, more… natural or accurate or something? fireflies of a truer intimacy flicker subtly amongst the hedgerows of dreary familiarity?

4 Likes

The Aleph code base is/was all written in C. There is a lot of info in the monome.org docs section on the aleph.

Are all the recent eurorack modules/teletype C as well?

1 Like

I believe so, yes. @xievub I think a lot of Mutable Instruments’ module code is also written in C! Also, of course, the Csound language, some parts of Pd, I think SuperCollider is in C++… etc.

1 Like

I would recommend additive synthesis as a fun first thing to explore if starting coding audio DSP from the ground up without pulling in a big library of existing tools.

Basic smoothing using a rolling average is another intuitive building block not requiring a degree in audio to work out from first principles.

The deep puzzles of audio DSP come in when you start messing around with resonant filters, distortion, sawtooth waves and playback/recording at different pitch/speed.

2 Likes

Promoting my own tools here, but I highly recommend checking out Soundpipe. I initially wrote it because as a musician, I wanted a very portable lightweight DSP library for making music, that would also be fun to hack on. The library written completely in C, with absolutely no C++ ever. It’s also designed to be fairly decentralized, as far as C libraries go. It’s fairly trivial to extract specific modules from the library rather than use the whole thing. Most of the modules are adopted from Csound opcodes, so the underlying algorithms are pretty legit. Musically, it’s comprehensive enough now that I’ve been able to use Soundpipe (and other tools I write) in place of Csound for my needs. It’s also used quite a fair bit in audio framework AudioKit. IIRC, the sound architecture of Synth One, an open source iPad synthesizer, is actually designed using Soundpipe.

It’s great to write simple filters, oscillators, and delay lines in C to get a low level sense of how things work. Anything more complex, and I switch to the FAUST language, which can generate highly optimized C/C++ code. It is much easier to learn and understand the underlying DSP in a functional language like FAUST. I’ve never had much luck groking much DSP staring at DSP C code.

I found composing music in pure C + Soundpipe to be a fairly slow and frustrating task, so I wrote a silly little language on top of Soundpipe called Sporth. In the context of what is already out there, it’s a minimal language. The core Sporth engine and parser is about 2k lines of code. The rest of the codebase are mostly just boilerplate code for the unit generators.

My personal take on writing your own music software:

Writing your own music tools is slow work. The problem I always run into is that I spend more time debugging my code and adding features than actually writing music. Still working out the balance to that. That being said, I still very much encourage it! There is still much to be explored in the intersection of computation, music, and creative expression. With pre-made music software, many creative decisions have been made for you before you even begin creating. When you write your own software, you call the shots every step of the way for how things should look, feel, behave, and sound. You make enough of these decisions, and pretty soon you’ll have something very special and unique. This has at least been my experience. I’m not a great composer by any stretch of the imagination, but I’ve certainly become a better one because of my experience writing tools like Soundpipe and Sporth.

20 Likes

Wow, glad and excited to know sound pipe exists and also glad I’m not the only Faust fan on here!

Aleph got me properly started on c programming, although I already had read k&r and written quite a few simple programs (controlling old rs232 lab equipment from Linux circa 2005-2009) but no audio programming experience prior to that. Just a desire to fulfill some vague dream of combining my live guitar looping act with electronic sounds and a desire to build my own version of like an ehx hog or something. I was inspired by the 60s band silver apples and kind of dreamt of building a similar live electronics machine.

Mucking around with synthesiser code for the last 5 years or so still hasn’t really yielded a setup I want to work with for electronic music and my perspective has shifted a lot hanging out on here. but I’ve had a renewed burst of enthusiasm for the endeavour and am actively hacking again.

Pm me if anyone is interested in developing Faust or C DSP in the norns ecosystem I might have something of interest…

3 Likes

FAUST generates C/C++ code that is really hard to read, so it’s not really a good way to understand how to implement code in C. I will say the best way to understand how to write DSP code in C, FAUST, or any other language, is to learn the underlying mathematical concepts first. I’ve found this especially true for things like filters. By the time filters are implemented, all the coefficients are fully derived and what you are left with are a bunch of arithmetic + trig operations with magic variables that don’t make much sense. Audio DSP code for the most part is actually surprisingly simple in terms of structure. As a colleague of mine once said, “it’s all multiplies and adds”.

Personally, I really enjoy writing in C for my audio-related work because it is simple and portable. Software typically has a very short shelf life before it crumbles out of neglect, but I believe that this is unacceptable for the computer musician. We need tools that can sustain our creativity for many decades, and can even be run for future generations of computer musicians to study and improve on. Because it really sucks having to learn a new thing every few years. I want to actually try to master something, yknow? It’s a very tall order, but when I write a tool or component in clean unfancy C89 C or C99 C, I feel like I have a chance at that kind of sustainability. I’d really like to see a push for more of this kind of thinking in the digital art world. Our ecosystems for so creation are so brittle right now…

I’ve always found that when I write things in C, I always come up with more elegant solutions. C gives you very little in terms of data structures and algorithms, so it’s up to you to make good thoughtful design decisions, and I quite like that.

I’ve always found C++ to be this huge complicated beast. I never feel like I can write anything small and simple in it, and I like writing small and simple things. It’s one of those weird languages where codebases have to agree on which features to use and which features not to use. Sometimes they can be difficult to compile because I’ll have a C++ compiler that is out of date. Also, no one is able to give a straight answer on how to learn C++ in 2019? You don’t really need any of that fancy stuff to do audio DSP. C is a language that allows you to build things without any of the nonsense (you get to build that nonsense yourself. heh.).

14 Likes