#28

Starting by building your code as max/msp externals might be a great way to abstract away the audio management and routing side of the code while you focus on the algorithms and control structures. Eric Lyon’s excellent book on the subject is a great help though a little dated now — the style and manner of coding for max/msp hasn’t changed, only a few types and constants to support 64-bit, which the author provides as supplementary info via his website.

6 Likes
#29

You’ll probably spend a lot of time just experimenting with the algorithm, for this a rapid prototyping environment like Matlab or Octave would be helpful. This step won’t produce anything real-time, but it will allow you to make a lot of decisions up front without the overhead of C++.

When you’re ready to make something interactive/realtime, I’ll second @equipoise’s recommendation to create a Max external. For historical reasons Max community likes C rather than C++ but if you’re going to make an app, best to embed a C++ delegate object in the “C” max object (it works) and then you’ll have something you can port anywhere else.

You may actually be fine with the Max implementation, especially as this is an academic project. You can create a simple GUI in Max or hook up controllers, and you already have real-time audio.

If you want to develop something with a nicer/custom UI, say for commercial development or just to go beyond what Max allows, then you’d use something like JUCE to create AU/VST etc.

for very basic, easily implemented pitch tracking (just a starting point… you may find more “updated” methods for comparison): http://audition.ens.fr/adc/pdf/2002_JASA_YIN.pdf (note: it’s not perfect and parameters will always need tweaking for your test cases/source material. but still pretty good for an easily implemented algorithm).

3 Likes
#30

existing implementations of a few standard pitch tracking algorithms

hm i dunno if that sounds like a likely thing to encounter off the shelf.

my favorite, (update to yin, i think? bit simpler?):
http://www.katjaas.nl/helmholtz/helmholtz.html

important bit on “special normalized autocorrelation”
[ http://miracle.otago.ac.nz/tartini/papers/Philip_McLeod_PhD.pdf ]

using matlab or scipy or something good idea,

same good practices in DSP as anywhere else. use abstraction judiciously - in this case FFT should be a well contained wrapper. kissfft is convenient for testing.

pitch tracking is one case where test suite makes sense. especially since you want to do something comparative anyway

3 Likes
#31

If I can read and write from sample buffers then I can write the algorithms, although having filters and existing implementations of a few standard pitch tracking algorithms would be nice.

A friend and colleague of mine wrote a paper that studies a few pitch tracking algorithms. You should definitely give it a read if you haven’t already: https://ccrma.stanford.edu/~kermit/website/scpitch.html

Soundpipe has adopted a pitch tracking algorithm originally written by Miller Puckette, adapted from the ptrack opcode in Csound. It is a very portable opcode written in c99-compliant C code. It is also the main pitch tracker used by AudioKit. You can study the code here:

(PSA: if anyone can make this better/faster, please consider contributing it to AK. They would be very very grateful!)

My advice to you would to make sure your algorithms can rendered offline to an audio file. Why? Because debugging audio DSP code is a slow and tedious process… you will need all the help you can get. Tracking down nils and trying to figure out why your filter is exploding is such a tedious experience. Getting things to work inside of a command-line program makes testing a whole lot easier. From there, it is a trivial process to get things up and running on a realtime system.

For the library to use, I highly recommend using libsndfile. It’s so simple and straight forward to use. Here is some sample code to get you started:

In fact, this is the very same sample code I started out with when I started to learn how to do audio programming as an undergrad.

10 Likes
#32

Looks very promising and indeed simpler, probably a better choice than YIN these days.

The point of these “classic algorithms” isn’t necessarily how good they perform in an unmodified state, but that they’re good pedagogical starting points to being thinking about the issues of pitch tracking in general. that is, after getting one of them to work, you’ll be in a much better shape to read other papers; you’ll have a good framework in which to think about modifications or even creating your own algorithm from scratch.

at some point you’ll need your own ideas because you’ll always have different assumptions, goals, materials in mind than did the authors. everything is specific to some degree.

also a great point. But it also highlights the need for a Matlab (or Octave/scipy) reference that also works on file I/O. You can make sure everywhere along the block diagram your Matlab/C input and output files are identical (although you may run into problems with how each deals with floating point characteristics – may not get them exactly “identical” on the level of bits).

I know this sounds like a lot of overhead and duplication – you’re putting in significantly more effort up front, but it will really save you in debugging.

4 Likes
#34

teleporting this discussion over here - this looks totally useful to me rn & not something I’ve heard of before. anyone else have experience w/ using audio APIs in a C environment as opposed to writing low level maths ?

also, are there any good places to find example code and algorithms for audio processes ?

the situation is: stating to get my handle on using C/C++, but still far away from understanding & implementing low/mid-level audio processes.

(Modern) C Programming Tips and Tricks
#35

i’m a bit confused. the STK is a pretty classic library and seems like a good starting place for your questions… e.g the STK tutorial?

note that STK is implemented directly in C++ though; not sure this is the right thread for it.

STK source (c++)
CSound source
SuperCollider source (c++, but the inner loops are basic)
PD source
musicdsp.org

or a book by
boulanger/lazzarini
richard moore
will pirkle (c++)

e.g., all the stuff already mentioned in this thread and similar threads:
https://llllllll.co/t/lower-level-audio-programming
https://llllllll.co/t/dsp-book-recommendations

of course… learning a specific library is a pretty different thing from learning the math behind the algorithms yourself.

2 Likes
#36

The Soundpipe library was built with that in mind (I am the author):

If you stick to just rendering audio files, Soundpipe by itself has everything you need.

2 Likes
#37

ahhh didn’t know about this thread, probably mis-migrated & that would be a decent lookthrough as well

#38

Per @zebra’s suggestion, I moved things over here :wink:

1 Like
#39

A big turning point for me was being able to jump into some of these codebases. It is more about pyschology than understanding, IMO. Being able to say “fuck it, lets jump in and try and see how this works” is a very useful mindset to have at any level. It also is helpful to look read random bits of code, as it helps you learn to skim code to get the gist of things. The more you do it, the more confidence you will get in jumping in.

If you aren’t sure if you are ready or not to jump into things, you are ready. Just download some code, open a file, and start going line by line to see what you can understand. 90% comprehension? Great. 10%? Great! Nothing at all? Also great. There is something to be said about just the physical act of moving eyeballs across code.

I have looked at the source code for the projects listed above at varying degrees. For those interested in doing some code spelunking, here are my thoughts on them below. If any one wants to contribute their own person thoughts and experiences, feel free:

STK:

I have spent minimal time looking at this code, but it seems fairly readable. The core codebase is very small, and written in a style of C++ which I call “C+”. Code basically reads like C with classes.

Csound:

I have spent a considerable amount of time examining the source code to Csound, and adapting some of the algorithms for soundpipe. There is a treasure trove of great DSP algorithms there, but it definitely took me a while to get used to reading. It has a range of coding styles. A lot of the older code can be cryptic to read, and the style should not be emulated. If you get the Audio Programming Book, I recommend reading John ffitches chapter on Opcode design (this is what I did to learn). His tutorial has code samples that boil away some of the weird bits of the production code that you don’t really need to understand. Opcodes can be in a variety of places, depending on what decade they were made. I would recommend grepping for the opcode in quoates to find the dictionary entry in the code, then finding the compute code that way.

SuperCollider:

I actually know the least about this one. I’ve had some SC friends of mine tell me that SC did a major refactoring effort a while back. Looking at the code, I certainly believe it. It is written in what looks to me like clean enough C++ code (I do not really know C++ code, so I can’t really speak much on this). C++ is one of those languages that keeps evolving, so I have no idea of it is “modern C++” or not. Like Csound, SC has a rich ecosystem of good DSP algorithms. C++ has a tendency for code to be more coupled together, so my biased opinion is that it is could be harder to extract DSP code from SC than from Csound? But what do I know here.

PD:

I am only a casual PD user, mostly using it as a prototying environment. I managed to learn enough PD guts to get a fork of PD vanilla embedded as a sort of PD file player without the use of the libpd. I have also managed to implement some undocumented features in some of my custom PD objects by looking at standard PD object code.

Look at PD vanilla source code, if you are going to look at PD. It is actually a pretty small codebase, so you could definitely digest a sizeable chunk of it if you wanted to. Many people will tell you that PD source code is hard to read, but honestly it is not too bad if you are just interested in looking at external code. There is a lot of similar-looking code you can use as a rosetta tablet. Similar to Csound, PD has a lot of old-school looking code and design choices. Both Csound and PD were written for very different kinds of computers that we use today! So just take in things with a grain of salt.

11 Likes
#40

VCV Rack (https://github.com/VCVRack/Rack), is another open source project in C++ worth exploring the source code – seems very well written and organized.

4 Likes
#41

! these are all solid suggestions !

1 Like
#42

agreed, it’s a really clean project. i didn’t include it cause the stuff actually included in Rack engine is pretty minimal (momstly in these headers.)

some great building blocks though (FIR, IIR, minblep) as well as some stuff that is wrapping heavy libraries (speex resampler.)

super nice infrastructure

#43

ooooo made me realize - was thinking about stealing functionality from some mutable instruments source code but it looks like the audible instruments source is basically a bunch of handy wrappers for their modules - might be pretty easy to wrap that into a max external (though I suppose that would be the less learning things route)

#44

this is kinda what i mean. most actual VCVR functionality is not in the framework, it’s in the modules. some of them are by the framework author (andrew belt) and many are not.

the biggest one by AB is Fundamental

and, less extensively, his ports of Eseries and Befaco

they’re all worth looking at

the Audible Instruments stuff is interesting case. it’s not in fact a wrapper around Mutable Instruments source (which is well worth looking at itself; emilie is a great developer); rather, AB appears to have ported (most of) the functionality from MI, using the VCVR framework components instead of the MI conponents (makes sense since the latter are highly focused on STM32 processors.) there are significant differences; like, at first glance VCVR Clouds seems to have a pretty different interpretation of the “quality” parameter.

(all of AB’s code is highly dependent on building blocks from VCVR. e.g., the Fundamental VCF module defines sample update loop and UI, but the routines to actually compute the filter coefficients (viz, the hard part) are in the dsp headers linked above.)

maybe worth noting that some of this gets definitely into C++ specific language features (templates) and libraries (algorithm), though nothing super crazy

5 Likes
#45

Not sure if a discussion of open-source module firmware is maybe a bit orthogonal but I was reminded today that 4ms has published source for several modules (notably Spectral Multiband Resonator, Spherical Wavetable Navigator, Dual Looping Delay), all in C and for STM32. A lot of this of course focuses on UI rather than “audio programming” proper, though that’s also pretty interesting.

2 Likes
#46

Hoping this is not too tangential for this thread, but where could one start in practically and efficiently learning STM32 programming with an eventual goal of using it for making audio hardware things? Mainly asking so I don’t end up needlessly reinventing the wheel and making things too hard for myself (I have a tendency to do that).

I haven’t touched it at all yet, but I recently got this breakout board as a cheap starting point. Now I’m wondering if I’ve gone into the deep-end too soon and should’ve gone with a discovery board or something.

I’ve also seen this blog post from Tom Whitwell and I have a Clouds currently in a box somewhere, so I also have a fully functional thing with knobs I could use to play around with this stuff, too.

1 Like
#47

i’d say it kinda depends if you want to use the STM HAL, or something else, like the Mutable HAL stack. if the latter, you’re good to go with a Clouds i think. but i dunno how easy it would be to adapt to a raw breakout board like that one.

4ms and whimsical are other companies whose repos you can look at, i think both build on the MI work.

if you want to use STM HAL, it’s probably easiest to get a Discovery board and use the stack of stuff maintained / supported by STM. i recently had a fine experience with stm32f746g-discovery which is pretty cheap (though not $14 cheap!) and has tons of peripherals that are helpful for prototyping. (like, a touchscreen.)

i’ve done two audio projects with STM HAL. a few years ago someone else set it up, last year i started over myself. it was, frankly, a total pain. unfortunately, both projects are proprietary… :see_no_evil:

in a nutshell, i managed to build a linux project by using cubemx to figure out some peripheral configuration, openSTM (aka System Workbench, a customized Eclipse) to clone demo projects and get things going, and then translated everything into CMake / Makefiles with the standard arm toolchain / GDB. this was all a huge drag and i would love to see a better way, but at least it’s possible with open source toolchains.

honestly if you are worried about spending time in rabbitholes and just want the quickest path to coding something, that’s probably just getting a discovery board and using the System Workbench stack on windows.

thing is, the HAL abstracts so much stuff that you’re not really dealing with the instruction set or the MCU architecture. that’s probably fine; grokking the HAL and the examples will force you to read plenty of the arch documentation anyways, which is a baseline for rolling your own stack or whatever.

also, @csboling is on the nose with the observation that the stm-specific stuff and the audio stuff are basically completely orthogonal. all the painful crap is about configuring timers, memory, I/O, blabla. the audio stuff can and should be prototyped elsewhere first.

5 Likes
#48

This I think bears (over)emphasizing for the broader question of “how do I develop/extend module firmware”. Virtually anything you can should probably be prototyped elsewhere first. Even if you have an in-system debugger you’re reasonably happy with and other developer conveniences, it is still usually a much better experience to have an operating system yell at you when you inevitably access memory out of bounds, rather than an embedded target just crashing or corrupting unrelated parts of the program or whatever, and the compile/check/revise cycle is almost certainly much faster in a desktop programming environment than when you have to reflash the target in between. The Teletype code is a nice example of this – all the ops and language evaluation logic are in src/ so you can run unit tests/debug, all the UI and hardware interaction stuff is in module/.

4 Likes