Per-harmonic additive synth effects — ideas?

sudara · 2022-10-17T21:10:08.189Z

I’ve been building a cross-platform additive synth plugin the last couple years and finally got to the fun part: per-harmonic effects!

I’ve started with adding per-harmonic trem rate/depth. When the depth is varied over the harmonics it’s sorta like having a filter driven by an LFO. When the rate is varied over the harmonics, it’s… sorta like swirling chaos (haven’t weeded out what sounds good there yet):

I’m just at the start of exploring what’s possible here (different waveforms up next), but figured there must be additive addicts here who might have fun ideas to try out or things they’ve seen in the wild.

I’m familiar with most (all?) of the commercial additive offerings… my engine has some unique constraints but also some unique advantages (primarily performance, which is what’s allowing me to have 20 voices x 36+ oscillators each with their own adsr/envelope etc). One idea I want to try is adding randomness to the harmonic amplitudes as a source of noise.

A “bigger” idea I’ve been musing about recently is that my engine essentially works in the “relative frequency domain” — meaning, I can cheaply/cleanly/precisely change the volume of any harmonic of any note at any point in time… the real question is… to do what? (I’m thinking it would be sweet to provide an “API” of sorts and be able to script some effects from the UI). I definitely will be adding “formants” (like the ole k5000 has, basically spectral envelopes or EQ).

Anyway! For now, my goal is just to throw things at the wall and see what sounds good, eventually rework the UI to bubble up the good stuff…thanks for listening!

Here’s a more musical example of the per-harmonic trem (and whole synth):

wheelersounds · 2022-10-17T21:48:14.926Z

Mostly chiming in here to say congrats on a beautiful and apparently very functional UI - I think I’d really enjoy using this synth.
In my additive adventures, one thing I find valuable is the ability to “weigh” envelope lengths to have the highs decay faster than the lows, or vice versa, for a kind of physical modeling approach. I also often morph between two preset overtone balances, though I don’t know if you have interpolation built into your engine…
I also like having comparatively more depth on the tremolo/vibrato on the higher partials for some reason.
Super excited to see what other folks suggest!

sudara · 2022-10-17T22:08:23.069Z

Thanks for the kind words!

wheelersounds:

In my additive adventures, one thing I find valuable is the ability to “weigh” envelope lengths to have the highs decay faster than the lows, or vice versa,

The middle ‘envelopes’ tab currently lets you push and pull the harmonics very precisely over time… there’s also an excel spreadsheet-esque advanced editing of each harmonic’s adshr, which is… intense, but good for preset design. I wanted to go the “direct manipulation” route (painting/pulling/pushing the actual harmonics in the spectrogram visualization), but the prototypes I have for that seem over complicated so far…

wheelersounds:

I also often morph between two preset overtone balances, though I don’t know if you have interpolation built into your engine…

Nice! Yeah, interpolation is built into the engine, but I’m torn on morphing… I know some additive synths have this as their primary modulation “vibe”. From the UI point of view, I’m a bit afraid of adding that additional layer of abstraction, sticking the the whole UI behind a “voice I”, “voice II” sort of thing… maybe it would be pretty doable with just the harmonic volumes (vs. the whole env section).

Thanks for the thoughts!

wminor · 2022-10-18T02:25:31.652Z

Looks and sounds stunning. Great work there.

static · 2022-10-18T03:14:23.211Z

The animation and colors are excellent, I can’t get over that. It sounds very beautiful too! Nice going with all of this!!

Starthief · 2022-10-18T03:17:07.618Z

This is cool, I look forward to the finished synth

Throwing out some other ideas…

Extending the tremolo to stereo panning seems like a natural fit.

Also phase offset or modulation (at LFO rates) comes to mind, with different ways to tilt the amplitude and rate of modulation. Or skip around so the modulation only applies to even harmonics, every 3rd or 4th, etc.

At audio rate, what if you can cascade AM or RM (or PM I suppose!) modulation from one partial upward or downward to the next (or in other patterns?) Kind of thinking about 4ms Ensemble Oscillator with its Cross FM.

I also have wondered about emulating FM in additive with Bessel functions… maybe with something that filters or quantizes sidebands to fit harmonically even with non-integer ratios, or otherwise “bend” the results? That might not be a useful techique at all or it might be rely cool, I have no idea

Sometimes I think about polyphony/chords on additive synths, and it seems like something could be done with the overlap. E.g. if you play G2 and C3, there will either be partials overlapping exactly (and will they be in phase, out of phase or is it random?) or close together and fighting a bit due to the tuning system… maybe they should merge in some way, have their levels limited etc? That could even happen gradually over time, so they slide into perfect harmony as you hold the notes.

sudara · 2022-10-18T11:10:16.219Z

Wow, thanks everyone for all the positivity and excitement! I’ve been working daily on this for over 2 years (a lot of that time spent on UI, animations, etc), so it really warms me to hear this kind of feedback. I’ll be sure to post here when it’s beta time…

sudara · 2022-10-18T11:16:58.535Z

Starthief:

Sometimes I think about polyphony/chords on additive synths, and it seems like something could be done with the overlap

Ooooh, I like all your trains of thought but this one has a special place for me. I think a lot about the differences between harmony and timbre at the psychoacoustic and spectrum levels (is there a difference technically? When do notes become distinct psychocoustically?) and get excited by the idea of blurring those lines. I love the idea of modulating between “a complex note” and polyphony.

I have this other synth idea I want to pursue where all partials all in notes are locked to equal temperament. It would be a fun performance optimization (only 12 fixed rate oscillators needed per octave), but how would it sound? (my guess is thin/sharp, as there will be much less variety in frequency relationships).

oot · 2022-10-18T11:56:22.508Z

Your UI looks really slick.

You asked about ideas and I have a lot - not sure how applicable these are to your particular plugin but here goes:

One thing I have been thinking about a lot is an additive synth with graph functions in the mod matrix, like the graph you have for amplitude per partial but mod amount for various sources (envelope, LFO, velocity curve, etc.). This means the user could be more granular in their definition of how the tremolo should be expressed.

Speaking of graphs, I think your graph looks really good but there are a couple things I’ve noticed that is clunky with editing graphs on the additive synths I’ve used and that is that they don’t have macros for doing simple edits like only editing even or odd harmonics. Something like being able lock or solo partials by a particular multiple would improve that workflow I think. I suppose with fewer partials it might be less of a problem however.

The additive soft synths I’ve used the most are Razot and Harmor and both of those have two features in common:

First is a really indepth phaser. Harmor’s is especially complex in that you can apply it both exponentially and linearly; and on both frequency and amplitude.

The other I would expect from a modern additive plugin and that is some way to offset the pitch of select partials to create rich as well as metallic sounds.

Here’s some more random ideas that might be impossible or stupid but sound cool in my head:

Per partial compression. Having partials behave as restricted to an on/off state with a bit of envelope would hopefully behave like a spectral contrast effect, that underwater sound bad audio compression has.
Per partial portamento/slew. Having partials detune before catching up would sound interesting in my opinion. Having a slew on all amplitude modulation of a partial could also sound interesting, like a smear in certain parts of the frequency spectrum.
Partial stealing. Like note stealing but weirder. I believe some additive synths already do this subtly but it would be interesting to hear it exaggerated as an effect.
Allocating partials to a second related frequency so you could get harmonic relationships the same way you could use two oscillators per voice on a subtractive synth.
Multiple partial mixes for effects outputs. This would probably need an internal effects engine but sending different harmonic mixes to different delay lines sounds fun to me.

sudara · 2022-10-18T12:19:54.179Z

All great thoughts!

oot:

This means the user could be more granular in their definition of how the tremolo should be expressed.

I plan on exposing the per-harmonic trem rates/depth (plus other fx like stereo width panning, etc) at the granular level, like I have with the envelope at present, sorta like this:

AudioPluginHost - 2022-10-18.46@2x1784×1372 212 KB

oot:

Something like being able lock or solo partials by a particular multiple would improve that workflow I think.

Agreed! This is on the roadmap:

Figma - 2022-10-18.41@2x1808×736 88.1 KB

(HQ means “all harmonics up to nyquist” — for now I’m showing only the first 36 in the UI, as these make up the majority of the note’s sound, after which there’s diminishing returns in individual manipulation)

oot:

Having partials behave as restricted to an on/off state with a bit of envelope would hopefully behave like a spectral contrast effect

Interested in hearing more about this… Each harmonic has its own env — would this come afterwards and restrict the output to a specified range?

oot:

Allocating partials to a second related frequency

I wanted to offer a few “custom” partials, but I’ve been uncertain how to label/offer input for their frequency. I guess I could just pick a nice and easy number like “100” for the fundamental and have people enter the relative freq they want. There’s also some overlap with “doubling” which I would like to do (along the lines of that second osc in a subtractive synth that you can tune)

oot · 2022-10-18T15:58:14.611Z

sudara:

I plan on exposing the per-harmonic trem rates/depth (plus other fx like stereo width panning, etc) at the granular level, like I have with the envelope at present, sorta like this:

AudioPluginHost - 2022-10-18.46@2x1784×1372 212 KB

This looks really powerful but it’s pretty much the example of where I think a graph would be an easier way of editing. Say you want all of the odd harmonics to quickly die out and all of the evens to ring out and get darker. All you would need to do is open the graph for decay, select “odd”, do one curve, then select “even” and do another taller curve.

sudara:

This is on the roadmap:

Figma - 2022-10-18.41@2x1808×736 88.1 KB

(HQ means “all harmonics up to nyquist” — for now I’m showing only the first 36 in the UI, as these make up the majority of the note’s sound, after which there’s diminishing returns in individual manipulation)

This looks excellent! Also agreed that harmonics beyond 30-something don’t require micro-editing, I’ve mostly had issues with them cluttering the interface in other synths.

sudara:

Interested in hearing more about this… Each harmonic has its own env — would this come afterwards and restrict the output to a specified range?

I looked at my comment again and I think I didn’t think through what I was saying properly.
What I was trying to describe would function more like a noise gate than a compressor, where all partials below a certain threshold would be reduced, so the emphasis on the most resonant frequencies would be exaggerated. And yes it would be after the envelope, really all amplitude modulation, as the interesting part of the effect would come from the interaction between modulation sources in my opinion.

What you’re talking about does exist on other synths though, Razor has a limiter per partial you can use to tame certain resonances.

sudara:

I wanted to offer a few “custom” partials, but I’ve been uncertain how to label/offer input for their frequency. I guess I could just pick a nice and easy number like “100” for the fundamental and have people enter the relative freq they want. There’s also some overlap with “doubling” which I would like to do (along the lines of that second osc in a subtractive synth that you can tune)

Perhaps something like a number and a dropdown menu for whether it represents cents/Hz/semitones/octaves? Both linear and exponential are useful and it lets the user pick the language they’re familiar with.

sudara · 2022-10-18T21:03:07.667Z

oot:

Say you want all of the odd harmonics to quickly die out and all of the evens to ring out and get darker. All you would need to do is open the graph for decay, select “odd”, do one curve, then select “even” and do another taller curve.

Ah sorry, I misunderstood. The “envelopes” tab provides tools for changing and altering the envelope sections across harmonics (and visualizing those changes in context of the whole note).

I’m not a super-fan of editing isolated parts of an envelope (like offset or attack) because they alter what happens further down the envelope and it becomes sort of difficult to mentally visualize, especially since it’s 3D (amplitude over time across harmonics). Having some common shapes have felt flexible enough (though pushing around custom shapes would be fun!)

oot:

more like a noise gate than a compressor, where all partials below a certain threshold would be reduced, so the emphasis on the most resonant frequencies would be exaggerated.

Interesting stuff! Thanks for sharing. I also really liked your per-partial portamento idea too — i have an old note about a “portamento deluxe” that migrates partials from one note to the nearest partial of another. Unfortunately pitch stuff is my engine’s current weak spot right now, so it’ll take probably some solid work to gain full independence there!

wminor · 2022-10-20T16:58:34.335Z

I don’t want to derail the thread, so I hope you’ll forgive this brief side quest:

I’m assuming this is written in JUCE, since I know you’re a JUCE user (we’ve interacted a little in the Audio Programmer Discord server). I’m particularly interested in the GUI (because, frankly, it is stunning). Is that all regular JUCE, and did you use any other tools to help you develop it?

Happy to take the conversation elsewhere if you’d prefer not to clutter up this thread.

alanholding · 2022-10-20T17:42:06.493Z

Hello. Your synth looks amazing. It must be super satisfying to code something like that and see it come to fruition. Congratulations.

The visualisation of the harmonics in the second video in your post reminds me of 2D water tank simulations, which makes me think if applying some physical simulations to modulate the harmonics would be interesting. Like use an LFO or some other modulations to rotate the “harmonic tank” and hear what effect the harmonics “sloshing around the tank” would have? And other 2D physics stuff like that?

Anyway. The UI looks amazing and the clips sound really nice. Very interested in this!

wminor · 2022-10-20T17:49:50.298Z

That is such a fun and cool idea!

sudara · 2022-10-20T19:30:47.280Z

wminor:

Is that all regular JUCE, and did you use any other tools to help you develop it?

Thanks for the nice words!

It’s all vector in JUCE with the caveat that… I spent an unreasonable amount of time on details like animating tooltips (which JUCE doesn’t do and is surprisingly difficult) and figuring out how to layer and cache blurs efficiently (the tip is to use a StackBlur like FigBug’s). IMO JUCE doesn’t really invest enough in UI tooling (my feature request for that is #3 on the forum)… seems common that 80% of dev time ends up in UI, fixing and reinventing wheels. Happy to chat via discord further!

alanholding:

The visualisation of the harmonics in the second video in your post reminds me of 2D water tank simulations

Oooh, where can I see what that looks like? Do you mean the “oscillator dots” in the footer or the “spectrogram” of the sound over time?

alanholding:

Like use an LFO or some other modulations to rotate the “harmonic tank” and hear what effect the harmonics “sloshing around the tank” would have?

This is blowing my mind right now hehe. I do have plans to show the visualization “full screen”, I’m all for some good ole tank sloshing!

sudara · 2023-02-27T19:44:51.154Z

After what feels like a dozen engine rewrites later, I think I’ve landed on a sustainable way to do both independent pitch and amplitude modulation for up to 512 harmonics per voice.

Here’s an example of taking a square/saw and then pitching the harmonics until they are constrained to the set of fundamental frequencies that make up a chord (in this case a m6/9)

https://twitter.com/sudara/status/1630236704694239233

Edit: unfortunately no twitter vid embed, here’s a shorter lofi version:

I just started playing with pitch, have come up with a few other terrifying things — always interested in trying things out in case people have ideas.

Things I plan on trying:

Having each harmonic start at 0/nyquist and very quickly bending down to settle in its position
Subtler string tension-ish things on attack
Checking out if it’s worth getting into the whole Railsback curve
Trying to discover more about the phase cancellation patterns that happen when harmonics within a tone go slightly out of tune with each other (this was a big surprise to me so far, how much cancellation can exist, the patterns that emerge)
Trying to hack together a reverb-ish pitched noise from adding randomness to both freq/amp of each harm.

Outspacer · 2023-03-01T15:06:26.864Z

Wow, what a unique looking synth. The design of the synth itself as well as the UI seem so thoughtful and different. Can’t wait to try it!