#61

generally it works, its when you start mashing the keyboard its leaves them hanging around.

and Ive just debugged with sendTrig, and the envelope is marked as done, it just not freeing the synth

SynthDef(\testSynth, {
		arg freq=440,gate=1,cutoff=440,amp=0.3;
		var sig,env;
		sig = Saw.ar(freq);
		env = EnvGen.ar(Env.adsr(),gate,doneAction:2);
	    SendTrig.kr(Impulse.kr(4), 0, Done.kr(env));
		sig = sig * env * amp;
		sig = LPF.ar(sig,cutoff.clip(10,24000));
		Out.ar(~outL,sig!2);
	}
).add;

[ /tr, 1125, 0, 1 ]
[ /tr, 1054, 0, 1 ]
[ /tr, 1125, 0, 1 ]
[ /tr, 1054, 0, 1 ]
[ /tr, 1125, 0, 1 ]
[ /tr, 1054, 0, 1 ]

thanks for the kr() tip, changed thatā€¦ but still the same behaviour :frowning:

EDIT: Env.perc() does the same :confounded:
EDIT2: correction
EnvGen.ar(Env.perc(),gate,doneAction:2); - does the same
EnvGen.ar(Env.perc(),doneAction:2); - works ok, it seems

#62

If it happens when mashing keys my guess is itā€™s a race condition. Probably a new synth gets created and inserted into the dictionary before the gate set to zero message is dispatched to the first. You could probably verify this with some logging.

#63

Iā€™m not really sure, but maybe itā€™s got something to do with bundling if osc messages to the server? Maybe try adding the line ā€œs.sync;ā€ after you set the gate to zero?

You can free also free a node if you know itā€™s id by sending an osc message to the server: s.sendMsg("/n_free", id);

#64

maybeā€¦
Ive noticed if I test with:

~notesfunc.value(nil,nil,60,100);
~notesfunc.value(nil,nil,60,0);
~notesfunc.value(nil,nil,60,100);
~notesfunc.value(nil,nil,60,0);

then if fired, individual it works, but fire in one goā€¦ and you get no notes, and synths that are just thereā€¦
but as ive seen dictionary is empty, its just the synths not freedā€¦
(Im wondering if perhaps the env doesnt even start?)

BUT if i do in a routine, with sync


Routine( {
~notesfunc.value(nil,nil,60,100);
	s.sync;
~notesfunc.value(nil,nil,60,0);
	s.sync;
~notesfunc.value(nil,nil,60,100);
	s.sync;
~notesfunc.value(nil,nil,60,0);
	s.sync;
~notesfunc.value(nil,nil,60,100);
	s.sync;
~notesfunc.value(nil,nil,60,0);
	s.sync;
}).play;

this worksā€¦

so whats happening hereā€¦ is Im calling notesfunc (indirectly) from a OSCdef
so Iā€™m wondering if I have to wrap that some how, or yield?

not quite sure exactlyā€¦ so I just yield, or sync in the OSCDef
seems inefficient to do for every osc message.
like this?

OSCdef( \OrganelleKeys,
	{
		arg msg, time, addr, recvPort;
//                ~notesfunc.value(msg[3],msg[4]);
               Routine({ ~notesfunc.value(msg[3],msg[3]); s.sync;}).play;
	},
	"/key",
	recvPort:4000
);

(this is simplified, i dont quite to do this, I actual use an object dependents to fire the events, but i suspect the OSC is the issue?)

#65

youā€™re right, this really does seem like a bug. all the self-freeing UGen methods seem to be unreliable when used with a gate argument. what a drag.

my test case is even simpler: just create a large number of threads, each starting and stopping a synth using a gate argument.

occasionally one just doesnā€™t get freed, and the choice self-freeing method doesnā€™t seem to make a difference, these all fail:

  • EnvGen(doneAction:2)
  • FreeSelfWhenDone
  • FreeSelf
  • DetectSilence(doneAction:2)
    &c

i think this should preclude any culpable race conditions in our client code, because each node is only ever controlled from one thread, and clearly the OSC messages to set the gate value are making it to the server - itā€™s just the node freeing itself that fails (and i donā€™t really know what the mechanism for that looks like, under the hood; i guess thatā€™s where weā€™d find the race condition.)

it also doesnā€™t seem to make a difference whether the server is local, internal, or remote.

i even rolled back to 3.6 and found the same behavior. pretty nasty; iā€™m inclined to go back to 3.2 on an older system and see if it persists.

as @frankchannel suggested, seems a viable workaround to free the nodes from the client using a SendTrig.
this works for me (on mac, 3.8; havenā€™t tried on pi/headless/dev yet); i can just keep executing the script until it maxes the scheduler or the node count, and it still frees everything:

s = Server.local;

s.waitForBoot { Routine {
	var interval, offset; // make it sound interesting at least

	SynthDef.new(\gated_sine, { arg out=0, hz=110, amp=0.5, gate=1, pan=0;
		var ampenv, snd, done;
		ampenv =  EnvGen.kr(Env.asr(), gate);
		snd = ampenv * SinOsc.ar(hz) * amp;
		done = Done.kr(ampenv);
		SendTrig.kr(done);
		Out.ar(out, Pan2.ar(snd, pan));
	}).send(s);
	s.sync;

	OSCdef(\free_when_done, { |msg, time|
		s.sendMsg("/n_free", msg[1]);
	}, \tr);

	/// make a bunch of notes and sustain them for random times
	// each is on a different thread
	interval = Array.fill(1 + 4.rand, { 2 + 8.rand });
	offset = 30 + 20.rand;
	200.do {
		Routine {
			var syn;
			0.4.rand.wait;
			syn = Synth.new(\gated_sine, [
				\hz, (offset + (10.rand * interval.choose)).midicps,
				\amp, 0.02, \pan, 1.0.rand2
			]);
			NodeWatcher.register(syn);
			1.0.rand.wait;
			syn.set(\gate, 0);
		}.play;
	}

}.play; }

/*
s.queryAllNodes
*/

whatā€™s bizarre is that, as youā€™ve found, it also works with EnvGen.ar(Env.perc, doneAction:2) - that is, the failure case is somehow dependent on using a gate argumentā€¦ hmā€¦ should keep pulling on that stringā€¦

(btw: i use audio rate envelopes all the time. any time when it needs to be faster than the audio block sizeā€¦ certainly this makes an audible difference for short percussive envelopes and larger block sizes.)

yeah, every Routine is a new thread so thatā€™s not great.

2 Likes
#66

Ugh thatā€™s pretty nasty that doneAction with gate is unreliable :confused:

#67

I cant recall Iā€™ve had these issues. Sounds like OSC messages are not synced up. I almost always wrap synth creation and set() calls in s.bind clauses. Does that help in your case?

(s.bind incurs the default Server latency as defined in s.latency. I lower the default and usually keep it at 0.05)

My reasoning: s.bind creates a bundle with time stamps. Iā€™m not sure non-bundled Synth.new and set() calls are time stamped.

2 Likes
#68

really is this necessary: (from bind doc)

(
s.bind {
    a = { |freq=100| SinOsc.ar(freq, LFTri.ar(freq)) * 0.2 }.play;
    s.sync;
    a.set(\freq, 400);
}
)

this seems ludicrous if its requiredā€¦

I guess I could put the OSCdef callback code in a bind, but having the users have to litter around s.sync if they call set, seems pretty horrible.

e.g. what Iā€™m thinking is:

OSCdef( \OrganelleKeys,
	{
		arg msg, time, addr, recvPort;
                s.bind({
                    if ( msg[1]>0,
                       {~notes.key_hit(msg[1],msg[2]);},
                       {~aux.key_hit(msg[1],msg[2]);}
                     );
                     s.sync;
                }
               );
	},
	"/key",
	recvPort:4000
);

what Im concerned about though, is this will only sync, between osc callbacks, which will probably solve this particular issue, but not if in the callback the user has to do sync between multiple calls to setā€¦ and what is this going to do to performance.

also there is something still a bit odd hereā€¦ in my particular case, the synth creation, and set of gate (to zero) , are coming form different OSC messages, i.e. multiple invocations of the OSDdef function, surely SC, would sync between them?
Im guessingā€¦ possibly the issues is, in the case of ā€˜mashing the keyboardā€™, perhaps two messages turn up in the same udp packet, so perhaps the OSCdef is done multiple times with no sync in betweenā€¦ might explain why its uncommon but still happens.
(I will note though, in this caseā€¦ these messages (/key) are not in an OSC bundle, though I dont think thats relevant)

I quite like the trig solution, as it could be slow, to just recover ā€˜lostā€™ synths, but my fear isā€¦ if this is a sync issue, is this the only time this occurs, or can it occur with things other than gate.

Iā€™ll have to play with itā€¦ but feels ā€˜wrongā€™, whilst as a programmer Iā€™m ok with technicalities like this (thread sync is bread n butter for me) , its not really for musicians/patch makersā€¦

Does anyone know of a doc which explain when sync is necessary, and when its not?
(bind/makebundle say how to do it , rather than whyā€¦ as far as i could see)

BTW: sorry if this is all off-topic, Im new to SC, so dont really know where the ā€˜support channelsā€™ are , so came here, as you guys are very knowledgeable, thanks for your input.

#69

I was unclear and just referred to documentation without any context, sorry.

I propose to wrap your Synth.new and set() messages in s.bind { } clauses (or equivalent) and see if the issue remains.

Iā€™m not suggesting to use s.sync. I believe the time stamps implicitly added in bundles created using s.bind may solve your problem.

OSC events of Synth.new() and set() invocations are AFAIK not guaranteed to arrive in the order they are made in SClang to the scsynth server. This might cause issues with messages without timestamps as events without timestamps are performed directly on the scsynth.

Time stamped ones makes the events timing deterministic (as long as server latency is not exceeded), since (again, AFAIK) the scsynth server will store and schedule the events at a given time point and not perform them directly upon arrival.

I cannot recreate the issues using @zebra 's posted code above so this is a guess. :slight_smile:

More on this:
http://doc.sccode.org/Guides/ServerTiming.html

1 Like
#70

The official mailing list is the best place to ask, I suggest posting your question there since it does seem a bit weird this isnā€™t working. Some of the developers who know more may be able to chime in.

Though IMO here is totally fine, but thereā€™s definitely a larger audience there.

1 Like
#71

sorry if i was unclear,the posted code above is working for me - thatā€™s the workaround that doesnā€™t rely on self-freeing synths.

here is a version that doesnā€™t work - occasionally it leaves some synths un-freed (though silent)

s = Server.local;

s.waitForBoot { Routine {

	var interval, offset;

	SynthDef.new(\gated_sine, { arg out=0, hz=110, amp=0.5, gate=1, pan=0;
		var ampenv, snd, done;
		ampenv =  EnvGen.ar(Env.asr(), gate, doneAction:2);
		snd = ampenv * SinOsc.ar(hz) * amp;
		Out.ar(out, Pan2.ar(snd, pan));
	}).send(s);

	s.sync;

	/// make a bunch of notes and sustain them for random times
	// each is on a different thread
	interval = Array.fill(1 + 4.rand, { 2 + 8.rand });
	interval.postln;
	offset = 30 + 20.rand;
	200.do {
		Routine {
			var syn;
			0.4.rand.wait;
			syn = Synth.new(\gated_sine, [
				\hz, (offset + (10.rand * interval.choose)).midicps,
				\amp, 0.02, \pan, 1.0.rand2
			]);
			1.0.rand.wait;
			syn.set(\gate, 0);
		}.play;
	}

}.play; }

/*
s.queryAllNodes
*/

i actually donā€™t think this has anything to do with the timing of client->server OSC. the failure state is the self-freeing, which should happen long after the gate argument is set - and it is being set since we canā€™t hear anything.

the failure conditions seem to be:

  • multiple self-freeing synths are finishing at the same time. (i assume)
  • they are using gate arguments at all - this is the really weird part to me.

so yeah, seems like a race condition all right, but one that is somehow internal to the server process (?)

i think this is definitely a good candidate for a bug report or query to sc-dev.

both AFAIKs tally with my understanding as well. in particular, non-timestamped messages will be executed at the start of the very next audio processing block after they were received.

(in practice, iā€™ve never actually had a problem with calling Synth.new(); Synth.set() - my assumption is that all queued, unstamped synth creation commands are executed before all queued, unstamped set commands, on each block? dunno.)

you do need to sync after server commands that complete asynchronously, and this includes SynthDef compilation { }.play is a convenience method that creates a SynthDef on the fly, creates a Synth using it, and returns the Synth object - which isnā€™t synchronous, so the Synth doesnā€™t really exist on the server yet! (and yeah, this is definitely one of those ugly misleading default cases that @jah mentions far above.)

so {}.play is really only useful (IMHO) for simple fire-and-forget cases, and its almost always better to manage SynthDefs explicitly.

1 Like
Norns: development
#72

Iā€™m pretty sure that sclang doesnā€™t communicate with scsynth via OSC if they are running on the same machine, thereā€™s a shared memory interface in use instead.

#73

alas, shm interface is only used for control busses and scope buffers, AFAICT

sc-dev thread
sclang interface
c++ header

#74

ok some interesting, and good (?) newsā€¦

so based on @jah 's suggestion i changed it tooā€¦

OSCdef( \OrganelleKeys,
	{
		arg msg, time, addr, recvPort;
                s.makeBundle(0.003, {
                    if ( msg[1]>0,
                       {~notes.key_hit(msg[1],msg[2]);},
                       {~aux.key_hit(msg[1],msg[2]);}
                     );
                });
	},
	"/key",
	recvPort:4000
);

and this fixes itā€¦

I decided to use makeBundle, as then I dont have to set the servers global latency timeā€¦
this also meant I was comfortable using much lower than the suggested 0.05, which was unusable for latency for playing a keyboard (50ms), but 3ms is kind of acceptable.

I did initially try 1-2ms, I found this would still leave a few synths would still be left over (after mashing the keyboard, for 400+ notes) ā€¦ probably in ā€˜realā€™ use this would be ok.

I dont think this explains why this is happening reallyā€¦
in my particular example, the gate is off, and the env is marked as done, so why is the doneAction not being triggered (and as @zebra noted, freeWhenDone also does not trigger).
something (server?) appears not to realise that the ā€˜doneā€™ of the envelope requires that the parent object (synth) should be freedā€¦ Im assuming this is the responsibility of the server/rather than client.
(or do doneactions get processed on the client end?)

Ive a nasty suspicion, that perhaps this makeBundle might not be fixing it, but rather introduces enough of a time delay to make it workā€¦ such is the nature of thread race conditions, slight timing changes often ā€˜fixā€™ the issue :wink:

#75

Interesting. The packets sent between the IDE process and scsynth donā€™t look like regular OSC messagesā€¦

Edit: nevermind, I was getting confused because for some reason thereā€™s an extra 4 bytes at the start of the message (0x0 0x0 0x0 0x0f) before the OSC type tag. But also there doesnā€™t seem to be an address either.

Edit2: Ah, I see the protocol is a modified version of OSC defined here: http://doc.sccode.org/Reference/Server-Command-Reference.html They use command numbers instead of path strings.

2 Likes
#76

Ive not read these bugs thoroughly, but I wonder if one or both might be related :


#77

yeah itā€™s all on the server; iā€™d blame this on some bug in the way NodeEnd events are handled. itā€™s a bit convoluted and i donā€™t fully grok it. but AFAICT between a UGen calling NodeEnd (e.g. here) and a Node actually being freed (e.g. here) is all on the same thread?

i know the ā€œgateā€ argument is treated specially in some way for the purpose of some ā€œfeatures,ā€ and at this point i can only guess that something about gate-argument processing is interfering with node deletion (node ID is getting zeroed prematurely, i dunno)

those issues you link do seem relatedā€¦
ā€¦ and hereā€™s another one

i think thatā€™s true. it ensures that two gate-close messages are never sent close enough together for the corresponding synths to hit their ā€œdoneā€ states at the same time? (if they have same release curve)

i have a new test in mindā€¦

ā€¦ ok, this is just so weird: if i take the script above, and donā€™t randomize the timings at all, then there are no stuck synths. not what i expected!

like, it breaks if one node is freeing itself at the same time as a different node gets gate=0 ??

anyway, iā€™m going to stick with the method of using SendTrig(Done.kr()) and explicitly freeing nodes from the client, since it seems most reliable right now, and the server messaging code looks like murky watersā€¦

1 Like
#78

Interesting. There appears to be open scsynth issues on gate messages not working as expected when handled within the same ā€œblockā€.

Iā€™ve not experienced hanging sounds due to the above but have had sync issues causing not-so-great-sounding audio modulations. These have always been resolved by making sure OSC messages sent from sclang at the same (logical) time are bundled and, most importantly, time stamped. BTW Iā€™ve never felt the need to communicate back node-lifecycle information from scsynth to sclang. Iā€™ve tried to keep synth creation and automation a one-way sclang -> scsynth thing and tried to work around issues within sclang.

The Client-Server, Scheduling, Server timing and Physical vs Logical time concepts in SuperCollider are tricky. Bundling stuff with timestamps has certain trade offs. As @TheTechnobear noted the added delay, if large, can be problematic for live triggering. The flip side is that sample accurate timing (more exact than, for instance, Pd) can be attained for certain use cases. OffsetOut will sample accurately trigger sounds within a ā€œblock bufferā€. OffsetOut + timestamps makes a lot of difference in tight drum sequencing.

@TheTechnobear Your suspicion seems correct. As long as timestamping things prevents this I dont worry to much about it.

One thing you might wanna try out to get around sloppy live triggering due to server delay would be to send note-ons without timestamp and not offs with timestamps possibly after a really small, fixed delay:

if (noteOn) {
	synth = Synth.new(\whatever);
} {
	fork {
		0.05.wait; // lower might work
		s.bind { // or s.makeBundle()
			synth.release; // a variant of set(\gate, 0)
		};
	}
}

ā€¦ or something (untested code). Reasoning: onsets are more important to trigger as fast as possible compared to note offs. You still might run into issues with this, though, since some messages do not have timestamps.

Also if you decide to use non-timestamped messaging, check if block size (s.options.hardwareBufferSize) can be lowered since the reported issues seems to relate to it.

@zebra I sure hope these issues are not due to one node affecting other nodes but rather isolated to a single node. Nodes affecting other nodes would be severe!

#79

hope these issues are not due to one node affecting other nodes

i would love to see a demonstration of this bug without >1 nodes (or indeed, many nodes), that would clarify things

i want to make sure the issue (my issue) is clear though:

  • no problem at all setting gate values, getting EnvGen to honor the gate value and become silent. iā€™ve never needed any special tricks for this (even bundling.)

  • once in a while, a synth will go silent all right, but not free itself as its supposed to from doneAction or &c.

  • i havenā€™t got this to happen with a single node[*], or even (weirdly) with a large number of nodes on different threads that are controlled in tandem. i can only get it to happen with a large number of nodes that are controlled at kinda random times.

  • the reproducing cases are definitely (i think) not due to missed gate msgs, nor (therefore?) to client-side race conditions when sending OSC.

  • its not even a problem in many circumstances, like typical performance patches that are run for an hour or two. (maybe 1/5K synths will silently hang.) but for an embedded supercollider app with arbitrarily long uptime, itā€™s unacceptable to have zombie synths even if they are silent, b/c the server will eventually stop working.

some of the issues reported in links above are definitely not about this, but about clear edge cases. like starting a synth with one gate value and setting to different gate val in same audio block. however, itā€™s all maybe a bit mixed up so seems good to at least glance at any issue even tangentially related.

[*] admittedly havenā€™t tried hard enough to reproduce with only 1 node at a time - seems like itā€™d take awhile!

(apologies for being so tedious about this)

1 Like
#80

I want to build a batteries-included ā€˜DX7 orchestraā€™ on top of @canncā€™s work(https://github.com/everythingwillbetakenaway/DX7-Supercollider) to bundle some prime soft-synth preset sounds with a grid-controlled looping midi sequencer Iā€™ve been working on. Thinking along the same lines as tidal/super-dirt.

I know nearly nothing about sc & the code for that thing looks scary! Hereā€™s what Iā€™m planning, and despite lack of existing sc skills I think it should be a very manageable amount of work:

  • audition sounds & curate a GM drumkit from the best of internet percussion presets
  • bundle some drum sounds to trigger from a single midi channel in some kind of GM-drum-like layout
  • curate 14 other midi instrument tones to put on remaining 14 channels
  • audition more presets, rinse, repeat
  • add UI to select between a couple of different ā€˜orchestrasā€™
  • there should be a stereo mixer onboard, jack-stereo-out for live & also individual jack outputs for certain instruments/instrument-groups to record into ardour.

Before I start fudging through learning enough sc to pull this off, can anyone say if I will get there without getting stuck into the internals of DX7-supercollider? sc should behave more or less like a general midi module when running the ā€˜DX7 orchestraā€™ program. Not looking for real-time param tweaks on midi CCs (yet!)

My other option is to start sifting through zynaddsubfx banks & hydrogen sample-kits but that feels like a bit of a cop-out somehow (sampling gives me the heebie-jeebies anyway)ā€¦ Equally I donā€™t want this to get too open-ended - looking for a quick win so I can stay focussed on the controller itself. Good sounds ā€˜in the boxā€™ would help a lot!

2 Likes