#1

I was a little bummed to discover recently that POSIX message queues aren’t available on macos. (I’m in the middle of a big rewrite on a project replacing python queues with posix queues, python threads with pthreads, and python process pools with openmp parallel for loops…)

In 2018, what do C programmers reach for on macos when in need of a standard blocking message queue? I started this rewrite with a linked list and locks, but opted to try to switch to posix queues when I discovered they exist! Maybe best to stick with the linked list? I suppose I’m a typical python programmer in that I’d assume there is a standard library for something as universal as a message queue, but maybe not?

Thanks!

#2

FWIW, I opted to stick with a linked list + mutex locks… seems the simplest way forward. :-/

1 Like
#3

That’s a shame - I was lurking in this topic to see if I could understand what you were talking about! :thinking::joy:

#4

I’m using first-in-first-out queues to delegate voice creation, among other things – there is a daemon process listening for commands, and if it gets (for example) a play message, it adds the message (and params) to a queue, and another thread waits for messages in the queue, spawning voices for playback when it gets play messages.

I’ve been prototyping using the queue.Queue implementation from the python standard library along with threads from the threading module and spawning voices with a ProcessPool from the concurrent futures library, but that creates a lot of overhead for voice creation – in some cases I was seeing more overhead on voice management than DSP…

So I’m moving the voice management logic into cython, which lets me release the GIL and use real posix threads, as well as openmp’s parallel for loops – the catch is that I can’t use python objects in those contexts, including the standard library queues…

I’ll share my implementation when I’m further along!

2 Likes
#5

sorry bit necro, but

the gaps in macOS posix implementation are v annoying. one rather obnoxious literal answer is to use Mach directly.

or roll your own Q with linked list and mutex.

or use c++ and Boost.

3 Likes
#6

I ended up switching back to python queues for this project, and so far optimizing elsewhere (eg making improvements on my entire approach to voice handling) has been OK, and the bottleneck I was originally fighting isn’t such a big deal anymore…

I’m still really interested in this, thank you for the necrobump!

I also bought “The Linux Programming Interface” this year hoping to get a better handle on queue implementation, for when I revisit all this…

1 Like