UX of Music Instruments & Tools

[papernoise]

@LNDF

it’s cool that people are making this stuff, but most of us would never come close to playing in room with the adequate equipment.

That’s a very good comparision! So far I’ve been happy to find venues where they actually had a PA haha
And yes, I also feel that we’re currently putting a lot of energy and money in the wrong place.

But maybe we’re getting a bit too far away from the original topic here, which is about the design of music instruments and tools.

[anon45924993]

@now_its_dark

I kinda question whether ‘we are not remotely about to approach’ an expression of display tech/form factor which would be acceptable for general computing— maybe depends on what is remotely, like 5 years, 10?

Making 5-10 year technology predictions is folly (unless you have supply chain insight on the level of Tim Cook). It certainly isn’t going to happen this year, I can confidently state that.

Think it through though, it may be physically impossible to solve the foveal distance problem no matter how many pixels you can paste to your eyeballs. It is a physiologically bad idea given the physics of eyesight.

@now_its_dark

Exactly how wide an FOV and how small a headset are required for it to arrive?

Weight and FOV aren’t the only factors. When you include all the factors my answer is “it isn’t on the way and will not be arriving”.

@LNDF

I found an interesting research paper from last year.

Excited to read this later today!

[G4B3]

@papernoise

But maybe we’re getting a bit too far away from the original topic here, which is about the design of music instruments and tools.

It’s at least a parallel conversation imo. I think there is going to be a similar boom/bust as touchscreens in cars with AR interfaces for music software. “The interface can be anything in physical space!” will quickly be replaced with “I just want to touch something real!”

There is a reason so few musicians play theremin. Waving your hands around with no touch feedback isn’t as fun as it looks.

[now_its_dark]

From the tone / seeming disdain for this tech, it kinda reads like you’re generally soured on it for some reason. In many ways it’s already pretty great and usable from my pov. (not vision pro specifically, just hmds in general) I’m curious, what exactly does it have to be for you to be ‘sold’ on it?

[papernoise]

@G4B3

There is a reason so few musicians play theremin. Waving your hands around with no touch feedback isn’t as fun as it looks.

Although we could also see this from another angle. The Theremin is an instrument, which – from a playing interface point o view – is much closer to traditional instruments like a violin. Of course it’s very different in the way you play it, but it’s definitely an instrument that requires some learning, practice and hence commitment.
It’s not like you can’t have some more immediate fun with it, but if you want to make some more specific, let’s call it deterministic, music with it, it will requite a bit of practice (and commitment).

Now synth interfaces often strive to reduce that part as much as possible. Synths and electronic instruments are generally loved because they lover that barrier and require less practice. Our electronic instruments are full of little helpers, for example: things like sequencers and time-quantization alllow you play in time even if your timing is very sloppy.

I imagine the AR apps you describe will probably have such helpers as well, maybe allowing you to hit the targets more easily, without requiring as much precision as a with a Theremin.

[anon45924993]

@now_its_dark

I’m curious, what exactly does it have to be for you to be ‘sold’ on it?

something that doesn’t make 50% of the population physically ill from use.

Also, ew, please don’t “sell” me on tech.

[vcvcvc_val]

@now_its_dark

I’m curious, what exactly does it have to be for you to be ‘sold’ on it?

I’m pretty much in lockstep with Jason on this one and I’ll say that I think people are taking the display portion of this hugely for granted.

Some kind of overlay/HUD would make me feel a LOT more comfortable than converting the whole world to a flat, close up pixel rendering of your POV. Strapping a display to one’s face will likely come with “please take a break” warnings but we all know how much those will be ignored given how our culture around tech and taking care of ourselves works.

It just feels irresponsible the way it’s being marketed. And it’s not even all on Apple or Meta – under our current system anyone that implements this tech has an automatic incentive to not report issues. Trying to mitigate that requires a herculean effort that’s way less efficient than fixing the incentive structure (which happens to be our entire economy so this conversation is now getting bigger in scope).

IMO if we want wearables/implants we have to ditch the profit motive or it’s gonna be a dystopian nightmare. That’s not a kneejerk either – I’m a tech fangirl and a huge nerd. I love gadgets and have early adopted on a lot of things. But even I have my limits and am starting to get called “Luddite” for my caution on body tech and crypto in particular. It’s… challenging.

[watney_sw]

I like to compare Theremin to the Ondes Martenot. Both afford the same amount of pitch expression, and both require practice and skill. However, on average I think that people sound a lot more on pitch with the Ondes than with the Theremin. The Ondes’ fingerboard with bumps and divots provides tactile feedback. It’s easier to find a pitch and also lock into a pitch once you’ve found it.

So, even within the realm of expressive instruments that take practice and skill, there’s room for the designer to maximize the amount of expressiveness you get and minimize the difficulty. And synthesizers I think have more room for this optimization than acoustic instruments.

[now_its_dark]

Lmao I’m so not trying to sell you on anything, it’s just a figure of speech. It’s not as though I’m some hardcore VR advocate or anything, it’s just surprising to me to see this kind of reaction.

[LNDF]

@watney_sw

So, even within the realm of expressive instruments that take practice and skill, there’s room for the designer to maximize the amount of expressiveness you get and minimize the difficulty. And synthesizers I think have more room for this optimization than acoustic instruments.

That’s a valid point, but it is also a double edge sword.

Synth and computer allow for a decoupling of the interface and the processing system, the form of a violin is hugely dependent on the physical contraints needed for it to emit (nice) sound whereas the GUI of a synth isn’t really limited by the DSP, so yeah infinite possibilities of ui exploration !

BUT
You gotta do all the mapping between UI and DSP explicitely.
The violon / theremin is a good example
the violon implies a lot of intricate mapping, the pressure of the box controls a ton of timbral variable, and the volume is controled by a tone of parameters. The theremin has a control for pitch and one for volume (I imagine there are modded theremin for timbre modulation but that’s not the norm). This simplicity can make it a bit less “exciting” and it doesn’t afford for a lot funny exploration like playing on the threshold of oscillation

[now_its_dark]

That makes total sense, there is an intimacy to it that is definitely concerning. As a social/cultural phenomenon it’s not hard to see how it could lead to a dystopian outcome. I guess my outlook is to take the good with the bad… maybe it’s naive, but if something has a potential for positive use, I tend to to look for that application, not with blinders on, but with some degree of optimism.

[karst]

@now_its_dark

I guess my outlook is to take the good with the bad

i think it’s maybe also worth looking at technology not just as “good” or “bad”.

spatial computing, i think in ways that are similar to each of the last few cycles of this is the new tech that changes everything (LLMs, NFTs, crypto, self-driving cars) is an attempt to create a facsimile of something (human reason, physical art, money, and decision-making) and discuss the merits of the technology as if it is already or is nearly a successful facsimile, rather than something that falls (perhaps dramatically) short of what it claims.

so even before getting into the merits of replacing multiple imaginary screens in your house with one very nice screen that has to be mounted to your face, it’s looking at questions like “is this actually compatible with human eyeballs?”.

---

i do think there’s an interesting tension here, because a deep relationship with an instrument feels to me like the migration from rational understanding to deeper instinct. the act of learning an instruments is almost sublimation, the state-change from some kind of concerted effort to understand to turning into something you don’t have to think about at all.

to @papernoise’s point, electronic instruments are sometimes outlier in this instance. the fact that they are generally parametric both in design and interaction can sometimes make it difficult to move from “i know what this instrument is and how it works” to “i can play this instrument”. additionally, the many ways the instrument can play itself changes that relationship in some real way.

in some ways, the explicitness of electronic instruments also prioritizes the role that your vision plays. the assessments are more visual and less haptic (sound itself is, i think, closer to touch than to seeing). if you accept the visual utility of an instrument, then a device that proposes to conjure the phantom of any object and lets you interact with it somehow seems very appealing, but i do get the sense that it is going in very much the wrong direction to give people that deeper connection with the act of making music.

skimming the paper that @LNDF shared, it also seems to be wrestling with this and points in an interesting direction, i think.

[Kirtai]

There are already examples of implanted tech that has been orphaned or abandoned by the companies that made and supported it so your concern is not misplaced.

BTW, Luddites were not against technology, they opposed dehumanising tech.

[Starthief]

Tactile or not, the violin is fiendishly difficult and in some ways poorly designed. But also supremely expressive, with centuries of tradition behind it and wonderful examples of music that features it in a variety of genres. And it’s often an option for schoolchildren to learn.

The theremin is still, unfortunately, seen mainly as a gimmick for making wobbly ghost noises. Most people haven’t heard of Clara Rockmore, or any of the handful of really excellent modern thereminists. Most theremin repertoire is just violin or vocal parts that have been adapted. Honestly I feel like this is less because it has no tacticle component, and more because it’s electronic but almost never used in rock/pop/hiphop/dance genres. It’s probably better known to the general public than the Ondes Martenot or the Trautonium or Clavioline after all…

Personally I feel like the violin is a bit more difficult instrument than the Theremin (having played both) but I’m having trouble imagining a middle school Theremin ensemble

[karst]

@Starthief

a middle school Theremin ensemble

aren’t middle school music teacher’s lives hard enough already?

[JES]

@Kirtai

There are already examples of implanted tech that has been orphaned or abandoned by the companies that made and supported it so your concern is not misplaced.

BTW, Luddites were not against technology, they opposed dehumanising tech.

It’s actually worse than that. The recent history of cochlear implants in India is a great example of what happens when maintenance isn’t part of the plan.

[vcvcvc_val]

@Kirtai

BTW, Luddites were not against technology, they opposed dehumanising tech.

Hmmm maybe it is a badge of honor…

[Starthief]

@karst

aren’t middle school music teacher’s lives hard enough already?

I don’t envy anyone who has to deal with middle school kids. But a theremin ensemble has to be less painful than a roomful of soprano recorders… :shudder:

[anon45924993]

I think I was 6-7 the first time I was handed a recorder in class? Some of the most excruciating sounds I’ve ever heard (and I’ve done factory and construction work)

[ghostclub]

Recorder class was an optional activity at my primary school. Of course I immediately signed up, excited to get my own recorder for free.

Except I didn’t get the tape recorder I was expecting, but a fiendish plastic instrument capable of producing some of the most unpleasant tones a human will ever endure.

I felt deceived, and I was inconsolable. The recorder lived in my closet and I never went to class again.

My parents were of course grateful that my recorder career was so short-lived, and they later bought me the Sony cassette recorder I’d been hoping for. I used the built-in microphone to record everything from my own private radio shows to the demo tape that secured my first label release on 7" at the age of 16.

All because of the humble, horrible recorder.