UX of Music Instruments & Tools


@strettara, I completely agree. And it points out that for a musical instrument, “obvious” isn’t necessarily “better”.


vboehm has ported fourses for max, maybe it could be of your interest and @Rodrigo’s: http://vboehm.net/downloads/

and here’s a max patch: https://www.dropbox.com/s/myc8ltkycuql1mq/fractalforces.zip?dl=0


Verrry interesting. Thanks!


Very cool. Similarly, there’s a gen version (though I don’t know how thorough it is as compared to the external) that a few folks on the c74 built:


So, I was going to say a piano keyboard as one of my choices, too.

So here’s the thing: yeah, it does help an advanced pianist. The problem with an isomorphic keyboard is the idea that you want to be constrained to a single scale.

Now: I love my Push, and I love its iso implementation; it leads to all sorts of things I’d never come to on a chromatic keyboard.

But similarly: all the “wrong” notes are actually where interest can lie: in tonal music, there are many possible ways to harmonize a melody, or to write a tune over chords: lots of things fit. Sometimes, happy accidents turn out not to be accidents at all, just a scale you hadn’t thought of. I can shift from A flat major into B minor in an instant’s thought on a piano - and only with some well timed button presses on many iso keyboards. (That’s before you get to the fact that whilst A minor and C major and D dorian (say) have the same notes in them, they have very different intents, and sometimes you come to a series of notes as you play, rather than deciding them up front: iso keyboards start out by forcing you to choose a starting point, even if you then use them in very different ways).

Similarly: whilst it’s lovely that same-chords have the same-shape… I do find inversions harder to think about on the iso layout, wheras they come instantly to me on a piano keyboard. Maybe that’s just experience, tho.

It’s also worth noting that different keys do feel different - at least, they do to me. I find D flat and A flat majors warm and lush, and G and C a bit dreary; temprament is a thing that has meaning; not all identical intervals are made equal, as it were. Too much jazz too early means my hands tend to gravitate to a B flat minor 9 - D flat major 7 - when I approach a piano keyboard. Lovely and rich. Iso keyboards have the effect of making me use less interesting harmony; usually, that’s a good thing, I stick far too many 7/9/11s in otherwise - but a choice that’s very hard to fight.

The advanced use is much like the advanced use of any tool or machine or instrument: having choice available and not using it. There are more options, but I have the control to limit myself to them - when I look at a keyboard during a solo, I see it isomorphically - obvious notes in the current chord, less obvious ones, transitions to the next one, obvious wrong ones. If you gave me an isomorphic keyboard, I’d also be able to solo on that - but it’d lose a few elements, some of which would be mistakes (knowing me) but others… others I’d perhaps rather have.

But: the choice of tool is a choice, and interesting things come form it. I’ve had a lot of fun wrapping my head around the Push iso layout, and use it a lot out of choice - with somewhat pianistic technique - and yet it’s also nice to force myself to return to a real chromatic keyboard.


(now I’ve read your post, having written my own splurge: yep, that).


Another ideal thought, about DAWs I’ve used: for a long while, I used Reason. It’s one of the few times skeuomorphic design has really worked for me. My first little studio was all hardware, so it mapped to how my head worked: inserts, sends, real mixers, and also the CV/gate model it used for its data passages began to make sense for me. Maybe it’s all a bit obvious, but it helped me consider the relations between the component pieces of software. What it never made me do - as is the case for so many skeuomorphs - is try to subvert those expectations, or break out of some of my common patterns of usage. One piece of magic it made very real for me, though, was the magic of connection: for starters, each time I dragged a cable from “jack” to “jack”, I realised that’d have cost me a fiver in the studio…

I’ve used Logic for a bit, and it’s fine enough, but it always feels like software: there’s varying levels of abstraction in it for me. By contrast, I moved to Live last year, and finally get it.

For years, Live befuddled me: I had no idea how anybody thought like that. But then I remembered my first sequencer - a Yamaha RM1x - and realised that the clip-based model was exactly what I used to use in hardware, with some interesting shifts. And slowly, Live’s UI began to make sense; much more than the Pro Tools/Logic/Cubase school of DAWs, Live really does feel more interested in the Studio As Instrument (to me): it doubles down on embracing its nature as software, not having some weird halfway house of an abstraction model, but being almost entirely abstract - and then using that to its advantage. Its integration with Max also reminds me that it wants me to build it in my own image: turn my ‘generic’ Live into whatever specific studio I need at any point in time.

That’s made me think a lot about musical UI: about the gap between offering ‘features’ (which is what software does) and offering potential, like an instrument: scope for mastery, as opposed to scope for usage.

I’m currently designing a physical, electronic instrument, and I think a lot (even if it doesn’t show in the object) about what Being An Instrument means, and I’m pretty sure that scope for mastery, scope for depth, a surface that affords expression, rather than just usage is part of that. More than any DAW I’ve used, Live feels like it has so much more of that by shrugging off the skeuomorph of Being A Studio In A Computer.


I’m reminded of this:

Long story short: there is no average sized pilot. Having an average measurement in one dimension doesn’t guarantee that you will have average measurements in other dimensions. So, to make an airplane cockpit that actually works you have to make things adjustable with straps, etc.

Adjustability in traditional musical instruments is generally of a different variety: expose all options simultaneously and require the player to ignore what they don’t need in any one moment. This is great because it keeps those skipped over elements available at a moment’s notice. But it can also make learnability more challenging and require a great deal more sophistication, both cognitively, and in terms of muscle memory/hand-eye coordination.

But that level of mind and muscle sophistication is probably precisely the thing that separates traditional musical instrument virtuosos from genre-recycling beginners.

What I’m getting out of this conversation is some confidence to discard some traditional UX-defined usability goals in favor of other goals. UX designers rarely get to design things specifically for advanced users. In a commercial environment where there is immense pressure to appeal to the broadest market possible, there is a tendency to work hard to make things instantly apprehensible, to cater to the novice. Ease of use is primary, and “time on task” (how quickly can the user achieve a specific goal) is a typical metric. These ideas simply don’t work when you’re trying to enable the maximum palette of potential creative expression to the advanced instrumentalist.


I’m also a big fan of Live personally. But it’s interesting in its lack of skeumorphism. It was flat before flat was cool (an aesthetic I feel was unconsciously adopted from its roots in Max/MSP).

Music operates largely in the abstract realm. Much of what makes music work is our mind’s ability to remember form and repetition over time, and to “notice” when patterns are formed and when patterns are broken. All of that temporal and spectral processing is entirely mental (and largely subconscious). The fact that we also relate this pattern processing to an emotional and/or narrative milieu is pretty astounding.

It makes sense to me that a more abstract representation may be more expressive in a musical instrument than more concrete (skeumorphic) representations.


Exactly. I learned a lot about this when I worked as a game designer; that’s another space where ease of use isn’t necessarily a key metric. Ease of understanding and approach is: a game usually (but not always) should be legible to the player - but then to use their learning may take skill, practice, and dexterity. The flipside to this is raising the mastery threshold. That doesn’t necessarily mean making the rules or interactions complex; it means making the possibility space larger.

(If we were in a pub, I would now probably talk about the design of competitive fighting videogames for about half an hour).

The example I sometimes use isn’t the isomorphic keyboard, but the Omnichord: to me, it always feels that by making ‘successful’ usage easier, it limits the expressiveness or level of mastery available to players. It’s sometimes the right tool - or sound - for the job - but it’s an instrument that limits itself to the lower ends of the mastery curve. A hurdy gurdy, by contrast, makes a very different set of choice - but by opening up the depth of interaction, and the possible level of mastery, it actually also allows for greater expression.

And remember: one form of “expression” is “playing it wrong”.


I’m a huge fan of Street Fighter and Soul Caliber. :wink:


I’m also a big fan of Live personally. But it’s interesting in its lack of skeumorphism. It was flat before flat was cool (an aesthetic I feel was unconsciously adopted from its roots in Max/MSP).

Skeuomorphs don’t always have to be aesthetic or visual, though; skeuomorphism runs deeper than that. Logic isn’t very visually skeuomorphic, bar its plugin design, but the whole model it ascribes to definitely is: there’s a linear tape, with fragments of audio on it, and each track of tape corresponds to a track of a mixer… that’s as much a skeuomorph as making the knobs on the mixer look like metal.

(For another interesting take on this, consider the OP-1’s “tape”: yes, it lets you ‘jump’ by defined sections, but it does at least simulate the motion of tape over a playhead - and thus you get the ‘crank’ mode, which is only possible when you bother to go all the way with your skeuomorph).

For that reason, I prefer Reason to the middle-ground DAWs at least because it’s entirely honest about how far it wants to go with the Skeuomorph idea. It pays off in the design of the Reason 6+ Mixer that says, hey, if you’re going to have a mixer channel, it damn well better have a proper EQ and a set of filters, because that’s what makes a studio mixer interesting, not a bare channel that needs EQ inserting as a choice. (You may disagree with this).

Live’s interesting because its aesthetic reflects its design philosophy: it goes to a slightly more abstract place, and comes back with something perhaps more interesting on its own terms, but less immediately approachable if your mental model is: here is tape, here are tracks, here is my virtual sellotape and razorblade.


I tend to think of skeumorphism as a handy metaphorical crutch to offer the novice so they feel more comfortable with software abstractions at the outset.

But you’re right that it can go deeper than that. When skeumorphism goes deep into the abstractions it can potentially serve to make those abstractions richer, more complete.

@Rodrigo, I’m curious to hear your thoughts about the potential for use of real-world metaphors in TPV2. Are concepts like “tape” (or others) applicable?


Interesting discussion! Coming from an background in anthrpology I’m very interested in human - tool/instrument interaction and development. Would be nice to compare the first instruments humans made and how it developed over time with the development of electronic music hardware and software.

What fascinates me about the electronic musician is the multidisciplinarity. Building an instrument, playing it and composing are intertwined. This leads to new kinds of creativity I think.


What’s also interesting about the interdisciplinary nature of this discussion: I’m getting ideas that apply outside the musical realm as well. In my day job I tend to design tools for software developers and sysadmins. Also advanced users that need maximal expressive capability. It’s making me think about potential new avenues for UX design as an industry to pursue.

Hear that lines? You may be influencing the software design industry as a whole! (ha! OK, calm down there little hubris monster.)


Yes. I think there’s been chat around these parts - and if not, I’m mistaken - about the time we spend with things; how long does an electronic musician spend learning their instrument(s), before they buy more, or wire the studio differently… compared to, say, someone spending as long as a concert pianist going deep into a single thing.

It’s the thing I find most interesting about Modular: that people are building their own little specific instruments. Or at least, that’s how I approach it - I’m always surprised by people Pokemonning up modules, when the simple addition of one new function exponentially changes what an instrument can do, and requires us to re-learn it; or, say, the usage of a thing over time eventually makes it clear what you’re really missing, rather than what New Shiny you’d like.

And then, sometimes, it’s the New Shiny that opens the doors for you, rather than the considered practice.

If you’re interested in things on learning interfaces, one of my favourite books is David Sudnow’s Pilgrim in the Microworld, in which Sudnow - known for his piano teaching course - describes getting addicted to Breakout in the 80s. It’s a great book; there’s a long sequence where he describes taking his new Atari VCS home and playing Breakout on it, slowly getting better, at one point putting masking tape on the TV to train himself… and eventually, he beats the game, and pats himself on the back for mastering it in two weeks…

…only he then remembers everything he knows as a professional pianist, and says: you can’t master anything in two weeks. It takes thousands of hours to master a new skill. And he gets disappointed, because he realise Atari have reminded him how to do something he already knows.

Anyhow, that’s chapter 2 or 3; he goes on to try to meet up with the designers, get disappointed at how simple the internal modelling of ball physics is, and more. It’s really worth hunting down a copy; there’s lots about mastery, interfaces, legibility in games, how people perceive systems from the outside, and the relationship to mastery of music in there. It’s fun.


Sounds like a very interesting book, thanks!


Lots of great quotes relevant to this discussion here.



Our discussion about skeumorphic metaphor reminded me of Bret Victor’s (AKA @worrydream) piece “Up and Down the Ladder of Abstraction”. Bret is one of my favorite design thinkers, I recommend reading basically every word he’s ever written.