The UX of grid-based applications

Picking up the discussion from the norns topic, I think this is somehow a valuable thing to talk about in general, not just related to the grid or other monome instruments.

To sum up what has been said so far, the discussion started with this post:

which was followed by Zebra stating this.

I think this is the central aspect of the whole discussion… affordance-based design as a poor fit? Is “affordance” really the right term here? But more about this later.

Never used any of the above mentioned “greatest hits” but I can only second this.
To those who are familiar with the wide world of monome, would you say that it has developed itself in an “organic growth” kind of way? From the outside that is how it looks like, and I think there’s great beauty in it.
There is this big divide, or I actually I’d call it conflict, that one often faces between designing something and just letting the thing design itself. Design usually is about control, about planning, conforming, standardizing and ultimately about making things fit into the boxes we have created for it to fit in. Design tends, by nature, towards being very “top down”.
On the other hand systems involving a community, where the design effort is not centralized in one or few people, often have the ability to grow and shape themselves in an organic manner.
This can of course produce anomalies, incongruence and idiosyncrasies, but it’s kind of part of the process and the value in it is that as things evolve further, some practices/solutions can be abandoned in favour of new, more efficient, more inspiring or more natural ones (depending on their nature). This means that we indeed need to keep observing and analysing. As far as I can tell nothing in the monome world is static, and actually a lot of it is already beyond the control of the original makers.

With the arrival of norns standards (if we want to call them like this) can indeed be redefined.

There’s a lot of discussion right now about how boring, bland, annoyingly edge-less everything has become due to Apple/Google dominating everything with their design guidelines.
Design has seen a new “brutalist” wave happen. Partially it’s just a fashion, but it’s also a reaction, and arguably it’s a needed one. A world where everything is perfectly designed would be a total horror to me.
The other aspect is that once we all start to just blindly follow design guidelines produced by big corporate companies, we stop to question the processes and principles of what these guidelines define as being good design. And lots of questioning is needed for design to stay healthy.

But let’s get back for a moment to the grid and to affordance-based design.
I mentioned earlier that I am not sure affordances is really what we should be talking about, can you maybe elaborate a bit more about what you mean here @zebra?
To me it looks like affordance-based design is the perfect fit for a minimal hackable interface canvas, since what Brian and Kelli have done with the Grid is almost to reduce the interface to the bare affordance one could say.

6 Likes

If you look at the monome grid in the context of its derivatives (apc, launchpad, push), I would agree that the monome grid is not one that obviously expresses its affordances.

This can be both difficult as well as liberating. Think of a cello vs a guitar. Frets announce affordances, but inhibit working outside of them. Meanwhile, a cello offers more pitch flexibility, but requires more practice to play in tune.

I’d suggest that it’s also worth looking at failed experiments too… like this abomination that would be nearly impossible to use without gaff tape

https://github.com/monome-community/collected-ms/blob/master/sepra/sepra_layout.jpg

5 Likes

But I’d argue that the affordance of the monome grid is its buttons and that these clearly express that they can be pushed.
The illumination patterns are signifiers of the grid and the application that is being used with the grid (which work as a two-part unity)

edit: I should add that I totally agree with Don Norman’s clarification on the whole affordance/signifier confusion in one of the last updates to Design of Everyday Things

1 Like

sure, of course. i completely agree with that

also agree.

the point i was trying to make is more about context, though… adding tape to the grid changes context. adding a notch to the center changes context. adding an extra set of round-row / round-col buttons with abletonic intentions changes context. having sliders under each column changes context. using variable brightness definitely also changes context (which i think was part of @andrew’s original point?)

maybe affordance isn’t the right word… maybe it’s an onion of affordances

1 Like

Oh man… so much to say.

Norman got the idea of “affordances” so wrong and it sent our whole field down this rabbit hole for many years. I haven’t read the update to his book, but it sounds like he realized where he got it wrong and tried to correct it…

If we want to talk about affordance we should go back to the source, Gibson. Gibson defined “affordance” as what the environment offers to the “animal”, including humans. What he describes is a combination of the physicality of the environment and objects combined with the animal’s abilities to interact with those things.

For example, a stick to a dog affords biting and carrying. A stick to a human affords holding and throwing, or hammering, or hunting.

With that in mind, the Grid is interesting. The buttons afford pressing, but the grid itself affords carrying, holding, and a lot more that we don’t really use as interface anymore.

When we talk about what sort of interface cues exist on the Grid, there are two layers worth teasing apart. One is the physical Grid itself, with the buttons and lights. The second is the software layer that uses those elements to create and communicate an interface and functionality.

In that light I totally agree with @zebra. The Grid is a vehicle for software, much light a touch screen. It has abilities and constraints for the design of that software, but isn’t the software itself. The design of the software interface is what determines use and visualization, how transparent or opaque the interactions are, etc.

Should there be interface standards for the Grid? I’m not sure it’s really necessary given the types of things we use it for and the variety of uses. There are definitely some design paradigms that you get used to if you use one application a lot, but that doesn’t mean others are invalid.

Interaction design is all about feedback loops.

imagehttps://walkertalky.files.wordpress.com/2014/05/bill-verplank.jpg

Bill Verplank talks about these stages as “What do we DO?”, “What do we FEEL?, and “What do we KNOW?” - meaning how do we act on the environment, what is the feedback that tells us the result of our action, and how does that feedback change our understanding of our agency and ability in the environment.

I also think that’s a fundamental difference between designing a software interface for playing music like a Grid app, and designing a traditional instrument. The Grid provides access to a complex system with states and modes, multiple interaction types, and much more. It’s more akin to designing an iOS app than a guitar. But a good software interface for musical expression needs to embody some of the ideals of an instrument, using “instrument” more as a metaphore than a literal state. Instruments can be opaque, but they are open to exploration and discovery. They provide immediate feedback, and they create musical systems that encourage certain ways of working. That’s what we want out of a great Grid or Arc interface married with software…

——

In the wider world of Interaction Design this is less and less true… There is a huge community practicing co-design and collaborative methods where designers work to facilitate ideas from a community and help them express their concepts using the best tools and deep knowledge of design history and practice. The work that comes out of these practices is amazing and can really help a community get their own ideas into a good place.

11 Likes

Yes totally. Also it changes how much of the initial blank canvas is left.
If I have understood andrew’s post correctly his conceren was mostly about interaction patterns being consistent across applications. Which in general is understandable and desirable, but not always easy to enforce in an open-ended system (the very word “enforce” sounds weird when in the same sentence with monome, doesn’t it?) and might sometimes not even be desirable… but I’ve said that already.

Did he get it wrong or did people get him wrong. No idea, just posing the question.
One thin is sure, the word has been used, misused and abused a lot.

Hehe I was about to say that the grid basically is a touch screen with very few pixels :smiley:

I don’t want to sound pessimistic, because I’m really not about this, but the interaction design world is still very much dominated by what big companies like Apple and Google dictate, which do have good interest in keeping things under control and very much top down.
Though I do agree that if I compare the design community now to how it was 15 years ago it’s changed quite a lot, and even more probably if you go further back in time.
I do agree with you that there’s a lot of interesting movement going on regarding this!

2 Likes

Not to get too far off topic… Apple and Google certainly dominate and have the most visibility for consumer electronics/software right now, but they don’t do the vast majority of design that’s happening in the world. I teach Interaction Design at two schools, one post-grad, and none of my students from the last 5 years have gone to work at either of those companies. There’s a huge world of interaction design happening, and Apple and Google only do a small portion of it, although a very visible portion.

8 Likes

You totally have a point there indeed.
Oh and thanks for reminding me about Gibson!

I think I’ve said this earlier, but I’m very interested in this discussion even if I never had a Grid. Part of my interest is also due to the fact that I would love to explore the Grid more (and maybe if I get a norns, a Grid might follow?).
So to get the discussion back to the topic, when talking about the Grid we should probably rather talk about the applications that run on/through it. Which of course opens up the big can of worms which is the question: what is the affordance in software? Probably, if we stick to the original definition of it, we shouldn’t use the term, since software by itself is not physical.
But I guess we can still somehow talk about affordances, can’t we? At least there is usually two layers in every software interaction, one which is the physical one (with its set of affordances, feedbacks etc.) and one which is the software itself. So the button affords pressing, but controls a sample player which affords playing a sample.
This distinction would be superfluous if the mapping between controls, the software and the feedbacks was totally 1:1 without modality, but in cases where a non-changing physical interface will work with a very wide range of software it’s likely easier to look at things separately.

My guess would be that the act of identifying the affordances and interpreting the feedbacks in grid-based software is partially something that happens thanks to previously acquired knowledge (reading the manual, asking people and practising so this knowledge sticks) and that each software has its own conceptual models, which can sometimes be different. Or how is that?

1 Like

Apologies for going lateral in the middle of a great discussion, please don’t allow me to derail you, and feel free to ignore my little interruption, but I thought I’d impart a a bit of my technique.

When I write interface guidelines I typically start with the controls. What are the basic elements of interaction? What do they look like and what do they allow you to do? (And how does the appearance suggest the function, ie: what affordances does it provide?) When is this interaction element called for? When is it inadvisable?

@scanner_darkly and @ithkaa have given us a head start. In @ithkaa’s Control we have a fader, a keyboard, a momentary button, a toggle button, and two variants of x/y control. @scanner_darkly’s Grid OPs bring us groups, LEDs, rectangles, buttons, faders, and x/y controls.

If we now look at kria, earthsea, meadowphysics, and mlr, do we see how these control patterns are being used? Are there additional interactions these apps contain that aren’t covered by these patterns? (Hint: yes, but they are probably arrived at by composing more atomic controls into more complex interaction patterns)

Try to name all the patterns you see in use. Next, try describing how they are being used. Can you identify any usage do’s and dont’s? Write those down too. Do you notice any differences in how the controls are implemented? Which of these differences should be a permanent part of the pattern? Which are optional variations? Which should be avoided?

Just a little window into my day job…

By the way, I highly recommend Brad Frost’s Atomic Design for thinking about composition of control

6 Likes

So this bit came out of some frustration with using other people’s grid applications from the max days, and not knowing how they work. I’ve always felt, and still do feel, that a manual or instructions are an unintuitive way of learning something. As sort of a natural design instinct I thought, “hey, there should be standards here,” that way I wouldn’t have to re-learn every application (sorry to bring in affordances there, forget how much of a loaded term that is). The natural way to use something changes from person to person, so we set up standards to try and normalize those differences so everyone can be happy. It’s good design. I still think it’s the right answer in the majority of cases, and most of the time when designing for screens.

But the grid is not most cases. It’s a tool for making art and that gives it less constraints. Thinking about it more I’m definitely seeing @zebra and others’ points how standards would be deviating from the grid’s intent. I feel like it’s an interface that’s supposed to be modified, so standards don’t need to exist. If a use of the grid doesn’t make sense to someone I think it’s simply meant to be changed. I think that’s part of why we see a separate language used for interaction on norns, it encourages customization.

Again the proper comparison point is an instrument. Part of playing an instrument is setting up a system of how to interact with it. This can be someone else’s system (like bach’s), or better, one’s own. I have my own system for playing instruments. I thought about publicizing it once, but then I realized that was a horrible idea because I honestly don’t think it would make sense to anyone but me.

I think in the context of a grid, seeing up the system is the actual act of programming it, and using the application is just playing based on that system. The grid is blank like an instrument so multiple systems can be set up. Because of this blankness another person’s system can be hard to learn, yea, but other people’s system’s aren’t really the point. The grid, like a guitar fretboard, is best approached with patience and self-experimentation.

5 Likes

Heh, but now you got my motor running and I want this grid pattern documentation we’ve been talking about!

But maybe I’ll do it myself (in my ample free time, ha!)

3 Likes

I’m a total bystander in this conversation—don’t have a grid, probably won’t get one for years although they’re beautiful and inspiring from this remove. But, I did also download a bunch of Max patches for grids—with the intention of figuring out how I could replace the grid interaction with hacky BeatStep Pro control—and have been frustrated by how I can see some aspects of what is going on, but in general would really have to commune with the source to find out what is going on, especially without being directly able to just press buttons.

As I sit here, I don’t really think that this is a complaint about the user interface, but rather how much of the knowledge of how these Old Things tick has been lost as the community and creators move on to New Things. And like, on some level, it’s fine, just a shame.

1 Like

I sincerely hope that the New Thing, due to use of many mature, text-based, open-source technologies, will be much less prone to backwards compatibility issues than the Old Things were. This should mean that the wisdom we start accumulating now stands a chance of surviving for decades in a completely usable form.

Hopefully that also means that some of the people responsible for that wisdom will also stick around!

2 Likes

From this perspective moving to Supercollider/linux is certainly a great move!
Still, it rests on the shoulders of everybody who will contribute to document – at least to some extend – their creations, and maintain that documentation.

2 Likes

that would be brilliant, yes!

off topic, but it’s funny how often I feel “green” lately. finally able to participate meaningfully in Big Projects and Conversations… only to realize how much there really is to catch up to in terms of skills and experience to really start to move things. I guess that’s probably just endemic to being in one’s 20s :sweat_smile:

1 Like

You’ll be gray and brown like the rest of us soon enough. Enjoy being green while you can!

1 Like

this is a side note, but it would be great if we had some shared repository of assets to help make visual interaction documentation easier for people. I know that @ether basically reverse engineered existing monome documentation to create a guide to @scanner_darkly’s ansible earthsea port, but it seems like we could make that easier. now with norns on the way, there’ll be another surface to document as well if we want to make interactions clear.

I guess a bunch of stuff could also live in the comment section for lua, but I like the idea of people being able to relatively quickly and easily put together a page or two on how to interact with their norns creations.

As I was describing interaction patterns above, I was imagining how I’d set up a Sketch library to make documenting those patterns easier.

This is sounding more and more like an actual project I’m actually going to pursue…

6 Likes

Yes, now you have to :smiley:

This said, the way things have developed on the Teletype are certainly a good “best practice” example in my opinion.

1 Like

Please do. I spent some time a few months ago trying to think through some arc functionality and wound up recreating the arc documentation visuals in keynote so I could play around with them, but it would be great to have a set of images anyone could just start with.

1 Like