I've used the same software for many years now (Logic and Max, and to a much lesser extent Sibelius), so it's hard to really be objective about UI/UX with these things, but here are some thoughts/experiences.
Starting with an old one, there was a massive shift in Sibelius around Sibelius6 I think where they moved to the 'ribbon' paradigm of control, which is one of the worst pieces-of-shit things I've encountered. It's like the worst of both worlds in terms of context and menu-driven UX (like you have to know which ribbon bit to enable, to THEN hunt around contextually for what you want to do...). Awful.
I think skeuomorph-y things tend to work well for audio UX stuff. Although I don't have/use any of them, the Madronalabs instruments look wonderful, and seem to invite 'play'. Similarly, Max is built around the paradigm of 'objects' and 'cables' (though in a much less playful manner!). Though this (play) is probably less important once you get your head around something. I guess something that you can get a clear overview of what's what, and then what to play with within that.
In working on TPV2 for the last month or so, I've spent a lot of time playing with the MuBu package for Max (for rebuilding some of the Combine-type stuff) and holy-moly has that been an exercise in banging my head against a wall. The combination of similar-but-different naming/structure (a
buffers, which can refer to Max
buffers~ (or not!)), a dense/informative (though unclear) UI, and general lack of structure has made it a real uphill battle.
I had the benefit of seeing one of the designers demo some of the stuff, so I knew it could (pretty much) do what I wanted to do, but even with that, I almost gave up several times.
I think this one is a case of smart people working on something that works well, but only if you have some kind of inside-track, because it's near impenetrable for outsiders.
I find the Waves plugins to be pretty shitty, as they are a grab-bag of unrelated UI/UX, with seemingly arbitrary names too. Again, powerful, but an impenetrable mess.
To this day, I still find Live very confusing. I know I should learn it, especially with M4L and all, but I open it up and it just looks like a mess to me, especially since it's a pretty mature (entrenched) piece of software now. (I should really bite the bullet on this one though)
Oh, a non-music, but related experience. The move from Final Cut Pro 7 to Final Cut Pro X. FCP7 felt like Live to me, so many things all over the place, with such a weird (coming from music) workflow, that made it difficult to do anything. I'd have to google/youtube just about everything I wanted to do, before I could do it. FCPX, I feel, is worlds better in terms of interface/usability (though I'm not a "pro"), but structurally it is really weird (Events/Projects hierarchy).
Ack, that got quite ramble-y, but I wanted to chime in with some thoughts.