That’s my understanding as well, and my situation is the opposite, for my personnal stuff I work at 44.1khz out of habit mostly, for work at 48khz because I work with video and there 48khz is a standard. Overall, for what I’m doing I don’t think anyone could care less whatever it is I use and yeah, no audible difference really.
At 96khz, you can have some practical use case if you mix tracks with lots of well recorded audio source and signal to noise ratio is a problem once they’re all treated / compressed, on top of eachother and all the background noise add up. Otherwise if it’s the occasional acoustic element in the instrumentation here and there, really, it’s no big deal.
Oh and also, background noise is sooo nice.
Edit : also, we live in a world where the realities of today aren’t those of yesterday, a lot of standard software now do 44.1khz to 48khz conversion seamlessly and it really works out ok in most cases so we’re really far from the days were it could ruin a whole project, play the tracks at the wrong speed and have a big impact. Funily enough as a counter argument I had the issue recently with my Norns on MLR if I remember correctly ^^