Gagaku is essentially descended from, or considered a part of the more ancient form of music & dance “kagura”. Kagura was (and still is) performed on stages within shrine complexes and was initially considered a method of summoning gods & spirits, so I’d say gagaku/kagura is a product of Shintoism (and thus probably can be considered “sacred” music, although gagaku was essentially performed for the Emperor and so is likely less sacred than kagura), and only vaguely related to Buddhism (if at all) as a result of the ‘fusion’ of Shinto and Buddhist practices that has happened in Japan over the centuries.
I’m also really interested in this idea of stillness and silence in gagaku though - I spent some time studying it here in Tokyo and find it really interesting to talk about with gagaku musicians I’ve had the opportunity to become friends with. At a very basic level Shintoism is based around the idea that gods & spirits live in trees, rocks, mountains and other natural things, and shrines were constructed initially as a way for villagers to acknowledge that certain spaces are “sacred” and are occupied by a multitude of spirits. They are incredibly dignified spaces that seem removed from the human world, and as things like clapping wood sticks together, ringing large bells and clapping while praying in front of deified shrines are thought to be ways of “calling” gods down from above, I’d say the opposite of sound making might also pretty important within shrine complexes so as not to disturb the congregation of spirits.
With regard to traditional Japanese instruments used in gagaku like the sho & biwa and their inherent philosophical characteristics, the composer Toru Takemitsu’s writings are really illuminating - there is a great English translation of some of his essays called Confronting Silence.
The closest thing maybe to Zen-related music would be something like “shomyo”, Buddhist chanting, I think. When I think about “zen” in music what instantly comes to mind is silence, but what is silence anyway? Doing menial repetitive tasks like raking pebble gardens is also considered zen practice, as I understand it to largely be about emptying the mind, so I often think that long, drone-like music, such as ambient music, might get us pretty close to what could be thought of as Zen.
I find it all great food for thought!