Here is my wishlist of books to get at some point (and some that I already have, but haven’t seen on this topic):
Alex Ross - The rest is noise
Andy Farnell - Desining sound
Brian Kane - Sound Unseen - Acousmatic Sound in Theory and Practice
Christoph Cox - all/any of his books
Curtis Roads - all/any of his books
Edward Pearsall - Twentieth century music - Theory and practice
Ernst Krenek - Modern music is unpopular
Brian Ferneyhoug - Collected writings
Ferrucio Busoni - Sketch of a new esthetic of music
Harry Partch - Genesis of a music
John Chalmers - Divisions of a tetrachord
Larry Austin - Source music of the avantgarde
Miller Puckette - The Theory and Technique of Electronic Music
Nicolas Slonimsky - Thesaurus of musical scales an pattens
Pascal Quignard - The hatred of music
Pierre Schaeffer - all/any of his books
Some of these are more contemporary classical music oriented, also theory books, but I still think they apply to the topic and might be interesting to folks.