Curtis Roads has written or contributed to many really great books on granular synthesis and beyond. I return often to his 2001 book Microsound which is really deep dive into granular synthesis and related techniques. It takes a recipe-like approach for lots of the material covered and is very practical to pick up and find some interesting directions to play with.
He also edited The Music Machine which is an amazing collection of articles from out of print issues of Computer Music Journal. Section V of the book is focused on synthesis and covers some of the first writings and original research on physical modeling, an overview of most standard forms of synthesis, and esoteric stuff like warped linear prediction.
He’s made a number of individual articles available on his website too.
Computer Music Journal is an endless resource for interesting research in synthesis. If you are lucky enough to have a JSTOR account every back issue is available there digitally, otherwise you can buy access to just CMJ back issues via JSTOR for something like ~$30 for a few months. (Your library might have access too.)
Also, if you find something in CMJ that looks interesting, google it. It’s not uncommon for the original author to post the article for free on their personal site.
I’ve found F. Richard Moore’s Elements of Computer Music and Gareth Loy’s Musimathics books incredibly useful as well, but they have a broader scope including stuff like acoustics and DSP fundamentals.
Last but not least I was about to write how difficult it is to track down but it looks like Trevor Wishart has made his amazing Audible Design book available as a free PDF download on his website!! This book is an incredible resource and the introduction of the microsound technique of waveset synthesis among other really interesting and creative approaches to sound synthesis you aren’t likely to find anywhere else.