Okay, I guess that might just be the root of the problem. After running for about two hours, the frequency got relatively stable. There are still some deviations in both directions, but no longer that continuous drifting upwards. Actually the frequency started drifting back downwards after leaving my windows open for a while, causing the room temperature sink a couple degrees.
So yes, seems as if the module is very sensitive to changes in temperature.
My Euroroack case still has a lot of free space, which I guess leads to even more unwanted air circulation. So I guesss the only solution is buying more modules until both rows are completely closed 