Also check out the Euroshield from 1010music, which puts a Teensy 3.2 or better onto the front of a module with TRS MIDI in and out and two CV/Gate/Audio ins and outs. It’s got two pots and a button on it for inputs.
You can use pjrc’s Audio Design Tool with it to patch up your idea and then finish it in Arduino. I’ve been having fun using it for simple things like an extra LFO, and a Turing Machine-like thing.