@lijnenspel This is awesome, I'll definitely be interested in a pcb (and panel if you are making one)!
I am myself designing an OSC to CV module, I was planning to use the new teensy MCU and LTC2668, which is 16-Channel, 16-/12-Bit DAC. It also has switchable voltage ranges and buffered outs so it doesn't need much in terms of external circuit. The advantage of your solution is the ability to reconfigure an output to be an input, which I haven't thought about.
My plan is to use it mainly with PureData and also for live performance using Lemur App (on a 23" flat screen with one of those cheap android TV sticks).
EDIT: Also though it would be good to include a PWM controlled LED with each jack socket, to show input/output voltage. The LED could be two colour for negative/positive voltages. All LEDs could be driven using one of the I2C chips with multiple PWM outputs.
EDIT2: Haha I just noticed on your buildblog you decided to add LEDs too, though you're going for an analog approach. Personally I would avoid that approach altogether (to cut down on external components) and just use I2C chip like Tlc5940 (16 PWM outputs - I haven't found one with more outputs but 2 can be chained together).