Yeah this is interesting. I think what’s going on here is probably that the same tuning table is being used, but the midi mode pitch calculation is not quite the same. In particular there is an additional pitch shift applied when assigning midi pitches, this spot in the code notes that there is a -2 octave shift. With the way the CV output code is structured this is effectively being applied as a pitch bend, ignoring the tuning table for those 2 octaves.
The note on this says this value (-3277) is chosen because this is N -24 on Teletype. This is true, but it does not match up with the pitch table being used – it turns out the Teletype N op uses another slightly different note -> DAC ticks lookup table.
Of the three tuning tables Teletype’s seems the most correct to me.
Comparison using a Python shell
Python 3.7.4 (tags/v3.7.4:e09359112e, Jul 8 2019, 19:29:22) [MSC v.1916 32 bit
(Intel)] on win32
# ansible ET method
# different slope from the others and discards 2 bits of precision
>>> [round(i * 4092 / 120) << 2 for i in range(121)]
[0, 136, 272, 408, 544, 680, 820, 956, 1092, 1228, 1364, 1500, 1636,
...
14732, 14868, 15004, 15140, 15276, 15412, 15548, 15688, 15824, 15960, 16096, 16232, 16368]
# ansible midi SEMI14 method
# uses integer truncation rather than rounding
>>> [int(i * 16384 / 120) for i in range(121)]
[0, 136, 273, 409, 546, 682, 819, 955, 1092, 1228, 1365, 1501, 1638,
...
14745, 14882, 15018, 15155, 15291, 15428, 15564, 15701, 15837, 15974, 16110, 16247, 16384]
# teletype N op method
# rounds and keeps full precision
>>> [round(i * 16384 / 120) for i in range(121)]
[0, 137, 273, 410, 546, 683, 819, 956, 1092, 1229, 1365, 1502, 1638,
...
14746, 14882, 15019, 15155, 15292, 15428, 15565, 15701, 15838, 15974, 16111, 16247, 16384]
The DAC hardware and control code should all be the same, so I think we should just move the table Teletype defines into libavr32 and use it for all the pitch calculations on Ansible as well as the Teletype note table. Looking at the DAC’s datasheet I think 16384 / 120 = 136.5333… ticks per semitone is the correct value rather than 4092 * 4 / 120 = 136.4 ticks per semitone, but there could be a consideration I’m overlooking, like something that’s different about Ansible’s design.
Do you have a Teletype or Crow connected to Ansible? As a short-term way to address the tuning issues while this is getting sorted out on the code side I believe you can use MIDI.SHIFT or a crow equivalent (does not look like this is wired up in the crow firmware yet but this is easy to add) to reprogram this -2 octave shift, then save it to flash on Ansible. You may have to experiment with this value, 15+ cents seems like a lot for this discrepancy to completely account for it but it’s at least part of the problem.