I poked at this again and with an FTDI cable in hand I quickly found that I had changed the function to copy a buffer to flash (e.g. a whole track’s worth of note values) so that it would copy the whole buffer at once, but I accidentally left the function in the loop that was copying the hex-decoded buffer to flash one byte at a time. After taking this out we are at about 40 seconds to save presets and 1 minute 45 seconds to load, down from 30 minutes!
(much improved version below)
ansible.hex (304.8 KB - d3c3527)
libavr32 @ 54c1713
This could probably be further improved by batching fields together to get a larger flash write. This I think would be achievable with an alternative implementation of
json_read_object that stores a struct for the whole object in its state (would have to be dynamically allocated somehow) or params (would have to be statically allocated and referenced in the
json_docdef_t) and passes a different
json_copy_cb down to its children for them to copy into RAM. Then it would have to call the original copy function to do the flash write right before returning
JSON_READ_OK. Similar struct-level caching might improve
json_write_object by writing a larger block to the USB disk at a time. Hopefully pretty doable, I’ll look into it cough “soon”.
ansible.hex (313.4 KB - 37cbeec)
libavr32 @ 094fb5d
- Implemented preset-level RAM caching during reads, using the “live” app state structs to store the cache. Load and save now each take under 10 seconds.
- Fixed a number of bugs during parsing long buffers, saving decimal values, and generally made things a lot more stable. The tokenizer has been modified so that tokens can span multiple reads. There is still a limitation on maximum buffer size imposed by the size of the working buffer used for hex encoding/decoding. I would like to eliminate this but it works fine for now and there is a possible performance tradeoff.
- The UI described in the original post in this thread has been implemented. “Cancel” is really more like “disarm” - if you have pressed either Key 1 or Key 2 once to arm reading or writing, the Mode key will return you to the disarmed state. Cancelling during disk access would complicate things considerably to ensure that we bail out safely and so the key is ignored. If an operation fails (e.g. malformed JSON, file not found/could not create file) then both the orange and white LEDs will turn on. Diagnostic information about the failure is written to the UART.
You’d be right to be curious about the memory footprint all this extra parsing code, document definition structures, buffers (including a 4k disk buffer), and state. Here’s the size summary from master:
text data bss dec hex filename
0x1579a 0x654 0x39334 323874 4f122 ansible.elf
and from this branch:
text data bss dec hex filename
0x1a54e 0x1844 0x38140 343762 53ed2 ansible.elf
Still room for improvement, but should be reasonably usable in this state.
Edit 2019/05/16: Eliminated requirement for a fixed size hex buffer and reduced the size of the disk buffer without an apparent impact on speed. Made a libavr32 PR.
Edit 2019/05/20: updated with new track direction & TT clock enable state ansible.hex (313.1 KB - 4420970 - PR)