And with that, you’ve reminded me that the keyboard will not be available during USB filesystem operation.
My goodness, how could I have forgotten:
Facet 7: The USB UI will need to be entirely operated with the PARAM knob and the button.
This also helps to clarify questions about script / pattern copies:
Facet 8: Script / pattern copy will need to either be performed from the current bank in edit mode, or from any bank in USB FS mode, or both.
This facet means that the USB FS mode will be menu-driven. Here’s my first crack at a menu:
Save Bank to USB
- Bank Selector
- Save (default selected)
- Cancel
Load Bank from USB
- Bank Selector
- Load (default selected)
- Cancel
Save Scene to USB
- Bank Selector
- Scene Selector
- Save / Cancel
Load Scene from USB
- Bank Selector
- Scene Selector
- Load / Cancel
Copy Script
- Source Bank / Scene / Script Selectors
- Target Scene / Script Selector
- Copy / Cancel
Copy Pattern
- Source Bank / Scene Selectors
- Target Scene Selector
- Copy / Cancel
The menu-driven idiom is unfortunate, but inevitable I think. Its downside can be somewhat ameliorated by good design.
Facet 9: USB FS mode menu options should have sane defaults.
Also, scratch Facet 2:
Yeah. I kept forgetting that too 
Perhaps some sort of hierarchical menu, param to scroll through the entries, the button to select.
(Ah, just seen your post with something similar)
Have you given much thought to the actual code that serialises and deserialises a scene to a text file? Would it be okay for me to share some thoughts on it? I actually think that it’s more important to get that code right, then it is to get the UI right as the UI is more easily changed going forward.
Not anything yet, but I have broken out the read and write operations to operate on a by-scene-and-filename basis.
I was taking the “If it ain’t broke, don’t fix it” approach, but as we can see on the issue tracker, it is broken in at least one way (# in scene description breaks deserialization).
curious to hear your thoughts on serialization! this really ought to be a part of libavr32 so we could add it easily to other modules as well. have you taken a look at orca code for that? pretty simple and forgiving to the syntax (which among other things means a user has more freedom to format it to whatever is more readable for them).
re: keyboard not being available - @sliderule have you thought of taking a stab at writing USB hub support? 
I doubt the serialisation / deserialisation code will ever be generic enough for inclusion in libavr32, but if anyone knows of a parser combinator library for C that doesn’t heap allocate I’m all ears.
Rather we need to separate the serialisation code from the IO code, that way we can move it from module where it’s incredible hard to test and debug to src where we can write unit tests and use gdb.
(Does everyone know what I mean when I say module and src?)
This is particularly important as we will need some sort of forward compatibility1 for presets, and maintaining that will be impossible without some sort of automated testing. And that in turn will give us the freedom and confidence to adjust the preset format for new features going forwards.
My suggestion is that we add 2 functions to src:
void tele_serialise_scene(tele_scene_t *scene, char[MAX_TEXT_SIZE] *output);
bool tele_deserialise_scene(char *input, tele_scene_t *scene, char[MAX_ERROR_SIZE] *error_text); // returns false on error
Neither function is allowed allocate or do any IO, that will be handled by the caller. AFAIK scene text files aren’t that big, we may choose to statically allocate some space for the buffer for the module code rather than deal with malloc.
The error text can either be displayed to the user, or saved to an error.log file on the USB stick (or both).
But most importantly it will allow us to build up a corpus of presets to test against. The easy path would be to just assert that they must parse. The harder route would be to assert that they parse and check the value. It’s possible that we could build some tools to turn a script into a C file with the structs in it so that we may quickly generate the data.
Anyway, that’s what I would do if I had the time. Alas, I don’t. So it isn’t for me to say whether you should go this way or not!
1 I suggest we do place some limitations on forward compatibility, e.g. we only support import presets saved under the previous X releases.
It really wasn’t on my radar, and may be significantly more complex than I’m willing to tackle given the breadth of USB devices that might be simultaneously connected (multiple USB drives, a keyboard, a grid).
Do I ever!
That sounds good, including the corpus of presets. The versioning will require that we change the file structure, so it will be breaking.
i disagree that we can’t have some generic enough functions for serializing / deserializing common types, which is where the bulk of the work is.
heap allocation - could you point where it is?
Maybe, it depends how @sliderule ends up implementing it. I would suggest letting it mature in the Teletype repo for a while before it gets moved though. It’s that much harder making changes to shared code.
I don’t follow this.
In my years, I have found that this sort of effort is not worth the time, at least in C. (Trying to make generic operations instead of specific ones)
i thought you were referring to the existing code, sorry.
i’m not convinced still, could you elaborate on the reasoning?
and take a look here: https://github.com/scanner-darkly/monome-mods/blob/master/orca/main.c#L3203
Excellent. You’ve still got the really messy bit of trying to demangle the existing functions.
Are you going to stick a magic number and a format version in the new presets going forwards?
I kind of agree with this. I’d vote for copying and pasting the code between modules.
Unless we can find a pre-existing library that targets embedded devices.
sorry, but i find this very strange reasoning. why do we have libavr32 at all then?
edit: maybe i didn’t make myself clear. i’m not advocating writing a full blown serializer/deserializer method, i would agree it couldn’t be done efficiently. but i think helper methods could certainly be re-used by module specific implementations of serialization.
Well. For sharing AVR32 specific code…
But it’s so hard to write generic parser code. Your one doesn’t look like it checks for the end of the buffer1, which might be fine for your use case, but not for something generic.
Also, the src bit of the Teletype code shouldn’t be limited to only running on the modules. If someone enterprising wanted to make it run on a Raspberry Pi with MIDI instead of CV and a Curses interface, that should be pretty doable.
1 Or rather it’s based on null terminated strings. Which isn’t necessarily the same as a file could contain a null character. (edit: or maybe I’m talking out of my bottom)
I see that the linked code is bound-constraining, but that will just happily siphon from an invalid file and produce a bad result instead of abandoning, so I might redesign this to include an error return value, which will lead to every output being pointer-driven. At that point, I don’t know why I wouldn’t use sscanf / whatever and do external bounds-constrainment.
Essentially, these type of exercises lead to replacing 4-5 lines of code with 2 using compromises to get there.
I understand the benefits of modular code and the downsides of long function bodies (and sscanf!), but at the end, you haven’t really saved yourself much of anything.
That’s just my opinion. I will take a good hard look at the teletype scene loading, which is much less complex than that orca preset load, and see if there is any hay to be made by modularizing any of the code.
it’s not just teletype code. it would be nice to make USB storage universally available on all, mp/ww/es/ansible. as it happens, teletype serialization is the simplest of them all, it’s mostly text with no actual parsing other than pattern data.
But do they need text based serialisation like Teletype does? Wouldn’t a structured file format like JSON/BSON/ProtoBuf/etc/etc (or even a binary dump with a magic number and version) be better suited?
on the other hand, it will be forgiving when you include extra spaces or edit it on different file systems with different file endings. i see it as a benefit. but yes, as mentioned already this mostly benefits something with a far more complex data structure.
worth mentioning that orca methods are far from being optimized, but they definitely saved me a lot of time when writing USB code.
if we were redesigning it from scratch i would go with JSON, yes
anyway, sorry for getting this discussion off track, we can discuss serialization elsewhere and keep this thread for its original purpose.