Teletype 3.0

csboling · 2018-12-02T21:09:39.416Z

scanner_darkly:

2nd possibility: f.fresh somehow gets overwritten when something else gets updated

Could this corruption happen due to flash_write getting called somehow with a preset_no arg equal to SCENE_SLOTS, causing flashc_memcpy((void *)&f.scenes[preset_no].scripts, ... to overwrite the next few members of nvram_data_t? The calls to flash_write from flash.c and usb_disk_mode.c are in for loops bounded by SCENE_SLOTS, but grid.c and preset_w_mode.c call it with the global variable preset_select, which gets set a number of places. Those include the SCENE op (via tele_scene()) and the function process_preset_r_preset, which in turn is called with calculated values based on grid coordinates and an ADC read. Seems like a preset save would have to be triggered from grid mode or preset write mode for this to explain it though.

scanner_darkly · 2018-12-02T22:12:07.171Z

this is a good guess! the grid calculations should be okay as it applies constrains before the calculation, and from what i can tell the ADC value shouldn’t cause it either but still possible, of course. in any case we should add a boundary check in flash_write!

alphacactus · 2018-12-03T01:50:55.704Z

not sure if I can contribute much, but I took a look at the teletype.map file to see if there was anything weird. my best guess is the problem as something to do with grid_data_t as that’s the only thing that changed with nvram_data_t in 3.0.

first thing I noticed is that grid.o is included under the flash_update_last_saved_scene and flash_write symbols.

Screen Shot 2018-12-02 at 2.15.08 PM.png1306×328 43.3 KB

additionally found grid.o as under the scene_text symbol.

Screen Shot 2018-12-02 at 8.46.59 PM.png1302×122 15.8 KB

not really sure what I’m looking for, or if this helps. figured I’d mention it incase something clicks @scanner_darkly since you know the code better. I’ll keep looking around. let me know if there’s something specific you’d like to be investigated.

csboling · 2018-12-03T07:55:18.543Z

It occurred to me that if anybody’s had this happen and did not re-flash their firmware since then, there could possibly be evidence of flash corruption left in the firmware image, since an overrun could have written to a part of flash that the reset wouldn’t blank out. If someone affected gets a chance to upload their .hex file (obtained from the procedure under firmware backups - teletype) I can try and do some forensics.

desolationjones · 2018-12-03T14:05:40.700Z

I was affected last week and I can perform this backup procedure tonight.

I am running 3.0 Keyboard fix version and have not reflashed since the event. The TT blanked itself after power cycling; had been left in pattern mode with no keyboard attached at the time (or recently). Since the event I have powered up/down several times and recreated a single script.

The TT is hooked to an I2C backpack and 2 x TXi, JF, ER-301, and Ansible. My script involved using one TXi and remapping its inputs at INIT. I have never performed a calibration OP.

desolationjones · 2018-12-02T15:50:28.843Z

My Teletype blanked itself Upon boot-up last night all my scripts were gone. I had backed up some of my WIP on paper but this still stings. Anything I can do to prevent this in the future? Paging @tehn…

EDIT: I should mention I am running 3.0 Keyboard fix version.

alanza · 2018-12-02T15:55:45.614Z

I think @scanner_darkly was investigating this happening to someone else a few weeks ago

scanner_darkly · 2018-12-02T18:08:56.969Z

some discussion here: Teletype 3.0

unfortunately at this point i don’t know why it happens. i’m going to post a new version soon that will have some additional protection around the part that wipes out all scenes (which should normally happen only when you flash new firmware). not a real fix but hopefully will prevent it from happening. i’m also going to remove saving the last mode completely, it’s the only thing that saves to flash other than saving scenes explicitly, so it’s the most likely candidate for somehow causing it (or to be more precise it’s the best guess i have in the absence of solid details or steps to reproduce).

i’m also going to repeat a call to other devs - if you have free cycles more eyes on it would be great…

desolationjones · 2018-12-02T19:16:15.022Z

I was using the teletype without a keyboard. Focus was on the pattern screen.

Edit: perhaps a mod could move these posts over to the 3.0 thread?

scanner_darkly · 2018-12-02T19:17:05.440Z

thanks for the additional details! let’s continue it on the teletype 3.0 thread as the main discussion is there.

edit: looks like the posts were moved to this thread. it’s a bit confusing having them out of chronological order after they’re moved though…

scanner_darkly · 2018-12-03T16:34:52.808Z

alphacactus:

first thing I noticed is that grid.o is included under the flash_update_last_saved_scene and flash_write symbols

tbh not sure why - flash files don’t reference grid in any way…

csboling:

If someone affected gets a chance to upload their .hex file (obtained from the procedure under firmware backups - teletype ) I can try and do some forensics

that would be great!

@tehn and i added some protection to the code yesterday. it will now display a warning before overwriting scenes, and you’ll need to explicitly confirm the action by pressing the front panel button. normally you will only ever have to do this the first time you run a beta version, official releases won’t be affected. and you should never see the warning again but in case there is a bug that somehow triggers this at least we’ll know about it, and this way you can opt out, which would hopefully leave your scenes intact. i’ve also added a boundary check to the flash write functions (@csboling - thanks for the idea!).

hopefully the above measures should prevent this from happening again, but in any case it would be good to investigate what might’ve caused it!

scanner_darkly · 2018-12-03T16:51:13.089Z

do you have any scripts that might also be setting it?

theelectricyouth · 2018-12-03T16:53:40.223Z

Oh. You right.

desolationjones · 2018-12-03T23:58:18.965Z

blanked_tt_120318.zip (216.9 KB)
Hex file, as promised.

scanner_darkly · 2018-12-04T02:13:00.829Z

a new beta with additional protection:

teletype.hex (568.5 KB)
teletype.zip (167.9 KB)

flashing this will overwrite your existing presets, make sure to do a backup first!
includes everything that was added since the official 3.0 release.

the 1st time you run it it will show you a warning that scenes are about to be overwritten - this is expected - this is the additional protection we added. since it’s a beta version you do want to overwrite, to confirm press the front panel button while the message is present. you should not see this message at any other point - if you do please pm me or post here. if you don’t press the button it will shut down, simply power cycle it to get the prompt again.

unity2k · 2018-12-04T04:48:57.508Z

Be aware that the prompt disappears fairly quickly.

csboling · 2018-12-04T06:02:07.676Z

Thanks for getting this and for the info about the last thing TT was doing before it happened, and sorry about your scripts! My thought was that the scene data structure is large enough that, if the out-of-bounds hypothesis above were correct, we would still have most of a scene written to an area of flash that should never be written to from the same write that trashed the “fresh” field. If this were the case, I thought it might even be possible to recover the scripts saved to that scene, since they’re in the last part of the scene data.

…But it doesn’t seem like this happened. After the calibration defaults, everything else in flash is 0xFF - flash bits are typically set to 1 when erased. So it doesn’t seem like a bulk write did it, at least not off the end like that. If there was some other off-by-one condition I thought maybe the padding bytes used to align struct members could also wind up getting set, but those are 0xFF in this firmware image too.

So, sad to say after reading flash.c a half dozen more times, I got nothing. If it’s useful to anybody the ‘repl’ tool here is the program I used for sifting through the firmware image.

scanner_darkly · 2018-12-07T00:16:15.234Z

thanks for checking, and thanks for the idea of doing boundary check on preset_no - i’ve added it in the latest beta!

i’ve also posted the latest beta in Teletype 3.+ feature requests and discussion thread to avoid confusion (as that’s currently the main thread for latest teletype betas).

Starthief · 2019-01-02T23:15:08.122Z

I found a bug with… whichever flavor of 3.0 I’m running here (says 25E14EE on startup).

INIT.SCENE (and probably other INIT commands) resets the value of M but not the actual metronome tick rate.

E.g.:

M 150
INIT.SCENE
M

(result: 1000)

Set the M script to TR.P 1 and you’ll see it fire every 150ms.

Jonny · 2019-01-02T23:54:58.357Z

Does INIT.DATA work?