#32

Hi @tehn do you think it’s likely the TIMELINE feature will be implemented in a V2? The length of the scripts and number of scripts is something I constantly find myself hitting a wall with. I imagine that if I were to be using just friends with the new commands or introduced the expander or ansible then I don’t think there will be enough space for scripting ideas that take full advantage of these new possibilities.

Cheers, Alex

#33

Thank you @zebra and @sam for your input and all that information. It clears up a few misconceptions I had and explains the limits based on the actual code (I only have experience with Python, C needs a lot of effort and feels obscure to me).I think just minor modifications to the font is probably the way to go. Will start a new thread and gauge interest, get some feedback.

Thanks again!

2 Likes
#34

Calculations time…

A teletype contains a AT32UC3B0512 MCU, it has 96kb of RAM and 512kb of flash. Pointers are 32bit. The flash can be split between ROM and storage, we are currently using approx. 80kb of ROM.

I’m going to ignore RAM usage, as it stands we have approx. 70kb free. Flash usage is what will constrain us.

Currently… a teletype contains:

  • 32 scenes, containing:
    • 10 scripts (1-8, I, M), containing:
      • 6 commands, containing:
        • 12 ‘datas’ (a.k.a. ops)

So a single scene contains 720 ops, and the entire teletype contains 23040 ops.

Each op is a defacto tagged union:

typedef enum { NUMBER, MOD, SEP, OP } tele_word_t;

typedef struct {
    tele_word_t t;
    int16_t v;
} tele_data_t;

sizeof(tele_word_t) == 1
sizeof(tele_data_t) == 4 // is 4 due to struct packing

So, 23040 ops * 4 bytes == 92160 bytes.

All the way at the back of the top of this thread we discussed how to store a reference to the op (or mod) in tele_data_t, currently it is an index of a table. If we were to switch to using a pointer to the op struct then sizeof(tele_data_t) == 8 (unless we can come up with some clever way of hiding the tag inside the pointer).

So I think I will keep the op and mod tables, and the new parser code will carry on returning an index for them rather than a pointer.

1 Like
> teletype : code exchange
#35

A big update on my parser branch…

https://github.com/samdoshi/teletype/tree/parser

The entire parser is now running via Ragel, both scanning for tokens and matching them.

Code sizes, master branch first, parser branch second:

section               size     size
.reset                8204     8204
.rela.got                0        0
.init                   26       26
.text                55728    66080
.exception             512      512
.fini                   24       24
.rodata              14732    15140
.lalign                  4        4
.dalign                  4        4
.ctors                   8        8
.dtors                   8        8
.jcr                     4        4
.got                     0        0
.data                 7492     7492
.balign                  0        0
.bss                  8136     8136
.heap                74456    74456
.comment                47       47
.stack                8192     8192
.flash_nvram        147076   147076

That makes the flash ROM 10.5kb bigger:

(66080 - 55728) + (15140 - 14732)  = 10352 + 408  = 10760 bytes

I have tested the code on device, but not thoroughly. I have tested loading in a slightly complex scene from USB. I also haven’t benchmarked it, but my gut (and brain) tell me that it should be several orders of magnitude faster.

What’s the next step then? Anyone want to help test? Or should I open up a PR and invite comments from there?

2 Likes
#36

amazing! i’d say 10k is no big deal at all given the improvement.

i’ll do some testing! it doesn’t seem that there are too many points of failure though?

1 Like
#37

Apart from the usual segmentation faults, etc. (which I don’t think there are too many opportunities for), it tends to be weird things, e.g. for a while TADD 1 1 would be the same as ADD 1 1. It would parse TADD as T followed by ADD. I added some code to count how many operators were matched. It’s possible that a better way would be to return after the first match, whilst also asserting that the match has consumed the entire token.

I would also feel more comfortable if you had a look through scanner.rl and match_token.rl and make sure you feel confident with how they work. I plan on sticking around to provide help and support, but you never know…

I’ll also freely admit to not really knowing how most of the bits of Ragel work. We’re in a special use case (the => operator for scanning) which feels quite orthogonal to all the other ways to use Ragel.

#38

Hi @tehn I noticed on another thread that you’re currently working on some teletype & ansible code.

Would it be possible to coordinate somehow as I think my parser changes might be a tricky rebase if you’re going to make the teletype changes I think you’re going to make (i.e. new ops for ansible).

Would you consider forking monome/teletype to tehn/teletype and working on it there? There would be a couple of advantages…

  • It’s socially acceptable to force push to your own fork, when you can’t to the main repo.
  • I could manually grab your changes and reimplement them on my branch before the merge to master on monome/teletype.
  • If you open up a PR from your branch to master you’ll get the benefit of the automated tests running and checking your branch. (You can also get this working on your fork for any push.)
  • If you wanted I’d be happy to review any code if you did open a PR.

Alternatively, could you make the changes on a new branch in the main repo.

Also while I’m “needy mode”, would it be possible to adopt semantic versioning going forward (i.e. 1.2.1 instead of 1.21).

#39

yes to all of the above!

though i’m afraid i’ve already added quite a bit of code-- i’ll figure out how to move files around to prevent any trouble and not commit directly. FYI i’m really only editing hardware.* and ii.h to get more ansible ii ops in there-- not sure that collides with your edits.

semantic versioning, yes! so after i add these ops we’ll be at 1.3.0

#40

ok, i’ve successfully moved all my changes over to my fork. i’ll from now on behave like a responsible community member on this project-- thanks for the push.

1 Like
#41

It’s not so much the edits to hardware.c as the edits to op.c to enable the extra ops that will mess with it.

Do you want to make your changes on tehn/monome and we can have a look and figure the best order to do things

#42

i’ll submit a PR when the new ops are ready-- hoping to finish this all by the end of the day. honestly not doing anything breaky, i think. also i’ll be sure to stay on top of my formatting.

#43

Okay, open the PR, and I will have a quick go at rebasing my changes this evening (or tomorrow morning) to make sure it’s not going to cause me too much grief. Sounds like it should be straightforward though.

#44

Okay I’ve added the PN versions of all the ops in. patterns.c has been heavily rewritten to reduce duplication, I’m really hoping I haven’t introduced any bugs, but it’s a lot tider now.

I’ve also added some tests to check that all ops are unique and that each op really manipulates the stack as it says it should. (FYI, while I was playing with merging the code, the test told me that LV.RES has the wrong .param size, it’s set to 0, it should be 1).

@tehn are your changes on tehn/teletype finished? I’ve had a quick go at trying to merge your branch and mine, and we’ll end up with a merge conflict. It’s not a particularly hard one to fix, and I’m happy to do the merge. We just need to decide which branch we merge to monome/teletype first.

Also FYI, the tests are broken on your master branch, you need to add the following to tests/main.c

void tele_ii_tx_now(uint8_t addr, uint8_t *data, uint8_t l) {}

And cough make format

3 Likes
#45

thanks for the tips. will have a minute later tonight to wrap it up with a PR

#46

late late late, but PR is in.

2 Likes
#47

Nice one, it’s (past) my bedtime now. I’ll have a quick look in the morning and make sure there is nothing that will cause me too many problems. Then you can merge the PR? And I will merge those changes into my branch.

#48

Looks good to me. (Also left the same message on PR.)

I’ll merge the changes into my parser branch this week. I’ll probably rename some of the structs to match the op name (I’ve made similar changes in my branch for other ops).

Do you have any more short term plans for the teletype code? I’m trying to decide if I should PR my changes soon, or wait until I’ve taken a stab at the ; code.

#49

no short term plans-- must push through more production work first. but also regroup with you all as to what the priorities are with the TT codebase and feature extensions.

4 Likes
#50

I’ve merged monome/master into my parser branch.

I’m going to try and do the ; multi command code over Christmas, and then go for a big PR in the new year. It would be nice to get most of it wrapped up before 2017 gets going.

4 Likes
#51

I have a brand new TT arriving tomorrow and very interested in the progression of the firmware and making life easier to add custom ops (one of my driving factors in picking one up) - so having the ragel-based parser is awesome. I’ve already built a version from your parser branch and plan to give it a run out of the gates, sam.

I’ll let you know if I run into anything via github. Good work on this!

4 Likes