#30

this font discussion is interesting and very glad for your insights! but (i feel obliged to point out) it is pretty off-topic, if the topic is the parser code.

but well, forging ahead… i thought i would raise my hand and take sheepish responsiblity for all the nasty rendering code in libavr32 and for the selection of Liquid in the first place (after a significant but not exhaustive consideration of different possibilities), though i did not work on the teletype code in particular.

Odd enough, I get 29 periods until teletype stops printing more on
screen. Which makes me think the limit is not based on how many of the
widest character one can fit on a line.

teletype seems to have a hard limit of 32 characters per line in the scene struct. the rendering library can end up padding the right margin by up to 6px if a glyph would otherwise be truncated. can’t spot any other limitation right now, it is a mystery :ghost:

I am not sure how the empty pixels defined to the sides of each character on the font is truncated by the library.

each entry in the glyph table contains:

  • number of blank columns between left edge and start of glyph
  • number of blank columns between end of glyph and right edge
  • full 6 columns of pixel data, whether they are zeroed or whatever.

the library can render in proportional mode - in which case reads however many columns are appropriate according to the table, then adds one blank column, or in fixed mode - rendering 6 columns for every glyph, plus one blank column. of course the latter doesn’t look great, a real monospace font would be better, but at least I’s and T’s are more or less centered.

in early ui experiments we decided pretty quickly that a proportional font was almost mandatory to fit any significant amount of data on the screen, and so didn’t bother going further with monospace. it would be super easy to integrate it though, if you were interested in taking on the design task.

([edit] oh, and i can’t think of a technical reason to require an even number of columns for a monospaced font. the rendering regions themselves must have even width, because of the way data is sent to the OLED controller (1byte==2px), but font rendering happens before that, in a flat buffer (1B==1px))

in aleph there is also support for antialiased fonts of arbitrary size, which has been hacked out for libavr32 but can be easily re-added.

and yeah, it’s the serif’d ‘I’ to better distinguish from ‘1’ :slight_smile:

2 Likes
#31

I’m happy for y’all to keep talking about fonts here, but you might get a wider audience if you start a new topic.

Whether you want a wider audience is another question…

@capieniz you are correct the number of characters per line is ~29 per line (I’d need to dig deeper into the code the give you the exact number for all scenarios).

It’s still 29 even if you type just M, e.g. type a lot of Ms until it goes off the end of the screen, then type a 1, then left arrow till you can see the cursor and delete some Ms, watch how the 1 appears.

(there is also a rather nasty overflow bug, e.g. type 29 :, left arrow several times, type some more characters (#4))

What makes things slightly more complex is that the actual internal limit is how many ops you can have per line not how many characters it takes up. Normally this is not a problem as the number of characters used is usual greater than the number of ops, but it is possible to construct lines which will fail, e.g.

LIM P 1 2 3 P 1 2 3 P 1 2 3

returns COMMAND TOO LONG. (FYI, numbers are ops too!)

I will say at the moment, issues relating to running out of characters seem to be few and far between. (Though I do have some ideas that might suddenly make it important.)

#32

Hi @tehn do you think it’s likely the TIMELINE feature will be implemented in a V2? The length of the scripts and number of scripts is something I constantly find myself hitting a wall with. I imagine that if I were to be using just friends with the new commands or introduced the expander or ansible then I don’t think there will be enough space for scripting ideas that take full advantage of these new possibilities.

Cheers, Alex

#33

Thank you @zebra and @sam for your input and all that information. It clears up a few misconceptions I had and explains the limits based on the actual code (I only have experience with Python, C needs a lot of effort and feels obscure to me).I think just minor modifications to the font is probably the way to go. Will start a new thread and gauge interest, get some feedback.

Thanks again!

2 Likes
#34

Calculations time…

A teletype contains a AT32UC3B0512 MCU, it has 96kb of RAM and 512kb of flash. Pointers are 32bit. The flash can be split between ROM and storage, we are currently using approx. 80kb of ROM.

I’m going to ignore RAM usage, as it stands we have approx. 70kb free. Flash usage is what will constrain us.

Currently… a teletype contains:

  • 32 scenes, containing:
    • 10 scripts (1-8, I, M), containing:
      • 6 commands, containing:
        • 12 ‘datas’ (a.k.a. ops)

So a single scene contains 720 ops, and the entire teletype contains 23040 ops.

Each op is a defacto tagged union:

typedef enum { NUMBER, MOD, SEP, OP } tele_word_t;

typedef struct {
    tele_word_t t;
    int16_t v;
} tele_data_t;

sizeof(tele_word_t) == 1
sizeof(tele_data_t) == 4 // is 4 due to struct packing

So, 23040 ops * 4 bytes == 92160 bytes.

All the way at the back of the top of this thread we discussed how to store a reference to the op (or mod) in tele_data_t, currently it is an index of a table. If we were to switch to using a pointer to the op struct then sizeof(tele_data_t) == 8 (unless we can come up with some clever way of hiding the tag inside the pointer).

So I think I will keep the op and mod tables, and the new parser code will carry on returning an index for them rather than a pointer.

1 Like
> teletype : code exchange
#35

A big update on my parser branch…

https://github.com/samdoshi/teletype/tree/parser

The entire parser is now running via Ragel, both scanning for tokens and matching them.

Code sizes, master branch first, parser branch second:

section               size     size
.reset                8204     8204
.rela.got                0        0
.init                   26       26
.text                55728    66080
.exception             512      512
.fini                   24       24
.rodata              14732    15140
.lalign                  4        4
.dalign                  4        4
.ctors                   8        8
.dtors                   8        8
.jcr                     4        4
.got                     0        0
.data                 7492     7492
.balign                  0        0
.bss                  8136     8136
.heap                74456    74456
.comment                47       47
.stack                8192     8192
.flash_nvram        147076   147076

That makes the flash ROM 10.5kb bigger:

(66080 - 55728) + (15140 - 14732)  = 10352 + 408  = 10760 bytes

I have tested the code on device, but not thoroughly. I have tested loading in a slightly complex scene from USB. I also haven’t benchmarked it, but my gut (and brain) tell me that it should be several orders of magnitude faster.

What’s the next step then? Anyone want to help test? Or should I open up a PR and invite comments from there?

2 Likes
#36

amazing! i’d say 10k is no big deal at all given the improvement.

i’ll do some testing! it doesn’t seem that there are too many points of failure though?

1 Like
#37

Apart from the usual segmentation faults, etc. (which I don’t think there are too many opportunities for), it tends to be weird things, e.g. for a while TADD 1 1 would be the same as ADD 1 1. It would parse TADD as T followed by ADD. I added some code to count how many operators were matched. It’s possible that a better way would be to return after the first match, whilst also asserting that the match has consumed the entire token.

I would also feel more comfortable if you had a look through scanner.rl and match_token.rl and make sure you feel confident with how they work. I plan on sticking around to provide help and support, but you never know…

I’ll also freely admit to not really knowing how most of the bits of Ragel work. We’re in a special use case (the => operator for scanning) which feels quite orthogonal to all the other ways to use Ragel.

#38

Hi @tehn I noticed on another thread that you’re currently working on some teletype & ansible code.

Would it be possible to coordinate somehow as I think my parser changes might be a tricky rebase if you’re going to make the teletype changes I think you’re going to make (i.e. new ops for ansible).

Would you consider forking monome/teletype to tehn/teletype and working on it there? There would be a couple of advantages…

  • It’s socially acceptable to force push to your own fork, when you can’t to the main repo.
  • I could manually grab your changes and reimplement them on my branch before the merge to master on monome/teletype.
  • If you open up a PR from your branch to master you’ll get the benefit of the automated tests running and checking your branch. (You can also get this working on your fork for any push.)
  • If you wanted I’d be happy to review any code if you did open a PR.

Alternatively, could you make the changes on a new branch in the main repo.

Also while I’m “needy mode”, would it be possible to adopt semantic versioning going forward (i.e. 1.2.1 instead of 1.21).

#39

yes to all of the above!

though i’m afraid i’ve already added quite a bit of code-- i’ll figure out how to move files around to prevent any trouble and not commit directly. FYI i’m really only editing hardware.* and ii.h to get more ansible ii ops in there-- not sure that collides with your edits.

semantic versioning, yes! so after i add these ops we’ll be at 1.3.0

#40

ok, i’ve successfully moved all my changes over to my fork. i’ll from now on behave like a responsible community member on this project-- thanks for the push.

1 Like
#41

It’s not so much the edits to hardware.c as the edits to op.c to enable the extra ops that will mess with it.

Do you want to make your changes on tehn/monome and we can have a look and figure the best order to do things

#42

i’ll submit a PR when the new ops are ready-- hoping to finish this all by the end of the day. honestly not doing anything breaky, i think. also i’ll be sure to stay on top of my formatting.

#43

Okay, open the PR, and I will have a quick go at rebasing my changes this evening (or tomorrow morning) to make sure it’s not going to cause me too much grief. Sounds like it should be straightforward though.

#44

Okay I’ve added the PN versions of all the ops in. patterns.c has been heavily rewritten to reduce duplication, I’m really hoping I haven’t introduced any bugs, but it’s a lot tider now.

I’ve also added some tests to check that all ops are unique and that each op really manipulates the stack as it says it should. (FYI, while I was playing with merging the code, the test told me that LV.RES has the wrong .param size, it’s set to 0, it should be 1).

@tehn are your changes on tehn/teletype finished? I’ve had a quick go at trying to merge your branch and mine, and we’ll end up with a merge conflict. It’s not a particularly hard one to fix, and I’m happy to do the merge. We just need to decide which branch we merge to monome/teletype first.

Also FYI, the tests are broken on your master branch, you need to add the following to tests/main.c

void tele_ii_tx_now(uint8_t addr, uint8_t *data, uint8_t l) {}

And cough make format

3 Likes
#45

thanks for the tips. will have a minute later tonight to wrap it up with a PR

#46

late late late, but PR is in.

2 Likes
#47

Nice one, it’s (past) my bedtime now. I’ll have a quick look in the morning and make sure there is nothing that will cause me too many problems. Then you can merge the PR? And I will merge those changes into my branch.

#48

Looks good to me. (Also left the same message on PR.)

I’ll merge the changes into my parser branch this week. I’ll probably rename some of the structs to match the op name (I’ve made similar changes in my branch for other ops).

Do you have any more short term plans for the teletype code? I’m trying to decide if I should PR my changes soon, or wait until I’ve taken a stab at the ; code.

#49

no short term plans-- must push through more production work first. but also regroup with you all as to what the priorities are with the TT codebase and feature extensions.

4 Likes