Max line length is no. of characters not how many pixels used. I think the single pixel width for : is a bug really.

I can’t type any more : on that line.

The data adds up though, 32 scenes * 10 scripts * 6 lines * 31 chars = 59520 bytes

(I’m not 100% sure it’s 31 characters, ± 1)

edit: maybe it’s not a bug, the . is also 1px wide, anyway, as you say we could manually adjust the : glyph to be wider, or use an alternative glyph for rendering

1 Like

also note how N takes 4 pixels (‘M’ is probably five? don’t have a tt nearby right now…), so it’s definitely proportional. that’s why i think the max length right now is coupled to how wide the widest character is to adhere to the constraint of not having horizontal scrolling. which is a nice constraint to have from the perspective of having a complete script fit on one screen (and forcing you to think creatively…) but maybe scrolling wouldn’t be so bad.

The up arrow is a ^ (also it’s a single quote followed by a double quote just before the ?).

I think I’ll get my Ragel code working with the existing <space>:<space> rules, then try to mull over the other options.

i’m happy to consider modifying the font, by the way. i’d defer to any type designers, if they are here. i do have some modification suggestions in my e-mail inbox, by they relate to legibility of a few letter characters.

re: storing scripts as text, seems like overkill. perhaps better to correct the typeface.

Not sure if this is the right place to ask, but what’s the rationale for the 6 line limit? Is this a memory footprint issue?

it’s a UI consideration, for no scrolling.

i proposed a new feature, TIMELINE, for v2. it’d facilitate all your super long script needs.

but this would be a big project.

4 Likes

I’m not a type designer, but I know a couple. I think you’re doing a great job given the resolution you’re dealing with.

so that’s the name of that TeX face. Now I have a name for the thing I, too, hate!

1 Like

My pleasure.

I generally like LaTeX, although it can feel a bit quirky by modern standards. Computer Modern is yuck though, although part my of dislike can probably be traced back to Analysis problem sheets at university.

Xe(La)Tex works well for modern font use, though you’re generally quite limited if you need serious maths support.

1 Like

I have pushed a working version of the code to my parser branch (commits, and diff).

This is only a replacement for the strtok part of parser, the strcmp bit (which now lives in match_token) will be next. I was wondering if people could have a look through the changes and make sure they’re happy with the use of Ragel before I start on the next bit.

I’ve tested the code on device and it seems to be working fine (I’ve even tested USB import).

1 Like

Anyone had a chance to look?

@tehn if I start on the next bit (the token identifier) will you be happy to merge all of it once it’s done?

just checked, looks great! looking forward to the merge-- thank you for taking this on. learning about ragel, didn’t see myself going down this road.

1 Like

monadic parsing, anyone? Obviously not directly relevant to teletype, but kind of interesting from p.o.v hand-rolling aleph patching language on host. ragel looks very handy, btw!

Got curious about the same early this year - this library kept me entertained for a while:

‘monadic’ parsing in lisp was pretty mind-melting, but I suspect very fast to prototype new ideas once out of the steep initial learning curve. Took it far enough to have a 95% complete common lisp parser & an algol-family language parser (including variable assignment, array syntax, arithmetic operator precedence etc) which spat out working lisp code.

Guessing monadic parsing technique is even more elegant & cooler in haskell context because of the lazy evaluation. Believed at the time that smug explodes in certain situations where a haskell parser would work fine, though since I don’t know much haskell, it’s hard to verify that!

2 Likes

Type designer here. I could look into this. Won’t be too difficult to modify things according to what people might prefer. Looking at the original Liquid fonts and what I could see on libavr32 there have already been some modifications and/or selection of certain specific variants of some characters (like the I with serifs).

Right now the font uses a maximum of 6 pixels horizontally per character (the 5 pixel wide M plus 1 pixel for spacing). Going lower than that would start affecting legibility in a bad way. Try to imagine an M using only 2 pixels between the vertical stems. It’s doable, but you definitely sacrifice legibility and end up having to rely a lot more on context to try to figure our what letter you are reading.

Basically, with an even number of pixels to represent the shapes, any glyph with odd-numbered stems or features (I, W, M, T, V, X, etc) will suffer either from bad spacing (on monospaced fonts) or nor enough pixels to show all the features that make the glyph recognisable.

As an example, here is a monospaced font with a situation like I described. Check the W and M for one solution to the not-enough-pixels issue. I and T suffer from spacing issues:

Considering we do have more space to use within those potential max 5+1 pixels, we could certainly try and give some letters a bit more breathing space. But that would only be true if the character limit is actually set by the M, which I doubt. See more below. I would also differentiate the O and the 0, either by making the O wider or by “slashing” the 0. For the : , we could always try to force more space within the glyph itself and then get rid of the spaces to the sides of it in code. I am not sure how the empty pixels defined to the sides of each character on the font is truncated by the library.

We could also start completely from scratch and use 4+1 pixels per character, though that would give us at most 3 more characters per line if the M theory is correct (which I think is not). I can currently fit 19 M across, which gives me 114 pixels of space in total. With 5 pixel wide characters we could fit 22. Not a huge gain. And legibility would definitely suffer.

Odd enough, I get 29 periods until teletype stops printing more on screen. Which makes me think the limit is not based on how many of the widest character one can fit on a line.

Anyway, I would gladly help make any modifications. I have to say I quite like the Liquid fonts, they manage to get decent legibility and a fair amount of personality with limited resources. And the lowercase is lovely too. But there is always space for improvement, especially when taking into account specific needs and uses.

Apologies for the long text.

Cheers,
José

5 Likes

this font discussion is interesting and very glad for your insights! but (i feel obliged to point out) it is pretty off-topic, if the topic is the parser code.

but well, forging ahead… i thought i would raise my hand and take sheepish responsiblity for all the nasty rendering code in libavr32 and for the selection of Liquid in the first place (after a significant but not exhaustive consideration of different possibilities), though i did not work on the teletype code in particular.

Odd enough, I get 29 periods until teletype stops printing more on
screen. Which makes me think the limit is not based on how many of the
widest character one can fit on a line.

teletype seems to have a hard limit of 32 characters per line in the scene struct. the rendering library can end up padding the right margin by up to 6px if a glyph would otherwise be truncated. can’t spot any other limitation right now, it is a mystery :ghost:

I am not sure how the empty pixels defined to the sides of each character on the font is truncated by the library.

each entry in the glyph table contains:

  • number of blank columns between left edge and start of glyph
  • number of blank columns between end of glyph and right edge
  • full 6 columns of pixel data, whether they are zeroed or whatever.

the library can render in proportional mode - in which case reads however many columns are appropriate according to the table, then adds one blank column, or in fixed mode - rendering 6 columns for every glyph, plus one blank column. of course the latter doesn’t look great, a real monospace font would be better, but at least I’s and T’s are more or less centered.

in early ui experiments we decided pretty quickly that a proportional font was almost mandatory to fit any significant amount of data on the screen, and so didn’t bother going further with monospace. it would be super easy to integrate it though, if you were interested in taking on the design task.

([edit] oh, and i can’t think of a technical reason to require an even number of columns for a monospaced font. the rendering regions themselves must have even width, because of the way data is sent to the OLED controller (1byte==2px), but font rendering happens before that, in a flat buffer (1B==1px))

in aleph there is also support for antialiased fonts of arbitrary size, which has been hacked out for libavr32 but can be easily re-added.

and yeah, it’s the serif’d ‘I’ to better distinguish from ‘1’ :slight_smile:

2 Likes

I’m happy for y’all to keep talking about fonts here, but you might get a wider audience if you start a new topic.

Whether you want a wider audience is another question…


@capieniz you are correct the number of characters per line is ~29 per line (I’d need to dig deeper into the code the give you the exact number for all scenarios).

It’s still 29 even if you type just M, e.g. type a lot of Ms until it goes off the end of the screen, then type a 1, then left arrow till you can see the cursor and delete some Ms, watch how the 1 appears.

(there is also a rather nasty overflow bug, e.g. type 29 :, left arrow several times, type some more characters (#4))

What makes things slightly more complex is that the actual internal limit is how many ops you can have per line not how many characters it takes up. Normally this is not a problem as the number of characters used is usual greater than the number of ops, but it is possible to construct lines which will fail, e.g.

LIM P 1 2 3 P 1 2 3 P 1 2 3

returns COMMAND TOO LONG. (FYI, numbers are ops too!)

I will say at the moment, issues relating to running out of characters seem to be few and far between. (Though I do have some ideas that might suddenly make it important.)

Hi @tehn do you think it’s likely the TIMELINE feature will be implemented in a V2? The length of the scripts and number of scripts is something I constantly find myself hitting a wall with. I imagine that if I were to be using just friends with the new commands or introduced the expander or ansible then I don’t think there will be enough space for scripting ideas that take full advantage of these new possibilities.

Cheers, Alex

Thank you @zebra and @sam for your input and all that information. It clears up a few misconceptions I had and explains the limits based on the actual code (I only have experience with Python, C needs a lot of effort and feels obscure to me).I think just minor modifications to the font is probably the way to go. Will start a new thread and gauge interest, get some feedback.

Thanks again!

2 Likes

Calculations time…

A teletype contains a AT32UC3B0512 MCU, it has 96kb of RAM and 512kb of flash. Pointers are 32bit. The flash can be split between ROM and storage, we are currently using approx. 80kb of ROM.

I’m going to ignore RAM usage, as it stands we have approx. 70kb free. Flash usage is what will constrain us.

Currently… a teletype contains:

  • 32 scenes, containing:
    • 10 scripts (1-8, I, M), containing:
      • 6 commands, containing:
        • 12 ‘datas’ (a.k.a. ops)

So a single scene contains 720 ops, and the entire teletype contains 23040 ops.

Each op is a defacto tagged union:

typedef enum { NUMBER, MOD, SEP, OP } tele_word_t;

typedef struct {
    tele_word_t t;
    int16_t v;
} tele_data_t;
sizeof(tele_word_t) == 1
sizeof(tele_data_t) == 4 // is 4 due to struct packing

So, 23040 ops * 4 bytes == 92160 bytes.

All the way at the back of the top of this thread we discussed how to store a reference to the op (or mod) in tele_data_t, currently it is an index of a table. If we were to switch to using a pointer to the op struct then sizeof(tele_data_t) == 8 (unless we can come up with some clever way of hiding the tag inside the pointer).

So I think I will keep the op and mod tables, and the new parser code will carry on returning an index for them rather than a pointer.

1 Like

A big update on my parser branch…

https://github.com/samdoshi/teletype/tree/parser

The entire parser is now running via Ragel, both scanning for tokens and matching them.

Code sizes, master branch first, parser branch second:

section               size     size
.reset                8204     8204
.rela.got                0        0
.init                   26       26
.text                55728    66080
.exception             512      512
.fini                   24       24
.rodata              14732    15140
.lalign                  4        4
.dalign                  4        4
.ctors                   8        8
.dtors                   8        8
.jcr                     4        4
.got                     0        0
.data                 7492     7492
.balign                  0        0
.bss                  8136     8136
.heap                74456    74456
.comment                47       47
.stack                8192     8192
.flash_nvram        147076   147076

That makes the flash ROM 10.5kb bigger:

(66080 - 55728) + (15140 - 14732)  = 10352 + 408  = 10760 bytes

I have tested the code on device, but not thoroughly. I have tested loading in a slightly complex scene from USB. I also haven’t benchmarked it, but my gut (and brain) tell me that it should be several orders of magnitude faster.

What’s the next step then? Anyone want to help test? Or should I open up a PR and invite comments from there?

2 Likes