Ii bus board, v2, with pullups

trickyflemming · 2017-04-09T19:51:57.209Z

Great. About to test that, but one more data point…

I just changed my M script to TR.TOG 5 (with the Ansible in Teletype mode) while using the rest of the Jumpy Edges script (no TO code in there… that was the only line in the M script). It crashed within ten seconds.

EDIT: As an aside, here’s my thought as to what is potentially happening. It seems that the Teletype is internally stable. I was smacking it pretty hard with the Varigate while the M script was flipping one of the main toggles. Everytime I do anything involving the ii bus on the M script while using the main TT script triggers it seems to crash.

I think there may be some weird race condition involved with the M, ii, and trigger sampling schedulers, but only when using all three.

With my “No Triggers” test (M script triggering an ii message every 50 ms), things were stable.
With my “Lots of Triggers” test (M script triggering internal message every 50 ms), things were stable.
As soon as I have triggers, M, and ii together, that’s when it seems to crash.

trickyflemming · 2017-04-09T20:05:16.272Z

Okay, just ran all of the #1 tests. It crashes with any number of cables (4, 3, 2, or 1). So, I suppose it’s only stable with no cables there.

On to test #2.

EDIT
Test #2, part 1.
Deleting scripts 1-4 but leaving the cables in leads to a crash. This is making me believe more in my scheduler race condition theory.

ANOTHER EDIT:
I’m about to test the M thing for Test #2, although I think it’s pretty similar to the TR.TOG M test that I ran above. I looked at the STACK_SIZE and STACK_OP_SIZE defines at https://github.com/samdoshi/teletype/blob/2.0/src/state.h and noticed that they’re defined to a constant 8. I’m not too familiar with the Teletype code, so this may be way off. However, my M script is sending 8 messages to the ii bus each tick. Is it possible that there’s a situation where the various schedulers are trying to push to an already full stack?

bpcmusic · 2017-04-09T21:24:44.054Z

@trickyflemming really interesting results. Apologies for not being more interactive this weekend, I was building furiously and now am on my way to South America for a couple of days. @sam - thanks for your efforts here as well!!

sam · 2017-04-10T06:52:48.623Z

trickyflemming:

I’m about to test the M thing for Test #2, although I think it’s pretty similar to the TR.TOG M test that I ran above. I looked at the STACK_SIZE and STACK_OP_SIZE defines at https://github.com/samdoshi/teletype/blob/2.0/src/state.h and noticed that they’re defined to a constant 8. I’m not too familiar with the Teletype code, so this may be way off. However, my M script is sending 8 messages to the ii bus each tick. Is it possible that there’s a situation where the various schedulers are trying to push to an already full stack?

So those stacks aren’t what you think.

STACK_OP_SIZE defines the maximum number of commands you can add to the S mod.

STACK_SIZE is how large the stack can grow inside the teletype language. 8 is plenty. e.g. L 1 8 : TO.TR.M I MUL I X will firstly use multiple different stacks, one for the PRE and one for each run of the loop. Loops from L are processed sequentially, not concurrently. In the given example the stack will never get larger than 2. The actual stack is defined in the linker script and is 8kb.

You’re right that there is an issue between the i2c code and triggers / M scripts. A few options (a.k.a. a bit of brainstorming):

The i2c code is not ‘thread safe’ enough. Maybe there are some critical bits that need masking from interrupts.
Possibly an issue with messages being sent and received out of order.
Long running code in an interrupt (which will be masked by default?) causing the i2c internal rx or tx buffers to overflow
Too many long running processes (e.g. i2c) which lead to the event queue overflowing or generally fubaring.

More simplifications if possible, can you try the following 3, all with empty scripts 1-8, but a fast Varigate trigger into input 1.

The I script is the same for all of them:

#I
M 50
M.ACT 1

Pulse TXo

#M
TO.TR.PULSE 1

Pulse non-existant TXo

#M
TO.TR.PULSE 9

Pulse Ansible

#M
TR.PULSE 5

If you remove the trigger do the crashes stop?

Leverkusen · 2017-04-10T21:51:08.449Z

I finished my powered i2c bus board tonight. Together with a simple one trigger to four random CV’s on output 1 - 4 this crashed within a minute:

sam:

#M
TR.PULSE 5

I have two Ansibles, ES and MP connected. When I started the Metro script Ansible was not in the right mode, which was no problem. I switched it to telepepe output mode then and it started to blink. But not for long.

I also have a lot of display errors with truncated lines. Like this:

IMG_4275.JPG2448×2448 793 KB

trickyflemming · 2017-04-11T02:17:11.305Z

Just ran all six scripts.

For all three with the cable inserted: failure.
For all three without the cable inserted: stability.

From all of these tests, I’ve noticed we have a set of three things:

External triggers (i.e. gates plugged into the TT from other modules).
The M script.
Anything ii related.

Right now, any two of these co-exist peacefully on my TT. The moment all three are in action, I experience a lock-up within 5 minutes or so.

Leverkusen · 2017-04-11T07:49:26.816Z

I can also report that the issues have changed a bit.

What I experience is that outputs get simply inactive while the whole system keeps working.

For example Ansible stops emitting CV changes transmitted form teletype but the trigger outputs still work as expected and patching in the grid still switches to kria. But no response on CV changes via scripts or in live mode. When I change the output numbers in the script from CV 5,6,7,8 to CV 1,2,3,4 while the script is running and repatch the four cables from Ansible CV to teletype everything is working again.

I also experience something like this on the teletype outputs occasionally, e.g. a trigger output just stays on and does not react to any commands.

It seems to me that this occurs with the newly build bus board but I could imagine that it is about the teletype firmware and I just did not use it that much while I was waiting for me building the new bus board.

sam · 2017-04-11T14:15:01.515Z

trickyflemming:

From all of these tests, I’ve noticed we have a set of three things:

External triggers (i.e. gates plugged into the TT from other modules).

The M script.

Anything ii related.

Right now, any two of these co-exist peacefully on my TT. The moment all three are in action, I experience a lock-up within 5 minutes or so.

I concur with this. Sorry for not helping with the testing myself, but my Teletype is currently out of my main case and has no other modules alongside it. I’m going to move this over to the Teletype Firmware i2c Debugging thread and tag various people as this is almost definitely a software bug in the i2c implementation.

@Leverkusen are you too only seeing the display errors when using triggers and an M script that transmits over the i2c bus?

Also is the display corruption happening all the time or just when using i2c? (fyi, display corruption like that is usually caused by memory corruption, which may be indicative of the type of i2c bug we’re dealing with). @trickyflemming have you seen any display artefacts like that?

bpcmusic · 2017-04-11T14:38:58.535Z

Leverkusen:

Ansible stops emitting CV changes transmitted form teletype

I’ve experienced this as well. You can see it in the video I shared the other day on the expander thread; the ansible should be mirroring the CV values for the teletype - but at some point the CV became frozen while the triggers continued to function:

I’m assuming this is an issue in the Ansible itself and not the Teletype as all other i2c messages are still reliably transmitting.

@trickyflemming and @sam - apologies as well for not testing myself. I’m in Argentina for business through tomorrow and have been away from my TT to try your scenarios. It looks like you have done an excellent job of narrowing down the issue. I’ll work to replicate when I’m back in town.

sam · 2017-05-20T14:51:49.471Z

Hi all. This seemed like the best thread.

I find myself needing to make some II bus board cables that are “just the right length”.

So I need some 2x3 2.54mm pitch IDC connectors right?

Like this one: RS-Online?

And also some 1.27mm pitch 6-way flat ribbon cable? I’m going to buy some 16-way for use in other projects. It’s easy enough to reduce down to 6 cables right?

tehn · 2017-05-20T17:47:14.080Z

both correct. ribbons separate quite easily.

sam · 2017-06-03T13:46:13.967Z

Yay, got my cables made!

I’ve got the following II cables and an unpowered bus board if anyone wants them?

ii_cables.jpg1024×768 138 KB

Free to anyone who wants them in the UK, or anywhere where it won’t cost too much to post.

edit: cables and bus board gone

Dewb · 2017-09-28T19:51:49.599Z

I’m about to order a set of V2 bus board PCBs if anyone else in Boston needs one.

Also, does teletype plus the bus board still fit comfortably in the isms case? I assume so, but wanted to confirm. Thanks for everyone’s hard work!

tehn · 2017-09-28T20:26:38.036Z

Dewb:

Also, does teletype plus the bus board still fit comfortably in the isms case?

absolutely, it fits just fine

xenus_dad · 2018-07-28T17:36:54.111Z

Apologies for necroposting, but: I am about to put an order into Oshpark for PCBs for these (along with boards for https://www.whimsicalraps.com/pages/slashes and some other goodies). My Teletype network is starting to grow.

Oshpark has a minimum order of 3 PCBs, so I will have 2 extras on hand. I’m happy to build out and ship the extras if folks are willing to split boards, components, and shipping.

This would more be a labor of love than a business proposition, especially as I am an amateur DIY builder - competent enough, but I would not sell my builds for profit So, let’s say I would build a limit of 4 boards maximum, if there is even that much interest. More than that and it starts to become work.

Alternately, if you would just like a single board, I’m amenable to that too, though it’s probably more cost-effective to get Oshpark to make them for you and to have the spares.

PM me for either if there is any interest. Thanks!

(Mods, let me know if this belongs in Trade and I can put it there instead, but this is definitely a zero-profit endeavor, including free labor.)

xenus_dad · 2018-08-21T01:14:04.187Z

Are 5 modules enough to cause the problems the pull up resolves? I have 2 W/, 2 JF, and an Ansible.

Looking specifically to test bus boards I’m building, so I would want to have a reliable failure scenario to be sure the boards are working.

alanza · 2018-08-21T01:32:41.859Z

I think the problem is intermittent in its nature: too much load on the bus leading to missed commands? that’s only coming from my reading on here though

sakul · 2018-08-21T08:53:09.708Z

Yes. I recall three units to be the number one should get a powered busboard.

Edit: some background in this thread Teletype Firmware i2c Debugging

SwainVoorman · 2018-08-24T21:14:48.717Z

Is anyone offering an iiBackpack or something similar for sale? I’m planning on having a Teletype connect to four different devices in the near future. I see that bpcmusic has in the past but is currently out of stock.

SwainVoorman · 2018-08-25T01:52:27.891Z

SwainVoorman:

Is anyone offering an iiBackpack or something similar for sale?

You all are awesome, thanks for the offers! Looking forward to getting one in the near future.