norns: "include" caching?

Monome
1

Hi all,

Noob norns dev question here.

I am making changes to a library that is read in to a script with include.
At one point during my editing, there was an error in the script:

lua: /home/we/dust/code/toga/lib/togaarc.lua:98: attempt to call a nil value (global 'dump')

The string dump does not occur in the file, so that was an error. Fair enough. I fixed it, got rid of the reference to dump.

But now whenever I run my script, I still get the same error. It is referencing the same line number which now is something different.

I even put a print() statement at the top of the included file that prints out a number that I increment every time I edit the file. According to those print statements, the latest version of the file is being run, but I still get that error.

I took a look at the code for include in /home/we/norns/lua/core/startup.lua:37 and it looks like it calls dofile under the hood, which is supposed to not use caching.

I notice other weird things. Like, I commented out some print() lines and added others, but I see all the lines in the repl output.

I am definitely saving my changes. I am even making the changes in vi in an ssh session to the norns, to be sure there is not an issue with smb or maiden.

If I restart the norns, the problem goes away, but this implies that I need to restart after every change - is that the case when editing a library (not a script)?

This is how I restart my script:

Press K1
Press K3 to choose SELECT
My script is already selected, I press K3 twice more to run it.

Is there something else I need to do (short of restarting after every change)?

Thanks.

2

strange. everything you say seems correct: include()/dofile() does not cache and you should not need to restart between edits.

this is normal workflow and i’ve never seen such an issue, but did a quick test anyway:

script dir called hellolib:

~/dust/code $ tree hellolib/

hellolib/
├── hello.lua
└── lib
    └── mylib.lua

1 directory, 2 files

hello.lua:

--- hellolib

local mylib = include("lib/mylib")

init = function()
    mylib.hello()
end

mylib.lua:

return { 
    hello = function () 
        print("hello from mylib")
    end
}

if i introduce an error to mylib by adding, say

        print(fu.bar)

i get an init error and stacktrace; if i remove the offending line and rerun from the norns menu, the error and stacktrace goes away; moving around print statements works as normal, etc etc.

i am using sshfs in VS Code to edit the script, and Chrome to view maiden. host is macOS 13.1.

hypothesis: something amiss with the actual output stream / websocket getting buffered / cached by the web browser?

test: open another ssh session and run journalctl -f. this will show the same stdout of norns processes that is sent to maiden. (matron and sclang outputs show up under separate instances of ws-wrapper which run the websockets on the norns side.) same oddness in that output?

1 Like
3

Thanks for replying and trying to reproduce the issue.

Regarding your hypothesis, I am not using a web browser - I am running maiden-repl on the command line in an ssh session.

I did try your test suggestion. In the journalctl -f window I see the same oddness. The stack trace has gone away since I restarted norns, but now I see the output of print statements that have been commented out, as well as new ones that were added after the old ones were commented out.

I also saw norns kernel: Under-voltage detected! (0x00050005) which seems like a problem. I thought it was not related to this issue, but decided to do a change what I plugged into anyway, and that particular log message went away. (FWIW I was plugged into a dedicated charging port of a USB hub which does not give connectivity to other devices; now I’m plugged directly into an outlet via a charger).

I tried your example files and I can’t reproduce it with those either.

It seems to be specific to the file I am working on, which is part of the toga library.
And worth mentioning, it’s not in the init() function, but in a callback function that is called when an OSC message is received.

Thanks again for trying to help get to the bottom of this…

4

ah. this might be worth pulling on - i’d guess a bug in the way OSC callback is registered (either in core or just in this script’s stack.) core osc.event may not be getting updated correctly.

5

I’ll look into the way the OSC callback is registered. I did not write that code so it may take a bit.

I also wonder - are there intermediate bytecode files lying around somewhere that are produced by the lua compiler? Maybe I can delete those and force a full recompilation?

6

no, we’re not storing intermediate bytecode on disk.

i am guessing this is a little bug in toga. it looks like it’s trying to save (and presumably restore?) the osc.event handler that was already defined when the script was run:

this shouldn’t be necessary and seems related to the bug. like, the old handler was just the compiled closure from the last time the script was run.

module event handlers like osc.event are intended to be overwritten each time a script is run, scripts and script-level libraries should not attempt to cache and restore them.

i haven’t read through this thoroughly so maybe i’m missing some reason why toga is doing this (some kind of internal chaining system or idk.)

anyway unless there is some other reason to do this, i’d try removing that logic and just setting osc.event normally.

1 Like
7

oh! right! this wasn’t happening until the forthcoming update tho, so maybe that’s why toga was caching? clean up script-defined `osc.event` callbacks between scripts by dndrks · Pull Request #1643 · monome/norns · GitHub

1 Like
8

this has the ring of truth to it

1 Like
9

OK, so is it OK to set osc.event in the old fashioned way or should I wait till the forthcoming update?

Thanks.

10

i think it is totally fine to just set it. the update only addresses the case where one script sets an OSC event, and the next script doesn’t, and the first one remains.

at least worth a try in this scenario.

anyways you can always just issue git pull in the norns repo on device since this change is already merged. (and i think udpate release is imminent if not already live.)

1 Like
11

OK thanks. Appreciate your help & that of all others who chimed in.

I tweaked stuff and went back to the old fashioned osc.event() and that seems to have done the trick, as far as this weird problem is concerned.

Now I just need to solve the original problem that had me diving into toga in the first place.

1 Like
12

sorry i’m afraid that one is a bit beyond my remit, good luck!

1 Like