Norns: ecosystem

Just thought it was worth mentioning that the Aleph is the first option when choosing what patches you want when you arrive at Patchstorage.

I don’t know much about aleph, but from my experience with C&G Organelle patches it’s a great spot to keep track of the new hotness and keep the discussions here on the forum.

It’s generally a compressed zip (that can be uncompressed on the Organelle itself), but I’d presume we’d be limited by larger codebase changes needed to support some of the scripts norns is capable of. It’d be easy enough if you just had to add the following:

/username/newscript.lua /lib/newscript_support.lua

… but the overhead is higher when you need to walk an inexperienced user through SuperCollider this and peripheral that.

I also tried to fix a few typos yesterday and since I’m not a collaborator on the repo, I got a permission denied error. Obviously not a big deal, but I’m sort of curious if there’s a funnel for being ‘accepted’ as a contributor?

This is a permission denied when trying editing file directly on github?

The normal workflow is to fork the repository on github, make the changes in your fork (copy) and then open a pull request to get it merged into the main repository.

There are several people who have contributed to maiden already via this workflow and it is how I expect to work now that the repos are public (despite being one of the people who can currently modify/mess up the main monome repos if I’m not careful!)

The term collaborators on github does gives the impression of exclusivity but in reality it is little more than saying whether or not one has write access to a repository. It does not restrict contributions.

1 Like

I’m wondering now if it was because it was copy from the website (/study-2), which might be on the monome repo itself and not into the norns repo directory. Which would make sense. And if so, oops!

I tried pushing changes for review as a new branch and to the master branch when that didn’t work, knowing that someone could reject or merge changes accordingly after I opened a pull request, but I got the error on the push, which means I couldn’t open a PR.

It sounds like you were working directly off of a clone of the main monome repository instead of a fork (then clone). If you had fork'ed the repository first you should be fine. For example I’m not a collaborator on the monome/docs repository so I would have to fork it (which I did some time in the past), make changes in my fork, then open up a PR to get those changes merged back in.

…but this is getting off topic so I’ll stop here and we can pick it up in a different thread if need be.

1 Like

Here’s a great thread!

yeah, that was something unsanctioned i did in an effort to consolidate our efforts after enjoying the simplicity of the platform

at the time i had an organelle as well and seeing the ease of upload/search/download asked them to create a new category for .scn and .hex files used for aleph apps

since nobody knew about it (i felt a bit guilty promoting the idea without tehn’s knowledge)
and most aleph users are quite comfortable with github…there’s really nothing there

1 Like

Continuing on what some others mentioned about “discoverability” of other users scripts…

I went searching at https://github.com/monome/dust/network/members but did not find that any of those forks had any “personal” folders or user scripts other than the regular distribution.

So… I guess I’m keen to see what others are working on - even if that’s not ready for primetime - so :+1: for some kinda of solution for finding/sharing scripts.

1 Like

The forks view is a bit quicker/easier for that purpose since it’ll give you an overview of the branches people have created as well :slight_smile:

2 Likes

Ooh… :bulb: I didn’t think to look at “Branches” - i think that’s where stuff is probably hiding (github is hard for noobs)

1 Like

Oops! Apologies for calling attention to it then. Say the word and I’ll kill the post.

I say familiarizing oneself with git workflow may just be part of the joy of contributing to any codebase and it seems like there’s a great group of people here to help out (like me!), which I adore.

I literally use Git everyday at work but still hadn’t realized I needed to FORK first before contributing. Makes perfect sense and it was great to get that support from these other great norners (don’t worry, that won’t catch on).

1 Like

For what it’s worth - I tested a bit this weekend and SuperCollider Quarks work well out-of-the-box on the norns. Any norns/dust package management would reasonably need to support Quarks dependencies anyway, so I wonder if it might make sense to simply piggyback on this existing system?

1 Like

To answer an earlier question:
Quarks is SuperCollider’s package manager. It’s fairly simple - it uses a central repository with a map of project names and git repos. From there, getting info about or installing a Quark simply involves pointing to the git repository (which can be third party) and cloning. Quark installs do basic dependency resolution, though the system doesn’t yet support soft dependency specifications like <= 4.1, and doesn’t really do any advanced version resolution (these things are on the roadmap, but historically few quarks are really versioned very rigorously, mostly you just pull from head). Since it’s just cloning a git repo, a quark can contain any type of file (though binary data, audio files etc. would take a little work as putting them in git is a no-no - lfs is the right solution here).
The current quarks system has some holes (mainly around versioning), but I was able to ssh into my norns on Sunday and run Quarks.install("Foo"), and immediately install a quark and all it’s dependents with no problem.

A system for norns would look something like:

  1. Create a norns / dust central repo, e.g. monome/dust-quarks
  2. Either point the quarks install location to a place where lua will pick up .lua files, or point lua to the place where quarks are currently being installed (lua could use sclang’s include directory list, which is a simple json file)
  3. Add a simple two-way OSC interface between sclang and lua to run the basic Quarks command (listing, fetching info, install and uninstall), and expose it in the norns system menu (maiden might be another place to put the UI for this)

There’s probably some improvement work to do on the sclang side to e.g. specify that a quark is for norns / a specific sclang version, and some better recovery if you manage to bork your class library. And, likewise, some norns work to help manage files, keep things organized, and iterate on code that’s already on the system.

2 Likes

i hear that and it’s one of my top priorities. it’s amazing how quickly this community generates content— the studies aren’t even done yet!

in the short term, i’d suggest using cyberduck or other FTP program for file sorting. it works well and it’s thorough. we’re just looking to do something more integrated into the system and more nicely designed.

(i should add this to the docs. it’s been mentioned elsewhere variously)

10 Likes

just wondering. will some of the community made scripts get integrated on the next firmware update?

those that get submitted to the dust repository, yes.

5 Likes

PR to dust seemed like a really clean strategy for keeping script sharing organized and tidy. But now I find myself wondering if scripts need to be versioned along with norns releases, and also wondering if said releases will be months apart.

In order to keep script sharing vibrant, it would be nice to have a formal way to express what a script depends on (including version info) and thus allow script release schedules to be independent of norns release schedules.

I see that PRs to dust are getting merged promptly. I’d like to feel confident that I can go ahead and use those scripts with my current version of norns without breakage.

Should I throw caution to the wind? Am I being unnecessarily insecure about dependencies and potential breakage? I know some changes with Midi are afoot, how will that impact scripts?

Just trying to get a better handle on the vision for norns/dust release process and what expectations I should have for the pace of change and the infrastructure intended/desired/needed for supporting that pace.

I want to dive in but I’ve been holding back. I’m waiting for controller studies and I’m waiting for some kind of enlightenment about the issues I just raised. Possibly a few gentle words might unstick me from my perceived impasse.

I’m also happy to contribute to any package management or other efforts that might be underway to address this stuff that I may simply be unaware of.

4 Likes

TL/DR: it should always be fine to pull dust/master and norns/master together.


maybe a little.

if a PR to dust or norns master is approved, it is because we are reasonably sure that it won’t break anything. experimental PRs are marked as do-not-merge. (consensus was that having a master ahead of update packages was overall better than having a seperate bleeding-edge branch and only merging to master when tagging and rolling an update.)

re: midi, what is happening is that a higher-level layer is being added with a responder pattern, but it shouldn’t invalidate the existing callback pattern or break scripts that use it.

next update will have some changes under the hood that make SC engines going forward incompatible with older system. so we will need to be a little careful and maybe time to create a develop branch.

this will not impact script compatibility, but if we do break a lua API then of course we will have to decide how to deal with all the existing scripts.


all that said, there is a sort of framework in there for lua module versioning. it hasn’t really come up yet, but if necessary we can make some sort of dependency structure. i worry that there is not enough core project management or dev time to make this really functional though. (every norns developer except brian is a volunteer. and brian is busy with other things as well…)

4 Likes

Thanks @zebra! That’s really reassuring.

yeah, this only works if you maintain, a ‘latest view’ principle (like arch linux) , which basically says if you want any updates you have to take them all.
in this case, you want new scripts from dust, you have to update your norns to the latest OS, regardless of if the script you want needs the new feature.
(anything else is potentially unsafe, esp, if you are not on the newest version of norn OS)

what I did with axoloti for the community and factory library, was to branch the libraries along with the release, this way, new additions could be added into either new or old releases (if old, we could merge them into new)
(the binary build also ‘knows’ what release it is, and so which branch of the library it should be using, this means we know any firmware dependancies are satisfied (a bit like norns/dust might have crone dependancies)

all works well, the only thing that gets complex, is because auto sync the community library and users are ‘unaware of git’ , when a new release happens, we can get into a situation where both new and old branch are being added to by users (as they are on different versions), this can get a little confusing for users.

but given norns is expecting users to be ‘git savvy’ this should not be an issue for norns.