#61

Ooh… :bulb: I didn’t think to look at “Branches” - i think that’s where stuff is probably hiding (github is hard for noobs)

1 Like
#62

Oops! Apologies for calling attention to it then. Say the word and I’ll kill the post.

I say familiarizing oneself with git workflow may just be part of the joy of contributing to any codebase and it seems like there’s a great group of people here to help out (like me!), which I adore.

I literally use Git everyday at work but still hadn’t realized I needed to FORK first before contributing. Makes perfect sense and it was great to get that support from these other great norners (don’t worry, that won’t catch on).

1 Like
#64

For what it’s worth - I tested a bit this weekend and SuperCollider Quarks work well out-of-the-box on the norns. Any norns/dust package management would reasonably need to support Quarks dependencies anyway, so I wonder if it might make sense to simply piggyback on this existing system?

1 Like
#65

To answer an earlier question:
Quarks is SuperCollider’s package manager. It’s fairly simple - it uses a central repository with a map of project names and git repos. From there, getting info about or installing a Quark simply involves pointing to the git repository (which can be third party) and cloning. Quark installs do basic dependency resolution, though the system doesn’t yet support soft dependency specifications like <= 4.1, and doesn’t really do any advanced version resolution (these things are on the roadmap, but historically few quarks are really versioned very rigorously, mostly you just pull from head). Since it’s just cloning a git repo, a quark can contain any type of file (though binary data, audio files etc. would take a little work as putting them in git is a no-no - lfs is the right solution here).
The current quarks system has some holes (mainly around versioning), but I was able to ssh into my norns on Sunday and run Quarks.install("Foo"), and immediately install a quark and all it’s dependents with no problem.

A system for norns would look something like:

  1. Create a norns / dust central repo, e.g. monome/dust-quarks
  2. Either point the quarks install location to a place where lua will pick up .lua files, or point lua to the place where quarks are currently being installed (lua could use sclang’s include directory list, which is a simple json file)
  3. Add a simple two-way OSC interface between sclang and lua to run the basic Quarks command (listing, fetching info, install and uninstall), and expose it in the norns system menu (maiden might be another place to put the UI for this)

There’s probably some improvement work to do on the sclang side to e.g. specify that a quark is for norns / a specific sclang version, and some better recovery if you manage to bork your class library. And, likewise, some norns work to help manage files, keep things organized, and iterate on code that’s already on the system.

2 Likes
#66

i hear that and it’s one of my top priorities. it’s amazing how quickly this community generates content— the studies aren’t even done yet!

in the short term, i’d suggest using cyberduck or other FTP program for file sorting. it works well and it’s thorough. we’re just looking to do something more integrated into the system and more nicely designed.

(i should add this to the docs. it’s been mentioned elsewhere variously)

10 Likes
#67

just wondering. will some of the community made scripts get integrated on the next firmware update?

#68

those that get submitted to the dust repository, yes.

5 Likes
#69

PR to dust seemed like a really clean strategy for keeping script sharing organized and tidy. But now I find myself wondering if scripts need to be versioned along with norns releases, and also wondering if said releases will be months apart.

In order to keep script sharing vibrant, it would be nice to have a formal way to express what a script depends on (including version info) and thus allow script release schedules to be independent of norns release schedules.

I see that PRs to dust are getting merged promptly. I’d like to feel confident that I can go ahead and use those scripts with my current version of norns without breakage.

Should I throw caution to the wind? Am I being unnecessarily insecure about dependencies and potential breakage? I know some changes with Midi are afoot, how will that impact scripts?

Just trying to get a better handle on the vision for norns/dust release process and what expectations I should have for the pace of change and the infrastructure intended/desired/needed for supporting that pace.

I want to dive in but I’ve been holding back. I’m waiting for controller studies and I’m waiting for some kind of enlightenment about the issues I just raised. Possibly a few gentle words might unstick me from my perceived impasse.

I’m also happy to contribute to any package management or other efforts that might be underway to address this stuff that I may simply be unaware of.

4 Likes
#70

TL/DR: it should always be fine to pull dust/master and norns/master together.

maybe a little.

if a PR to dust or norns master is approved, it is because we are reasonably sure that it won’t break anything. experimental PRs are marked as do-not-merge. (consensus was that having a master ahead of update packages was overall better than having a seperate bleeding-edge branch and only merging to master when tagging and rolling an update.)

re: midi, what is happening is that a higher-level layer is being added with a responder pattern, but it shouldn’t invalidate the existing callback pattern or break scripts that use it.

next update will have some changes under the hood that make SC engines going forward incompatible with older system. so we will need to be a little careful and maybe time to create a develop branch.

this will not impact script compatibility, but if we do break a lua API then of course we will have to decide how to deal with all the existing scripts.

all that said, there is a sort of framework in there for lua module versioning. it hasn’t really come up yet, but if necessary we can make some sort of dependency structure. i worry that there is not enough core project management or dev time to make this really functional though. (every norns developer except brian is a volunteer. and brian is busy with other things as well…)

4 Likes
#71

Thanks @zebra! That’s really reassuring.

#72

yeah, this only works if you maintain, a ‘latest view’ principle (like arch linux) , which basically says if you want any updates you have to take them all.
in this case, you want new scripts from dust, you have to update your norns to the latest OS, regardless of if the script you want needs the new feature.
(anything else is potentially unsafe, esp, if you are not on the newest version of norn OS)

what I did with axoloti for the community and factory library, was to branch the libraries along with the release, this way, new additions could be added into either new or old releases (if old, we could merge them into new)
(the binary build also ‘knows’ what release it is, and so which branch of the library it should be using, this means we know any firmware dependancies are satisfied (a bit like norns/dust might have crone dependancies)

all works well, the only thing that gets complex, is because auto sync the community library and users are ‘unaware of git’ , when a new release happens, we can get into a situation where both new and old branch are being added to by users (as they are on different versions), this can get a little confusing for users.

but given norns is expecting users to be ‘git savvy’ this should not be an issue for norns.

#73

That’s not at all how I interpreted what @zebra said:

1 Like
#74

sorry, i should revise that. of course you should pull dust/master and norns/master together, because new scripts could require new features. i meant to underscore that so far there are no breaking API changes. if there are we will deal with them in an appropriate way, probably by requiring version checks on the scripts as you say.

1 Like
#75

OK. Not quite as reassuring then. It means that to try a new, WIP script, we also need to try a new, WIP system.

I suppose it’s simple enough to rollback if it proves to be unstable though. But it would be great to have looser coupling someday. I hope it’s a goal of some kind.

#76

For those of us that want to get the latest dust scripts, does that just mean running the regular update process even if the os hasn’t been updated?

As @jasonw22 says, what if we want the new scripts but the norns/master repo is mid update?

1 Like
#77

i guess all i’m saying is that we’re trying to be very careful that whenever anything is merged with dust/master, it is compatible with current norns/master. sometimes that means two separate PRs (which does bother me.)

2 Likes
#78

Cool. How stable is norns/master HEAD? is it usually in a playable state? Apologies for not doing my own research. Music time has been at a premium lately.

#79

And norns/master is always non-broken?

#80

yes, norns/master is always playable and non-broken to the best of our ability. hence the rather slow rate of PR approval on norns - this has caused some frustration in fact.

however, it is a bleeding-edge branch, not a RC branch, for exactly the reason that we want people to not be hung up on trying new stuff in between update packages.

PRs to norns requiring changes to dust should be few and far between. any such changes should of course be accompanied by a PR on dust.

also note that most contributions to dust don’t depend on changes to norns.

2 Likes
#81

Cool. Sounds like the rigor I was looking for in version/package management is instead being placed in merging of PRs. That works for me!