#81

Cool. Sounds like the rigor I was looking for in version/package management is instead being placed in merging of PRs. That works for me!

#82

@zebra, any reason not to fork dust alongside the release? (say fork name = tag name of release)
then users could submit changes for the current release, and you then merge this released branch to dust/master when you release.

(your update script would then checkout the new release branch)

so currently…
dust would have a fork v1.0.1 (=norns current tag)
adding scripts- ‘normal’ users could submit PRs to 1.0.1 (*)
updating script - at any time, ‘normal users’ , could do a git pull on dust to get dust/v1.0.1 - this could even be a ‘menu option’ in norns.
(this can all happen ‘mid release cycle’)

on norns release…
change norns update script to use dust/v1.0.2 (i.e. git checkout v1.0.2)
release norns = v1.0.2
merge dust/v1.0.1 -> dust/master (you could do this periodically anyway)
create branch dust v1.0.2 off dust/master (so you get 1.0.1 changes and any dev changes in dust/master)

this similar to what we do on axoloti (but without the user contribution side )

(*) normal users, using released norns - devs using norns/master (or dev) would submit to dust/master as this is development.

an alternative approach (that we also do in axoloti) , is to allow users to use alternative repo in addition.
so rather than stuffing everything into dust, you make it easy for users to add other users repos…
this has the advantage that it removes the central management point (monome accepting PRs) and moves it to the developers of the scripts… this is particularly helpful if breaking api (or behavioural) changes are made, as it puts the original developer in a position to fix it rather than having to go via a middle man.

#83

i don’t have a problem with a version-branch flow in principle. (btw, s/fork/branch in your post makes it easier for me to grok.) and i understand the benefits. (you can do something similar with a develop branch and tags.)

whether or not this model is a good fit for norns / monome is not really my call, because it implies a certain role to fill in managing contributions now from potentially many branches. the benefit of the current system (such as it is) is that there is one point of contributions (or rather two, PRs to monome:norns/master and monome:dust/master) and anyone in the monome group can approve them.

i also think it’s convenient to be able to review all pending contributions on one GH page. (ok, two)

i also think we should be de-emphasizing “releases” since the schedule for them will be sporadic, and focus on a master branch that is always functional.

either way, i agree with the sentiment that this structure needs work and attention to encourage the health longevity of the open-source project. it also needs realistic expectations.

in that light, i’m going to go ahead put a bit of background here:

  • i am a “core dev” for norns due to having designed and implemented the first iteration of the architecture (C, SC and lua) way back in 2016 under contract with monome. (as i’ve contributed to various other monome projects over the years.)

  • i am not a monome employee. (as far as i know, there currently are no monome employees besides brian and kelli.)

  • in the months before hardware release brian brought in some more contributors (who are responsible for great work especially on maiden and linux) and i worked a bit on that push.

  • however, i have to apologize that the last two months i have had very little time to do norns development (and none at all to make music with it), playing catch-up very hard with professional and personal life.

so i want to apologize for anyone who has had PRs in limbo or been waiting patiently for updates, it’s partly on me.

to my mind, what is missing right now is a more solid understanding of release process, and a consistent schedule around reviewing PRs. any process we can come up with is going to involve some PM attention. there’s really no one who can do this right now besides @tehn. but i’m going to suggest that one good criteria is that it should allow approval / integration to be a distributed task within the monome dev group, and should minimize the role of an “integration czar.”

i’m sorry but i don’t really want to bikeshed this question much more. i realize it is important but i don’t wanna be the PM for norns, i want to focus my limited time on answering dev questions about the codebase i made, and on making my own contributions to that same codebase.

10 Likes
#84

Thanks for the time you have spent answering my questions! Certainly no expectation of any time-bound response from me. I understand the nature of volunteer projects.

To say this is bike-shedding, well, it probably is for you. But understanding this stuff is important to making myself feel comfortable making code contributions, and I imagine the same may be true for others. So, it’s potentially relevant to the vibrance of this community.

But I don’t want to overcook it! And I’m feeling a lot more confident about the thought process at work now. So, thanks again!

2 Likes
#85

sorry yes, didn’t mean to disparage (it is very important). just that i am going to step out of the conversation now. i’d be happy to support whatever contribution flow, but realistically am not going to be in charge of maintaining it.

2 Likes
#86

week-long trip ending, i apologize for my absence. many many posts coming tomorrow morning during the plane flight home.

10 Likes
#87

@jasonw22 It’s nothing scary :slight_smile:
Basically master of both repos should be stable and they should be updated in tandem. These updates can be done in larger increments (the releases that have been made so far) or in small increments by using git directly. The latter is effectively the same as how rolling release distro’s work (you update in small increments, but you do always update everything).

One thing we could consider is adding an option to the UI to update both repos through git to make it easier to get the current code.
There might be some technical challenges though like what to do with the running code if it changes.

2 Likes
#88

Some git shortcuts in maiden would certainly be helpful to those less familiar with git (and handy for those moments when you aren’t in a command line mood).

@simonvanderveldt I’m really quite experienced with software development, but not everybody here is. I try not to make assumptions about what people do or don’t know. Having spent a while hanging around the monome community, I know things can at times get quite messy, and that can become a frightening situation for the less confident. I wanted to understand what measures were being taken to allow people to feel safe to experiment, but also confident in the idea that their musical instrument will continue to function. I’m good now!

2 Likes
#89

The one and only time I will update (for the time being) is when I see an official release here. There are too many moving parts otherwise, and I do not have the time to become expert at judging how they all play together.

Thank you to all of you who make this platform all it can be.

2 Likes
#90

I can relate to this. But I’m getting the feeling from what @zebra said that PRs aren’t getting merged unless they’re non-breaking, so I’m feeling like I can play on the bleeding edge without obsessively tracking everything, because it sounds as though efforts are being made to keep the edge from getting too bloody. So, I’ll live dangerously until I regret it. And then I’ll roll back to an official release and adopt your strategy, which is totally valid and understandable.

1 Like
#91

i first want to apologize that project management has been one of my failures since release— the workload has been impressive and i was unsuccessful at quick assessment and delegation. i aim to remedy this immediately and delegate where i can’t keep up.

on the structural things as of now: master should always be stable. perhaps after some discussion we’ll consider using a dev branch, making pushes to master less often.

what’s most important is that the process is easy and encourages collaboration. (and of course, not seeing a mess encourages collaboration as well).

re: multiple repos. maiden should be able to stay on its own repo as there shouldn’t be breaking changes with norns.

dust however is not an ideal situation. we put together some proposals for having dust be libraries fundamentally (in which case these should possibly be just rolled into norns) and establishing a different method for sharing/updating user scripts (such as those currently in dust). i’ll type up this proposal in a git issue— it basically is a “project” structure where a script also can include data files (text/audio) and have a standard folder arrangement for presets/etc.

thanks for all of your patience while we gather momentum

15 Likes
#92

This sounds like a great idea for managing scripts/projects. One model that has worked well for me is how Node.js handles libraries. A library has a set structure, but resides it its own git repository (not linked to any core Node repo). There is a package manager that allows you to add libraries from a main library registry, or just add them from a git URL (or you can just put your own files in the right place).

In this way you can install whichever ones you want, remove them any time, and easily make your own that are distributed or private.

1 Like
#93

Agreed. nnnnnnnpm install thepartyvan#v0.1.1

2 Likes
#95

Anyone familiar with the text editor by github, Atom?

While I admit when I first started to think about my ideal interface to browse & install community provided scripts, I thought about command line interfaces (thinking Vundle for Vim specifically), reading this thread got me thinking more about a happy medium: something that would allow those of us who are not comfortable with CLIs to take advantage of the breadth of scripts made by the community without necessarily needing to ssh into norns itself… but something similarly that is more granular and allows a greater degree of individual user control than getting all the scripts all the time.

(Maybe it’s because I’m easily distracted? But the more loaded with functionality that I didn’t add a tool is the more likely I’ll just reach for something else in the first place.)

Atom has an awesome UI for managing packages. I wonder if we could add something similar to Maiden, but still fundamentally take advantage of git / github under the hood?

This is something I’d be interested in helping out with, if there was community consensus.

Either way, I will keep an eye out for the impending github-issue-proposal @tehn mentions above so as to contribute / follow along with the convo there.

2 Likes
#96

I had a parallel thought: that you could put a git repo or tarball URL into maiden and have it download and place the corresponding script somewhere logical, and have a button that lets you ‘refresh’ that script (by git pull or re-downloading the tarball if it’s changed), so sharing a script becomes a matter of sharing a URL and providing a new version of a script is just updating the content at that URL.

2 Likes
#97

Exactly. And, if we provided some sort of registration mechanism—this could even be just a list stored & updated in the Maiden repo that pointed to repos on GitHub that adhere to the package structure—we could provide a UI that prevented the end user from having to provide the URL.

#98

this is kind of what the Bela Web IDE does.
(Bela IDE is very similar to maiden really, allows running/compiling of ‘patches’ , editing, changes of preferences - and even an oscilloscope - could be a good source inspiration for Maiden)

each bela project is on the Bela board, and it can optionally be ‘backed’ by a GitHub repo, and it provides the primary git functions , like init/push/pull…

this works well, it opens up and additional workflow (compared to editing in the ide), where you can edit things in your favourite editor on your desktop/laptop, push them via git, and just pull/sync from the ide.
(they even have a script where, it detects files changing on the board, and then automatically recompiles, re-runs the patch)

im not sure that a ‘project’ per norn script is the right model,
as i said before, i think, better would be a list of sources for scripts.
so you could have dust as one, and another for your ‘private’ repo, and say another you are sharing with a friend as a collab.

git works well for this (although easily argued its a bit too ‘developer oriented’)

a package system (like deb, npm etc) is useful , if you start having dependencies between script packages… (e.g. mymlr:0.2 depends on dust:0.6) - this will happen if users start sharing libs/frameworks, and others build on them. then a package system is imperative… and you also need to support having multiple versions of packages installed (to avoid breakages)
… that all seems like a pretty big step from where norns is at the moment, but perhaps should be a goal?!

2 Likes
#99

i’m hesitant to take on creating a full-blown package manager for norns at this point.

but leveraging git more effectively seems like the best path, in addition to the ability to drop-in scripts from other sources.

also remember, while it’s not perfectly elegant, it’s possible to just copy a script (off lines or a gist etc) and then paste it into maiden and push run. of course this doesn’t work for engines at this point.

here’s the current thinking for the UI redesign, but i need to update it with further thoughts. comments highly welcomed as this is a pretty imperative user collaboration issue: https://github.com/monome/norns/issues/446

4 Likes
#100

fwiw @tehn I am not advocating a full-blown package manager, but I think we could gain a lot by providing a UI for managing what scripts are installed, and I think we could get some big gains with a relatively small amount of overhead.

I’ll put together a more thorough proposal as a github issue in maiden to discuss.

3 Likes
#101

@robbbiecloset this existing issue seems relevant.