#1

Hello dear community!

I wanted to know how you all more seasoned and professional coders do manage and organise your code, snippets, projects and so on?

I have almost no experience in programming things but thought I’ll use my time in social distancing to finally learn Supercollider properly.
To put my endeavors on a solid basis I thought about documenting my code snippets and projects in some kind of database or maybe a personal wiki for future uses.

I have a very limited knowledge of using wiki software but I’d really like some kind of idiot-proof and quickly accessible software, preferably without having to switch between learning Supercollider and a markup language.

Even better would be the addition of media-organising,e.g. for sample files, DAW-projects, or pictures from handwritten notes.

In the past I’ve been using the macOS finder with its tag-system for organising projects, but right now I’m stuck with windows 10, and I have no clue if the explorer is suitable for something like this.

On Linux I’ve been also using Tomboy for organising notes for projects, which I can recommend, but it has its limits of course.
But a nice thing was, that it automatically linked “topics” to keywords in the notes.
It would be amazing to have something that automatically links “//comments” from the SC code.

I haven’t found a thread regarding this topic, so I’m hoping for a lively discussion.

Greetings, tilman.

7 Likes
#2

I fell like this kinda matches the way I use git / GitHub. usually that’s thought of as a code sharing / collaboration tool for open source projects, but I’ve found it pretty useful just for keeping my own code organized. sometimes I maintain branches for different variations of a project. and there’s the bonus that all your code is (")safe(") online. there’s also a tag system that may or may not be useful

2 Likes
#3

My professional coding organizes itself by necessity… as in, I do a lot of product maintenance and my tasks are organized in a bug/feature tracking system, so things just go into the existing structure :slight_smile:

Familiarity with the code base and searching for particular things I remember helps me find specifics of syntax and patterns if I want to reuse something. Though for more common items, I also keep a couple of big text files with snippets and reminders of where to find stuff. One of them is right in my main notes file where I also keep track of the different tasks I’m juggling and current debugging notes.

The bit of audio coding I did at home was relatively simple and self-contained, and there was no organization, only chaos :grin:

#4

I structure things like a sketchbook/playground using source control and a mono-repo. The idea is to get creative ideas out as quickly as possible; Minimize friction between idea and implementation. Each idea/session in the playground folder gets a subfolder, and I drop code in there. Some ideas don’t go anywhere, and some do. Eventually, some sketches graduate the playground and get turned into a composition or some re-usable bit of code. I tend to have flat directories full of things. What often happens is categories (and eventually useful folder structures) will organically emerge from the mess.

I rely on org-mode for notes for projects and snippets. I often will use a casual literate programming style to blur the two together.

I’m in the early stages of building software for myself that utilizes SQLite to organize things like code snippets and samples. We’ll see where that takes me.

5 Likes
#5

Thanks for the suggestions so far!

I found out, that you can add text and even pictures to items in Reaper, which seems useful to me.
Ok, the formatting of SC code gets lost a bit, but I was planning to always record my steps in SC to Reaper anyway.

That leads me to some observations I made of myself and me organising things:
I think, that documenting the code, samples and so on, will lead myself to commit more easily to final compositions, kinda paradoxically.

In the past I’ve been using Ableton live for basically everything, which led me for a “I’ll do it later” or “I must perfect everything” mentality and now I’m stuck with hundreds of ableton-projects, which sometimes don’t work anymore, because dependencies or samples are missing.

I guess documenting everything in plain txt/wavs in a preferably open source system leads to a better traceability but also to more commitment to “concrete” results.

Sometimes it’s also fun to leave stuff behing, to make a clean cut: I once had a native instruments Maschine but decided to sell it, but before selling it, I rendered just a few bars of the output sum of each project, which kind of freed me creatively, when I didn’t have to worry about changing things anymore.

Ok, these were just a few, quick thoughts about documentation/organising projects… :innocent:

#6

As @andrew already mentioned, I can also highly recommend learning some git basics and building a habit of commiting your code on a regular basis. It’s a very powerful tool and the amount of features can be a bit daunting at first, but you really don’t need that much to get started. The official git book is available online for free and will provide you with all the information you need.

I’m not sure if you are already familiar with version-control systems, but basically they allow you to take snapshots of your code to which you can return anytime you like. This comes in very handy if something breaks and you just want to revert to the last working version of your code. Branches are another very useful feature, they allow you to safely work on a copy of your codebase, which is great for trying out new ideas or different solutions for a task. You can anytime switch between branches and merge them into each other. This is one of the slightly more advanced features but was a huge game changer for me when I finally started using it.

Feel free to ask, if you have any questions :slight_smile:

2 Likes
#7

+1 to git. If you have a problem with Gut Hub (some people do :shrug:), there are also Gitlab and Bitbucket.

FWIW, I use Notable as a kind of personal wiki for work-related stuff. Very light weight, all local, driven by Markdown (so plain text, but can be as simple or complex as you want to make it). Also, Markdown is good for, um, “inverse literate programming” (I just made that up) - you can easily embed nicely-formatted code samples into your Markdown docs.

#8

Ok, sounds like there’s no way around learning Git Hub, thanks for the book-recommendation @Isky , I’ll try to read it first, before asking further questions.

Also @ground_state Notable is looking exactly like what I was searching for in the beginning, Thank you!

2 Likes
#9

It all depends on the scope of the project.

Most projects are small sketches, so I don’t bother with formal version control. Instead I just save off a new version every time I make some change, for instance _v8, _v9, etc.

With collaborations, formal version control is a must, as is some kind of issue tracking system. Git is needlessly complex, but it’s what everyone knows these days.

if I have a project that makes it past the ‘sketch’ stage; i.e. something that I now use regularly and wish to maintain, then I switch over to git. The only problem is that the first checked-in version is usually already quite mature.

For personal projects I use Atlassian/bitbucket, which has a reduced version of the Jira issue tracking system. All subsequent work is managed not only by version control, but by issue tracking.

With issue tracking very change to the code, including adding new features, is considered an ‘issue’. That is, there’s no difference between fixing a bug and adding new functionality – both are work items.

Of course, many times there’s just open experimentation, and the ‘issue’ description is determined or mostly filled in post hoc. This is rare to nonexistent in professional projects, but happens all the time with personal creative work.

The point is all code checkins become associated with a particular issue. Commit messages reference the issue#, and briefly discuss what code was changed.

In the spirit of open experimentation, a previously ‘resolved’ issue can be cloned and made to express something else – the new ideas or changes. So one can use the system either a more or a less disciplined manner.

Professional projects tend to be overly disciplined or thought out ‘in advance’ – the ‘agile’ strategies were supposed to prevent this but instead are widely misused as prods to make people work faster with less attention to detail and with less thoughtful reflection on what they are making; i.e. Zuckerberg’s ‘move fast and break things.’, or the general attitude of ‘shut up and code’, or even more broadly blind and destructive assumptions regarding ‘progress’, ‘changing the world’, ‘disruption’ and so on, which prevent critical reflection upon what

so the trick is to make use of professional tools while being mindful of the broader implications and pitfalls of such professionalization, and being especially mindful of late-capitalist ideological constructs literally encoded in such tools.

Anyway, I find issue tracking much more helpful in the day-to-day than the very rare cases I have to go back and retrieve an old version, it also helps streamline communication among multiple people, and it can be somewhat creatively ‘misused’ in ways that preserve the openness of creative work.

also, as I indicated at the beginning the bar for transitioning to formal version control/issue tracking is indeed quite high – when I make such a decision, I feel that the result has already crossed a threshold and become a ‘release’, ‘finished work’ etc. rather than just an exercise or exploring an idea.

2 Likes
#10

I think git is as complex as you want or need it to be. Yes, I still remember spending three hours with two colleagues fixing an incorrect and force-pushed rebase. But that’s advanced stuff that you can (and probably should) ignore as a beginner, especially if you’re the only person working on your project.

git init, git add . and git commit <message> are enough to get you started. And then you just learn the next command/technique whenever you need it.

That being said, I totally agree with the advice to keep things simple for non-professional projects. I don’t want to know how much time I’ve spent thinking about the perfect tooling instead of actually writing code.

And if git seems to complex or unnecessary for now, just leave it until you feel like it coud be of use :slight_smile:

3 Likes
#11

Git is great and quite ubiquitous version control system, but in the spirit of variety here are a few other systems I’ve personally used in the past.

SLorK manages all their code in a giant CVS/SVN mono-repository. CVS is old software that predates Git. There are few reasons to start a new CVS repository in 2020, but one of the advantages of using CVS is that you can check out individual elements of the repo rather than the whole code-base.

For my own personal work, I’ve been using Fossil. It’s kind of a nifty thing, and a welcome vacation from git. It has a few interesting things like a built-in web interface, integrated ticketing system, and a wiki. It’s built by the SQLite devs to manage SQLite, so it should be unsurprising to learn that the repository is managed as a SQLite database. I actually take advantage of this and will treat some Fossil-managed projects as more of an archive with history metadata. Since it’s also SQLite, it is also possible to write your own tables to the file, so I use this to store things like preset data and musical patterns.

2 Likes
#12

I actually prefer cvs/svn because of simplicity, but it’s difficult working with others because they will generally just want to use git. Same issue with the python language, there really seem to be no special or redeeming characteristics except everyone else seems to use it nowadays :wink:

For huge, truly commercial scale projects some of git’s added complexity would be helpful, but I don’t personally work at that scale.

1 Like
#13

I think one of the biggest advantages of git is the much easier and much more lightweight branching/merging workflow. But I don’t have much experience with SVN, so I can’t really make a fair comparison between the two…

And yes, “everyone is using it” really is no reliable indicator of quality. But in the case of git I think it’s not without reason, that it hase become so popular and almost completely replaced SVN at enterprise scale.

Anyways, I’m by no means a zealot about these things, in the end they’re all just tools and you have to figure out for yourself what works best for the task at hand. And it’s always worth taking a look at alternatives, even if it’s just for the joy of trying out something new :slight_smile:

#14

it is literally a far more complex piece of software than subversion/svn. but i know what you mean. simple usage of git has simple workflow and syntax. but there are also more “powerful” features, flip side of which is many ways to get in trouble and some kinds of trouble that are hard to get out of.

man, i haven’t used svn in a long time. but it’s true that it’s simpler in many ways: architecturally, syntactically &c. (with svn, each checkout is like a shallow clone in git, containing a complete snapshot but not the entire history.) i think branching is a pain point in svn.

re: usability, it’s telling that a lot of git users end up making aliases for common combinations of / arguments to git commands. (or rather, subset of git commands needed for particular workflow.) and of course the number of options makes it easy to footshoot (i still make “expensive” mistakes all the time whenever i depart from the “usual” sequence of steps.)

anyways on-topic, i also use local SCM and mono-repos to consolidate related bits of WIP or ideas, and emacs org-mode to do planning and any spreadsheet-like stuff. i have lots of private GH repos but also make backup snapshots on other servers because cloud services are what they are.

i am not the most diligent planner or documenter on private or “leisure” projects, (or even OSS heh sorry) and would like to do better.

pretty much every day-job i’ve worked uses atlassian suite for issue tracking and collaboration, in various ways. presently that means Bitbucket for SCM hosting and review tools, Jira for bug triage and milestone/issue tracking, and Confluence for more in-depth documentation of research and planning. (blog/article/wiki-like. i sort of hate Confluence, and think this is too many tools.)

i’m curious about literate-programming experiences / setups.

2 Likes
#15

I can certainly elaborate on my setup a bit.

As I mentioned before, I use org-mode + emacs to do literate programming. I make heavy use of the noweb style syntax. The built-in org-tangle is painfully slow and inefficient for anything more than a small script or config file, so I wrote my own portable tangler that does the job much much faster on a limited subset of org. Most of the time, I’m using it to generate C code from org files.

With higher-level interpreted languages like Scheme, I take advantage of emacs + org-babel a bit more and often don’t need to tangle code. I have things set up so that C-c C-c can send contents of an org codeblock to an emacs-spawned REPL to be evaluated. I have it configured to do noweb expansion, so often I’ll have org files structured with a “top” noweb code block that expands to all the code.

The org-mode code block evaluation has been so convenient for me, that I’ve begun writing my one-off “playground” code using a literate style. This is tricky sometimes because many of my playgrounds will wander and mutate in weird ways. When you’re playing programmer AND writer/editor, things can get very convoluted very quickly!

1 Like
#16

For “expert-mode users”: I use Vimwiki with markdown, which I love, but is command line only. It stores everything in a flat folder. I must have a few hundred pages in my end, but they’re organized into categories and nav in my vimwiki. I use git with a very simple alias that essentially git adds all , git commits with the date and git pushes to a private (free) Keybase git repo. If you use gitlab instead of Keybase you’d also have the ability to update your notes on your phone too.

As an aside: I have a daily code sketching practice all hosted on the same webpage / github page in a monorepo. I make a sequential numbered sub folder for each daily code sketch project. and each project usually starts with code from my previous project, then I make additions/alterations. These are stored here: Everyday code sketches and my github code repo is Here. Oftentimes I am working on something and need past code and can just jump to the right sub folder here and grab code out of it. I wrote extensively about my code sketching process here for anyone interested.

For the original poster: git is probably overly complicated for your needs. Pick the simplest and most intuitive system for yourself. You can change your system later as your needs grow.

2 Likes