I wanted to know how you all more seasoned and professional coders do manage and organise your code, snippets, projects and so on?
I have almost no experience in programming things but thought I’ll use my time in social distancing to finally learn Supercollider properly.
To put my endeavors on a solid basis I thought about documenting my code snippets and projects in some kind of database or maybe a personal wiki for future uses.
I have a very limited knowledge of using wiki software but I’d really like some kind of idiot-proof and quickly accessible software, preferably without having to switch between learning Supercollider and a markup language.
Even better would be the addition of media-organising,e.g. for sample files, DAW-projects, or pictures from handwritten notes.
In the past I’ve been using the macOS finder with its tag-system for organising projects, but right now I’m stuck with windows 10, and I have no clue if the explorer is suitable for something like this.
On Linux I’ve been also using Tomboy for organising notes for projects, which I can recommend, but it has its limits of course.
But a nice thing was, that it automatically linked “topics” to keywords in the notes.
It would be amazing to have something that automatically links “//comments” from the SC code.
I haven’t found a thread regarding this topic, so I’m hoping for a lively discussion.
I fell like this kinda matches the way I use git / GitHub. usually that’s thought of as a code sharing / collaboration tool for open source projects, but I’ve found it pretty useful just for keeping my own code organized. sometimes I maintain branches for different variations of a project. and there’s the bonus that all your code is (")safe(") online. there’s also a tag system that may or may not be useful
My professional coding organizes itself by necessity… as in, I do a lot of product maintenance and my tasks are organized in a bug/feature tracking system, so things just go into the existing structure
Familiarity with the code base and searching for particular things I remember helps me find specifics of syntax and patterns if I want to reuse something. Though for more common items, I also keep a couple of big text files with snippets and reminders of where to find stuff. One of them is right in my main notes file where I also keep track of the different tasks I’m juggling and current debugging notes.
The bit of audio coding I did at home was relatively simple and self-contained, and there was no organization, only chaos
I structure things like a sketchbook/playground using source control and a mono-repo. The idea is to get creative ideas out as quickly as possible; Minimize friction between idea and implementation. Each idea/session in the playground folder gets a subfolder, and I drop code in there. Some ideas don’t go anywhere, and some do. Eventually, some sketches graduate the playground and get turned into a composition or some re-usable bit of code. I tend to have flat directories full of things. What often happens is categories (and eventually useful folder structures) will organically emerge from the mess.
I rely on org-mode for notes for projects and snippets. I often will use a casual literate programming style to blur the two together.
I’m in the early stages of building software for myself that utilizes SQLite to organize things like code snippets and samples. We’ll see where that takes me.
I found out, that you can add text and even pictures to items in Reaper, which seems useful to me.
Ok, the formatting of SC code gets lost a bit, but I was planning to always record my steps in SC to Reaper anyway.
That leads me to some observations I made of myself and me organising things:
I think, that documenting the code, samples and so on, will lead myself to commit more easily to final compositions, kinda paradoxically.
In the past I’ve been using Ableton live for basically everything, which led me for a “I’ll do it later” or “I must perfect everything” mentality and now I’m stuck with hundreds of ableton-projects, which sometimes don’t work anymore, because dependencies or samples are missing.
I guess documenting everything in plain txt/wavs in a preferably open source system leads to a better traceability but also to more commitment to “concrete” results.
Sometimes it’s also fun to leave stuff behing, to make a clean cut: I once had a native instruments Maschine but decided to sell it, but before selling it, I rendered just a few bars of the output sum of each project, which kind of freed me creatively, when I didn’t have to worry about changing things anymore.
Ok, these were just a few, quick thoughts about documentation/organising projects…
As @andrew already mentioned, I can also highly recommend learning some git basics and building a habit of commiting your code on a regular basis. It’s a very powerful tool and the amount of features can be a bit daunting at first, but you really don’t need that much to get started. The official git book is available online for free and will provide you with all the information you need.
I’m not sure if you are already familiar with version-control systems, but basically they allow you to take snapshots of your code to which you can return anytime you like. This comes in very handy if something breaks and you just want to revert to the last working version of your code. Branches are another very useful feature, they allow you to safely work on a copy of your codebase, which is great for trying out new ideas or different solutions for a task. You can anytime switch between branches and merge them into each other. This is one of the slightly more advanced features but was a huge game changer for me when I finally started using it.
+1 to git. If you have a problem with Gut Hub (some people do :shrug:), there are also Gitlab and Bitbucket.
FWIW, I use Notable as a kind of personal wiki for work-related stuff. Very light weight, all local, driven by Markdown (so plain text, but can be as simple or complex as you want to make it). Also, Markdown is good for, um, “inverse literate programming” (I just made that up) - you can easily embed nicely-formatted code samples into your Markdown docs.
Ok, sounds like there’s no way around learning Git Hub, thanks for the book-recommendation @Isky , I’ll try to read it first, before asking further questions.
Also @ground_state Notable is looking exactly like what I was searching for in the beginning, Thank you!
I think git is as complex as you want or need it to be. Yes, I still remember spending three hours with two colleagues fixing an incorrect and force-pushed rebase. But that’s advanced stuff that you can (and probably should) ignore as a beginner, especially if you’re the only person working on your project.
git init, git add . and git commit <message> are enough to get you started. And then you just learn the next command/technique whenever you need it.
That being said, I totally agree with the advice to keep things simple for non-professional projects. I don’t want to know how much time I’ve spent thinking about the perfect tooling instead of actually writing code.
And if git seems to complex or unnecessary for now, just leave it until you feel like it coud be of use
Git is great and quite ubiquitous version control system, but in the spirit of variety here are a few other systems I’ve personally used in the past.
SLorK manages all their code in a giant CVS/SVN mono-repository. CVS is old software that predates Git. There are few reasons to start a new CVS repository in 2020, but one of the advantages of using CVS is that you can check out individual elements of the repo rather than the whole code-base.
For my own personal work, I’ve been using Fossil. It’s kind of a nifty thing, and a welcome vacation from git. It has a few interesting things like a built-in web interface, integrated ticketing system, and a wiki. It’s built by the SQLite devs to manage SQLite, so it should be unsurprising to learn that the repository is managed as a SQLite database. I actually take advantage of this and will treat some Fossil-managed projects as more of an archive with history metadata. Since it’s also SQLite, it is also possible to write your own tables to the file, so I use this to store things like preset data and musical patterns.
I think one of the biggest advantages of git is the much easier and much more lightweight branching/merging workflow. But I don’t have much experience with SVN, so I can’t really make a fair comparison between the two…
And yes, “everyone is using it” really is no reliable indicator of quality. But in the case of git I think it’s not without reason, that it hase become so popular and almost completely replaced SVN at enterprise scale.
Anyways, I’m by no means a zealot about these things, in the end they’re all just tools and you have to figure out for yourself what works best for the task at hand. And it’s always worth taking a look at alternatives, even if it’s just for the joy of trying out something new
it is literally a far more complex piece of software than subversion/svn. but i know what you mean. simple usage of git has simple workflow and syntax. but there are also more “powerful” features, flip side of which is many ways to get in trouble and some kinds of trouble that are hard to get out of.
man, i haven’t used svn in a long time. but it’s true that it’s simpler in many ways: architecturally, syntactically &c. (with svn, each checkout is like a shallow clone in git, containing a complete snapshot but not the entire history.) i think branching is a pain point in svn.
re: usability, it’s telling that a lot of git users end up making aliases for common combinations of / arguments to git commands. (or rather, subset of git commands needed for particular workflow.) and of course the number of options makes it easy to footshoot (i still make “expensive” mistakes all the time whenever i depart from the “usual” sequence of steps.)
anyways on-topic, i also use local SCM and mono-repos to consolidate related bits of WIP or ideas, and emacs org-mode to do planning and any spreadsheet-like stuff. i have lots of private GH repos but also make backup snapshots on other servers because cloud services are what they are.
i am not the most diligent planner or documenter on private or “leisure” projects, (or even OSS heh sorry) and would like to do better.
pretty much every day-job i’ve worked uses atlassian suite for issue tracking and collaboration, in various ways. presently that means Bitbucket for SCM hosting and review tools, Jira for bug triage and milestone/issue tracking, and Confluence for more in-depth documentation of research and planning. (blog/article/wiki-like. i sort of hate Confluence, and think this is too many tools.)
i’m curious about literate-programming experiences / setups.
As I mentioned before, I use org-mode + emacs to do literate programming. I make heavy use of the noweb style syntax. The built-in org-tangle is painfully slow and inefficient for anything more than a small script or config file, so I wrote my own portable tangler that does the job much much faster on a limited subset of org. Most of the time, I’m using it to generate C code from org files.
With higher-level interpreted languages like Scheme, I take advantage of emacs + org-babel a bit more and often don’t need to tangle code. I have things set up so that C-c C-c can send contents of an org codeblock to an emacs-spawned REPL to be evaluated. I have it configured to do noweb expansion, so often I’ll have org files structured with a “top” noweb code block that expands to all the code.
The org-mode code block evaluation has been so convenient for me, that I’ve begun writing my one-off “playground” code using a literate style. This is tricky sometimes because many of my playgrounds will wander and mutate in weird ways. When you’re playing programmer AND writer/editor, things can get very convoluted very quickly!
For “expert-mode users”: I use Vimwiki with markdown, which I love, but is command line only. It stores everything in a flat folder. I must have a few hundred pages in my end, but they’re organized into categories and nav in my vimwiki. I use git with a very simple alias that essentially git adds all , git commits with the date and git pushes to a private (free) Keybase git repo. If you use gitlab instead of Keybase you’d also have the ability to update your notes on your phone too.
As an aside: I have a daily code sketching practice all hosted on the same webpage / github page in a monorepo. I make a sequential numbered sub folder for each daily code sketch project. and each project usually starts with code from my previous project, then I make additions/alterations. These are stored here: Everyday code sketches and my github code repo is Here. Oftentimes I am working on something and need past code and can just jump to the right sub folder here and grab code out of it. I wrote extensively about my code sketching process here for anyone interested.
For the original poster: git is probably overly complicated for your needs. Pick the simplest and most intuitive system for yourself. You can change your system later as your needs grow.
