#30

Maybe the new site should just be a wiki?

#31

I like that… wiki with pull requests sounds rather nice to avoid bot-posting and implementation of captcha’s. Still agree with @mateo’s sentiment that it should be a git repo

#32

Well, github supports wikis, so that content could be supported there.

1 Like
#33

Does lock us in specifically to github though. (Unless the github wikis are portable too?)

Edit, yeah looks like the github wikis are git-backed too. That’s cool. So we could maybe do editing / updates directly in github and have some skinned mirror of it running at music.dsp.systems?

#34

Yeah, Github wikis are just repositories themselves.

Yeah, it would be pretty simple to setup a static site generator that pulls the data from github and builds html files whenever there’s a merge to master. It could be hosted in Github itself (via pages) or another service like Netlify

1 Like
#35

I suppose since there seems to be enough interest to make this happen, and plans are starting to form, maybe we should let Bram know?

A database dump would be easier to work with to bootstrap this than a scraped clone of the site…

2 Likes
#36

20 characters of yes definitely!

1 Like
#37

I’ll send him a message today, but keep in mind this current discussion was spawned as a direct result of my initial conversation with Bram.

1 Like
#38

Right on, I’d be happy to get in touch too, just seemed like better to be up front even if he doesn’t want to participate or doesn’t have time to make a dump. Maybe just posting about it on the music-dsp list would be better. Wasn’t trying to drop this on you. :slight_smile:

1 Like
#39

No problem at all, he got back to be rather quickly and I’ll direct him to this topic

1 Like
#40

Gentlemen,

Let’s not scrape that makes no sense. I’m trying to get access again to the database, that should be a lot easier :wink: The server was moved from one machine to another and I get a sneaky suspicion that something scraping or spamming musicdsp.org is killing that machine.

I want to replace to replace the whole site with something static.

I do like the github idea, but contrary to what someone here said that, the wikis in github are NOT part of the repo. So, if musicdsp would be replaced with a wiki it would mean maintaining and cleaning up the wiki from abuse - blergh. So it would just have to be a plan repository with maybe markdown snippets?? Let’s see.

Maybe just plain old HTML with a bootstrap CSS should already be rather nice.

greetings,

Bram

6 Likes
#41

apologies for even throwing the scraping idea out there, if it has intensified the bandwidth problem. Glad that you’re still ‘about’ and that there is hope of recovering the site in a sensible way.

I’m doing some hobby synth coding again and feel like I progressed to a stage with it where (especially) the optimisation recipes I saw on wayback could come in really handy…

#42

Just to clarify, what I meant to say was that the wikis in github are repositories themselves but they are a seperate from the code.

I wouldn’t recommend putting the main content there, but I was under the impression that there was some content that would make more sense as a wiki…

#43

FWIW, I’ve only attempted to scrape from the archive.org snapshot never musicdsp.org – mostly because I haven’t managed to catch it when it’s been up long enough to try. It wasn’t me, I swear! :slight_smile:

Glad you’re involved @bdejong!

Storing all the content as flat files sounds great to me. Seems like we could stick a wiki interface on top of that for editing if that would help automate the process of maintaining the site – still restricted by at least the requirement of having a github account, if we use github, and maybe further by using a github organization? Spam should be pretty easy to avoid by just leaving the public editing option off:

By default, only people with write access to your repository can make changes to wikis, although you can allow everyone on GitHub to contribute to a wiki in a public repository.

I just cloned a random wiki to look, and the repo is just a directory full of markdown files. We could then use the main repo to host whatever frontend we use to publish the wiki as HTML/CSS.

Otherwise I suppose we’d just do updates via pull requests with someone(s) in charge of merging the submissions into the main repo – or just grant folks direct write access to the repo?

The wikis seem perfectly suited to what’s being described above, and I think would work well as a place to hold all the content for the site IMHO. I like to edit my markdown offline in a text editor, but it couldn’t hurt to have a wiki-style editor attached for free for those who prefer it.

1 Like
#44

Hope I didn’t kill the momentum, just posting here to say I still think this would be a really nice thing to work on.

#45

Hey all,

I finally managed to score the mysql dump and all the files on the site. The database consists of 99% spam. :+1: :man_facepalming: This also explains why the site is going down continuously.

My current idea is the following: I will create some python scripts that can dump the whole DB and everything to markdown files, more specifically using sphinx. Then dump all of this into github.
That way people who understand markdown, github and pull requests will be able to submit more notes (comments) or new recipes (code, …) as pull requests.That should be a damned good measure against spam :wink:

And what the hell, I might still add some ads ;-)))

Anyone have a better idea, let me know!

Bram

4 Likes
#46

Well, that’s not too bad…

mysql> select count(*) from archive;
+----------+
| count(*) |
+----------+
|      266 |
+----------+
1 row in set (0.00 sec)

mysql> select count(*) from archivecomments;
+----------+
| count(*) |
+----------+
|     1674 |
+----------+
1 row in set (0.00 sec)
3 Likes
#47

Thanks for the efforts on this!! Sincerely appreciated. :slight_smile:

#48

So, I’ve imported the SQL dumps and all the files into a private github repo. Is there anyone here who wants to help cleaning the dumps up? Basically, I need to go through 1600 comments, deleting the ones that are spam and the ones that are… useless. Anyone here volunteering to help? It’s still private because the dumps contain a lot of email addresses… :’(

If you want to help, just shoot me an email at bram.dejong at (ends with mail.com and starts with a g). I can only add 3 private collaborators, so if you volunteer, be serious about it :wink:

cheers,

Bram

5 Likes
#49

A suggestion: you could do a first pass using an API like aksimet and save a lot of time

1 Like