Data management: backups and storage

[guillaume]

I don’t.

The last few years have been a struggle for me to shed my tendency to hoard things - both physical (eurorack modules, records, books, …) and digital.

For digital stuff, that means that I have a Dropbox with the stuff that absolutely matters, which has to be <1 TB (max Dropbox size). Of course, when I am working on a project, I might have much more than 1TB of files - but once the project reaches its end, I delete pretty much everything except for the deliverables/documentation of the project.

A nice side effect of that is that all my digital devices could burst into flames right now - all I have to do is go to the Apple store, buy a new MacBook, install Dropbox and i’m ready to go. That’s been a nice peace of mind of its own.

Does that mean there are times where I find myself wishing I had some source code or source video from years ago? Sure. Do I believe that these small, rare frustrations are worth not having to worry about backing up everything, worrying about a RAID server, etc? Certainly. And not having a file from 5 years ago has never been a show stopper (besides, when I used to archive everything obsessively, there was still the frustration that many years later the software was incompatible with the data files, and then you’re trying to archive systems rather than data which is its own maddening process…)

Nothing is permanent, and trying to fight that is just too much stress and overhead for me.

[abalone]

important discussion
digital data has to exist
in more than one location
or it doesn’t really exist

pro tip:
'a master and two back-ups

[Random]

I have a daily-to-weekly on-site backup routine (using Time Machine).

Every four months (end of term) I also back up to an offsite copy - stays at the office for home machines, stays at home for office machines.

Every year I do a careful ‘stuff that really matters’ archive, manually, to another offsite in case of systematic corruption e.g. by Time Machine.

Am I paranoid? Probably not enough. I’ve had complete system fails before. Both my work and many of my hobbies are on my machines, so…

[mateo]

So here’s what I’m thinking: All the important stuff on the NAS, which is set for mirroring, along with a drive attached to it for backups and an additional backup to a cloud service. Then, a copy of my samples on a direct attached drive, and I’ll keep time machine and dropbox for my Mac, which should cover all the bases!

[disquiet]

Wanted to mention something I hadn’t earlier. I use the schedule feature on Synology so that my Dropbox is sync’d only between 1am and 6am. That way the process doesn’t tie up my Internet connection when I’m awake.

[felipevareschi]

I am currently using a western digital EX4 but I will probably upgrade to Synology since the stability is much better, also those new Seagate helium drives are super cheap for the capacity (10TB enterprise-grade drives…).

As an additional layer you could use something like Amazon snowball or glacier to backup your entire NAS once every 3 months or so to the cloud, so in case of a disaster, you can get everything back.

I’d say if you are only storing music and other general stuff a NAS with a good network switch is a good option, for anything more resource intensive or if you need gigantic storage build your own monster with 45 drive inclosures, you can do crazy stuff like NVME SSD cache and petabyte-level storage with Limetech Unraid.

[geoffroy]

I’d like to buy a 3 or 4 TB NAS soon. Any recommandation in 2020 ?
It looks like the Western Digital My Cloud EX2 Ultra would be OK for my storage needs.
If someone has it, is it quiet ? Can it make a backup from time to time to Amazon Glacier ?

[duckpow]

Had an older model of MyCloud series which I won’t recommend… The newer models might be different but for the older models they are using a custom version of the ext4 filesystem which is fine as long as the control board works.
It is very tough to get any data out of the HDD if the control board dies. Even from a Linux machine… Ended up recovering at most 50% of my files.

[rycolos]

I looked hard at one of those as well as a Synology. I ended up buying a 2014 Mac Mini and hanging two 8tb externals off of it. Not only is it working great as a NAS and handling my time machine backups like a champ, it’s also running as a print, plex, and file server. Very happy I went this route.

[geoffroy]

Thanks for your answer !

[andrew]

anybody have advice for a more DIY web storage approach ? want something to use as a web backup for a hard drive but also as a private file-sharing server for friends and for weird PHP/node.js stuff with said files

It would probably be cheap to set up an AWS server but slightly worried about maybe having to take care of security myself + maybe accidentally locking myself out. (which I did on digital ocean )

but then again their service is pretty massive enough that I can probably find what I need + I can be good about making hard backups. idk! will investigate myself