Data management: backups and storage

[disquiet]

Weird, I do the same thing: Dropbox 1TB for current stuff, and Synology for the vast backlog.

[shellfritsch]

yeah NAS is good for mass storage and backup. i keep a huge library of sound FX on it for my film stuff, and pull files as needed. but when i’m editing it all goes on a usb 3 ssd.

[dansimco]

My approach used to be a Dropbox kind of setup but I ended up getting bogged down with syncing so took the lesser (to me) evil of using an external hdd just for music, which travels with me between locations. I mainly keep project storage and libraries on there but individual machines have their own vst setups and libraries so if I forget the drive I can still get something down. I use backblaze on my desktop which makes an incremental backup whenever the drive is plugged in. Apart from eating up a USB port it’s working well so far.

[abalone]

digital data has to exist in at least two places at once, or it doesn’t really exist

pro tip from twenty years ago…
'a master and two back-ups

[philmaguire]

I have a Time Machine backup of my laptop drive, and a couple of other drives with photography work, music sketches, and part of my music collection. I’d like some way of backing up all of this, but not sure what would be best. Someone suggested a NAS drive to me before, but that seems like overkill in some ways (also £££). What do you use?

[ermina]

daily or so: a simple 1To usb hd that stays in a bookshelf at home. I carry my laptop so this is really to preveny any unfortunate loss of the unit or failure of the ssd.

monthly: i sync that usb disc to another one. then one of the discs moves with me to the office and gets synced to a NAS. This NAS also has a periodical automatic “offsite” backup (another NAS in a technical room across the street).

I also have lots of “old” and badly duplicated data on various hdds, dvds, flash drives, so it’s still a mess

[declutter]

The dropbox 1TB plan is my main live backup; with occasional superduper clones of my hard drive which I could plug in and run in the event of a lost laptop or catastrophic ssd failure.

[iain]

I don’t. 20 chars of YOLO!

[x2mirko]

I use a cloud service (Backblaze) for backups. Currently I have about 6TB backed up. The initial backup obviously takes a while depending on the size of your hard drives (it was about half a year for me ), but after that, it just constantly backs up new things and changes in the background and everything’s always stored remotely. In case of data loss, you can either download it or pay them to ship you a hard drive with all your data on it.

This solution also has the advantage of having access to all your files where ever you go, as you can access them through a web interface. Sharing links to specific files is also possible. I’m very happy with the solution. There’s a monthly payment connected with it, but I wouldn’t be paying much less if i was just buying external hard drives from time to time and that wouldn’t have the additional benefits.

[razzkazz]

I have 2 identical Toshiba Hds (2 TB) in separate HD caddies. I use a software called Carbon Copy Cloner. You can configure the software to copy in various ways but I use it in a simple way. 1 HD is my working drive the other is a clone. As soon as I plug in the clone the software recognises it, scans for new/updated files and copies them to the clone. When these drives are full I will put 2 new drives in the caddies.

[infovore]

NAS drives aren’t really a valid backup plan - they’re more like a networked version of the other external drives you currently have. However, because they’re always-on-tiny-servers, they enable a few other useful things, notably, backing themselves up to cloud storage.

So, having just overhauled my own backup strategy, I can share:

• I have Dropbox but I mainly use it for collaboration rather than storage.
• I have a Time Machine clone of my laptop that gets done about… once a week. Not quite bootable, but good enough.
• I have Arq running backups of everything in my home folder but iTunes off that machine to Backblaze B2 storage, fairly all-the-time. So that’s offsite versions of key things.
• I also have a 1TB usb drive that, like you, I don’t backup yet. It’s about ten years old and this just feels like a massive liability for all my offloaded photographs, and probably offloaded music stuff too. So my plan is to replace that with a big NAS with a pair of disks mirroring each other (RAID1). RAID is not backup, but it’s a guard against mechanical failure. And then I’m going to let the NAS software also back that up to Backblaze B2.

Coupled with: a lot of my life is online (GMail) or code (Github) and can be recovered somehow via those means.

That covers multiple locations and strategies for all my key stuff, and I think that’s a good start. (Of course, Gary Bernhardt would point out at this point that if you haven’t tried to restore from one of your backups, it can’t really be counted as backup, but I’m a trusting sort).

[apeirophobe]

TimeMachine backups for my laptops, iMac and USB/Firewire drives. TM has saved my ass several times over the past several years but I would still like some kind of online backup solution.

[guillaume]

I don’t.

The last few years have been a struggle for me to shed my tendency to hoard things - both physical (eurorack modules, records, books, …) and digital.

For digital stuff, that means that I have a Dropbox with the stuff that absolutely matters, which has to be <1 TB (max Dropbox size). Of course, when I am working on a project, I might have much more than 1TB of files - but once the project reaches its end, I delete pretty much everything except for the deliverables/documentation of the project.

A nice side effect of that is that all my digital devices could burst into flames right now - all I have to do is go to the Apple store, buy a new MacBook, install Dropbox and i’m ready to go. That’s been a nice peace of mind of its own.

Does that mean there are times where I find myself wishing I had some source code or source video from years ago? Sure. Do I believe that these small, rare frustrations are worth not having to worry about backing up everything, worrying about a RAID server, etc? Certainly. And not having a file from 5 years ago has never been a show stopper (besides, when I used to archive everything obsessively, there was still the frustration that many years later the software was incompatible with the data files, and then you’re trying to archive systems rather than data which is its own maddening process…)

Nothing is permanent, and trying to fight that is just too much stress and overhead for me.

[abalone]

important discussion
digital data has to exist
in more than one location
or it doesn’t really exist

pro tip:
'a master and two back-ups

[Random]

I have a daily-to-weekly on-site backup routine (using Time Machine).

Every four months (end of term) I also back up to an offsite copy - stays at the office for home machines, stays at home for office machines.

Every year I do a careful ‘stuff that really matters’ archive, manually, to another offsite in case of systematic corruption e.g. by Time Machine.

Am I paranoid? Probably not enough. I’ve had complete system fails before. Both my work and many of my hobbies are on my machines, so…

[mateo]

So here’s what I’m thinking: All the important stuff on the NAS, which is set for mirroring, along with a drive attached to it for backups and an additional backup to a cloud service. Then, a copy of my samples on a direct attached drive, and I’ll keep time machine and dropbox for my Mac, which should cover all the bases!

[disquiet]

Wanted to mention something I hadn’t earlier. I use the schedule feature on Synology so that my Dropbox is sync’d only between 1am and 6am. That way the process doesn’t tie up my Internet connection when I’m awake.

[felipevareschi]

I am currently using a western digital EX4 but I will probably upgrade to Synology since the stability is much better, also those new Seagate helium drives are super cheap for the capacity (10TB enterprise-grade drives…).

As an additional layer you could use something like Amazon snowball or glacier to backup your entire NAS once every 3 months or so to the cloud, so in case of a disaster, you can get everything back.

I’d say if you are only storing music and other general stuff a NAS with a good network switch is a good option, for anything more resource intensive or if you need gigantic storage build your own monster with 45 drive inclosures, you can do crazy stuff like NVME SSD cache and petabyte-level storage with Limetech Unraid.

[geoffroy]

I’d like to buy a 3 or 4 TB NAS soon. Any recommandation in 2020 ?
It looks like the Western Digital My Cloud EX2 Ultra would be OK for my storage needs.
If someone has it, is it quiet ? Can it make a backup from time to time to Amazon Glacier ?

[duckpow]

Had an older model of MyCloud series which I won’t recommend… The newer models might be different but for the older models they are using a custom version of the ext4 filesystem which is fine as long as the control board works.
It is very tough to get any data out of the HDD if the control board dies. Even from a Linux machine… Ended up recovering at most 50% of my files.