[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]
On 19/07/13 20:25, Simon Avery wrote: > Bad Apple's a real diplomat, isn't he? Bless... > > One of life's lessons: Storage hates you. Hard drives hate you with a deep > and dark loathing. They only appear to store your data long enough to lull > you into a false sense of security so that when they do die (note when, not > if), you will be typically unprepared. > > What we do at work, (and also myself at home). I write this in the full > knowledge that BA or anyone else competent can pull it apart and find holes > in the strategy. But this is the thing - NO BACKUP STRATEGY IS FOOLPROOF > > IMO, backups should be fully automated. It's no good relying on weak humans > to remember. It should also be sophisticated enough to complain if > something goes wrong and alert you. > > And you should schedule human checking now and then to ensure all this > wonderful automation is actually working. That should include restoring > occasionally. > > So... > > We have a dedicated server that runs backuppc for the network (Smaller > geographic lans use the single server per site). > > Backuppc is one of those wonderful pieces of software that when you learn > to love it, you don't look elsewhere. Some might think it's heavy for > personal use, I don't. I back up my home desktop and two laptops to it. > > It'll run various types of backup and "pull" from the remote. Windows PCs > are dealt with reliably using the cygwin rsync service. (It'll do SMB too > (and NFS), but Vista onwards caused a whole bunch of permission problems > for me, while rsync just walks everywhere without fear). It'll keep > cyclical backups and manage them. > > It also backs up our servers. > > And you can reinstate individual files, whole directories or whatever using > the web interface on the server. Works well. > > And of course we do some usb-in-a-physical-safe backups for some really > important bits on top of that. Oh, and occasionally burn to DVD, although > longevity of this is very suspect. I also re-use old pata and sata HDD's in > a caddy for backups. Bit-rot affects everything, but IME these will last > longer that DVD or CD backups which degrade at an alarming rate. > > Also, we use backup-manager to backup debian servers to a local directory. > This can be on the same hdd (not useful if the hdd dies, but good for > accidental deletes), on another hdd on the same machine (psu might > potentially fry both hdd's. In practice fairly remote chance). Might be on > another server, but if it's nearby consider the building burning down. > > There are two types of people in disaster planning; > > 1. Those who really enjoy it. These are the dangerous ones. You'll have > great backups, but it'll cost a fortune as they predict riots, floods, > fire, tornados and nuclear war. > > 2. Those who get fed up and only predict basic things and don't check. > These are also the dangerous ones. They don't prepare enough. > > Oh, and don't forget cloud backups. The usual caveats about safety and > accessibility of your data when it lives on somebody else's computer, but > it is an option. > I completely endorse this email! Couldn't have written it any better myself, and it does pretty much chime with my methodology. Rsync is your friend for backups - the vast majority of *nix based systems ultimately use it under the covers. For windows you might want to look in VSS (volume shadow copy services) which is excellent, but really needs leveraging through frequently hideously expensive commercial backup suites. BackupPC is excellent, Bacula is similar and decent too. For Macs, just use TimeMachine and call it a day (well, kind of: that's what my DJ customer did - unfortunately I'm now currently in possession of his main data drive *and* the TimeMachine volume he was backing it up to). > Storage hates you. Hard drives hate you with a deep and dark loathing. This is 100% true. > NO BACKUP STRATEGY IS FOOLPROOF This is also 100% true - the problem is not necessarily our strategies, but that we always tend to underestimate the true power of fools. As the quote goes: "Don’t argue with idiots. They drag you down to their level and beat you with experience." So partly in response to Phil, but in general, here are The Rules of Backup: # You must always have two copies at minimum. So many times I have seen people move old data from their PC to their backup store (usually a single USB drive) and call it a day. THAT IS NOT A BACKUP. That is an archive. There is a huge difference. You must maintain identical copies of all your treasured data on at least two separate locations, at all times. Otherwise, if your backup USB dies, it takes your archived versions with it and you can't just retrieve the original from your PC. Multiple copies on multiple sources. This is the single most important rules of backups. # You must test your backups, preferably quite regularly. Otherwise, when the dreaded restore-for-real event happens, you will be screwed. This is the most common failing of backup strategies. # Keep separate indexes somewhere else: even a straight "ls -alhR / > mymassiveindex.txt" is better than nothing. Put in on 3 of your USB sticks or a CD. You could even just encrypt the file and email it to yourself for example. Hopefully you will, never, ever need it, but if you do (worst case scenario, you have a restore fail and need to start figuring out what you've potentially lost in the first place) having simple file lists to diff will save you countless hours. Trust me on this one. And that's pretty much it to be honest. I would add as a personal note: RAID is *not* a backup, it's an archive (see above for distinction). Once again, people will complain about cost, but if you want to archive all your stuff on a big RAID array - which is a splendid idea, and exactly what I do - you also need another even bigger RAID array to back that up to. Too expensive? Tough. Count yourself lucky you don't have to buy a tape library. As Simon pointed out bitrot is the enemy of all your data, and it really is the silent killer of your lolcat pictures and pizza recipes. I trust exactly one thing, and one thing only to deal with this: checksumming. This is almost impossible to correctly implement yourself, so don't bother with tripwire, home made shell scripts or whatever (I have done all of these in the past) because you're about as likely to successfully achieve this as reimplementing hard crypto yourself. Don't Do It. In a word, ZFS. BTRFS may be ready one day, but for now, anything that I really, really care about lives on a ZFS RAIDZ2 box in the garage with continuous end to end checksumming, COW, data validation, dedupe and pool scrubbing. What does it backup to? Another ZFS RAIDZ2 box in a colo. I also archive to tape frequently and they live in a friendly customer's roomy firesafe. I should maybe point out that 15 years in to my IT career, I have *NEVER* lost any of my data: not one single tiny bit of it. To answer Phil directly, SpiderOak are pretty cool actually but in light of recent PRISM revelations, I wouldn't take their claim to be unable to access their own data that seriously. Use any cloudy host you like the look of, but definitely encrypt it yourself before you upload to them. My advice on the most simple, most cost-effective backup strategy of all? Find a friend (this is definitely something the LUG could be great for organising) who is like-minded, and wants to properly backup. Both of you buy *two* external disks, in USB enclosures or whatever you need. Immediately both of you settle on the tool you like (backuppc is genuinely great, anything rsync related or possibly LVM/BTRFS/ZFS snapshot based can also work but is more complicated), practice with it a bit until you're totally confident in how to use it AND restore from it and then commence backing up to drive A. Meet with friend, swap your A drives and keep them at each other houses. Continue on B drives and when you next meet, swap your drives over again. Rinse, repeat. You now both have redundant, rotating backups that will save you from even complete disasters such as burglary and house fires. This solution scales very high with bigger groups of friends and has the advantage of providing an excuse for beer: "Sorry love, I'm out to the pub tonight - taking care of your backups!" Finally, use a cheap/free cloud service to upload the absolutely critical life-changing stuff to, fully encrypted of course. I mean financial stuff, business records, passport scans and so on. It won't be that big. In case of utter disaster (both you and your friends house somehow burn down on the same night) the absolute essentials are still safe. 95% of your data is likely your MP3 and film collections, and even if you lose them, using the index file you generated and saved as mentioned above, you have a list of lost stuff that you can then set about retrieving from Amazon, iTunes, TPB or whatever. It's really only your unique, personal stuff that's priceless and irreplaceable. Regards -- The Mailing List for the Devon & Cornwall LUG http://mailman.dclug.org.uk/listinfo/list FAQ: http://www.dcglug.org.uk/listfaq