Backups: My "uh oh" moment and what I do now for prevention and recovery

End users generally think, "I've saved the document, I'm all good". They put it into the computer and it'll always be there, ready for when they need it.

My "uh oh" wake up moment with regards to data integrity and backups was a long time ago now. It all started with an Athlon 1.4 system I built for myself. I also built a similarly equipped unit for my parents. Both from parts I purchased at a computer "swap meet" (very rarely these days involving any actual swapping). They both consisted of an MSI motherboard, the AMD CPU, RAM, Sony CD Burner, an ASmart Nvidia based AGP4x card, Creative SoundBlaster Live (which would later fail and make everything sound like cartoon chipmunks), a cheap PSU that came with the case ("Hairong"??), Microsoft Windows 2000 Professional and an IBM hard disk - a 40GB DeskDeathstar (I can sense some of you out there groaning now).

Previously to this, I'd used several machines with several types of hard disk (Maxtor, Seagate, WD, Fujitsu etc.) and never had a problem. I hadn't yet suffered a drive failure and so never really considered it would be a problem.

Well, I copped the Deathstar curse.. bad!


When I installed Windows 2000 I chose to employ Windows' encryption. I mean, what self respecting person doesn't encrypt their files (so I naively thought at the time anyway)? What Windows 2000 didn't readily highlight at the time was the need to back up one's decryption keys, so which part of the Deathstar drive do you think failed first? That's right.. the decryption keys! I lost a LOT of data.. but it was a good lesson to learn.

I now tell people

• Your important data should always be backed up
  
• Think beyond tax paperwork
  Your important data doesn't only include your accounting. Photos, videos - a lot of irreplaceable stuff is deserving of backup
  
• Think beyond your computer
  Do you still keep a paper phonebook? If your phone crashed tomorrow, would you be able to salvage all your contacts? Your appointments? Your music? Family photos? Videos?
  
• RAID is not a backup, although it may help you handle a failure before you have to bring in backups
  
• Backups should always be verified after being performed
  
• You should regularly test restoring data from backups
  
• Preferably always, but especially if a backup is being stored on rewritable media (E.g. tape, disk or solid state) there should be at least two backups, such that one can remain away from the online storage at all times (otherwise if you bring the backup medium to the online storage location and it burns to the ground or the computer suffers a catastrophic PSU failure etc. you'll have lost your backup and your online copy in one hit)
  
• Archival is not backup. If you delete important files to free up hard disk space, then you need to make sure that there's the main archived copy + another in case the archive copy fails. Lesser use doesn't guarantee an archive copy won't fail, just that it's less likely.
  
• Never label optical media with markers not specifically fit for the purpose
  
• Backup media don't like extremes of UV, temperature, humidity or shock so keep them in a dark, cool, dry, stable, secure place.
  
• Snapshot utilities aren't backup if they gradually "drop off" files. If by some accident a file is deleted from your working copy and you don't notice it, eventually as more data ends up in the system, the deleted file will be deleted and you'll be none the wiser.

These days, the amount of data we all generate in our daily lives is increasing rapidly. E.g.

• Identification
  Access credentials, contact lists, account details
  
• Interaction
  Emails, associated documents
  
• Photos and Videos
  Point and shoot digital cameras are capable of cranking out 12 megapixels and higher with full HD (1080p) video.
  
• Electronic documentation
  Utility bills, banking statements

With an increase in primary online storage capacity to cope, so too must backup storage capability increase. The first home computer I used, a Tandy TRS-80, had around about 125kiB storage capacity per side of a 5.25" disk (want to use both sides of the disk? Take it out and flip it over!). I can now take photos on my mobile phone that are bigger than that.

Where to now with backup media?

• Floppy disks are all but dead now. I don't know if you can even buy them anymore
  
• Optical media capacity seems to jump every so often and then double inbetween the major increments by adding layers, but can be slow to access and very sensitive to scratches and solvents used in marker pens - I think it was silly when the industry discarded the idea of caddies, I observed first hand in a school environment how safely they kept the discs. It's really heading out now. I reckon Blu-ray will be the last of the widely available optical formats.
  
• Solid state storage (USB + others) is getting faster but still some people are concerned about write cycles and then there's some issues of speed as they age. I'm not entirely convinced about them as a long term storage media. That said I think it's important to note that not all solid state storage is created equal. A cheap USB stick is exactly like buying a cheap floppy disk. It may fail.. quickly.. and without warning.
  
• Linear magnetic media - Tape
  Tape was keeping up reasonably well in a capacity sense, but on a cost per GB, it's still very very high - well out of the reach of a lot of places. If stored correctly, in climate controlled environment, it can last a long time, but it's nowhere near as fast as...
  
• Hard disks
  Once upon a time I'd advocated upon a "backup to a different type of media" approach, so that if online storage was a hard disk, you'd use tape or disk. But I think there's merit in "disk to disks" backup, if managed appropriately. As data densities increase, however, the likelihood of failure increases. There are drives available now where the recommended duty cycles are such that
  
• Online
  This is kind of a "meta media" in that we know, in the end, the data's probably going to be stored on traditional, multi-plattered hard disks in some farm somewhere. The advantage here, is on a rented basis, you outsource the hassle of having to manage the multiple copies, the separate geographical areas etc. The downside is that you're letting the data out of your sight (you might want to encrypt it first if it's sensitive) and if you're frequently generating large volumes, in Australia it's going to eat into your internet bill unless you have a suitable internet plan.

In that last point, I touched on another issue with backups - encryption. When you send your data off for commodity storage and processing, how secure is it? Which jurisdictions has it travelled through? Is your information commercial in confidence? Might it be leveraged for identity theft? When storing data elsewhere, consider how valuable it is, and whether or not it should be encrypted before it leaves your network. This may not be a consideration for you, but think of it this way. Would you be comfortable with the contents of your backups being posted online publicly? If the answer is no, then I recommend that you consider encryption prior to upload. This does, however, mean that you also need to keep a separate, secure backup of your encryption keys.