{"id":24614,"date":"2015-08-20T15:00:44","date_gmt":"2015-08-20T13:00:44","guid":{"rendered":"https:\/\/mamchenkov.net\/wordpress\/?p=24614"},"modified":"2015-08-20T15:03:21","modified_gmt":"2015-08-20T13:03:21","slug":"backup-your-data-unless-you-are-google","status":"publish","type":"post","link":"https:\/\/mamchenkov.net\/wordpress\/2015\/08\/20\/backup-your-data-unless-you-are-google\/","title":{"rendered":"Backup your data! Unless you are Google"},"content":{"rendered":"\n

BBC reports<\/a> that one of the Google data centers experienced a data loss, after a nearby power power facility was struck by lightnings four times in a row. \u00a0Only about 0.000001% of total disk space was permanently affected, it is said.<\/p>\n

A thing called “backup” immediately comes to mind. \u00a0This was something I had to deal with in pretty much every company I worked for as a sysadmin. \u00a0Backup your data or lose it, right?<\/p>\n

Well, maybe. \u00a0For most of those companies a dedicated storage or a couple of tape drives could easily solve the problem. \u00a0But Google is often special in one way or the other.<\/p>\n

A quick Google search (hehe, yup) for how much data Google stores, brings up this article<\/a> from last year, linking to this estimation approach<\/a> – there are no officially published numbers, so the estimate is all we can do: 10-15 exabytes. \u00a0(10-15 exabytes, Carl!<\/a>) \u00a0And that’s from the last year.<\/p>\n

Using this method, they determined that Google holds somewhere around 10-15 exabytes<\/a> of data. If you are in the majority of the population that doesn\u2019t know what an exabyte is, no worries. An exabyte equals 1 million terabytes, a figure that may be a bit easier to relate to.<\/p><\/blockquote>\n

Holy Molly, that’s a lot of data! \u00a0To back this up, you’ll need at least double the storage. \u00a0And some really lightning-fast (pan intended) technology. \u00a0Just to give you an idea, some of the fastest tape drives have a throughput of about 1 TB \/ hour and a native capacity of about 10 TB (have a look here<\/a>, for example). \u00a0The backup process will take about … forever to complete.<\/p>\n

So if tapes are out, then we are backing up onto another storage. \u00a0Having the storage in the same data center sort of defeats the purpose (see above regarding “lightning”). \u00a0Having a storage in another data center (or centers) means you’ll need some super fast networks.<\/p>\n

You could probably do quite a bit of optimization with incremental and differential backups<\/a>, but you’d still need quite a substantial infrastructure.<\/p>\n

Simpler, I guess, just spread your data across many data centers with several copies all over the place, and hope for the best.<\/p>\n

But that’s for Google. \u00a0For the rest of us, backup is still an option. \u00a0(Read some of these horror stories<\/a> if you are not convinced yet.)<\/p>\n

And since we are on the subject of backups, let me ask you this: how are you doing backups? \u00a0Are you still with tapes, or local NAS, or, maybe, something cloud-based? \u00a0Which software do you use? What’s your strategy?<\/p>\n

For me, dealing with mostly small setups, Amazon S3 with HashBackup<\/a> is sufficient enough. \u00a0I don’t even need to rotate the backups anymore. Just do a full daily.<\/p>\n\n","protected":false},"excerpt":{"rendered":"\n

BBC reports that one of the Google data centers experienced a data loss, after a nearby power power facility was struck by lightnings four times in a row. \u00a0Only about 0.000001% of total disk space was permanently affected, it is said. A thing called “backup” immediately comes to mind. \u00a0This was something I had to … Continue reading Backup your data! Unless you are Google<\/span><\/a><\/p>\n\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"Backup your data! Unless you are Google #Google #sysadmin #backup","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false,"_links_to":"","_links_to_target":""},"categories":[1,133,62],"tags":[251,2068,2289],"keyring_services":[],"class_list":["post-24614","post","type-post","status-publish","format-standard","hentry","category-general","category-sysadmin","category-technology","tag-backup","tag-google","tag-web-hosting"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t