{"id":11574,"date":"2009-06-13T18:09:12","date_gmt":"2009-06-13T16:09:12","guid":{"rendered":"https:\/\/mamchenkov.net\/blog\/?p=3"},"modified":"2009-06-13T18:09:12","modified_gmt":"2009-06-13T16:09:12","slug":"what-happened","status":"publish","type":"post","link":"https:\/\/mamchenkov.net\/wordpress\/2009\/06\/13\/what-happened\/","title":{"rendered":"What happened?"},"content":{"rendered":"\n

The long story short : I lost my blog<\/strong>, as well as a few other web sites.<\/p>\n

Here goes the longer version.\u00a0 I have been moving a whole bunch of web sites from my reseller hosting account at EuroVPS<\/a> to a brand new VPS account at VAServ<\/a>.\u00a0 Site by site, blog by blog, database by database.\u00a0 To keep things simple, once I made sure that the site was moved properly, I deleted the copy from the old hosting (after a week or so).<\/p>\n

When I was almost done with the move and there were just a few more left, something really bad happened a VAServ<\/a>.\u00a0 All company’s servers got compromised.\u00a0 The attackers gained control over thousands of VPS accounts across hundreds of servers, and then they deleted whatever they could.\u00a0 The effect of this was so devastating that it was extensively covered in the news<\/a>.<\/p>\n

According the VAServ, hackers utilized a security hole in the HyperVM software, which was written by LXLabs.\u00a0 Apparently, HyperVM is known for its poor security, but things never went wrong at this scale. (In other news, LXLabs founder was found dead<\/a> in a suspected suicide a day or so later.\u00a0 And the rumour has it that the break-in had nothing to do with HyperVM<\/a>, but was VAServ negligence)<\/p>\n

Now for the most interesting part of the story – the lost data.\u00a0 How did that happen?\u00a0 OK, the company got hacked and all data was deleted.\u00a0 But what about the backups?\u00a0 It turned out, there were no tape backups.\u00a0 The only backups VAServ had were on the network storage.\u00a0 And, of course, that data got deleted by the attackers.\u00a0 Imagine that.\u00a0 Web sites, databases, emails, DNS records.\u00a0 Everything is gone.\u00a0 Well, not everything – they managed to recover some servers, but not all by far.<\/p>\n

My sites were on one of those servers which experienced 100% data loss, and which had no backup.\u00a0 That was when I contacted EurVPS support and asked them to restore my recently deleted sites from tapes.\u00a0 After all, it’s better to lose a few weeks of work, rather than a few years.\u00a0 Guess what?\u00a0 It turned out, EuroVPS has no backups either.\u00a0 When I insisted, saying that backups are a part of my hosting plan, they corrected themselves and said that they have backups, but, as advertised on the site<\/a> – weekly only.<\/p>\n

\"Screenshot\"<\/a>
Screenshot<\/figcaption><\/figure>\n

Let me ask you a simple question.\u00a0 How do you understand the phrase “weekly backups on tape”?\u00a0 My understanding was that there’s a scheduled backup task (every weekend or\u00a0 so), which dumps data on tapes, and those tapes are moved out of the building somewhere.\u00a0 Eventually, of course, they are rotated (monthly, or annually, or so).\u00a0 But there is a certain period which you can go back to and restore from those weekly tapes.<\/p>\n

It so happened, my understanding was wrong.\u00a0 Weekly tape backup means one backup within a week on tape.\u00a0 That is, there is no way to go more than one week back using tape backups.\u00a0 I was shocked a bit, but there was still a chance to get something.\u00a0 I clearly remember that I deleted two sites five days ago.\u00a0 I asked EuroVPS support to restore at least those.\u00a0 To which they replied that those two sites aren’t on the backups either.<\/p>\n

What?\u00a0 How? Err…\u00a0 I know, of course, that the loss of data is my fault as much as theirs. I should have done my own backups, downloading them to my own machine.\u00a0 And I’m deeply sorry for not doing so.\u00a0 But on the other hand, having paid for hosting, I do expect uninterrupted power, redundant network connection, and properly organized backups.\u00a0 If that’s not how commercial hosting is different from home servers, than I don’t know how.<\/p>\n

Currently, I am setting up a new VPS host, reconfiguring domains for the new IP, installing a bunch of WordPress blogs, and issuing a whole lot of apologies.\u00a0 Those things that can be recovered, will be recovered.\u00a0 Those things that were important and were lost, will be started a new.\u00a0 And those things that were not important and were lost, will remain lost.<\/p>\n

Let this be yet another painful lesson on the importance of backups.<\/p>\n\n","protected":false},"excerpt":{"rendered":"\n

The long story short : I lost my blog, as well as a few other web sites. Here goes the longer version.\u00a0 I have been moving a whole bunch of web sites from my reseller hosting account at EuroVPS to a brand new VPS account at VAServ.\u00a0 Site by site, blog by blog, database by … Continue reading What happened?<\/span><\/a><\/p>\n\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false,"_links_to":"","_links_to_target":""},"categories":[1,1335],"tags":[251,200,2289],"keyring_services":[],"class_list":["post-11574","post","type-post","status-publish","format-standard","hentry","category-general","category-site-news","tag-backup","tag-security","tag-web-hosting"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t