{"id":29013,"date":"2018-11-23T17:40:46","date_gmt":"2018-11-23T15:40:46","guid":{"rendered":"https:\/\/mamchenkov.net\/wordpress\/?p=29013"},"modified":"2018-11-23T17:40:46","modified_gmt":"2018-11-23T15:40:46","slug":"archiving-web-sites","status":"publish","type":"post","link":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/","title":{"rendered":"Archiving web sites"},"content":{"rendered":"<!-- google_ad_section_start -->\n<p>LWN runs <a href=\"https:\/\/lwn.net\/Articles\/766374\/\">an interesting article<\/a>, covering different ways of archiving a website.\u00a0 It sounds trivial, but it&#8217;s not.\u00a0 Even the simplest of ways &#8211; wget &#8211; will probably take you a few dozen attempts to figure out the following:<\/p>\n<pre class=\"brush: plain; light: true; title: ; notranslate\" title=\"\">\r\n$ wget --mirror --execute robots=off --no-verbose --convert-links \\\r\n       --backup-converted --page-requisites --adjust-extension \\\r\n       --base=.\/ --directory-prefix=.\/ --span-hosts \\\r\n       --domains=www.example.com,example.com http:\/\/www.example.com\/\r\n<\/pre>\n<p>There a few other interesting tools (like <a href=\"https:\/\/github.com\/webrecorder\/pywb\">pywb<\/a>) mentioned.<\/p>\n<!-- google_ad_section_end -->\n","protected":false},"excerpt":{"rendered":"<!-- google_ad_section_start -->\n<p>LWN runs an interesting article, covering different ways of archiving a website.\u00a0 It sounds trivial, but it&#8217;s not.\u00a0 Even the simplest of ways &#8211; wget &#8211; will probably take you a few dozen attempts to figure out the following: $ wget &#8211;mirror &#8211;execute robots=off &#8211;no-verbose &#8211;convert-links \\ &#8211;backup-converted &#8211;page-requisites &#8211;adjust-extension \\ &#8211;base=.\/ &#8211;directory-prefix=.\/ &#8211;span-hosts \\ &hellip; <a href=\"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Archiving web sites<\/span><\/a><\/p>\n<!-- google_ad_section_end -->\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"Archiving web sites #WebDev #SysAdmin #hosting #backup #CLI","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false,"_links_to":"","_links_to_target":""},"categories":[1,6,133,62,1334,60],"tags":[251,1960,1330,2289],"keyring_services":[],"class_list":["post-29013","post","type-post","status-publish","format-standard","hentry","category-general","category-linux","category-sysadmin","category-technology","category-web-work","category-wordpress","tag-backup","tag-command-line","tag-web-development","tag-web-hosting"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 4.9.9 - aioseo.com -->\n\t<meta name=\"description\" content=\"LWN runs an interesting article, covering different ways of archiving a website. It sounds trivial, but it&#039;s not. Even the simplest of ways - wget - will probably take you a few dozen attempts to figure out the following: [code light=&quot;true&quot;] $ wget --mirror --execute robots=off --no-verbose --convert-links \\ --backup-converted --page-requisites --adjust-extension \\ --base=.\/ --directory-prefix=.\/\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Leonid Mamchenkov\"\/>\n\t<meta name=\"google-site-verification\" content=\"VHvdD0_usx1_4DzKy_QCVcICVgX2EgA2ybELT-wl7kQ\" \/>\n\t<link rel=\"canonical\" href=\"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 4.9.9\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"Leonid Mamchenkov - Life, universe, and everything else\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"Archiving web sites - Leonid Mamchenkov\" \/>\n\t\t<meta property=\"og:description\" content=\"LWN runs an interesting article, covering different ways of archiving a website. It sounds trivial, but it&#039;s not. Even the simplest of ways - wget - will probably take you a few dozen attempts to figure out the following: [code light=&quot;true&quot;] $ wget --mirror --execute robots=off --no-verbose --convert-links \\ --backup-converted --page-requisites --adjust-extension \\ --base=.\/ --directory-prefix=.\/\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t\t<meta property=\"og:image:height\" content=\"1024\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2018-11-23T15:40:46+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2018-11-23T15:40:46+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/MamchenkovBlog\" \/>\n\t\t<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n\t\t<meta name=\"twitter:site\" content=\"@mamchenkov\" \/>\n\t\t<meta name=\"twitter:title\" content=\"Archiving web sites - Leonid Mamchenkov\" \/>\n\t\t<meta name=\"twitter:description\" content=\"LWN runs an interesting article, covering different ways of archiving a website. It sounds trivial, but it&#039;s not. Even the simplest of ways - wget - will probably take you a few dozen attempts to figure out the following: [code light=&quot;true&quot;] $ wget --mirror --execute robots=off --no-verbose --convert-links \\ --backup-converted --page-requisites --adjust-extension \\ --base=.\/ --directory-prefix=.\/\" \/>\n\t\t<meta name=\"twitter:creator\" content=\"@mamchenkov\" \/>\n\t\t<meta name=\"twitter:image\" content=\"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg\" \/>\n\t\t<script type=\"application\/ld+json\" class=\"aioseo-schema\">\n\t\t\t{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"BlogPosting\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#blogposting\",\"name\":\"Archiving web sites - Leonid Mamchenkov\",\"headline\":\"Archiving web sites\",\"author\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/#author\"},\"publisher\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#person\"},\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#articleImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g\",\"width\":96,\"height\":96,\"caption\":\"Leonid Mamchenkov\"},\"datePublished\":\"2018-11-23T17:40:46+02:00\",\"dateModified\":\"2018-11-23T17:40:46+02:00\",\"inLanguage\":\"en-US\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#webpage\"},\"isPartOf\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#webpage\"},\"articleSection\":\"All, Linux, Sysadmin, Technology, Web work, WordPress, backup, command line, web development, web hosting\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#breadcrumblist\",\"itemListElement\":[{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress#listItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/#listItem\",\"name\":\"Technology\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/#listItem\",\"position\":2,\"name\":\"Technology\",\"item\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/linux\\\/#listItem\",\"name\":\"Linux\"},\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress#listItem\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/linux\\\/#listItem\",\"position\":3,\"name\":\"Linux\",\"item\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/linux\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#listItem\",\"name\":\"Archiving web sites\"},\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/#listItem\",\"name\":\"Technology\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#listItem\",\"position\":4,\"name\":\"Archiving web sites\",\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/linux\\\/#listItem\",\"name\":\"Linux\"}}]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#person\",\"name\":\"Leonid Mamchenkov\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#personImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g\",\"width\":96,\"height\":96,\"caption\":\"Leonid Mamchenkov\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/#author\",\"url\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/\",\"name\":\"Leonid Mamchenkov\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#authorImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g\",\"width\":96,\"height\":96,\"caption\":\"Leonid Mamchenkov\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#webpage\",\"url\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/\",\"name\":\"Archiving web sites - Leonid Mamchenkov\",\"description\":\"LWN runs an interesting article, covering different ways of archiving a website. It sounds trivial, but it's not. Even the simplest of ways - wget - will probably take you a few dozen attempts to figure out the following: [code light=\\\"true\\\"] $ wget --mirror --execute robots=off --no-verbose --convert-links \\\\ --backup-converted --page-requisites --adjust-extension \\\\ --base=.\\\/ --directory-prefix=.\\\/\",\"inLanguage\":\"en-US\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#website\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2018\\\/11\\\/23\\\/archiving-web-sites\\\/#breadcrumblist\"},\"author\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/#author\"},\"creator\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/#author\"},\"datePublished\":\"2018-11-23T17:40:46+02:00\",\"dateModified\":\"2018-11-23T17:40:46+02:00\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#website\",\"url\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/\",\"name\":\"Blog of Leonid Mamchenkov\",\"description\":\"Life, universe, and everything else\",\"inLanguage\":\"en-US\",\"publisher\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#person\"}}]}\n\t\t<\/script>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"Archiving web sites - Leonid Mamchenkov","description":"LWN runs an interesting article, covering different ways of archiving a website. It sounds trivial, but it's not. Even the simplest of ways - wget - will probably take you a few dozen attempts to figure out the following: [code light=\"true\"] $ wget --mirror --execute robots=off --no-verbose --convert-links \\ --backup-converted --page-requisites --adjust-extension \\ --base=.\/ --directory-prefix=.\/","canonical_url":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/","robots":"max-image-preview:large","keywords":"","webmasterTools":{"google-site-verification":"VHvdD0_usx1_4DzKy_QCVcICVgX2EgA2ybELT-wl7kQ","miscellaneous":""},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"BlogPosting","@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#blogposting","name":"Archiving web sites - Leonid Mamchenkov","headline":"Archiving web sites","author":{"@id":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/#author"},"publisher":{"@id":"https:\/\/mamchenkov.net\/wordpress\/#person"},"image":{"@type":"ImageObject","@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#articleImage","url":"https:\/\/secure.gravatar.com\/avatar\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g","width":96,"height":96,"caption":"Leonid Mamchenkov"},"datePublished":"2018-11-23T17:40:46+02:00","dateModified":"2018-11-23T17:40:46+02:00","inLanguage":"en-US","mainEntityOfPage":{"@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#webpage"},"isPartOf":{"@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#webpage"},"articleSection":"All, Linux, Sysadmin, Technology, Web work, WordPress, backup, command line, web development, web hosting"},{"@type":"BreadcrumbList","@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#breadcrumblist","itemListElement":[{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress#listItem","position":1,"name":"Home","item":"https:\/\/mamchenkov.net\/wordpress","nextItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/#listItem","name":"Technology"}},{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/#listItem","position":2,"name":"Technology","item":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/","nextItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/linux\/#listItem","name":"Linux"},"previousItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress#listItem","name":"Home"}},{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/linux\/#listItem","position":3,"name":"Linux","item":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/linux\/","nextItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#listItem","name":"Archiving web sites"},"previousItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/#listItem","name":"Technology"}},{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#listItem","position":4,"name":"Archiving web sites","previousItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/linux\/#listItem","name":"Linux"}}]},{"@type":"Person","@id":"https:\/\/mamchenkov.net\/wordpress\/#person","name":"Leonid Mamchenkov","image":{"@type":"ImageObject","@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#personImage","url":"https:\/\/secure.gravatar.com\/avatar\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g","width":96,"height":96,"caption":"Leonid Mamchenkov"}},{"@type":"Person","@id":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/#author","url":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/","name":"Leonid Mamchenkov","image":{"@type":"ImageObject","@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#authorImage","url":"https:\/\/secure.gravatar.com\/avatar\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g","width":96,"height":96,"caption":"Leonid Mamchenkov"}},{"@type":"WebPage","@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#webpage","url":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/","name":"Archiving web sites - Leonid Mamchenkov","description":"LWN runs an interesting article, covering different ways of archiving a website. It sounds trivial, but it's not. Even the simplest of ways - wget - will probably take you a few dozen attempts to figure out the following: [code light=\"true\"] $ wget --mirror --execute robots=off --no-verbose --convert-links \\ --backup-converted --page-requisites --adjust-extension \\ --base=.\/ --directory-prefix=.\/","inLanguage":"en-US","isPartOf":{"@id":"https:\/\/mamchenkov.net\/wordpress\/#website"},"breadcrumb":{"@id":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/#breadcrumblist"},"author":{"@id":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/#author"},"creator":{"@id":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/#author"},"datePublished":"2018-11-23T17:40:46+02:00","dateModified":"2018-11-23T17:40:46+02:00"},{"@type":"WebSite","@id":"https:\/\/mamchenkov.net\/wordpress\/#website","url":"https:\/\/mamchenkov.net\/wordpress\/","name":"Blog of Leonid Mamchenkov","description":"Life, universe, and everything else","inLanguage":"en-US","publisher":{"@id":"https:\/\/mamchenkov.net\/wordpress\/#person"}}]},"og:locale":"en_US","og:site_name":"Leonid Mamchenkov - Life, universe, and everything else","og:type":"article","og:title":"Archiving web sites - Leonid Mamchenkov","og:description":"LWN runs an interesting article, covering different ways of archiving a website. It sounds trivial, but it's not. Even the simplest of ways - wget - will probably take you a few dozen attempts to figure out the following: [code light=&quot;true&quot;] $ wget --mirror --execute robots=off --no-verbose --convert-links \\ --backup-converted --page-requisites --adjust-extension \\ --base=.\/ --directory-prefix=.\/","og:url":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/","og:image":"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg","og:image:secure_url":"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg","og:image:width":1024,"og:image:height":1024,"article:published_time":"2018-11-23T15:40:46+00:00","article:modified_time":"2018-11-23T15:40:46+00:00","article:publisher":"https:\/\/www.facebook.com\/MamchenkovBlog","twitter:card":"summary_large_image","twitter:site":"@mamchenkov","twitter:title":"Archiving web sites - Leonid Mamchenkov","twitter:description":"LWN runs an interesting article, covering different ways of archiving a website. It sounds trivial, but it's not. Even the simplest of ways - wget - will probably take you a few dozen attempts to figure out the following: [code light=&quot;true&quot;] $ wget --mirror --execute robots=off --no-verbose --convert-links \\ --backup-converted --page-requisites --adjust-extension \\ --base=.\/ --directory-prefix=.\/","twitter:creator":"@mamchenkov","twitter:image":"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg"},"aioseo_meta_data":{"post_id":"29013","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"breadcrumb_settings":null,"limit_modified_date":false,"ai":null,"created":"2023-07-20 05:32:41","updated":"2026-01-15 13:52:44","seo_analyzer_scan_date":null},"aioseo_breadcrumb":"<div class=\"aioseo-breadcrumbs\"><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/mamchenkov.net\/wordpress\" title=\"Home\">Home<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/\" title=\"Technology\">Technology<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/linux\/\" title=\"Linux\">Linux<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\tArchiving web sites\n\t\t<\/span><\/div>","aioseo_breadcrumb_json":[{"label":"Home","link":"https:\/\/mamchenkov.net\/wordpress"},{"label":"Technology","link":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/"},{"label":"Linux","link":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/linux\/"},{"label":"Archiving web sites","link":"https:\/\/mamchenkov.net\/wordpress\/2018\/11\/23\/archiving-web-sites\/"}],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":26208,"url":"https:\/\/mamchenkov.net\/wordpress\/2016\/06\/27\/lets-encrypt-on-centos-7-and-amazon-ami\/","url_meta":{"origin":29013,"position":0},"title":"Let&#8217;s Encrypt on CentOS 7 and Amazon AMI","author":"Leonid Mamchenkov","date":"June 27, 2016","format":false,"excerpt":"The last few weeks were super busy at work, so I accidentally let a few SSL certificates expire. \u00a0Renewing them is always annoying and time consuming, so I was pushing it until the last minute, and then some. Instead of going the usual way for the renewal, I decided to\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":28216,"url":"https:\/\/mamchenkov.net\/wordpress\/2017\/11\/13\/github-archiving-repositories\/","url_meta":{"origin":29013,"position":1},"title":"GitHub : Archiving Repositories","author":"Leonid Mamchenkov","date":"November 13, 2017","format":false,"excerpt":"Last week, GitHub introduced archiving of repositories. While it might not seem like a news worthy feature, it is quite useful for both individuals and teams.\u00a0 Two particular scenarios that I find helpful are: Indicate that a particular repository \/ project is obsolete and is not maintained.\u00a0 This should save\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"GitHub archive repositories","src":"https:\/\/i0.wp.com\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2017\/11\/GitHub-archive-repositories-500x238.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":7930,"url":"https:\/\/mamchenkov.net\/wordpress\/2004\/09\/20\/gmane-mail-to-news-and-back-again\/","url_meta":{"origin":29013,"position":2},"title":"Gmane &#8211; Mail To News And Back Again","author":"Leonid Mamchenkov","date":"September 20, 2004","format":false,"excerpt":"Gmane - is an interesting project that provides archiving and comfortable interface to a huge number of mailing lists. What they do is subscribe to a huge number of mailing lists, receive and index messages. They also provide archives over web and NNTP (news). And they even allow to post\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":22001,"url":"https:\/\/mamchenkov.net\/wordpress\/2014\/06\/17\/httpie-command-line-http-client-a-user-friendly-curl-replacement\/","url_meta":{"origin":29013,"position":3},"title":"HTTPie &#8211; command line HTTP client, a user-friendly cURL replacement","author":"Leonid Mamchenkov","date":"June 17, 2014","format":"link","excerpt":"HTTPie - command line HTTP client, a user-friendly cURL replacement. Main features: Expressive and intuitive syntax Formatted and colorized terminal output Built-in JSON support Forms and file uploads HTTPS, proxies, and authentication Arbitrary request data Custom headers Persistent sessions Wget-like downloads Python 2.6, 2.7 and 3.x support Linux, Mac OS\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"httpie","src":"https:\/\/i0.wp.com\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2014\/06\/httpie-500x500.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":12626,"url":"https:\/\/mamchenkov.net\/wordpress\/2010\/06\/22\/google-services-via-command-line\/","url_meta":{"origin":29013,"position":4},"title":"Google services via command line","author":"Leonid Mamchenkov","date":"June 22, 2010","format":false,"excerpt":"I don't know how I missed the announcement and why there is no noise around this release, but here it goes. \u00a0GoogleCL is a tool that you use from the command line to access Google web services. \u00a0I just installed it on my Fedora 13 laptop and its awesome! Here\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":25965,"url":"https:\/\/mamchenkov.net\/wordpress\/2016\/03\/23\/good-bye-google-chrome-hello-chromium\/","url_meta":{"origin":29013,"position":5},"title":"Good bye Google Chrome, hello Chromium","author":"Leonid Mamchenkov","date":"March 23, 2016","format":false,"excerpt":"Google dropped the support of its Google Chrome browser on 32-bit Linux operating systems. \u00a0This is very unfortunate, but not deadly. \u00a0This change doesn't affect the Chromium browser - the Open Source project behind Google Chrome. The two are very compatible. \u00a0In fact, if you use the Google Sync in\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"Chromium-vs-Google-Chrome","src":"https:\/\/i0.wp.com\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2016\/03\/Chromium-vs-Google-Chrome-500x247.jpg?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]}],"jetpack_sharing_enabled":true,"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/posts\/29013","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/comments?post=29013"}],"version-history":[{"count":0,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/posts\/29013\/revisions"}],"wp:attachment":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/media?parent=29013"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/categories?post=29013"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/tags?post=29013"},{"taxonomy":"keyring_services","embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/keyring_services?post=29013"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}