{"id":16511,"date":"2012-07-18T00:38:09","date_gmt":"2012-07-17T22:38:09","guid":{"rendered":"https:\/\/mamchenkov.net\/wordpress\/?p=16511"},"modified":"2012-07-18T00:38:09","modified_gmt":"2012-07-17T22:38:09","slug":"boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages","status":"publish","type":"post","link":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/","title":{"rendered":"boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages"},"content":{"rendered":"<!-- google_ad_section_start -->\n<p><a href=\"http:\/\/code.google.com\/p\/boilerpipe\/\">boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages<\/a><\/p>\n<blockquote><p>The boilerpipe library provides algorithms to detect and remove the surplus &#8220;clutter&#8221; (boilerplate, templates) around the main textual content of a web page.<\/p>\n<p>The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual problem settings.<\/p>\n<p>Extracting content is very fast (milliseconds), just needs the input document (no global or site-level information required) and is usually quite accurate.<\/p><\/blockquote>\n<!-- google_ad_section_end -->\n","protected":false},"excerpt":{"rendered":"<!-- google_ad_section_start -->\n<p>boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages The boilerpipe library provides algorithms to detect and remove the surplus &#8220;clutter&#8221; (boilerplate, templates) around the main textual content of a web page. The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual &hellip; <a href=\"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages<\/span><\/a><\/p>\n<!-- google_ad_section_end -->\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"link","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false,"_links_to":"","_links_to_target":""},"categories":[1,18,62,1334],"tags":[1190,1330],"keyring_services":[],"class_list":["post-16511","post","type-post","status-publish","format-link","hentry","category-general","category-programming","category-technology","category-web-work","tag-html","tag-web-development","post_format-post-format-link"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 4.9.10 - aioseo.com -->\n\t<meta name=\"description\" content=\"boilerpipe - Boilerplate Removal and Fulltext Extraction from HTML pages The boilerpipe library provides algorithms to detect and remove the surplus &quot;clutter&quot; (boilerplate, templates) around the main textual content of a web page. The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Leonid Mamchenkov\"\/>\n\t<meta name=\"google-site-verification\" content=\"VHvdD0_usx1_4DzKy_QCVcICVgX2EgA2ybELT-wl7kQ\" \/>\n\t<link rel=\"canonical\" href=\"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 4.9.10\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"Leonid Mamchenkov - Life, universe, and everything else\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"boilerpipe \u2013 Boilerplate Removal and Fulltext Extraction from HTML pages - Leonid Mamchenkov\" \/>\n\t\t<meta property=\"og:description\" content=\"boilerpipe - Boilerplate Removal and Fulltext Extraction from HTML pages The boilerpipe library provides algorithms to detect and remove the surplus &quot;clutter&quot; (boilerplate, templates) around the main textual content of a web page. The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t\t<meta property=\"og:image:height\" content=\"1024\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2012-07-17T22:38:09+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2012-07-17T22:38:09+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/MamchenkovBlog\" \/>\n\t\t<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n\t\t<meta name=\"twitter:site\" content=\"@mamchenkov\" \/>\n\t\t<meta name=\"twitter:title\" content=\"boilerpipe \u2013 Boilerplate Removal and Fulltext Extraction from HTML pages - Leonid Mamchenkov\" \/>\n\t\t<meta name=\"twitter:description\" content=\"boilerpipe - Boilerplate Removal and Fulltext Extraction from HTML pages The boilerpipe library provides algorithms to detect and remove the surplus &quot;clutter&quot; (boilerplate, templates) around the main textual content of a web page. The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual\" \/>\n\t\t<meta name=\"twitter:creator\" content=\"@mamchenkov\" \/>\n\t\t<meta name=\"twitter:image\" content=\"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg\" \/>\n\t\t<script type=\"application\/ld+json\" class=\"aioseo-schema\">\n\t\t\t{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"BlogPosting\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#blogposting\",\"name\":\"boilerpipe \\u2013 Boilerplate Removal and Fulltext Extraction from HTML pages - Leonid Mamchenkov\",\"headline\":\"boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages\",\"author\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/#author\"},\"publisher\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#person\"},\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#articleImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g\",\"width\":96,\"height\":96,\"caption\":\"Leonid Mamchenkov\"},\"datePublished\":\"2012-07-18T00:38:09+02:00\",\"dateModified\":\"2012-07-18T00:38:09+02:00\",\"inLanguage\":\"en-US\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#webpage\"},\"isPartOf\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#webpage\"},\"articleSection\":\"All, Programming, Technology, Web work, HTML, web development, Link\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#breadcrumblist\",\"itemListElement\":[{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress#listItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/#listItem\",\"name\":\"Technology\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/#listItem\",\"position\":2,\"name\":\"Technology\",\"item\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/programming\\\/#listItem\",\"name\":\"Programming\"},\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress#listItem\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/programming\\\/#listItem\",\"position\":3,\"name\":\"Programming\",\"item\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/programming\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#listItem\",\"name\":\"boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages\"},\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/#listItem\",\"name\":\"Technology\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#listItem\",\"position\":4,\"name\":\"boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages\",\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/category\\\/technology\\\/programming\\\/#listItem\",\"name\":\"Programming\"}}]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#person\",\"name\":\"Leonid Mamchenkov\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#personImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g\",\"width\":96,\"height\":96,\"caption\":\"Leonid Mamchenkov\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/#author\",\"url\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/\",\"name\":\"Leonid Mamchenkov\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#authorImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g\",\"width\":96,\"height\":96,\"caption\":\"Leonid Mamchenkov\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#webpage\",\"url\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/\",\"name\":\"boilerpipe \\u2013 Boilerplate Removal and Fulltext Extraction from HTML pages - Leonid Mamchenkov\",\"description\":\"boilerpipe - Boilerplate Removal and Fulltext Extraction from HTML pages The boilerpipe library provides algorithms to detect and remove the surplus \\\"clutter\\\" (boilerplate, templates) around the main textual content of a web page. The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual\",\"inLanguage\":\"en-US\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#website\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/2012\\\/07\\\/18\\\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\\\/#breadcrumblist\"},\"author\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/#author\"},\"creator\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/author\\\/leonid\\\/#author\"},\"datePublished\":\"2012-07-18T00:38:09+02:00\",\"dateModified\":\"2012-07-18T00:38:09+02:00\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#website\",\"url\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/\",\"name\":\"Blog of Leonid Mamchenkov\",\"description\":\"Life, universe, and everything else\",\"inLanguage\":\"en-US\",\"publisher\":{\"@id\":\"https:\\\/\\\/mamchenkov.net\\\/wordpress\\\/#person\"}}]}\n\t\t<\/script>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"boilerpipe \u2013 Boilerplate Removal and Fulltext Extraction from HTML pages - Leonid Mamchenkov","description":"boilerpipe - Boilerplate Removal and Fulltext Extraction from HTML pages The boilerpipe library provides algorithms to detect and remove the surplus \"clutter\" (boilerplate, templates) around the main textual content of a web page. The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual","canonical_url":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/","robots":"max-image-preview:large","keywords":"","webmasterTools":{"google-site-verification":"VHvdD0_usx1_4DzKy_QCVcICVgX2EgA2ybELT-wl7kQ","miscellaneous":""},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"BlogPosting","@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#blogposting","name":"boilerpipe \u2013 Boilerplate Removal and Fulltext Extraction from HTML pages - Leonid Mamchenkov","headline":"boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages","author":{"@id":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/#author"},"publisher":{"@id":"https:\/\/mamchenkov.net\/wordpress\/#person"},"image":{"@type":"ImageObject","@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#articleImage","url":"https:\/\/secure.gravatar.com\/avatar\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g","width":96,"height":96,"caption":"Leonid Mamchenkov"},"datePublished":"2012-07-18T00:38:09+02:00","dateModified":"2012-07-18T00:38:09+02:00","inLanguage":"en-US","mainEntityOfPage":{"@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#webpage"},"isPartOf":{"@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#webpage"},"articleSection":"All, Programming, Technology, Web work, HTML, web development, Link"},{"@type":"BreadcrumbList","@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#breadcrumblist","itemListElement":[{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress#listItem","position":1,"name":"Home","item":"https:\/\/mamchenkov.net\/wordpress","nextItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/#listItem","name":"Technology"}},{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/#listItem","position":2,"name":"Technology","item":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/","nextItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/programming\/#listItem","name":"Programming"},"previousItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress#listItem","name":"Home"}},{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/programming\/#listItem","position":3,"name":"Programming","item":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/programming\/","nextItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#listItem","name":"boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages"},"previousItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/#listItem","name":"Technology"}},{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#listItem","position":4,"name":"boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages","previousItem":{"@type":"ListItem","@id":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/programming\/#listItem","name":"Programming"}}]},{"@type":"Person","@id":"https:\/\/mamchenkov.net\/wordpress\/#person","name":"Leonid Mamchenkov","image":{"@type":"ImageObject","@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#personImage","url":"https:\/\/secure.gravatar.com\/avatar\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g","width":96,"height":96,"caption":"Leonid Mamchenkov"}},{"@type":"Person","@id":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/#author","url":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/","name":"Leonid Mamchenkov","image":{"@type":"ImageObject","@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#authorImage","url":"https:\/\/secure.gravatar.com\/avatar\/3cf6df002a284d78fb6e9d8222ca4d102e0832035ed6bc8447008bd234e131a4?s=96&d=identicon&r=g","width":96,"height":96,"caption":"Leonid Mamchenkov"}},{"@type":"WebPage","@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#webpage","url":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/","name":"boilerpipe \u2013 Boilerplate Removal and Fulltext Extraction from HTML pages - Leonid Mamchenkov","description":"boilerpipe - Boilerplate Removal and Fulltext Extraction from HTML pages The boilerpipe library provides algorithms to detect and remove the surplus \"clutter\" (boilerplate, templates) around the main textual content of a web page. The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual","inLanguage":"en-US","isPartOf":{"@id":"https:\/\/mamchenkov.net\/wordpress\/#website"},"breadcrumb":{"@id":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/#breadcrumblist"},"author":{"@id":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/#author"},"creator":{"@id":"https:\/\/mamchenkov.net\/wordpress\/author\/leonid\/#author"},"datePublished":"2012-07-18T00:38:09+02:00","dateModified":"2012-07-18T00:38:09+02:00"},{"@type":"WebSite","@id":"https:\/\/mamchenkov.net\/wordpress\/#website","url":"https:\/\/mamchenkov.net\/wordpress\/","name":"Blog of Leonid Mamchenkov","description":"Life, universe, and everything else","inLanguage":"en-US","publisher":{"@id":"https:\/\/mamchenkov.net\/wordpress\/#person"}}]},"og:locale":"en_US","og:site_name":"Leonid Mamchenkov - Life, universe, and everything else","og:type":"article","og:title":"boilerpipe \u2013 Boilerplate Removal and Fulltext Extraction from HTML pages - Leonid Mamchenkov","og:description":"boilerpipe - Boilerplate Removal and Fulltext Extraction from HTML pages The boilerpipe library provides algorithms to detect and remove the surplus &quot;clutter&quot; (boilerplate, templates) around the main textual content of a web page. The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual","og:url":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/","og:image":"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg","og:image:secure_url":"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg","og:image:width":1024,"og:image:height":1024,"article:published_time":"2012-07-17T22:38:09+00:00","article:modified_time":"2012-07-17T22:38:09+00:00","article:publisher":"https:\/\/www.facebook.com\/MamchenkovBlog","twitter:card":"summary_large_image","twitter:site":"@mamchenkov","twitter:title":"boilerpipe \u2013 Boilerplate Removal and Fulltext Extraction from HTML pages - Leonid Mamchenkov","twitter:description":"boilerpipe - Boilerplate Removal and Fulltext Extraction from HTML pages The boilerpipe library provides algorithms to detect and remove the surplus &quot;clutter&quot; (boilerplate, templates) around the main textual content of a web page. The library already provides specific strategies for common tasks (for example: news article extraction) and may also be easily extended for individual","twitter:creator":"@mamchenkov","twitter:image":"https:\/\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2026\/03\/leonid-sailing-beer.jpg"},"aioseo_meta_data":{"post_id":"16511","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"BlogPosting","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"breadcrumb_settings":null,"limit_modified_date":false,"ai":null,"created":"2023-07-19 11:04:52","updated":"2026-01-15 08:00:55","seo_analyzer_scan_date":null},"aioseo_breadcrumb":"<div class=\"aioseo-breadcrumbs\"><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/mamchenkov.net\/wordpress\" title=\"Home\">Home<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/\" title=\"Technology\">Technology<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/programming\/\" title=\"Programming\">Programming<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\tboilerpipe \u2013 Boilerplate Removal and Fulltext Extraction from HTML pages\n\t\t<\/span><\/div>","aioseo_breadcrumb_json":[{"label":"Home","link":"https:\/\/mamchenkov.net\/wordpress"},{"label":"Technology","link":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/"},{"label":"Programming","link":"https:\/\/mamchenkov.net\/wordpress\/category\/technology\/programming\/"},{"label":"boilerpipe &#8211; Boilerplate Removal and Fulltext Extraction from HTML pages","link":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/18\/boilerpipe-boilerplate-removal-and-fulltext-extraction-from-html-pages\/"}],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":16540,"url":"https:\/\/mamchenkov.net\/wordpress\/2012\/07\/22\/weekly-digest-2012-07-22\/","url_meta":{"origin":16511,"position":0},"title":"Weekly digest &#8211; 2012-07-22","author":"Leonid Mamchenkov","date":"July 22, 2012","format":false,"excerpt":"Tribute to MS Outlook http:\/\/t.co\/XjIhglhJ # Stash - privately hosted Git repositories http:\/\/t.co\/Ms8Y0ZVR # Requiem For A Digg http:\/\/t.co\/SBiUFv8w # The Most Important Tech Company You've Never Heard Of http:\/\/t.co\/KEvwjBJP # This is What Snake Venom Does to Blood! http:\/\/t.co\/Z6HRcelU # New note : \u041a\u0430\u043a \u0432\u044b\u0433\u043b\u044f\u0434\u044f\u0442 \u043a\u0440\u044f\u043a\u043e\u0437\u044f\u0431\u0440\u044b? http:\/\/t.co\/TYM79cYA # New\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":23814,"url":"https:\/\/mamchenkov.net\/wordpress\/2015\/04\/06\/styleguide-boilerplate-patterns\/","url_meta":{"origin":16511,"position":1},"title":"Styleguide &#038; Boilerplate Patterns","author":"Leonid Mamchenkov","date":"April 6, 2015","format":"link","excerpt":"Styleguide & Boilerplate Patterns - feature comparison of many CSS templates and frameworks.","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":22048,"url":"https:\/\/mamchenkov.net\/wordpress\/2014\/06\/23\/google-web-starter-kit-boilerplate-tooling-for-multi-device-development\/","url_meta":{"origin":16511,"position":2},"title":"Google Web Starter Kit &#8211; Boilerplate &#038; Tooling for Multi-Device Development","author":"Leonid Mamchenkov","date":"June 23, 2014","format":"link","excerpt":"Google Web Starter Kit - Boilerplate & Tooling for Multi-Device Development","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":18020,"url":"https:\/\/mamchenkov.net\/wordpress\/2013\/04\/15\/wordpress-themes-roots\/","url_meta":{"origin":16511,"position":3},"title":"WordPress themes : Roots","author":"Leonid Mamchenkov","date":"April 15, 2013","format":"link","excerpt":"WordPress themes : Roots Roots is a WordPress starter theme based on HTML5 Boilerplate & Bootstrap from Twitter.","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":28490,"url":"https:\/\/mamchenkov.net\/wordpress\/2018\/04\/18\/wordpress-plugin-boilerplate-a-standardized-organized-object-oriented-foundation-for-building-high-quality-wordpress-plugins\/","url_meta":{"origin":16511,"position":4},"title":"WordPress Plugin Boilerplate &#8211; a standardized, organized, object-oriented foundation for building high-quality WordPress Plugins","author":"Leonid Mamchenkov","date":"April 18, 2018","format":false,"excerpt":"WordPress is an excellent system for a whole lot of different projects and needs.\u00a0 It's widely used, fast, and flexible.\u00a0 However it does show its age in many ways.\u00a0 One of the areas where things could be a lot better and simpler is the WordPress plugin development. WordPress plugins are\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":28179,"url":"https:\/\/mamchenkov.net\/wordpress\/2017\/11\/06\/php-ml-machine-learning-library-for-php\/","url_meta":{"origin":16511,"position":5},"title":"PHP-ML &#8211; Machine Learning library for PHP","author":"Leonid Mamchenkov","date":"November 6, 2017","format":false,"excerpt":"PHP-ML is a machine learning library for PHP.\u00a0 Given, PHP is probably not the best choice when it comes to machine learning, but sometimes one is limited in technology stack choices, so it's good have options like this one. Fresh approach to Machine Learning in PHP. Algorithms, Cross Validation, Neural\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/posts\/16511","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/comments?post=16511"}],"version-history":[{"count":0,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/posts\/16511\/revisions"}],"wp:attachment":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/media?parent=16511"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/categories?post=16511"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/tags?post=16511"},{"taxonomy":"keyring_services","embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/keyring_services?post=16511"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}