{"id":26279,"date":"2016-07-24T22:25:54","date_gmt":"2016-07-24T20:25:54","guid":{"rendered":"https:\/\/mamchenkov.net\/wordpress\/?p=26279"},"modified":"2016-07-24T22:25:54","modified_gmt":"2016-07-24T20:25:54","slug":"ssh-multiplexing-and-ansible-via-bastion-host","status":"publish","type":"post","link":"https:\/\/mamchenkov.net\/wordpress\/2016\/07\/24\/ssh-multiplexing-and-ansible-via-bastion-host\/","title":{"rendered":"SSH multiplexing and Ansible via bastion host"},"content":{"rendered":"<!-- google_ad_section_start -->\n<p>It never ceases to amaze me how even after years and years of working with some technologies I keep finding out about super useful features in those technologies, that could have saved me lots of time if I knew about them earlier. \u00a0Today was a day just like that.<\/p>\n<p>I was working on the Ansible setup for a new hosting environment. \u00a0One particular thing I wanted to utilize more was a bastion host &#8211; a single Linux machine with exposed secure shell (SSH) port, which will be used for managing the configurations of all the servers within the environment. \u00a0I sort of done that before, but the solution wasn&#8217;t as elegant as I wanted it to be.<\/p>\n<p>So, I came across this article &#8211;\u00a0<a href=\"http:\/\/blog.scottlowe.org\/2015\/12\/24\/running-ansible-through-ssh-bastion-host\/\">Running Ansible Through an SSH Bastion Host<\/a>. \u00a0Which, among other things taught me about a feature that I didn&#8217;t know nothing about. \u00a0Literally. \u00a0Haven&#8217;t even heard about it. \u00a0<a href=\"https:\/\/en.wikibooks.org\/wiki\/OpenSSH\/Cookbook\/Multiplexing\">Multiplexing in OpenSSH<\/a>:<\/p>\n<blockquote><p>Multiplexing is the ability to send more than one signal over a single line or connection. With multiplexing, OpenSSH can re-use an existing TCP connection for multiple concurrent SSH sessions rather than creating a new one each time.<\/p><\/blockquote>\n<p>This doesn&#8217;t sound too useful for when you are working in command line, one server at a time. \u00a0Who cares how many TCP connections do you need? It&#8217;ll be one, or two, or five. \u00a0Ten, if you are really involved. \u00a0But by that time you&#8217;ll probably be running background processes, and <a href=\"https:\/\/www.wikivs.com\/wiki\/screen_vs_tmux\">screen or tmux<\/a> (which are apparently called &#8220;<a href=\"https:\/\/en.wikipedia.org\/wiki\/Terminal_multiplexer\">terminal multiplexers<\/a>&#8220;).<\/p>\n<p>It&#8217;s when you are going deeper into automation, such as in my case with <a href=\"https:\/\/www.ansible.com\/\">Ansible<\/a>, when you&#8217;ll need OpenSSH multiplexing. \u00a0Ansible, being a configuration manager, can run a whole lot of commands one after another. \u00a0It can run them on multiple servers in parallel as well. \u00a0That&#8217;s where reusing the connections can make quite a bit of a difference. \u00a0If every command you run connects to the remote server, executes, and then disconnects, you can benefit from not needing to connect and disconnect multiple times (tens or hundreds of times, every playbook run). \u00a0 Reusing connection for parallel jobs is even better &#8211; and that&#8217;s a case with bastion host, for example.<\/p>\n<p>Here are a few useful links from that article, just in case the ether eats it one day:<\/p>\n<ul>\n<li><a href=\"http:\/\/dotfiles.tnetconsulting.net\/articles\/2015\/0506\/empowering-openssh.html\">Empowering OpenSSH<\/a><\/li>\n<li><a href=\"http:\/\/blog.scottlowe.org\/2015\/12\/11\/using-ssh-multiplexing\/\">Using SSH Multiplexing<\/a><\/li>\n<li><a href=\"https:\/\/gist.github.com\/seansawyer\/8fe009e67f7e01344328\">Managing OpenStack instances with Ansible through an SSH bastion host<\/a><\/li>\n<\/ul>\n<p>Armed with those, I had my setup running in no time. \u00a0The only minor correction I had to do for my case was the SSH configuration for the bastion host. \u00a0The example in the article is <strong>NOT<\/strong> wrong:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\nHost 10.10.10.*\r\n  ProxyCommand ssh -W %h:%p bastion.example.com\r\n  IdentityFile ~\/.ssh\/private_key.pem\r\n\r\nHost bastion.example.com\r\n  Hostname bastion.example.com\r\n  User ubuntu\r\n  IdentityFile ~\/.ssh\/private_key.pem\r\n  ForwardAgent yes\r\n  ControlMaster auto\r\n  ControlPath ~\/.ssh\/ansible-%r@%h:%p\r\n  ControlPersist 5m\r\n<\/pre>\n<p>It&#8217;s just that in my case, I use hostnames both for the bastion host and the hosts which are managed through it. \u00a0So I had to adjust it as so:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\nHost *.example.com !bastion.example.com\r\n  ProxyCommand ssh -W %h:%p bastion.example.com\r\n  IdentityFile ~\/.ssh\/private_key.pem\r\n\r\nHost bastion.example.com\r\n  Hostname bastion.example.com\r\n  User ubuntu\r\n  IdentityFile ~\/.ssh\/private_key.pem\r\n  ForwardAgent yes\r\n  ControlMaster auto\r\n  ControlPath ~\/.ssh\/ansible-%r@%h:%p\r\n  ControlPersist 5m\r\n<\/pre>\n<p>Notice the two changes:<\/p>\n<ol>\n<li>Switch of the first block from IP addresses to host names, with a mask.<\/li>\n<li>Negation of the bastion host configuration.<\/li>\n<\/ol>\n<p>The reason for the second change is that if there are multiple Host matches in the configuration file, OpenSSH will combine all options from the matched configurations (something I didn&#8217;t find in the <a href=\"http:\/\/man.openbsd.org\/ssh_config\">ssh_config manual<\/a>). \u00a0Try this example <em>ssh.conf<\/em> with some real hosts of yours:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\nHost bastion.example.com\r\n\tUser someuser\r\n\r\nHost *.example.com\r\n\tPort 2222\r\n<\/pre>\n<p>You&#8217;ll see the output similar to this:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\n$ ssh -F ssh.conf bastion.example.com -v\r\nOpenSSH_7.2p2, OpenSSL 1.0.2h-fips  3 May 2016\r\ndebug1: Reading configuration data ssh.conf\r\ndebug1: ssh.conf line 1: Applying options for bastion.example.com\r\ndebug1: ssh.conf line 4: Applying options for *.example.com\r\ndebug1: Connecting to bastion.example.com &#x5B;1.2.3.4] port 2222.\r\n^C\r\n<\/pre>\n<p>Once you negate the bastion host from the wildcard configuration, everything works as expected.<\/p>\n<p>You might also try using &#8220;%r@%h:%p&#8221; for the socket to be different for each remote username that you will concurrently connect with, but that&#8217;s just nit-picking.<\/p>\n<!-- google_ad_section_end -->\n","protected":false},"excerpt":{"rendered":"<!-- google_ad_section_start -->\n<p>It never ceases to amaze me how even after years and years of working with some technologies I keep finding out about super useful features in those technologies, that could have saved me lots of time if I knew about them earlier. \u00a0Today was a day just like that. I was working on the Ansible &hellip; <a href=\"https:\/\/mamchenkov.net\/wordpress\/2016\/07\/24\/ssh-multiplexing-and-ansible-via-bastion-host\/\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">SSH multiplexing and Ansible via bastion host<\/span><\/a><\/p>\n<!-- google_ad_section_end -->\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"SSH multiplexing and Ansible via bastion host #SysAdmin #DevOps #SSH #Linux #Ansible #security","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"_links_to":"","_links_to_target":""},"categories":[1,6,133,62],"tags":[3383,200,3436],"keyring_services":[],"class_list":["post-26279","post","type-post","status-publish","format-standard","hentry","category-general","category-linux","category-sysadmin","category-technology","tag-ansible","tag-security","tag-ssh"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":25985,"url":"https:\/\/mamchenkov.net\/wordpress\/2016\/04\/01\/checking-out-ansible-sorry-puppet\/","url_meta":{"origin":26279,"position":0},"title":"Checking out Ansible. Sorry Puppet","author":"Leonid Mamchenkov","date":"April 1, 2016","format":false,"excerpt":"It's Thursday evening of a particularly difficult week at work. \u00a0Tomorrow is a public holiday, effectively making this\u00a0- a Friday. \u00a0My brain is blank and exhausted, so I can't do anything productive. \u00a0And I'm too tired to go out. \u00a0But I can still learn a thing or two. First things\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":28211,"url":"https:\/\/mamchenkov.net\/wordpress\/2017\/11\/13\/ssh-via-bastion-host\/","url_meta":{"origin":26279,"position":1},"title":"SSH via bastion host","author":"Leonid Mamchenkov","date":"November 13, 2017","format":false,"excerpt":"A while back I wrote this blog post on the subject of using SSH via bastion hosts.\u00a0 If you are into this sort of thing, have a look at this blog post by my brother.\u00a0 He is providing a few more explanations and clarifications, as well as covers a tricky\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":25184,"url":"https:\/\/mamchenkov.net\/wordpress\/2016\/02\/19\/wtf-with-amazon-and-tcp\/","url_meta":{"origin":26279,"position":2},"title":"WTF with Amazon and TCP","author":"Leonid Mamchenkov","date":"February 19, 2016","format":false,"excerpt":"Here goes the story of me learning a few new swear words and pulling out nearly all my hair. \u00a0Grab a cup of coffee, this will take make a while to tell... First of all, here is a diagram to make things a little bit more visual. As you can\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"wtf","src":"https:\/\/i0.wp.com\/mamchenkov.net\/wordpress\/wp-content\/uploads\/2016\/02\/wtf-500x169.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":26004,"url":"https:\/\/mamchenkov.net\/wordpress\/2016\/04\/07\/ansible-safety-net-for-dns-wildcard-hosts\/","url_meta":{"origin":26279,"position":3},"title":"Ansible safety net for DNS wildcard hosts","author":"Leonid Mamchenkov","date":"April 7, 2016","format":false,"excerpt":"After using Ansible for only a week, I am deeply in love. \u00a0I am doing more and more with less and less, and that's exactly how I want my automation. Today I had to solve an interesting problem. \u00a0Ansible operates, based on the host and group inventory. \u00a0As I mentioned\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":26931,"url":"https:\/\/mamchenkov.net\/wordpress\/2016\/11\/21\/using-ansible-to-bootstrap-an-amazon-ec2-instance\/","url_meta":{"origin":26279,"position":4},"title":"Using Ansible to bootstrap an Amazon EC2 instance","author":"Leonid Mamchenkov","date":"November 21, 2016","format":false,"excerpt":"This article - \"Using Ansible to Bootstrap My Work Environment Part 4\" is pure gold for anyone trying to figure out all the moving parts needed to automate the provisioning and configuration of the Amazon EC2 instance with Ansible. Sure, some bits are easier than the other, but it takes\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":26860,"url":"https:\/\/mamchenkov.net\/wordpress\/2016\/11\/08\/install-ansible-2-0-on-amazon-ami\/","url_meta":{"origin":26279,"position":5},"title":"Install Ansible 2.0+ on Amazon AMI","author":"Leonid Mamchenkov","date":"November 8, 2016","format":false,"excerpt":"Today, while upgrading some of my Ansible roles I've hit the problem. \u00a0Some of the newer roles require Ansible 2.0. \u00a0My Amazon AMI machine that runs the playbooks was still on version 1.9. \u00a0EPEL repository doesn't seem to have the newer Ansible version yet. \u00a0Gladly, Google brough in this StackOverflow\u2026","rel":"","context":"In &quot;All&quot;","block_context":{"text":"All","link":"https:\/\/mamchenkov.net\/wordpress\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/posts\/26279","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/comments?post=26279"}],"version-history":[{"count":0,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/posts\/26279\/revisions"}],"wp:attachment":[{"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/media?parent=26279"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/categories?post=26279"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/tags?post=26279"},{"taxonomy":"keyring_services","embeddable":true,"href":"https:\/\/mamchenkov.net\/wordpress\/wp-json\/wp\/v2\/keyring_services?post=26279"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}