Let’s Encrypt on CentOS 7 and Amazon AMI

The last few weeks were super busy at work, so I accidentally let a few SSL certificates expire.  Renewing them is always annoying and time consuming, so I was pushing it until the last minute, and then some.

Instead of going the usual way for the renewal, I decided to try to the Let’s Encrypt deal.  (I’ve covered Let’s Encrypt before here and here.)  Basically, Let’s Encrypt is a new Certification Authority, created by Electronic Frontier Foundation (EFF), with the backing of Google, Cisco, Mozilla Foundation, and the like.  This new CA is issuing well recognized SSL certificates, for free.  Which is good.  But the best part is that they’ve setup the process to be as automated as possible.  All you need is to run a shell command to get the certificate and then another shell command in the crontab to renew the certificate automatically.  Certificates are only issued for 3 months, so you’d really want to have them automatically updated.

It took me longer than I expected to figure out how this whole thing works, but that’s because I’m not well versed in SSL, and because they have so many different options, suited for different web servers, and different sysadmin experience levels.

Eventually I made it work, and here is the complete process, so that I don’t have to figure it out again later.

We are running a mix of CentOS 7 and Amazon AMI servers, using both Nginx and Apache.   Here’s what I had to do.

First things first.  Install the Let’s Encrypt client software.  Supposedly there are several options, but I went for the official one.  Manual way:

# Install requirements
yum install git bc
cd /opt
git clone https://github.com/certbot/certbot letsencrypt

Alternatively, you can use geerlingguy’s lets-encrypt-role for Ansible.

Secondly, we need to get a new certificate.  As I said before, there are multiple options here.  I decided to use the certonly way, so that I have better control over where things go, and so that I would minimize the web server downtime.

There are a few things that you need to specify for the new SSL certificate.  These are:

  • The list of domains, which the certificate should cover.  I’ll use example.com and www.example.com here.
  • The path to the web folder of the site.  I’ll use /var/www/vhosts/example.com/
  • The email address, which Let’s Encrypt will use to contact you in case there is something urgent.  I’ll use ssl@example.com here.

Now, the command to get the SSL certificate is:

/opt/letsencrypt/certbot-auto certonly --webroot --email ssl@example.com --agree-tos -w /var/www/vhosts/example.com/ -d example.com -d www.example.com

When you run this for the first time, you’ll see that a bunch of additional RPM packages will be installed, for the virtual environment to be created and used.  On CentOS 7 this is sufficient.  On Amazon AMI, the command will run, install things, and will fail with something like this:

WARNING: Amazon Linux support is very experimental at present...
if you would like to work on improving it, please ensure you have backups
and then run this script again with the --debug flag!

This is useful, but insufficient.  Before you can run successfully, you’ll also need to do the following:

yum install python26-virtualenv

Once that is done, run the certbot command with the –debug parameter, like so:

/opt/letsencrypt/certbot-auto certonly --webroot --email ssl@example.com --agree-tos -w /var/www/vhosts/example.com/ -d example.com -d www.example.com --debug

This should produce a success message, with “Congratulations!” and all that.  The path to your certificate (somewhere in /etc/letsencrypt/live/example.com/) and its expiration date will be mentioned too.

If you didn’t get the success message, make sure that:

  • the domain, for which you are requesting a certificate, resolves back to the server, where you are running the certbot command.  Let’s Encrypt will try to access the site for verification purposes.
  • that public access is allowed to the /.well-known/ folder.  This is where Let’s Encrypt will store temporary verification files.  Note that the folder starts with dot, which in UNIX means hidden folder, which are often denied access to by many web server configurations.

Just drop a simple hello.txt to the /.well-known/ folder and see if you can access it with the browser.  If you can, then Let’s Encrypt shouldn’t have any issues getting you a certification.  If all else fails, RTFM.

Now that you have the certificate generated, you’ll need to add it to the web server’s virtual host configuration.  How exactly to do this varies from web server to web server, and even between the different versions of the same web server.

For Apache version >= 2.4.8 you’ll need to do the following:

SSLEngine on
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem

For Apache version < 2.4.8 you’ll need to do the following:

SSLEngine on
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem
SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem

For Nginx >= 1.3.7 you’ll need to do the following:

ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

You’ll obviously need the additional SSL configuration options for protocols, ciphers and the like, which I won’t go into here, but here are a few useful links:

Once your SSL certificate is issued and web server is configured to use it, all you need is to add an entry to the crontab to renew the certificates which are expiring in 30 days or less.  You’ll only need a single entry for all your certificates on this machine.  Edit your /etc/crontab file and add the following (adjust for your web server software, obviously):

# Renew Let's Encrypt certificates at 6pm every Sunday
0 18 * * 0 root (/opt/letsencrypt/certbot-auto renew && service httpd restart)

That’s about it.  Once all is up and running, verify and adjust your SSL configuration, using Qualys SSL Labs excellent tool.

diff-so-fancy – the best-lookin’ diffs

Here is a cool tool to spice up your regular boring looking diffs – diff-so-fancy.  Don’t get spooked by the npm installation instructions – the meat of it is all in perl/shell and you can install it as any other ~/bin/ script.  Have a look at what you are missing:

diff-so-fancy

Git 2.9

Git 2.9 has been released a few days, bringing in some very useful functionality, such as showing renamed files in git diff and git log, forbidding the merge of two branches that have no common ancestors, configurable path to hooks, and more.  All are welcome changes, making the life of a developer easier.

But what I found interesting is how two largest git companies – GitHub and BitBucket – reflect on it.  Surely, the new release is important to both, but it’s insightful to see which features each of them looks at first.  Have a look:

 

Chrome apps … mind blown!

Don’t ask me how, but I’ve ended up in the Google Chrome Web Store, where I spent the last three hours – especially in the Productivity -> Developer Tools category.  I knew, there were plenty of apps to make Chrome OS / Chrome Browser super awesome, but it seems it’s been a while since I looked in there … My mind is officially blown!

I don’t need much from my Fedora laptop – a browser, a terminal, and some instant messaging apps.  But these days apparently that’s too much.  A lot of the things I do through the regular day can be handled right from the browser apps.

mysql

Here are some examples.

  1. Text editors.  There is a slew of them!  Simple and complex, specialized and generic, fast and … not so much.  Have a look at Caret for example.  It’s Sublime-like editor, based on the Ace editing component.  It offers a selection of themes, syntax highlighting for all the major languages, multiple tabs, project settings, and more!
  2. SSH client.  Yup, that’s right.  You can connect to your remote servers right out of the browser, using, for example, ServerAuditor.
  3. MySQL clients.  Choose between a simple command-line one, like MySQL Console.  Or a full-featured one, with ERDs and database browser, like Chrome MySQL Admin.
  4. Git, GitHub, and Gist tools.  Which there is a variety of…
  5. Web server (yes, really, a web server running in the web browser!) – Web Server fro Chrome, debugger (Xdebug), and compiler (Compiler.work).

Most of these offer session saving, networking synchronization, Google Drive data saving, social network integration, etc.

Wow!  The browser world has come a long way since Netscape 3 …

 

Deploying with git

Git is an excellent version control, but it’s more than just that.  A lot of people use it to deploy their projects as well.  Most suggestions (for example, this tutorial from Digital Ocean) around the web employ the post-commit (or other) hooks to push the code to a remote server.  While this works well, I prefer to do it differently.  I like the pull model better, where the deployment is triggered outside of git, and relies on git to fetch the code updates and run some sort of a build script, which handles database schema changes, cache resets, filesystem permissions, etc.  Such approach also allows to limit remote access to the servers (especially the production ones), and separate responsibilities of a developer and a deployer.

With the many pull, merge, fetch, and update options that git provides, it is sometimes difficult to choose what’s the right set of commands to use.  I’ve figured it out via a rather lengthy trial-and-error process.  But if you don’t want to go through all the pain of that, here’s a nice blog post that tells you exactly how to do that.  I’m copy-pasting the commands here just for the future reference.

cd "${DEPLOY_TREE}"
git fetch --all
git checkout --force "${TARGET}"
# Following two lines only required if you use submodules
git submodule sync
git submodule update --init --recursive
# Follow with actual deployment steps (run fabric/capistrano/make/etc)

And I suggest you read the full article for the explanation of why this is a better way and what are some of the issues with other strategies.

 

Explain Shell

Here’s a good resource for all of those who is trying to learn shell and/or figure out complex commands with lots of parameters and pipes – Explain Shell.

ExplainShell

You just paste the command and hit the “Explain” button, and the site will decompose the command into parts, providing relevant parts from the manual pages.  There are a few examples to try it out on too.

28 Ways to Secure WordPress Website

28 Ways to Secure WordPress Website covers, as the title says, quite a few ways to make your WordPress website more secure.  There is no absolute security, and there are always more that you can do, but this is a good start.  Apart from all the useful advice, the article also tells you why you should care:

“Why would anyone hack my site?” – you ask

Let’s be clear. Your site is likely not special. Unless your firm’s name is CNN.

The fact is that most – or the great majority, rather – of attacks are automated. This means that various bots (pieces of software) developed by hackers crawl the web and look for vulnerable sites.

Then if they’re successful, the site gets added to the hacker’s portfolio, so to speak, and can be used for various purposes.

In other words, your site by itself is no special, but 10,000 sites just like yours is pure gold for a hacker. Such a network of hacked sites can be used for things like black hat SEO, mass email sending, database scraping (to get your users’ personal info), and so on.

You really shouldn’t feel overly safe just because/if you run a relatively small website.

Hackers don’t discriminate.

DNS / Whois Record Lookup Tool

Here is yet another DNS / WHOIS record lookup tool.  It’s quick and simple – just type the website’s URL and submit a form.  You’ll get a result with all the DNS records and WHOIS information, all on one page.

The term DNS stands for Domain Name System, the largest digital database which contains all websites information on the internet. Every domain has authoritative DNS server which publishes information about that domain and the name server for the domain.

Our DNS / Whois record lookup tool will grab A, MX, SOA, NS, TXT and Whois records for a domain name.

Let’s Encrypt is not in Beta anymore

Let’s Encrypt – anew Certificate Authority, which is free, open, and automated – announced that it’s leaving beta.  Just look at how many SSL certificates they’ve issued, and at what rate!

Issuance-April-10-2016

I’ve first written about Let’s Encrypt back in November 2014.  It hasn’t been that long ago, but boy, what a journey!