I’ve started using Let’s Encrypt for the SSL certificates a while back. I installed it on all the web servers, irrelevant of the need for SSL, just to have it there, when I need it (thanks to this Ansible role). One of those old web servers needed an SSL certificate recently, so I thought it’d be no problem to generate one.
But I was wrong. The letsencrypt-auto tool got outdated and was failing to execute, throwing some Python exception about missing zope.interface module. A quick Google search brought this StackOverflow discussion, with the exact issue I was having.
Traceback (most recent call last): File "/root/.local/share/letsencrypt/bin/letsencrypt", line 7, in <module> from certbot.main import main File "/root/.local/share/letsencrypt/local/lib/python2.7/dist-packages/certbot/main.py", line 12, in <module> import zope.component File "/root/.local/share/letsencrypt/local/lib/python2.7/dist-packages/zope/component/__init__.py", line 16, in <module> from zope.interface import Interface ImportError: No module named interface
However, the solution didn’t fix the problem for me:
unset PYTHON_INSTALL_LAYOUT /opt/letsencrypt/letsencrypt-auto -v
Even pulling the updated version from the GitHub repository didn’t solve it.
After poking around for a while more, I found this bug report from the last year, which solved my problem.
- Running rm -rf /root/.local/share/letsencrypt. This removes your installation of letsencrypt, but keeps all configuration files, certificates, logs, etc.
- Make sure you have an up to date copy of letsencrypt-auto. It can be found here.
- Run letsencrypt-auto again.
If you get the same behavior, you can try installing zope.interface manually by running:
/root/.local/share/letsencrypt/bin/pip install zope.interface
Hopefully, next time I’ll remember to search my blog’s archives …
Update (May 31, 2017): check out my brother’s follow up post with even better way of fixing this issue.
Picking the right settings for your SSL certificates and SSL configuration on your webserver is confusing. As far as I understand it there are about 3 billion settings. Here is an example of an SSL Labs result for mail.google.com. There is all this stuff like
OLD_TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256on that page (for real, that is a real thing.). I’m happy there are tools like SSL Labs that help mortals make sense of all of it.
# /opt/letsencrypt/certbot-auto renew Creating virtual environment... Installing Python packages... Installation succeeded. Traceback (most recent call last): File "/root/.local/share/letsencrypt/bin/letsencrypt", line 7, in <module> from certbot.main import main File "/root/.local/share/letsencrypt/local/lib/python2.7/dist-packages/certbot/main.py", line 12, in <module> import zope.component File "/root/.local/share/letsencrypt/local/lib/python2.7/dist-packages/zope/component/__init__.py", line 16, in <module> from zope.interface import Interface ImportError: No module named interface
My first though was to install the system updates. It looks like something is off in the Python-land. But even after the “yum update” was done, the issue was still there. A quick Google search later, thanks to the this GitHub issue and this comment, the solution is the following:
pip install pip --upgrade pip install virtualenv --upgrade virtualenv -p /usr/bin/python27 venv27
Running the renewal of the certificates works as expected after this.
P.S.: I wish we had fewer package and dependency managers in the world…
“S3 static site with SSL and automatic deploys using Travis” is a goldmine of all those simple technologies tied into a single knot for an impressive result. It has a bit of everything:
- Jekyll – simple, blog-aware, static sites engine, for managing content.
- GitHub – for version control of the site’s content and for triggering the deployment chain.
- Travis CI – for testing changes, building and deploying a new version.
- Amazon S3 – simple, cheap, web-enabled storage of static content.
- Amazon CloudFront – simple, cheap, geographically-distributed content delivery network (CDN).
- Amazon Route 53 – simple and cheap DNS hosting and domain management.
- Amazon IAM – identity and access management for the Amazon Web Services (AWS).
- Let’s Encrypt – free SSL/TLS certificate provider.
When put altogether, these bits allow one to have a fast (static content combined with HTTP 2 and top-level networking) and cheap (Jekyll, GitHub, Travis and Let’s Encrypt are free, with the rest of the services costing a few cents here and there) static website, with SSL and HTTP 2.
This is a classic example of how accessible and available is modern technology, if (and only if) you know what you are doing.
The last few weeks were super busy at work, so I accidentally let a few SSL certificates expire. Renewing them is always annoying and time consuming, so I was pushing it until the last minute, and then some.
Instead of going the usual way for the renewal, I decided to try to the Let’s Encrypt deal. (I’ve covered Let’s Encrypt before here and here.) Basically, Let’s Encrypt is a new Certification Authority, created by Electronic Frontier Foundation (EFF), with the backing of Google, Cisco, Mozilla Foundation, and the like. This new CA is issuing well recognized SSL certificates, for free. Which is good. But the best part is that they’ve setup the process to be as automated as possible. All you need is to run a shell command to get the certificate and then another shell command in the crontab to renew the certificate automatically. Certificates are only issued for 3 months, so you’d really want to have them automatically updated.
It took me longer than I expected to figure out how this whole thing works, but that’s because I’m not well versed in SSL, and because they have so many different options, suited for different web servers, and different sysadmin experience levels.
Eventually I made it work, and here is the complete process, so that I don’t have to figure it out again later.
We are running a mix of CentOS 7 and Amazon AMI servers, using both Nginx and Apache. Here’s what I had to do.
First things first. Install the Let’s Encrypt client software. Supposedly there are several options, but I went for the official one. Manual way:
# Install requirements yum install git bc cd /opt git clone https://github.com/certbot/certbot letsencrypt
Alternatively, you can use geerlingguy’s lets-encrypt-role for Ansible.
Secondly, we need to get a new certificate. As I said before, there are multiple options here. I decided to use the certonly way, so that I have better control over where things go, and so that I would minimize the web server downtime.
There are a few things that you need to specify for the new SSL certificate. These are:
- The list of domains, which the certificate should cover. I’ll use example.com and www.example.com here.
- The path to the web folder of the site. I’ll use /var/www/vhosts/example.com/
- The email address, which Let’s Encrypt will use to contact you in case there is something urgent. I’ll use firstname.lastname@example.org here.
Now, the command to get the SSL certificate is:
/opt/letsencrypt/certbot-auto certonly --webroot --email email@example.com --agree-tos -w /var/www/vhosts/example.com/ -d example.com -d www.example.com
When you run this for the first time, you’ll see that a bunch of additional RPM packages will be installed, for the virtual environment to be created and used. On CentOS 7 this is sufficient. On Amazon AMI, the command will run, install things, and will fail with something like this:
WARNING: Amazon Linux support is very experimental at present... if you would like to work on improving it, please ensure you have backups and then run this script again with the --debug flag!
This is useful, but insufficient. Before you can run successfully, you’ll also need to do the following:
yum install python26-virtualenv
Once that is done, run the certbot command with the –debug parameter, like so:
/opt/letsencrypt/certbot-auto certonly --webroot --email firstname.lastname@example.org --agree-tos -w /var/www/vhosts/example.com/ -d example.com -d www.example.com --debug
This should produce a success message, with “Congratulations!” and all that. The path to your certificate (somewhere in /etc/letsencrypt/live/example.com/) and its expiration date will be mentioned too.
If you didn’t get the success message, make sure that:
- the domain, for which you are requesting a certificate, resolves back to the server, where you are running the certbot command. Let’s Encrypt will try to access the site for verification purposes.
- that public access is allowed to the /.well-known/ folder. This is where Let’s Encrypt will store temporary verification files. Note that the folder starts with dot, which in UNIX means hidden folder, which are often denied access to by many web server configurations.
Just drop a simple hello.txt to the /.well-known/ folder and see if you can access it with the browser. If you can, then Let’s Encrypt shouldn’t have any issues getting you a certification. If all else fails, RTFM.
Now that you have the certificate generated, you’ll need to add it to the web server’s virtual host configuration. How exactly to do this varies from web server to web server, and even between the different versions of the same web server.
For Apache version >= 2.4.8 you’ll need to do the following:
SSLEngine on SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
For Apache version < 2.4.8 you’ll need to do the following:
SSLEngine on SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem SSLCertificateFile /etc/letsencrypt/live/example.com/cert.pem SSLCertificateChainFile /etc/letsencrypt/live/example.com/chain.pem
For Nginx >= 1.3.7 you’ll need to do the following:
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
You’ll obviously need the additional SSL configuration options for protocols, ciphers and the like, which I won’t go into here, but here are a few useful links:
- Apache Configuration Example
- How To Secure Nginx with Let’s Encrypt on Ubuntu 14.04
- Certbot User Guide
- Using the webroot domain verification method
- Guide to Deploying Diffie-Hellman for TLS
Once your SSL certificate is issued and web server is configured to use it, all you need is to add an entry to the crontab to renew the certificates which are expiring in 30 days or less. You’ll only need a single entry for all your certificates on this machine. Edit your /etc/crontab file and add the following (adjust for your web server software, obviously):
# Renew Let's Encrypt certificates at 6pm every Sunday 0 18 * * 0 root (/opt/letsencrypt/certbot-auto renew && service httpd restart)
That’s about it. Once all is up and running, verify and adjust your SSL configuration, using Qualys SSL Labs excellent tool.
I’ve first written about Let’s Encrypt back in November 2014. It hasn’t been that long ago, but boy, what a journey!
Electronic Frontier Foundation reports some excellent news:
Today EFF is pleased to announce Let’s Encrypt, a new certificate authority (CA) initiative that we have put together with Mozilla, Cisco, Akamai, IdenTrust, and researchers at the University of Michigan that aims to clear the remaining roadblocks to transition the Web from HTTP to HTTPS.
With a launch scheduled for summer 2015, the Let’s Encrypt CA will automatically issue and manage free certificates for any website that needs them. Switching a webserver from HTTP to HTTPS with this CA will be as easy as issuing one command, or clicking one button.