sunweaver's NET

There are as many ways of doing the Let's Encrypt thing as there are site admins on this planet. So here is my way of doing it, mainly as a documentation for myself and as a tutorial for a supervision class I'll be teaching tomorrow morning.

TL;DR;

This blog post describes how to obtain certificates from Let's Encrypt on a production web server in a non-privileged user context. We use the small and well-readable acme-tiny [1] Python script for it.

Assumptions

• You know how e.g. Apache2 gets configured (in general)
• and you have a host running Apache2 that is reachable on the internet
• and it least has one DNS hostname associated with its public IP address.
• You have an idea about OpenSSL, requesting a signed certificate
• You know what privileges on a *nix system are and why it is bad mostly to run self-updating scripts under a privileged user account (e.g. root)... (finger pointing at certbot development...)

Creating SSL Key and the Certificate Signing Request

For creating an SSL Key and a corresponding Certificate Signing Request, I use this little script, named web_certrequest.sh:

#!/bin/bash

FQDN="$1"
FQDNunderscores="$(echo $FQDN | sed 's/\./_/g')"

base="$(pwd)"

test -d "$base/private" || mkdir -p "$base/private"

if [ -f "$base/private/${FQDNunderscores}.key" ]; then
        openssl  req  -config "$base/openssl::${FQDNunderscores}.cnf" \
                      -nodes  -new \
                      -key "$base/private/${FQDNunderscores}.key" \
                  -out "$base/${FQDNunderscores}.csr"
else
    openssl  req  -config "$base/openssl::${FQDNunderscores}.cnf" \
                  -nodes  -new \
                  -keyout "$base/private/${FQDNunderscores}.key" \
                  -out "$base/${FQDNunderscores}.csr"
fi

If you are doing all this for the first time, create an empty folder, put the script in it and make the script executable:

$ chmod u+x web_certrequest.sh

For each host (FQDN) that I need SSL certificates for I have a small openssl::<fqdn-with-underscores-instead-of-dots>.cnf configuration file:

[ req ]
default_bits            = 4096                  # Size of keys
distinguished_name      = req_distinguished_name
req_extensions          = v3_req
x509_extensions         = v3_req

[ req_distinguished_name ]
# Variable name           Prompt string
#----------------------   ----------------------------------
0.organizationName      = Organization Name (company)
organizationalUnitName  = Organizational Unit Name (department, division)
emailAddress            = Email Address
emailAddress_max        = 40
localityName            = Locality Name (city, district)
stateOrProvinceName     = State or Province Name (full name)
countryName             = Country Name (2 letter code)
countryName_min         = 2
countryName_max         = 2
commonName              = Common Name (hostname, IP, or your name)
commonName_max          = 64

# Default values for the above, for consistency and less typing.
# Variable name                   Value
#------------------------------   ------------------------------
0.organizationName_default      = My Company              # adapt as needed
localityName_default            = My City                 # adapt as needed
stateOrProvinceName_default     = My County/State         # adapt as needed
countryName_default             = C                       # country code: e.g. DE for Germany
commonName_default              = host.example.com        # hostname of the webserver
emailAddress_default            = hostmasters@example.com # adapt as needed
organizationalUnitName_default  = Webmastery              # adapt as needed

[ v3_req ]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment, keyEncipherment, dataEncipherment, keyAgreement

# Some CAs do not yet support subjectAltName in CSRs.
# Instead the additional names are form entries on web
# pages where one requests the certificate...
subjectAltName          = @alt_names

[alt_names]
### host.example.com is the FQDN
DNS.1 = host.example.com
DNS.2 = www.example.com
DNS.3 = example.com
DNS.4 = wiki.example.com
DNS.5 = www.old-company-name.biz

The above config I place into the same directory as the web_certrequest.sh script and name it openssl::host_example_com.cnf.

Note: I will use host.example.com further down as my host's FQDN. Adapt to your host name, please.

You then run the above script (you need to answer several questions on the way...)...

$ ./web_certrequest.sh host.example.com

and you'll get two files from that:

• private/host_example_com.key: The private key belonging to the to-be-signed certificate
• host_example_com.csr: The file containing the Certificate Signing Request. This file we will send to the Let's Encrypt signing engine...

Note: Once you have the .key and the .csr file, you only on rare occasions need to recreate them:

• .key file recreation:

  • if you need more crypto strengths (e.g. more than 4096 bits)
  • except from that, leave it as is...

• .csr file recreation:

  • only if you recreate the .key file
  • if you need to include another host alias in subjectAltName
  • whenever any of the DNS names in the certificate is not in DNS anymore
  • if the certificate's meta data is wrong/bad/needs to be adapted (e.g. company names sometimes change)

Next step: The host_example_com.csr we will soon send to the Let's Encrypt signing engine, but let's prepare the webserver first.

Setting up the Webserver

With Let's Encrypt there are several ways of getting your mastery over a machine on the internet confirmed. One is to request a challenge from the LE API that you will answer via a webserver you control. Another approach is responding to an LE challenge via a DNS server you control. This blog post looks at web server based responses to the Let's Encrypt challenge.

Place the private key on the web server

Copy the private host_example_com.key file to the web server under /etc/ssl/private. Make sure, file permissions are 0600 and the file is owned by root:root.

Create a letsencrypt system user

For the acme-tiny call we need a non-privileged system user account:

$ sudo adduser --system --home /var/lib/letsencrypt --shell /bin/bash letsencrypt

Setup this letsencrypt user's home directory in the following way:

(root@host) {/var/lib/letsencrypt} # ls -al 
total 40
drwx--x---  8 letsencrypt www-data 4096 Mar  1  2017 .
drwxr-xr-x 80 root        root     4096 Apr 21 08:01 ..
drwx------  2 letsencrypt nogroup  4096 Mar  6  2017 bin
drwx--x---  2 letsencrypt www-data 4096 May  1 00:00 challenges
drwx------  2 letsencrypt nogroup  4096 Mar  6  2017 .letsencrypt

The bin/ subfolder contains this:

(root@host) {/var/lib/letsencrypt/bin} # ls -al 
total 12
drwx------ 2 letsencrypt nogroup  4096 Mar  6  2017 .
drwx--x--- 8 letsencrypt www-data 4096 Mar  1  2017 ..
-rwxr-xr-x 1 root        root   464 Mar  6  2017 letsencrypt-renew-certs

The letsencrypt-renew-certs script has this content:

#!/bin/bash

INTERMEDIATE=lets-encrypt-x3-cross-signed.pem

acme-tiny --account-key ~/.letsencrypt/account-host_example_com.key \
          --csr ~/.letsencrypt/host_example_com.csr \
          --acme-dir ~/challenges/ \
          1> ~/.letsencrypt/host_example_com.crt && \
    \
    cat ~/.letsencrypt/host_example_com.crt \
        ~/.letsencrypt/${INTERMEDIATE} \
        1> ~/.letsencrypt/host_example_com.fullchain.crt

Note: The script above needs the acme-tiny tool, so don't forget to install it:

$ sudo apt-get install acme-tiny

And the .letsencrypt/ subfolder contains this:

(root@host) {/var/lib/letsencrypt} # ls -al .letsencrypt/
total 32
drwx------ 2 letsencrypt nogroup  4096 Mar  6  2017 .
drwx--x--- 8 letsencrypt www-data 4096 Mar  1  2017 ..
-rw-r--r-- 1 root        root     3247 Mar  1  2017 account-host_example_com.key
-rw-r--r-- 1 root        root     1984 Mar  1  2017 letsencryptauthorityx3.pem
-rw-r--r-- 1 root        root     1647 Nov 16  2016 lets-encrypt-x3-cross-signed.pem
-rw-r--r-- 1 root        root     2480 Mar  1  2017 host_example_com.csr

The file host_example_com.csr needs to be copied over. We just created it above. Remember?

The files letsencryptauthorityx3.pem [2] and lets-encrypt-x3-cross-signed.pem [3] are intermediate certificates for the two different certificate chains offered by Let's Encrypt. For more info, see here [4].

The file account-host_example_com.key is our account key for Let's Encrypt. Personally, I prefer one account key per web server I run, but some people have one account.key file per admin. The file can be created with this command (do this as the letsencrypt system user we just created in subdir .letsencrypt/):

$ openssl genrsa 4096 > account-host_example_com.key   

Note: When immitating the above, make sure that the file permissions and file ownerships on your web server system match with what you read above.

Getting started with Apache2

For retrieval of the first certificate, nearly nothing needs to be configured in Apache2, except from it being reachable on port 80 and except from the challenges path being available as URL http://host.example.com/.well-known/acme-challenge:

Create /etc/apache2/conf-available/acme-tiny.conf with this content:

Alias /.well-known/acme-challenge/ /var/lib/letsencrypt/challenges/

<Directory /var/lib/letsencrypt/challenges>
    Require all granted
    Options -Indexes
</Directory>

... and enable it:

$ sudo a2enconf acme-tiny

... and restart Apache2:

$ sudo invoke-rc.d apache2 restart

Obtaining the Certificate from Let's Encrypt

Then, for (regularly) obtaining / updating our certificate, we create a script that can be found via $PATH and should normally be called with root privileges (/usr/local/sbin/letsencrypt-renew-certs), don't forget making it executable (chmod 0700 /usr/local/sbin/letsencrypt-renew-certs):

#!/bin/bash

su - letsencrypt -c ~letsencrypt/bin/letsencrypt-renew-certs

invoke-rc.d apache2 restart

This script drops privileges for the certificate file update part and then re-launches Apache2. This is the only part that requires root privileges on the web server. Point is: after the certificate has been updated, we need to restart (or reload?) the web server and this requires root privileges.

If all is correctly in place, you should be able to obtain (and later on update) the web server's SSL certificate from Let's Encrypt:

$ sudo letsencrypt-renew-certs

After the successful acme-tiny call, the .letsencrypt/ subfolder in /var/lib/letsencrypt looks like this:

(root@host) {/var/lib/letsencrypt} # ls -al .letsencrypt/
total 32
drwx------ 2 letsencrypt nogroup  4096 Mar  6  2017 .
drwx--x--- 8 letsencrypt www-data 4096 Mar  1  2017 ..
-rw-r--r-- 1 root        root     3247 Mar  1  2017 account-host_example_com.key
-rw-r--r-- 1 root        root     1984 Mar  1  2017 letsencryptauthorityx3.pem
-rw-r--r-- 1 root        root     1647 Nov 16  2016 lets-encrypt-x3-cross-signed.pem
-rw-r--r-- 1 letsencrypt nogroup     0 May  1 00:00 host_example_com.crt
-rw-r--r-- 1 root        root     2480 Mar  1  2017 host_example_com.csr
-rw-r--r-- 1 letsencrypt nogroup  4704 Apr  1 00:17 host_example_com.fullchain.crt

Plumbing it all into the Apache2 Config

First, the webserver needs to know where to find the SSL certificate file and the corresponding key file.

Adapting the basic Apache2 SSL Setup

The key file we earlier copied over into /etc/ssl/private/host_example_com.key.

The certificate file should now exist in /var/lib/letsencrypt/.letsencrypt. Personally, I prefer having it symlinked to /etc/ssl/certs/host_example_com.fullchain.crt:

$ sudo ln -s /var/lib/letsencrypt/.letsencrypt/host_example_com.fullchain.crt /etc/ssl/certs/host_example_com.fullchain.crt

Apache2 needs to be pointed to these two files (the .key and the .crt file). Enable the SSL module in Apache2, enable the default SSL site and modify the default SSL config like this:

diff --git a/apache2/sites-available/default-ssl.conf b/apache2/sites-available/default-ssl.conf
index 7e37a9c..4e8784a 100644
--- a/apache2/sites-available/default-ssl.conf
+++ b/apache2/sites-available/default-ssl.conf
@@ -29,8 +29,10 @@
                #   /usr/share/doc/apache2/README.Debian.gz for more info.
                #   If both key and certificate are stored in the same file, only the
                #   SSLCertificateFile directive is needed.
-               SSLCertificateFile      /etc/ssl/certs/ssl-cert-snakeoil.pem          
-               SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
+               #SSLCertificateFile     /etc/ssl/certs/ssl-cert-snakeoil.pem
+               #SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
+               SSLCertificateFile      /etc/ssl/certs/host_example_com.fullchain.crt
+               SSLCertificateKeyFile /etc/ssl/private/host_example_com.key

                #   Server Certificate Chain:
                #   Point SSLCertificateChainFile at a file containing the

Enable the SSL module:

$ sudo a2enmod ssl

Enable the default-ssl site:

$ sudo a2ensite default-ssl

Redirecting all traffic to https://

In the webserver, we now need to make sure that the URL (folder) http://host.example.com/.well-known/acme-challenge/ is always reachable for the Let's Encrypt signing engine on a non-encrypted connection.

Note 1: Once you have Let's Encrypt deployed, you want all requests coming in on http:// be redirected to https://.

Note 2: The challenges URL should be reachable without encryption (http://, rather than https://). Why? If your certificate upgrade fails some day, and you want to obtain a new certificate, you only can do it over a non-encrypted connection (as your https:// certificate has expired).

This is the configuration snippet that you need to place into all VirtualHost definitions (for non-encrypted access to the webserver, so normally under <VirtualHost <address>:80>) Watch out! Manual adaptation needed in the third line:

RewriteEngine on
RewriteCond %{REQUEST_URI} !/\.well-known/acme-challenge/.*
RewriteRule /(.*) https://vhost-server-name.example.com/$1 [L,NC,NE]

Enable the rewrite module:

a2enmod rewrite

Then restart Apache2:

invoke-rc.d apache2 restart

Testing in a Web Browser

Now open the URL http://host.example.com in a web browser and it should switch to https://host.example.com and the certificate should be trustworthy.

Testing another Let's Encrypt Signing Request

If the browser test above works ok, then try the sudo letsencrypt-renew-certs run again. It should get you another new certificate just fine (like when you ran it just before). You might want to check time stamps under /var/lib/letsencrypt/.letsencrypt.

If you do this re-test many many times, the Let's Encrypt / ACME site will block you at some point. Don't worry, this is only temporary and a DDoS protection mechanism.

Getting a new Certificate every Month

Every month I run the CRON job below. The CRONTAB entry (use e.g. VISUAL=mcedit crontab -e) looks like this:

0 0  1 * * /usr/local/sbin/letsencrypt-renew-certs 2>&1 1>/root/letsencrypt-renewal.log

References

• [1] https://github.com/diafygi/acme-tiny/blob/master/acme_tiny.py
       https://tracker.debian.org/pkg/acme-tiny
• [2] https://letsencrypt.org/certs/letsencryptauthorityx3.pem.txt
• [3] https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem.txt
• [4] https://letsencrypt.org/certificates/