Image copyright 2016 by Kharnagy, licensed under the Creative Commons Attribution-Share Alike 4.0 International license.

Introduction

This is the third in a series of occasional posts about security and DevOps. The ultimate goal of this series is to show how to build a reasonably secure Apache web server using the popular DevOps automation tool Chef. The server will be suitable for serving static content such as that generated by OctoPress. Each post explores a new aspect of Chef.

If you read the first and second posts in this series, you learned how to set up the Chef workstation and server; created webserver and base roles; created a test environment and a virtual machine; and built a partially hardened server called tester.local. This server has a minimized Apache configuration, and a restricted OpenSSH configuration.

In this post, I will demonstrate one of the most challenging aspects of any server automation project: copying sensitive keying materials, such as SSL private keys, to server nodes. Although SSL certificates themselves are not sensitive, certificate private keys are. In order to use Chef to truly “build security in,” these materials must be securely conveyed from the Chef server to the target server nodes. To do this, you will use Chef’s encrypted data bag feature and an add-on feature called chef-vault. You will create a custom cookbook recipe that performs all of the necessary decryption and file-creation actions on the target node. At the end of this post, you will possess a repeatable, reliable and secure method for conveying SSL keying materials or other secrets to target nodes.

Generate self-signed SSL certificate

To use SSL with your webserver, you must have an SSL certificate. In production environments, you will likely use a certificate signed by a public certificate authority, such as VeriSign, Thawte, or GoDaddy. But you can also use a self-signed certificate, which you will do here. At the command line, change to your chef-repo/.chef directory. Type:

openssl genrsa -aes128 -out tester.local.key-with-password 2048

This creates a 2048-bit RSA key and wraps it with 128-bit key secured by a password. You should see output similar to the following (and will be prompted to enter a password):

Generating RSA private key, 2048 bit long modulus
........+++
..+++
e is 65537 (0x10001)
Enter pass phrase for tester.local.key-with-password:
Verifying - Enter pass phrase for tester.local.key-with-password:

In most situations it isn’t desirable to have a password protecting the actual key file, because when you start Apache, it will block until the password is entered. If you have a large-scale infrastructure, you don’t want to type in passwords every time some random server starts up. (As a compensating control — later on in this post — you will make the key file accessible only to root.) To ensure that Apache starts cleanly, let’s remove the password. Type:

openssl rsa -in tester.local.key-with-password -out tester.local.key

Enter the password when prompted. A new key file tester.local.key will be created. Remove the old password-protected key file:

rm tester.local.key-with-password

Next, generate a certificate signing request (CSR). Type:

openssl req -new -key tester.local.key -out tester.local.csr

Enter data used in the certificate: country name, state, locality, organization name, OU name, common name, and email address, as shown in the sample output below:

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:Massachusetts
Locality Name (eg, city) []:Boston
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Markerbench
Organizational Unit Name (eg, section) []:SSL Test Certificate Directorate
Common Name (e.g. server FQDN or YOUR name) []:tester.local
Email Address []:nobody@markerbench.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Most of the fields are mandatory, but only one really matters: the common name. This must match the name of the webserver host; in this case, tester.local.

The CSR will be written to tester.local.csr. After it is created, create a self-signed certificate by typing:

openssl x509 -req -days 365 -in tester.local.csr -signkey tester.local.key -out tester.local.pem

You will see this output:

Signature ok
subject=/C=US/ST=Massachusetts/L=Boston/O=Markerbench/OU=SSL Test Certificate Directorate/CN=tester.local/emailAddress=nobody@markerbench.com
Getting Private key

The certificate will be written to tester.local.pem. You can verify the certificate contents by using OpenSSL’s x509 command:

openssl x509 -in tester.local.crt -noout -text

You should see output similar to this:

Certificate:
Data:
    Version: 1 (0x0)
    Serial Number:
        90:aa:b2:e4:06:ca:50:32
    Signature Algorithm: sha1WithRSAEncryption
    Issuer: C=US, ST=Massachusetts, L=Boston, O=Markerbench, OU=SSL Test Certificate Directorate, CN=tester.local/emailAddress=nobody@markerbench.com
    Validity
        Not Before: Oct  5 21:05:32 2013 GMT
        Not After : Oct  5 21:05:32 2014 GMT
    Subject: C=US, ST=Massachusetts, L=Boston, O=Markerbench, OU=SSL Test Certificate Directorate, CN=tester.local/emailAddress=nobody@markerbench.com
    Subject Public Key Info:
        Public Key Algorithm: rsaEncryption
        RSA Public Key: (2048 bit)
            Modulus (2048 bit):
                00:c4:3b:79:55:78:15:c2:82:a6:e3:e9:f0:64:c7:
                …(content omitted)
                56:e1:57:0d:b0:e0:37:31:19:ee:31:95:8f:2f:a6:
                c3:3b
            Exponent: 65537 (0x10001)
Signature Algorithm: sha1WithRSAEncryption
    1e:a6:1a:27:d3:5d:08:bc:ad:00:df:4e:6a:5b:4c:a4:be:80:
    …(content omitted)
    23:0e:02:be:3e:e8:89:75:58:03:7d:70:ac:13:a3:f4:d5:02:
    2e:d8:58:7f

Congratulations. You have created a self-signed certificate called tester.local.pem in the local directory, and a corresponding private key file tester.local.key.

Installing Chef-vault for distributing secrets

With the certificate and private key created, the next challenge is to use Chef to copy these two files to the correct locations on the web server. SSL certificates are stored at /etc/ssl/certs; private keys are stored at /etc/ssl/private.

There are many ways to convey the SSL certificate and key to the webserver. The easiest way would be to include these files in a cookbook as file resources, then use a recipe to copy them to the correct locations on the server. That is easy, but not very secure because the keying materials would be stored as part of the cookbook, and therefore in the clear. (The Git source tree contains your cookbooks and all of their supporting files. If the SSL private key were checked in as an unencrypted regular file, other people would be able to see it.)

It would be much nicer to store the SSL private key in an encrypted form — somehow — so that it can be copied to the server without worrying about who sees it (an attacker would just see ciphertext). After it is copied the server note, it can be decrypted in-place and moved to the correct destination in /etc/ssl/private.

Chef has some tools that make conveying secret materials easier. It provides a construct called a data bag for storing custom configuration items and other materials that target nodes need. Data bags are essentially hash maps (otherwise known as associative arrays, or in Rubyspeak, “hashes”) that are stored on the Chef server and retrieved by target nodes when chef-client runs. Data bags items can be encrypted. To encrypt a data bag item, you pass an symmetric encryption key (or password) to the knife data bag create command. For example:

knife data bag create certs tester_local_key --secret-file /tmp/my_secret_key

…where my_secret_key is a secret key generated, for example, by OpenSSL. To decrypt the item on target nodes, the target nodes perform an equivalent decryption operation, passing in the same secret key used to originally encrypt the item.

Go back and re-read that last paragraph. See anything problematic? Nodes that need to decrypt the data bag item need the the secret key used to encrypt it. That might sound obvious, but the it raises a question: how does the secret key actually get copied to the nodes? The Chef documentation is silent about how this is done; it leaves the problem of key management as an exercise for the reader. We are left to assume that the secret key is “somehow” copied to the target nodes.

How should this be done? You could copy the secret key using a Chef recipe, but you’d have the same problem all over again — sensitive keying material would be exposed. You could, instead, copy the secret key manually to the node over SSH, but that defeats the whole point of Chef — automation of configuration tasks.

It would be much nicer if there was a way to encrypt sensitive materials without requiring lots of complicated key management. Sadly, Chef does not provide such a method. However, a clever programmer named Kevin Moser, who works for Nordstrom, has created Chef plugin called chef-vault that solves the key management problem rather elegantly.

Kevin’s chef-vault tool takes a clever approach. Chef-vault uses a type of key encapsulation to protect secret materials using the public keys of target nodes that need them. These public keys are the same ones the nodes use to authenticate to Chef. Because these keys must, by definition, already exist, using them for encryption creates no extra work for you. Essentially, Chef-vault’s encrypt operation does the following:

Creates a symmetric encryption key (“secret key”). This secret key encrypts the plaintext (the thing you want to encrypt), and creates a ciphertext.
Adds the ciphertext to the data bag.
For each target node, encrypts the secret key with the node’s public key, creating an encapsulated key blob. The same operation is repeated for authorized users using their public keys as well
Adds each encapsulated key blob to the data bag

The result of Chef-vault’s encrypt operation is a data bag that contains an encrypted item for the secret being protected, and, for each authorized user and target node, an encrypted data blob that allows each user or node (and only that user or node) to recover the encryption key and, thus, decrypt the encrypted item. Chef-vault (essentially) extends Chef’s data bag structures to use its own public-key encryption system so that secret keys can be conveyed securely to target nodes. This rather neatly solves the secret-key distribution problem.

Enough talk. Let’s install chef-vault. Normally, you would use the Ruby command gem install chef-vault to install it. However, as of this writing, only chef-vault version 2.1 has the ability to encrypt entire files. That ability is missing in the version of chef-vault in the Gem repositories. So you must build it yourself using the latest version from Github.

At the command line, change to a directory outside of your chef-repo directory. Type:

git clone https://github.com/Nordstrom/chef-vault.git

You will see output similar to this:

Cloning into 'chef-vault'...
remote: Counting objects: 667, done.
remote: Compressing objects: 100% (343/343), done.
remote: Total 667 (delta 324), reused 619 (delta 279)
Receiving objects: 100% (667/667), 107.30 KiB | 0 bytes/s, done.
Resolving deltas: 100% (324/324), done.

Change to the new chef-vault directory and build the Gem:

cd chef-vault
gem build chef-vault.gemspec

You will see output similar to this:

WARNING:  no homepage specified
WARNING:  description and summary are identical
  Successfully built RubyGem
  Name: chef-vault
  Version: 2.1.0
  File: chef-vault-2.1.0.gem

Install chef-vault by typing:

gem install chef-vault-2.1.0.gem

Ruby will report successful installation as follows:

Successfully installed chef-vault-2.1.0
1 gem installed
Installing ri documentation for chef-vault-2.1.0...
Installing RDoc documentation for chef-vault-2.1.0...

Creating an encrypted vault for the SSL certificate and key

With chef-vault installed, you can now use it to encrypt sensitive materials and convey them securely to nodes. As an example, let’s encrypt the SSL certificate file. Change back to the chef-repo directory. Type the following, where username is your Chef username:

knife encrypt create certs tester_local_pem --mode client --file .chef/tester.local.pem -A "username"

This Knife encrypt command tells chef-vault to encrypt the contents of file .chef/tester.local.pem (the SSL certificate) and to authorize the user username to decrypt or update its contents. You can use any valid Chef username, or multiple usernames separated by commas. (If you need to find out your Chef username is, type knife user list.)

The contents of the encrypt operation are added to a vault named certs. The vault is backed by a data bag with the same name. You can verify that the data bag certs exists by typing:

knife data bag list

You will see the data bag certs in the list. You can see the items added to the data bag via the knife data bag show command. Type:

knife data bag show certs

You will see the following items:

tester_local_pem
tester_local_pem_keys

The first item, tester_local_pem is a hash that contains the encrypted contents of the file. The second item, tester_local_pem_keys, is a hash containing the list of authorized nodes, users and their associated public-key-encrypted blobs.

Take a look at the encrypted file contents. The command for viewing data bag items is knife data bag show name-of-data-bag name-of-item. Type:

knife data bag show certs tester_local_pem

You should see output similar to the following:

file-content:
  cipher:         aes-256-cbc
  encrypted_data: QpB63Qv2650jwmWfj3IX4iXAIoGz8WZkggoV+wbLyI0T4nUivD5QBovdjtJU
  YkhI9QOrbW55HVwew7tLW+ee0cetjZm+Amaa0Gyo8ehBsTbRAeY3jkdWv8Ia
  …
  (content omitted)
  …
  jGAUa+xcdDedmBSiRxoUwrjSq85hnAGwmKovXqKZeK4=
  
  iv:             kOrZ5kIrTCmwRloUodCtgA==
  
  version:        1
file-name:
  cipher:         aes-256-cbc
  encrypted_data: kEL5rHzmx85diXKC1AL7EXdEID+SC1E58GuNFBeu9lK1k+Bv5GbcQXK/iDtS
  L8tQ
  
  iv:             xEhV676bjE4SwVYZZwkFtw==
  
  version:        1
id:           tester_local_pem

As you can see, the file_content hash key contains the a child hash containing the cipher (AES-256-CBC), initialization vector, and the encrypted data blob. The file-name hash key contains similar data that corresponds to the original file name (which is also encrypted).

Let’s look at the encryption keys. Type:

knife data bag show certs tester_local_pem_keys

You should see output similar to the following:

admins:  arj
arj:     SDqZuaFrpy28YOSDDhkyDmDBLPZHuRSDXjOgHklnaetDjl8QI7zuTvznmg1Q
…
(content omitted)
…
f+s7gdSVBZ0el7Uc9gDhOFZA0hz0ADqcIPd2hA90PQ==

clients:
id:      tester_local_pem_keys

Here, the admins entry’s value is arj, indicating that the user arj is authorized to decrypt or update the contents. The arj entry contains the secret key, encrypted with arj’s public key. Of course, instead of seeing arj you will see your own username. Note that the clients entry is empty because no nodes are authorized to decrypt yet.

You can decrypt the secret by using the knife decrypt command. Type:

knife decrypt certs tester_local_pem file-content --mode client

This command decrypts the payload stored in the key path tester_local_pem > file-content in data bag certs. Because your Chef user is an authorized user, you should be able to see the decrypted content. It starts with the string -----BEGIN CERTIFICATE-----. Compare the output to the contents of the file .chef/tester.local.pem; the contents should be identical.

At this point, only one party (your Chef user) is authorized to decrypt the certificate. Of course, the web server needs to be authorized also! To authorize more users or nodes after the original knife encrypt create operation, use the knife encrypt update command. In this case, you want to authorize the tester.local node that will actually use the SSL certificate. Type:

knife encrypt update certs tester_local_pem --mode client --file .chef/tester.local.pem -S "name:tester.local"

You can examine the contents of the data bag by typing knife data bag show certs tester_local_pem again. If you do that, you will see that the contents of the data bag item are much the same as before, although the initialization vector file-content > iv and encrypted data blocks encrypted_data are different. That is because the vault has re-encrypted the contents with different keys.

Examine the tester_local_pem_keys entry. Type:

knife data bag show certs tester_local_pem_keys

You will see that the contents of this entry are now a little different:

admins:       arj
arj:          SDqZuaFrpy28YOSDDhkyDmDBLPZHuRSDXjOgHklnaetDjl8QI7zuTvznmg1Q
…(content omitted)
f+s7gdSVBZ0el7Uc9gDhOFZA0hz0ADqcIPd2hA90PQ==

clients:      tester.local
id:           tester_local_pem_keys
tester.local: y7DM7oQZj9+Yd5oRLFA4eSVOZ/+g/NYUNjfMJvsxxd1Nv85yLigzjb1JlaYm
…(content omitted)
yYWtFXX47765NivPGNTszfJQ8igNNBy1+YvfQn/wNw==

You can see that the clients entry contains the value tester.local, and that a corresponding encrypted data blob named tester.local has been added. Splendid!

With the certificate correctly added to the vault, let’s add the private key. Instead of doing a two-step process of creating the encrypted data items and then authorizing the node, let’s do it in one step by supplying both the user and node to the create operation. Type:

knife encrypt create certs tester_local_key --mode client --file .chef/tester.local.key -A "arj" -S "name:tester.local"

Substitute your own username instead of arj, of course.

If you want to, you can verify that the SSL private key was added successfully by typing the now-familiar knife data bag show certs tester_local_key command.

Creating a cookbook for configuring SSL

At this point you have added the SSL certificate and its corresponding private key to the encrypted data vault certs. Now you need to get the vault’s contents over to the target nodes so you can create the certificate and private key files.

First, create a new cookbook called ssl-config:

knife cookbook create ssl-config

Add the new cookbook to the webserver role so that it is executed whenever chef-client runs:

knife role edit webserver

Add the recipe for ssl-config to the role by editing the run_list as follows:

"run_list": [
  "recipe[apt]",
  "recipe[apache2]",
  "recipe[ssl-config]"
],

Next, edit the default recipe file ssl-config/recipes/default.rb as follows:

chef_gem 'chef-vault'
require 'chef-vault'

directory '/etc/ssl/certs' do
  recursive true
  owner 'root'
  group 'root'
  mode '0755'
end

directory '/etc/ssl/private' do
  owner 'root'
  group 'root'
  mode '0700'
end

# Certificate entries equal to hostname but with _ replaced by .
vault         = 'certs'
hostname      = node['fqdn']
cert_prefix   = hostname.sub('.','_')
cert_cert     = "#{cert_prefix}_pem"
cert_key      = "#{cert_prefix}_key"
cert_chain    = "#{cert_prefix}_chain"
puts "Creating certificates for #{hostname} using vault #{vault}."

# Decrypt certificate
puts "Decrypting certificate from hash item #{cert_cert}."
begin
  item = ChefVault::Item.load(vault,cert_cert)
  file "/etc/ssl/certs/ssl-cert-snakeoil.pem" do
    owner 'root'
    group 'root'
    mode '0444'
    content item['file-content']
  end
rescue ChefVault::Exceptions::KeysNotFound
  raise ChefVault::Exceptions::ItemNotFound,
    "Certificate not found at #{vault}/#{cert_cert}!"
end

# Decrypt certificate chain
puts "Decrypting certificate chain."
begin
  item = ChefVault::Item.load(vault,cert_chain)
  file "/etc/ssl/certs/#{hostname}.chain" do
    owner 'root'
    group 'root'
    mode '0444'
    content item['file-content']
  end
rescue ChefVault::Exceptions::KeysNotFound
  Chef::Log.warn("No certificate chain in #{vault}/#{cert_chain}.")
end

# Decrypt private key
puts "Decrypting key from hash item #{cert_key}."
begin
  item = ChefVault::Item.load(vault,cert_key)
  file "/etc/ssl/private/ssl-cert-snakeoil.key" do
    owner 'root'
    group 'root'
    mode '0400'
    content item['file-content']
  end
rescue ChefVault::Exceptions::KeysNotFound
  raise ChefVault::Exceptions::ItemNotFound,
    "Private key not found at #{vault}/#{cert_key}!"
end

# Configure the SSL default site if enabled
apache_site "default-ssl" do
  enable node['apache']['default_site_enabled']
end

There might appear to be a lot going on in this recipe, but it is actually quite simple. First, the chef_gem and require lines tell the target node’s Chef client to download the chef-vault Gem.

The directory '/etc/ssl/certs' do block creates the directory that should contain the SSL certificate /etc/ssl/certs if it does not already exist. Directory ownership is changed to root and it is made world-readable.

The directory '/etc/ssl/private' do block creates the directory that should contain the SSL private key /etc/ssl/private if it does not already exist. Directory ownership is changed to root and it is made readable only by root.

The next part of the recipe assigns variables used for looking up and decrypting the node’s certificate, private key and certificate chain. The key names for these items are equal to the fully-qualified domain name of the node with periods escaped as underscores, plus the _pem, _key, and _chain suffixes, respectively. For example, for your test VM tester.local these values are tester_local_pem, tester_local_key, and tester_local_chain. (In case you were wondering: that is why the knife encrypt create commands you typed earlier created items named tester_local_pem and tester_local_key.)

The next three code blocks (each beginning with the comment # Decrypt) actually decrypt the file contents and save them to files. Let’s look the first of these.

In the first decryption block, the line ChefVault::Item.load(vault,cert_cert) decrypts the certificate object and assigns the result to the variable item. The value of item will be a hash. The next 6 lines that begin with file "/etc/ssl/certs/ssl-cert-snakeoil.pem" do create the certificate file, assign ownership to root, make it world-readable, and set the contents to item’s hash entry named file-content. Note that all of this code is enclosed in a begin/rescue/end block, so that the ChefVault::Exceptions::KeysNotFound exception can be trapped. ChefVault::Item.load throws this exception if the vault does not contain the expected entry, in this case one whose key is tester_local_pem. If the entry is not found (for example, because you forgot to add the certificate to the vault), the recipe will throw and exception and fail — as it should.

The second decryption block decrypts and saves the certificate chain, if one was added to the vault. Because tester.local’s SSL certificate was self-signed, it does not need a certificate chain. However, in production situations you might have one, and if you do, you can ensure that it is copied to the server by adding it to vault using the usual knife encrypt create command and specifying an item named nodename_chain, where nodename is the escaped form of the fully-qualified domain name (periods replaced by underscores). Unlike the first decryption block, however, the recipe does not crash and burn if the certificate chain item is not found. Instead, the recipe simply warns that no chain was found.

The third decryption block decrypts and saves the private key. As with the first decryption block, the recipe fails if the key’s expected entry is not found in the vault.

The last block turns on the default-ssl site in Apache, which is preconfigured to use the various ssl_snakeoil certificate files and private keys.

The ssl-config recipe is fairly bare-bones, but sufficiently flexible that it will work with any SSL-enabled web server node. As discussed above, all you must do is (1) ensure that the node’s SSL certificate and private key are added to the vault correctly, and (2) configure the node’s run-list so that it executes the ssl-config recipe.

Copying SSL certificates to the server

With all of the prep work out of the way, it is time to finally configure the server. Upload the ssl-config cookbook to the Chef server:

knife cookbook upload ssl-config

SSH into tester.local:

vagrant ssh

Once in, run chef-client:

sudo su
chef-client

Many console messages will scroll past you at a dizzying pace. Look for these lines:

Recipe: ssl-config::default
…
Creating certificates for tester.local using vault certs.
Decrypting certificate from hash item tester_local_pem.
Decrypting certificate chain.
[2013-10-06T23:29:50+00:00] WARN: No certificate chain in certs/tester_local_chain.
Decrypting key from hash item tester_local_key.

These indicate that the recipe worked as expected. If there is a problem finding or decrypting the certificate or private key, the output will show an exception. Assuming the recipe ran successfully, the output will also contain lines showing that the certificate and private key files were created also. Look for lines similar to these, which shows the certificate file was created:

- create new file /etc/ssl/certs/ssl-cert-snakeoil.pem
- update content in file /etc/ssl/certs/ssl-cert-snakeoil.pem from none to 53f4ae
    --- /etc/ssl/certs/ssl-cert-snakeoil.pem	2013-10-06 23:29:51.663590528 +0000
    +++ /tmp/.ssl-cert-snakeoil.pem20131006-9127-hmw8o1	2013-10-06 23:29:51.667592528 +0000
    @@ -0,0 +1,23 @@
    +-----BEGIN CERTIFICATE-----

…and these, which shows the private key file was created:

- create new file /etc/ssl/private/ssl-cert-snakeoil.key
- update content in file /etc/ssl/private/ssl-cert-snakeoil.key from none to 60fcbe
    --- /etc/ssl/private/ssl-cert-snakeoil.key	2013-10-06 23:29:51.727622527 +0000
    +++ /tmp/.ssl-cert-snakeoil.key20131006-9127-m9p4zk	2013-10-06 23:29:51.731624527 +0000
    @@ -0,0 +1,27 @@
    +-----BEGIN RSA PRIVATE KEY-----

After the recipe runs, you can verify the files were correctly created by cat-ing the files /etc/ssl/certs/ssl-cert-snakeoil.pem and /etc/ssl/private/ssl-cert-snakeoil.key. The files should be owned by root/root; permissions should be restricted to 444 and 400, respectively.

Testing the webserver

To test that the webserver is working as it should, we need to do two more things: edit the webserver role to enable SSL and the default site. Then, we re-push the cookbook and restart the server.

First, edit the role as follows using the usual command knife role edit webserver. As shown below, add SSL as an enabled module by adding "ssl", to the default_modules array, and turn set the default_site_enabled value to true:

"override_attributes": {
    "apache": {
      "allow_override": "None",
      "contact": "nobody@example.com",
      "default_modules": [
        "alias",
        "cgi",
        "deflate",
        "dir",
        "log_config",
        "logio",
        "mime",
        "rewrite",
        "ssl",
        "setenvif"
      ],
      "default_site_enabled": true,
....

Also, enable port 443 in the listen_ports section:

      "listen_ports": [
        "80",
        "443"
      ],

On tester.local, run chef-client as root again and watch the node converge using these new settings.

Then, open your browser to tester.local using regular HTTP. You should see a page that screams It works!. Try using HTTPS; you should see the same message (and likely after getting an SSL warning about an untrusted certificate).

Save your work

You are done. Back up your nodes, roles, data bags and environments from the Chef server to your local workstation. Type:

knife backup export

You will see the following output:

Backing up nodes
Backing up nodes tester.local
Backing up roles
Backing up roles base
Backing up roles webserver
Backing up data bags
Backing up data bag certs item tester_local_key
Backing up data bag certs item tester_local_key_keys
Backing up data bag certs item tester_local_pem
Backing up data bag certs item tester_local_pem_keys
Backing up environments
Backing up environments testing

Next, edit .gitignore in your chef-repo directory so that your SSL certificate and private key are not stored in Git. Add this line somewhere near the top (for example, underneath the line .chef/*.pem):

.chef/*.key

Finally, commit your work. You can see what files were modified with the usual command git status; if you do, you will see that some new files have been added:

.chef/chef_server_backup/data_bags/certs/
cookbooks/ssl-config/

You will see that a few have also been modified. Commit everything:

git commit -am "DevOps Secuity Handbook Part 3"

Remember, the keying materials (the *.key and *.pem files in the .chef directory) are not versioned in your Git repository. This is both a feature and a bug. You can safely move the tester.local.pem and tester.local.key files to offline media now, if you wish; they are safely encrypted in the data bag certs and no longer need to be in the local filesystem.

Next: Adding custom content

If you have completed the instructions in this post, you learned how to do some very useful things. You created a self-signed SSL certificate and private key for tester.local. You installed the chef-vault plugin for storing the SSL certificate and private key as encrypted data bag items. You authorized the user arj and node tester.local to decrypt these items. And you created a cookbook that decrypts the certificate, private key and certificate chain and creates files in the correct locations on the server.

In the next post, you will use Chef to configure Apache for serving custom content. You will create a non-privileged user whose home directory stores static HTML. This directory will be served up by Apache as the default website. In keeping with the SSH configuration introduced in this post, the user account will be configured to use SSH public keys for authentication rather than passwords.

The DevOps Security Handbook: Building Security in With Chef, Part III

This third post in a series about security and DevOps describes how to configure web servers with TLS.

Image copyright 2016 by Kharnagy, licensed under the Creative Commons Attribution-Share Alike 4.0 International license.