A Private Docker Registry with SSL on an Offline Docker Swarm

A part of my master’s thesis is to set up a Docker Swarm to parallelize reinforcement learning experiments. For this I needed a registry hosted by the swarm. This is because the swarm is unfortunately offline and I somehow have to distribute images across nodes.

Many tutorials online show how to set up a registry with SSL certificates and authentication using nginx. However, I wanted something a little simpler. Further, I don’t have a domain name that I can set as common name (CN), so I have to use the IP address for the certificate. This has to be added as SAN (subject alternate name), something that the usual tutorials don’t describe.

Note: “Also as of the Effective Date, the CA SHALL NOT issue a certificate with an Expiry Date later than 1 November 2015 with a subjectAlternativeName extension or Subject commonName field containing a Reserved IP Address or Internal Server Name.” – Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.0 Thus, it is necessary to use self-signed certificates in this scenario.

The process breaks down into 3 simple steps:

  1. Create a self-signed SSL certificate with IP SAN
  2. Setup the registry service using the certificate
  3. Give the Nodes in the Swarm Access to the Certificate

Setup

For this post my “swarm” will be a single manager node running in a VM.

The last command gives the IP that has to be named in the certificate. I first setup everything I need on the host machine, then deploy it to the swarm. This is fancy talk for “make a folder with all the good stuff and scp it’s content to the VM” — poor mans deploy and as we know, all students are poor.

Create a self-signed SSL certificate with IP SAN

The IP for the node running my registry is: 192.168.99.102. Your one may be different. I wrote a custom openssl.cnf based off the example at /etc/ssl/openssl.cnf and placed it into ~/my_docker_registry_deploy_folder/

Remember that the IP in the last line may differ in your case. With that I could generate a private key and the certificate:

When prompted to enter some information I left everything blank.  Usually the CN has to be equal to the domain name, but in this case the SAN takes care of this. Quickly verify that the SAN is specified:

openssl x509 -in certs/certificate.crt -text -noout

The important line is:

X509v3 Subject Alternative Name:
IP Address:192.168.99.102

That’s all for the certificates.

Setup the registry service using the certificate

To make the deployment easy, I wrote a small docker-compose file that starts the registry service:

This launches the registry as a service on port 433. A potential flaw is that this container isn’t constrained to a specific machine. This is because volumes are not shared between nodes and thus all stored images would be lost if the container migrates. In the toy example there is only 1 machine; However, with an actual swarm (that’s the point of this exercise, right?) one would introduce such placement constraints or use a storage solution that migrates with the container.

Time for deployment:

The certificate will be wiped on reboot if we leave it in the home directory. Thus, I place it in a persistent location. Which is also the location the container reads it from.

Give the Nodes in the Swarm Access to the Certificate

The registry is ready and set up, however, when pushing or pulling docker will issue a self-signed certificate error, because it can not verify the certificate. To fix this, each client that wants to interact with the server needs a copy of the certificate. I installed the certificate on each client into

/etc/docker/certs.d/192.168.99.102/certificate.crt

Then, I restarted the docker on the client, to reload certificates:

sudo service docker restart

Thats it! Now I can push to this registry just like any other registry.

docker tag registry:2 192.168.99.102/registry:2
docker push 192.168.99.102/registry:2
docker pull 192.168.99.102/registry:2

Thanks for reading and happy coding!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s