A part of my master’s thesis is to set up a Docker Swarm to parallelize reinforcement learning experiments. For this I needed a registry hosted by the swarm. This is because the swarm is unfortunately offline and I somehow have to distribute images across nodes.
Many tutorials online show how to set up a registry with SSL certificates and authentication using nginx. However, I wanted something a little simpler. Further, I don’t have a domain name that I can set as common name (CN), so I have to use the IP address for the certificate. This has to be added as SAN (subject alternate name), something that the usual tutorials don’t describe.
Note: “Also as of the Effective Date, the CA SHALL NOT issue a certificate with an Expiry Date later than 1 November 2015 with a subjectAlternativeName extension or Subject commonName field containing a Reserved IP Address or Internal Server Name.” – Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.0 Thus, it is necessary to use self-signed certificates in this scenario.
The process breaks down into 3 simple steps:
- Create a self-signed SSL certificate with IP SAN
- Setup the registry service using the certificate
- Give the Nodes in the Swarm Access to the Certificate
For this post my “swarm” will be a single manager node running in a VM.
The last command gives the IP that has to be named in the certificate. I first setup everything I need on the host machine, then deploy it to the swarm. This is fancy talk for “make a folder with all the good stuff and
scp it’s content to the VM” — poor mans deploy and as we know, all students are poor.
Create a self-signed SSL certificate with IP SAN
The IP for the node running my registry is: 192.168.99.102. Your one may be different. I wrote a custom
openssl.cnf based off the example at
/etc/ssl/openssl.cnf and placed it into
Remember that the IP in the last line may differ in your case. With that I could generate a private key and the certificate:
When prompted to enter some information I left everything blank. Usually the CN has to be equal to the domain name, but in this case the SAN takes care of this. Quickly verify that the SAN is specified:
openssl x509 -in certs/certificate.crt -text -noout
The important line is:
X509v3 Subject Alternative Name: IP Address:192.168.99.102
That’s all for the certificates.
Setup the registry service using the certificate
To make the deployment easy, I wrote a small docker-compose file that starts the registry service:
This launches the registry as a service on port 433. A potential flaw is that this container isn’t constrained to a specific machine. This is because volumes are not shared between nodes and thus all stored images would be lost if the container migrates. In the toy example there is only 1 machine; However, with an actual swarm (that’s the point of this exercise, right?) one would introduce such placement constraints or use a storage solution that migrates with the container.
Time for deployment:
The certificate will be wiped on reboot if we leave it in the home directory. Thus, I place it in a persistent location. Which is also the location the container reads it from.
Give the Nodes in the Swarm Access to the Certificate
The registry is ready and set up, however, when pushing or pulling docker will issue a self-signed certificate error, because it can not verify the certificate. To fix this, each client that wants to interact with the server needs a copy of the certificate. I installed the certificate on each client into
Then, I restarted the docker on the client, to reload certificates:
sudo service docker restart
Thats it! Now I can push to this registry just like any other registry.
docker tag registry:2 192.168.99.102/registry:2 docker push 192.168.99.102/registry:2 docker pull 192.168.99.102/registry:2
Thanks for reading and happy coding!