Last week Docker released a new version, 19.03, which brings a few exciting features with it.
One of the features affects GitLab CI/CD when using the Docker in Docker
workflow.
As of version 19.03, docker:dind
will automatically generate TLS certificates and require using them for
communication. This is from Docker's official
documentation:
Starting in 18.09+, the dind variants of this image will automatically generate TLS certificates in the directory specified by the DOCKER_TLS_CERTDIR environment variable. Warning: in 18.09, this behavior is disabled by default (for compatibility). If you use --network=host, shared network namespaces (as in Kubernetes pods), or otherwise have network access to the container (including containers started within the dind instance via their gateway interface), this is a potential security issue (which can lead to access to the host system, for example). It is recommended to enable TLS by setting the variable to an appropriate value (-e DOCKER_TLS_CERTDIR=/certs or similar). In 19.03+, this behavior is enabled by default.
When you upgrade to 19.03 (which is done automatically if using
docker:dind) you may start seeing an issue like:
docker: Cannot connect to the Docker daemon at tcp://docker:2375. Is the docker daemon running?.
To fix the problem above you have two options:
- Configure GitLab Runner to use TLS.
- Explicitly turn off TLS.
The shared Runners available on GitLab.com support both workflows, which are described in detail below.
You may notice that we are now also suggesting a specific version such as
docker:19.03.0-dind and not docker:dind. This is to help prevent users'
jobs randomly failing when a new update comes out.
Configure TLS
Since the service docker:dind will create the certificates, we need to
have the certificate shared between the service and the job container.
To do this we have to add a mount inside of the
volumes
under the [runners.docker] section.
For example:
[[runners]]
name = "My Docker Runner"
url = "http://gitlab.com"
token = ""
executor = "docker"
[runners.custom_build_dir]
[runners.docker]
privileged = true
volumes = ["/certs/client", "/cache"]
shm_size = 0
If you're a GitLab.com user, we've already done the config change above for you on the Shared Runners.
Also, update .gitlab-ci.yml accordingly to specify the
DOCKER_TLS_CERTDIR
image: docker:19.03.0
variables:
DOCKER_DRIVER: overlay2
# Create the certificates inside this directory for both the server
# and client. The certificates used by the client will be created in
# /certs/client so we only need to share this directory with the
# volume mount in `config.toml`.
DOCKER_TLS_CERTDIR: "/certs"
services:
- docker:19.03.0-dind
before_script:
- docker info
build:
stage: build
script:
- docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests
Disable TLS
You might not have access to update the volume mounting inside of the
config.toml, so the only option is to disable TLS. You can do this by
setting the environment variable DOCKER_TLS_CERTDIR to an empty value.
For GitLab.com Shared Runners users this is done already using the environment settings, which works the same way.
image: docker:19.03.0
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
services:
- docker:19.03.0-dind
before_script:
- docker info
build:
stage: build
script:
- docker build -t my-docker-image .
- docker run my-docker-image /script/to/run/tests
We would like to thank the rest of the community with all the feedback and help throughout #4501.
