How to use GitLab CI to deploy to multiple environments

This post was originally published on 2016-08-26, but is still so popular we updated it, thanks to Cesar Saavedra!

This post is a success story of one imaginary news portal, and you're the happy owner, the editor, and the only developer. Luckily, you already host your project code on GitLab.com and know that you can run tests with GitLab CI/CD. Now you’re curious if it can be used for deployment, and how far can you go with it.

To keep our story technology stack-agnostic, let's assume that the app is just a set of HTML files. No server-side code, no fancy JS assets compilation.

Destination platform is also simplistic – we will use Amazon S3.

The goal of the article is not to give you a bunch of copy-pasteable snippets. The goal is to show the principles and features of GitLab CI so that you can easily apply them to your technology stack.

Let’s start from the beginning. There's no continuous integration (CI) in our story yet.

At the starting line

Deployment: In your case, it means that a bunch of HTML files should appear on your S3 bucket (which is already configured for static website hosting).

There are a million ways to do it. We’ll use the awscli library, provided by Amazon.

The full command looks like this:

aws s3 cp ./ s3://yourbucket/ --recursive --exclude "*" --include "*.html"

Pushing code to repository and deploying are separate processes

Important detail: The command expects you to provide AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. Also you might need to specify AWS_DEFAULT_REGION.

Let’s try to automate it using GitLab CI.

The first automated deployment

With GitLab, there's no difference on what commands to run. You can set up GitLab CI in a way that tailors to your specific needs, as if it was your local terminal on your computer. As long as you execute commands there, you can tell CI to do the same for you in GitLab. Put your script to .gitlab-ci.yml and push your code – that’s it: CI triggers a job and your commands are executed.

Now, let's add some context to our story: Our website is small, there is 20-30 daily visitors and the code repository has only one default branch: main (sometimes referred to as the master branch).

Let's start by specifying a job with the command from above in the .gitlab-ci.yml file:

deploy:
  script: aws s3 cp ./ s3://yourbucket/ --recursive --exclude "*" --include "*.html"

No luck:

It is our job to ensure that there is an aws executable. To install awscli we need pip, which is a tool for Python packages installation. Let's specify Docker image with preinstalled Python, which should contain pip as well:

deploy:
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://yourbucket/ --recursive --exclude "*" --include "*.html"

You push your code to GitLab, and it is automatically deployed by CI

The installation of awscli extends the job execution time, but that is not a big deal for now. If you need to speed up the process, you can always look for a Docker image with preinstalled awscli, or create an image by yourself.

Also, let’s not forget about these environment variables, which you've just grabbed from AWS Console:

variables:
  AWS_ACCESS_KEY_ID: "AKIAIOSFODNN7EXAMPLE"
  AWS_SECRET_ACCESS_KEY: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
deploy:
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://yourbucket/ --recursive --exclude "*" --include "*.html"

It should work, but keeping secret keys open, even in a private repository, is not a good idea. Let's see how to deal with this situation.

Keeping secret things secret

GitLab has a special place for secret variables: Settings > CI/CD > Variables

Whatever you put there will be turned into environment variables. Checking the "Mask variable" checkbox will obfuscate the variable in job logs. Also, checking the "Protect variable" checkbox will export the variable to only pipelines running on protected branches and tags. Users with Owner or Maintainer permissions to a project will have access to this section.

We could remove variables section from our CI configuration. However, let’s use it for another purpose.

How to specify and use variables that are not secret

When your configuration gets bigger, it is convenient to keep some of the parameters as variables at the beginning of your configuration. Especially if you use them in more than one place. Although it is not the case in our situation yet, let's set the S3 bucket name as a variable for the purpose of this demonstration:

variables:
  S3_BUCKET_NAME: "yourbucket"
deploy:
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://$S3_BUCKET_NAME/ --recursive --exclude "*" --include "*.html"

So far so good:

In our hypothetical scenario, the audience of your website has grown, so you've hired a developer to help you. Now you have a team. Let's see how teamwork changes the GitLab CI workflow.

How to use GitLab CI with a team

Now, that there are two users working in the same repository, it is no longer convenient to use the main branch for development. You decide to use separate branches for both new features and new articles and merge them into main when they are ready.

The problem is that your current CI config doesn’t care about branches at all. Whenever you push anything to GitLab, it will be deployed to S3.

Preventing this problem is straightforward. Just add only: main to your deploy job.

You don't want to deploy every branch to the production website

But it would also be nice to preview your changes from feature-branches somehow.

How to set up a separate place for testing code

The person you recently hired, let's call him Patrick, reminds you that there is a featured called GitLab Pages. It looks like a perfect candidate for a place to preview your work in progress.

To host websites on GitLab Pages your CI configuration file should satisfy three simple rules:

• The job should be named pages
• There should be an artifacts section with folder public in it
• Everything you want to host should be in this public folder

The contents of the public folder will be hosted at http://<username>.gitlab.io/<projectname>/

After applying the example config for plain-html websites, the full CI configuration looks like this:

variables:
  S3_BUCKET_NAME: "yourbucket"

deploy:
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://$S3_BUCKET_NAME/ --recursive --exclude "*" --include "*.html"
  only:
  - master

pages:
  image: alpine:latest
  script:
  - mkdir -p ./public
  - cp ./*.html ./public/
  artifacts:
    paths:
    - public
  except:
  - master

We specified two jobs. One job deploys the website for your customers to S3 (deploy). The other one (pages) deploys the website to GitLab Pages. We can name them "Production environment" and "Staging environment", respectively.

All branches, except master, will be deployed to GitLab Pages

Introducing environments

GitLab offers support for environments (including dynamic environments and static environments), and all you need to do it to specify the corresponding environment for each deployment job:

variables:
  S3_BUCKET_NAME: "yourbucket"

deploy to production:
  environment: production
  image: python:latest
  script:
  - pip install awscli
  - aws s3 cp ./ s3://$S3_BUCKET_NAME/ --recursive --exclude "*" --include "*.html"
  only:
  - master

pages:
  image: alpine:latest
  environment: staging
  script:
  - mkdir -p ./public
  - cp ./*.html ./public/
  artifacts:
    paths:
    - public
  except:
  - master

GitLab keeps track of your deployments, so you always know what is currently being deployed on your servers:

GitLab provides full history of your deployments for each of your current environments:

Now, with everything automated and set up, we’re ready for the new challenges that are just around the corner.

How to troubleshoot deployments

It has just happened again. You've pushed your feature-branch to preview it on staging and a minute later Patrick pushed his branch, so the staging environment was rewritten with his work. Aargh!! It was the third time today!

Idea! Let's use Slack to notify us of deployments, so that people will not push their stuff if another one has been just deployed!

Using Slack notifications for deployments

Setting up Slack notifications is a straightforward process.

The whole idea is to take the incoming WebHook URL from Slack...

...and put it into Settings > Integrations > Slack notifications together with your Slack username:

Since the only thing you want to be notified about is deployments, you can uncheck all the checkboxes except the "Deployment" in the settings above. That’s it. Now you’re notified for every deployment:

Teamwork at scale

As the time passed, your website became really popular, and your team has grown from two people to eight people. People develop in parallel, so the situation when people wait for each other to preview something on Staging has become pretty common. "Deploy every branch to staging" stopped working.

It's time to modify the process one more time. You and your team agreed that if someone wants to see their changes on the staging server, they should first merge the changes to the "staging" branch.

The change of .gitlab-ci.yml is minimal:

except:
- master

is now changed to

only:
- staging

People have to merge their feature branches before preview on the staging server

Of course, it requires additional time and effort for merging, but everybody agreed that it is better than waiting.

How to handle emergencies

You can't control everything, so sometimes things go wrong. Someone merged branches incorrectly and pushed the result straight to production exactly when your site was on top of HackerNews. Thousands of people saw your completely broken layout instead of your shiny main page.

Luckily, someone found the Rollback button, so the website was fixed a minute after the problem was discovered.

Rollback relaunches the previous job with the previous commit

Anyway, you felt that you needed to react to the problem and decided to turn off auto-deployment to Production and switch to manual deployment. To do that, you needed to add when: manual to your job.

As you expected, there will be no automatic deployment to Production after that. To deploy manually go to CI/CD > Pipelines, and click the button:

Fast forward in time. Finally, your company has turned into a corporation. Now, you have hundreds of people working on the website, so all the previous compromises no longer work.

Time to start using Review Apps

The next logical step is to boot up a temporary instance of the application per feature branch for review.

In our case, we set up another bucket on S3 for that. The only difference is that we copy the contents of our website to a "folder" with the name of the the development branch, so that the URL looks like this:

http://<REVIEW_S3_BUCKET_NAME>.s3-website-us-east-1.amazonaws.com/<branchname>/

Here's the replacement for the pages job we used before:

review apps:
  variables:
    S3_BUCKET_NAME: "reviewbucket"
  image: python:latest
  environment: review
  script:
  - pip install awscli
  - mkdir -p ./$CI_BUILD_REF_NAME
  - cp ./*.html ./$CI_BUILD_REF_NAME/
  - aws s3 cp ./ s3://$S3_BUCKET_NAME/ --recursive --exclude "*" --include "*.html"

The interesting thing is where we got this $CI_BUILD_REF_NAME variable from. GitLab predefines many environment variables so that you can use them in your jobs.

Note that we defined the S3_BUCKET_NAME variable inside the job. You can do this to rewrite top-level definitions.

Visual representation of this configuration:

The details of the Review Apps implementation varies widely, depending upon your real technology stack and on your deployment process, which is outside the scope of this blog post.

It will not be that straightforward, as it is with our static HTML website. For example, you had to make these instances temporary, and booting up these instances with all required software and services automatically on the fly is not a trivial task. However, it is doable, especially if you use Docker containers, or at least Chef or Ansible.

We'll cover deployment with Docker in a future blog post. I feel a bit guilty for simplifying the deployment process to a simple HTML files copying, and not adding some hardcore scenarios. If you need some right now, I recommend you read the article "Building an Elixir Release into a Docker image using GitLab CI."

For now, let's talk about one final thing.

Deploying to different platforms

In real life, we are not limited to S3 and GitLab Pages. We host, and therefore, deploy our apps and packages to various services.

Moreover, at some point, you could decide to move to a new platform and will need to rewrite all your deployment scripts. You can use a gem called dpl to minimize the damage.

In the examples above we used awscli as a tool to deliver code to an example service (Amazon S3). However, no matter what tool and what destination system you use, the principle is the same: You run a command with some parameters and somehow pass a secret key for authentication purposes.

The dpl deployment tool utilizes this principle and provides a unified interface for this list of providers.

Here's how a production deployment job would look if we use dpl:

variables:
  S3_BUCKET_NAME: "yourbucket"

deploy to production:
  environment: production
  image: ruby:latest
  script:
  - gem install dpl
  - dpl --provider=s3 --bucket=$S3_BUCKET_NAME
  only:
  - master

If you deploy to different systems or change destination platform frequently, consider using dpl to make your deployment scripts look uniform.

Five key takeaways

1. Deployment is just a command (or a set of commands) that is regularly executed. Therefore it can run inside GitLab CI.
2. Most times you'll need to provide some secret key(s) to the command you execute. Store these secret keys in Settings > CI/CD > Variables.
3. With GitLab CI, you can flexibly specify which branches to deploy to.
4. If you deploy to multiple environments, GitLab will conserve the history of deployments, which allows you to rollback to any previous version.
5. For critical parts of your infrastructure, you can enable manual deployment from GitLab interface, instead of automated deployment.