Jeff Butler - Senior DevOps Systems Engineer | Hyper-Local AI Modeling & Machine Learning

Deploy Dockerized Application to EC2 via GitLab CI/CD

When it comes to modern software development, a well-oiled CI/CD pipeline can be the difference between frustration and flow. Whether you're deploying a small web app or a complex microservices architecture, automating your processes ensures consistency, efficiency, and reliability.

In this post, we'll explore a practical example of a GitLab CI/CD pipeline for Dockerized applications. This setup handles building Docker images and deploying them to an EC2 instance on AWS seamlessly. We'll dive into the details, so whether you're a beginner or looking to refine your existing pipeline, there's something here for you.

Why Automate with GitLab CI/CD and Docker?

Automation in DevOps isn't just a convenience—it's a necessity. By leveraging GitLab CI/CD, you can integrate directly with your Git repository to create pipelines that build, test, and deploy your code. Combined with Docker's portability and consistency, you get a workflow that's robust, scalable, and easy to replicate.

Imagine pushing a single commit to your master branch and having your app automatically built and deployed to your production server. No manual file uploads. No repetitive commands. Just smooth, automated magic.

Breaking Down the Pipeline

Let’s take a look at a real-world GitLab CI/CD configuration. This setup covers everything from building Docker images to deploying them on an EC2 instance. Below, you'll find a detailed explanation of each section of the pipeline configuration.

Pipeline Overview


stages:
  - build
  - deploy

image: docker:latest

services:
  - docker:dind

variables:
  DOCKER_HOST: "tcp://docker:2375/"
  DOCKER_TLS_CERTDIR: ""

before_script:
  - echo "Logging into GitLab Container Registry..."
  - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"

build:
  stage: build
  script:
    - echo "Building Docker image..."
    - docker build -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA" -f docker/Dockerfile .
    - echo "Pushing Docker image..."
    - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"

deploy:
  stage: deploy
  image: ubuntu:latest
  before_script:
    - apt-get update && apt-get install -y openssh-client docker-compose
    - echo -e "$SSH_PRIVATE_KEY" > ./deploy_key.pem
    - chmod 600 ./deploy_key.pem
    - eval "$(ssh-agent -s)"
    - ssh-add ./deploy_key.pem
    - ssh-add -L  # Verify key added
  script:
    # Copy necessary files to the EC2 instance
    - scp -i ./deploy_key.pem -o StrictHostKeyChecking=no docker-compose.yml ubuntu@your_ec2_ip:/home/ubuntu/your_project/
    - scp -i ./deploy_key.pem -o StrictHostKeyChecking=no -r src/ ubuntu@your_ec2_ip:/home/ubuntu/your_project/
    - scp -i ./deploy_key.pem -o StrictHostKeyChecking=no -r docker/ ubuntu@your_ec2_ip:/home/ubuntu/your_project/
    # SSH into EC2 and deploy with Docker Compose
    - ssh -i ./deploy_key.pem -o StrictHostKeyChecking=no ubuntu@your_ec2_ip "
        docker login registry.gitlab.com -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD &&
        cd /home/ubuntu/your_project &&
        docker-compose pull &&
        docker-compose up -d --build
      "
  only:
    - master

Defining Stages

stages:
  - build
  - deploy

build: In this stage, a Docker image is built and pushed to the GitLab Container Registry.
deploy: This stage deploys the built Docker image to a remote server using Docker Compose.

Base Image and Services

image: docker:latest

services:
  - docker:dind

image: docker:latest: Specifies the Docker environment in which the pipeline will run.
services: docker:dind: Activates Docker-in-Docker (DinD), enabling the pipeline to build and manage Docker images within the CI/CD environment.

Global Variables

variables:
  DOCKER_HOST: "tcp://docker:2375/"
  DOCKER_TLS_CERTDIR: ""

DOCKER_HOST: Configures the Docker daemon to communicate via a TCP socket.
DOCKER_TLS_CERTDIR: Disables Docker’s TLS directory requirement for DinD.

Authentication Before Scripts

before_script:
  - echo "Logging into GitLab Container Registry..."
  - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"

docker login: Authenticates with the GitLab Container Registry using credentials provided by GitLab’s CI/CD environment.

Build Stage

build:
  stage: build
  script:
    - echo "Building Docker image..."
    - docker build -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA" -f docker/Dockerfile .
    - echo "Pushing Docker image..."
    - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"

Steps: Build the Docker Image: The image is tagged with the commit hash ($CI_COMMIT_SHORT_SHA) to make it easier to identify and roll back to specific builds if needed. Push the Image: The built image is pushed to the GitLab Container Registry for storage and subsequent deployment.

Deploy Stage

deploy:
  stage: deploy
  image: ubuntu:latest
  before_script:
    - apt-get update && apt-get install -y openssh-client docker-compose
    - echo -e "$SSH_PRIVATE_KEY" > ./deploy_key.pem
    - chmod 600 ./deploy_key.pem
    - eval "$(ssh-agent -s)"
    - ssh-add ./deploy_key.pem
    - ssh-add -L  # Verify key added

Install Prerequisites: Ensures ssh and docker-compose are available in the deployment environment. SSH Key Setup: Stores the private key from the CI/CD environment in a file deploy_key.pem. Adds the key to the SSH agent to facilitate secure connections to the server.

Deployment Steps

script:
    # Copy necessary files to the EC2 instance
    - scp -i ./deploy_key.pem -o StrictHostKeyChecking=no docker-compose.yml ubuntu@your_ec2_ip:/home/ubuntu/your_project/
    - scp -i ./deploy_key.pem -o StrictHostKeyChecking=no -r src/ ubuntu@your_ec2_ip:/home/ubuntu/your_project/
    - scp -i ./deploy_key.pem -o StrictHostKeyChecking=no -r docker/ ubuntu@your_ec2_ip:/home/ubuntu/your_project/
    # SSH into EC2 and deploy with Docker Compose
    - ssh -i ./deploy_key.pem -o StrictHostKeyChecking=no ubuntu@your_ec2_ip "
        docker login registry.gitlab.com -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD &&
        cd /home/ubuntu/your_project &&
        docker-compose pull &&
        docker-compose up -d --build
      "

File Transfer:

File Transfer
- The configuration (docker-compose.yml) and application files (src/ and docker/) are transferred to the target EC2 instance using scp.
Remote Commands:
- SSHs into the remote EC2 instance.
- Logs into the GitLab Container Registry to pull the latest Docker image.
- Executes docker-compose pull and docker-compose up to update and start the application.

Conditions

only:
  - master

Specifies that the deployment stage runs only for changes pushed to the master branch.

Key Takeaways

Automation: The pipeline automates the build and deployment process, ensuring consistency and reducing manual intervention.
Security: Securely handles sensitive data (e.g., SSH keys, Docker credentials) using environment variables.
Reusability: The structured pipeline can be adapted for similar Dockerized applications across projects.
Efficiency: The use of Docker Compose streamlines the deployment on remote servers, enabling rapid updates and scalability.

This CI/CD configuration is a powerful example of modern DevOps practices, combining containerization with GitLab’s robust automation capabilities.

What Makes This Workflow Effective?

Simplicity: With just a few stages (build and deploy), the pipeline focuses on doing a few things well.
Security: Environment variables handle sensitive credentials like Docker registry login details and SSH keys, keeping your secrets out of the codebase.
Flexibility: This pipeline can be easily adapted for various environments and projects, whether you're deploying to EC2, Kubernetes, or other platforms.
Scalability: Docker Compose ensures that your app is deployed in a consistent environment, no matter where it's running.

Tips for Getting the Most Out of Your CI/CD Pipeline

Keep Your Dockerfiles Lean: A smaller image means faster builds and deployments.
Use Environment Variables: Store secrets like API keys securely in GitLab CI/CD's settings.
Test Locally Before Pushing: Debugging a pipeline can be time-consuming, so catch issues early by testing locally.
Monitor Your Pipeline: Keep an eye on execution times and failure rates to optimize your workflow.

Ready to Build Your Pipeline?

By adopting automation and tools like GitLab CI/CD and Docker, you empower your team to focus on what matters—building and improving your application. This pipeline example is just the start. With a little customization, you can adapt it to suit your exact needs, whether you're running a simple blog or a multi-service platform.

Streamlining Your CI/CD Workflow with GitLab and Docker