Uncloud and Gitlab integration

The Uncloud Howto delves deep(er) into what you get when you have an Uncloud cluster to play with.

Keeping services within a cluster up to date is essential, but manually updating versions can quickly become tedious. This guide explains how to connect a GitLab project to an Uncloud cluster to automate the deployment of new images and services.

Prerequisites

1. An Uncloud cluster. For Gitlab to have access the config.yaml should:

   current_context: test
   contexts:
     test:
       connections:
         - ssh: uc@uncloud5.vm.science.ru.nl
           ssh_key_file: $UNCLOUD_SSH_PRIVATE_KEY
           machine_id: 0b104736144524a903ae4d523c0e0ed5
   

   The idea of the $UNCLOUD_SSH_PRIVATE_KEY variable is that it is populated by Gitlab.
2. SSH access to the cluster. See the access section. As the SSH private key need to be uploaded into GitLab it is advisable to just create another SSH key of type login for cluster access.
3. Make sure your application is available as a container image. See our image project for an example. You’ll need a runner that is able to run docker or docker-in-docker. Of course you are also free to run container images from the Docker registry.
4. A compose.yaml with the services you want to deploy. See compose.yaml changes for Uncloud specific differences below in this document.

CI/CD Configuration

Next, we need to teach Gitlab how to connect to the cluster. Lets create the following variables in the Gitlab project:

And the .gitlab-ci.yml part:

deploy_uncloud:
  stage: deploy
  environment: test
  image: registry.science.ru.nl/cncz/sys/image/uc:latest
  before_script:
    - mkdir -p ~/.docker
    - |
      cat <<EOF > ~/.docker/config.json
      {
          "auths": {
              "$CI_REGISTRY": {
                  "auth": "$(echo -n $CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD | base64)"
              }
          }
      }
      EOF
    - chmod 600 $UNCLOUD_SSH_PRIVATE_KEY
  script:
    - uc deploy -f compose.yaml -y

In this job:

• environment: test: environments allow you the use same CI/CD variables with different values, see Gitlab CI/CD Environments for more.
• image: registry.science.ru.nl/cncz/sys/image/uc:latest: this image is one that includes the uc CLI tooling. Using this image saves you from installing uncloud/uc on every pipeline run.
• Create a ~/.docker/config.json if you are using a private registry.

     - |
     cat <<EOF > ~/.docker/config.json
     {
         "auths": {
             "$CI_REGISTRY": {
                 "auth": "$(echo -n $CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD | base64)"
             }
         }
     }
     EOF
  

• chmod 600 $UNCLOUD_SSH_PRIVATE_KEY: To make SSH accept the key (file). Years old bug in Gitlab.
• uc deploy -f compose.yaml -y: Deploy the services from the compose file to the cluster. Using the -y flag is needed to omit the confirmation prompt.

Compose Changes

An example compose file for Uncloud use:

services:
  test-web:
    container_name: test-web
    image: registry.science.ru.nl/cncz/.../test-web/test:latest
    environment:
      APP_ENV: "test"
      LOG_STACK: "daily,graylog"
    env_file:
      - ${UNCLOUD_ENV_FILE}
    pull_policy: always
    x-ports:
      - test.web.u.science.ru.nl:80/https

This file differs from a standard Docker Compose file in several ways; the most important changes are listed below:

• container_name : the service and container names must be unique in a cluster.
• image: if you use a private registry, make sure ~/.docker/config.json is correct.
• environment: environment variables that you can use in your compose file. This file usually is committed in Git; do not put secrets in this file
• env_file: using a GitLab CI/CD file variable with secrets. Note that environment takes precedence before env_file. Note that Uncloud has a standard way to properly handle secrets, which might work even better.
• pull_policy: always: default is missing, but that doesn’t work with a latest tag.
• x-ports: format: url:80/https, the cluster webserver (Caddy) create TLS certificates and is the reverse proxy (mapping container:80 -> external https). De URL must be unique within a cluster. See Uncloud’s documentation for details.
  • URLs not in the u.science.ru.nl zone must be created by C&CZ (for now - this might be automated at some point).
• No networks: all services in a cluster share the internal network. This is way the names need to be unique.