Gitlab Installation, Registry and Runner with Docker

First part of a series where we build a CI eco system with Gitlab and Kubernetes to deploy a basic Go service.

NOTE: you need docker installed.

In this part, we first create self-signing certs for https access, we then install a dockerized gitlab and an integrated registry making use of these certs. We then add a runner and a custom docker image to allow running docker commands in the CI pipeline.

Self Signed Certs

openssl req -x509 -newkey rsa:4096 -sha256 -nodes -keyout gitlab.lightphos.com.key -out gitlab.lightphos.com.crt -subj "/CN=gitlab.lightphos.com" -days 3600

NOTE: if /CN does not work (eg in windows git bash) omit it and then enter the domain in the CN prompt

IP address (ipconfig/ifconfig)

Add an entry to /etc/hosts

{host ip} gitlab.lightphos.com

Gitlab and Registry Installation

The details  below can be found at:

https://gitlab.com/lightphos/gitlabops

Either clone the above repo or create directory gitlabops/gitlab and copy as needed.

Copy the gitlab key and crt files from the self signed certs step above to the directory gitlabops/gitlab/ssl

In gitlabops, create file docker-compose-gitlab.yml with the following

version: '2'
services:
  gitlab:
    image: 'gitlab/gitlab-ce:latest'
    restart: always
    hostname: 'gitlab.lightphos.com'
    container_name: gitlab
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'https://gitlab.lightphos.com:30080'
        registry_external_url 'https://gitlab.lightphos.com:5555'
        gitlab_rails['gitlab_shell_ssh_port']=30022
	    registry_nginx['enable'] = true
        registry_nginx['listen_port'] = 5555
        registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.lightphos.com.crt"
        registry_nginx['ssl_certificate_key'] =  "/etc/gitlab/ssl/gitlab.lightphos.com.key"
    ports:
      - '30080:30080'
      - '5555:5555'
      - '30022:22'
    volumes:
      - './gitlab/config:/etc/gitlab'
      - 'gitlab-logs:/var/log/gitlab'
      - 'gitlab-data:/var/opt/gitlab'
      - './gitlab/ssl:/etc/gitlab/ssl'
volumes:
  gitlab-logs:
    external: true
  gitlab-data:
    external: true

The above docker compose file includes setting up of the integrated docker registry (port 5555) and add the correct certs and keys. Start it up as follows (note the use of docker volumes for persistence between restarts).

cd gitlabops
docker volume create gitlab-logs
docker volume create gitlab-data
docker-compose -f docker-compose-gitlab.yml up -d

Takes a while to start, check progress with

docker logs -f gitlab

(Ctrl+C to exit)

You can then navigate on the browser with

https://gitlab.lightphos.com:30080

Add master password, register (username I chose was sr, used later) and create a project.

Add ssh key to gitlab:

https://subscription.packtpub.com/book/application_development/9781783986842/2/ch02lvl1sec20/adding-your-ssh-key-to-gitlab

You can then use git ssh to add or change files etc as normal.

Multiple Remotes

If you wish you can have multiple remotes (in addition to origin):

Create the same project on the gitlab.lightphos.com:30080, then add another remote:

git remote add local ssh://git@gitlab.lightphos.com:30022/sr/gitlabops.git
git pull local master --allow-unrelated-histories 
git push local master

Docker Registry Access

For linux/mac:
Copy the gitlab.lightphos.crt file to ~/.docker/certs.d/gitlab.lightphos.com:5555/ca.crt

In windows:
Right click on the cert,  install -> local -> trusted store.

In ubuntu:

sudo cp gitlab.lightphos.com.crt /usr/local/share/ca-certificates/gitlab.lightphos.com.crt

sudo update-ca-certificates

Restart docker. Check if you can login:

docker login gitlab.lightphos.com:5555 # username/password of gitlab account
$ Login successful

Gitlab Runner

This is on a mac (/Users/Shared is a mac directory,  for other OSs see below)

docker run -d --name gitlab-runner --restart always \
-v /Users/Shared/gitlab-runner/config:/etc/gitlab-runner \
-v /Users/Shared/gitlab-runner/certs:/etc/gitlab-runner/certs \
-v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest

Register a docker runner (note this sibling connection to host with docker.sock, also not the use of the shared directory /Users/Shared)

docker run --rm -ti -v /Users/Shared/gitlab-runner/config:/etc/gitlab-runner \
gitlab/gitlab-runner register \
  --url https://gitlab.lightphos.com:30080/ \
  --registration-token REGISTRATION_TOKEN \
  --executor docker \
  --description "Runner" \
  --docker-image "docker:19.03.1" \
  --docker-volumes /var/run/docker.sock:/var/run/docker.sock

The details for the url and token can be found in your gitlab project -> settings -> CI/CD -> Runners.

You should see an active (green) runner in settings:

Set the runner to run untagged jobs:
Click on the pencil:

Check CI Pipeline

Create a .gitlab-ci.yml file.

variables:
  CONTAINER_IMAGE: ${CI_REGISTRY}/${CI_PROJECT_PATH}:${CI_BUILD_REF_NAME}_${CI_BUILD_REF}
  CONTAINER_IMAGE_LATEST: ${CI_REGISTRY}/${CI_PROJECT_PATH}:latest
  DOCKER_DRIVER: overlay2

docker-build-master:
  image: docker:19.03.1
  stage: build
  before_script:
    - echo  $CI_BUILD_TOKEN | docker login -u gitlab-ci-token --password-stdin ${CI_REGISTRY}
  script:
    - docker images

Commit and push. Docker login and images should be visible  in the pipeline console.

Shared Directories

For other OSs you need to copy to the corresponding directories as shown:

VirtualBox	Linux	/home	/hosthome
VirtualBox	macOS	/Users	/Users
VirtualBox	Windows	C://Users	/c/Users

Useful command to remove exited docker processes:

docker rm -f $(docker ps -aq -f status=exited)

In the next post we will look at how to deploy Go service to the gitlab CI/registry infrastructure we have built.