How are Gitlab CI service ports exposed?

Question

I have a .gitlab-ci.yml file:

integration_test:
  services:
    - name: registry.gitlab.com/group/project/testmailserver:1.1
      alias: "mail.email"
  stage: test
  script:
    - ./gradlew -g /cache/.gradle --stacktrace --info integrationTest

The service is a full stack email server based on this: tvial/docker-mailserver:latest. Locally with my docker-compose config I'm able to run it and connect to it.

version: '2'

services:
  mail:
    image: registry.gitlab.com/group/project/testmailserver:1.1
    hostname: mail
    domainname: localhost
    ports:
      - "25:25"
      - "143:143"
      - "587:587"
      - "993:993"
    environment:
      - ONE_DIR=1
      - DMS_DEBUG=0
      - MAIL_USER=invoicereader
      - MAIL_PASS=invoicereader
    cap_add:
      - NET_ADMIN

If I run it with docker-compose up and connect to it via IMAP on port 993 it works fine. Also the integration test runs smoothly

However, if the integration test is executed by gitlab CI, it fails. The only exception I could get is Connection refused.

Can it be that the ports of the service are not exposed properly? How does the CI server determine the ports it has to open to said service?

What might be the problem when running with CI? How can I test it differently?

Sorry for the lot of questions, I'm just hopelessly lost..

Protectorate

Hey @Protectorate did you solve your problem? If so, could you share the solution? Many thanks. – Golda

It was a long time ago so I don't really remember the solution but it wasn't the ports IIRC. If you have the same Gitlab CI issue, I'd suggest getting the gitlab-runner, and execute your job locally with gitlab-runner exec docker jobname to test it. To debug or play around with it, insert the following line just before the part where it fails: tail -F /dev/null. This will cause the execution of the job to hang for 30 minutes. While it hangs, attach to the container with docker exec -ti container_name /bin/bash. Hope this helps. – Protectorate

Answer 1

From the official documentation:

The services keyword defines just another Docker image that is run during your job and is linked to the Docker image that the image keyword defines. This allows you to access the service image during build time.

There is no image keyword in your .gitlab-ci.yml file. Therefore, it's actually unpredictable where your job integration_test runs.

If your job runs inside a Docker container, this container is linked to the container of your service. Links are a legacy feature of Docker, but it's quite similar to two containers that are connected via a separate network. This is quite similar to multiple services in one compose file that communicate with each other.

Everything that is executed in your job's container can access your service via its name or alias (mail.email). Have a look at the Dockerfile of you mail server to see which ports the service listens to:

EXPOSE 25 587 143 465 993 110 995 4190

There is no need to manually expose any ports here. The ports keyword in your compose file exposes ports of a container to your host system. If you do something similar in your CI pipeline, it will expose the ports to the system that runs the job. This is certainly not what you want.

In short: use something like mail.email:993 to connect to your mail server via IMAP from within your CI job.

Ori

Answer 2

I was facing this issue. EXPOSE <port> will work if you own the docker image and in position to update the Dockerfile. In my case, I was to use multiple wiremock images to launch in .gitlab-ci.yml under services section.

services doesn't have any straight way of port mapping like 8080:8080 as in docker-compose.yml.

However there is a work around to pass it as a command as shown below

services:
- name: wiremock/wiremock:main-alpine
  alias: mock-server1
  command: [ "--port","9010","--https-port","9040","--verbose"]
- name: wiremock/wiremock:main-alpine
  alias: mock-server2
  command: [ "--port","9020","--https-port","9050","--verbose"]
- name: wiremock/wiremock:main-alpine
  alias: mock-server3
  command: [ "--port","9030","--https-port","9060","--verbose"]

Please read the GitLab's official document on services, below is the snippet for command option https://docs.gitlab.com/ee/ci/services/

Swetlana

This doesn't work for me. The command option basically corresponds to the command part of: docker [options] run [image] [command] [arg] So it only runs some bash command inside the container but not exposing the port. – Oat

The code in my response is working code in my project, so your issue could be very specific. – Swetlana