GitLab CI script needs entry in /etc/hosts

Question

I have a GitLab CI docker runner to execute my automated tests when I push. One of my tests requires a custom entry in /etc/hosts. I can't figure out how to get the entry into that file.

Here's basically what my .gitlab-ci.yml file looks like:

before_script:
  - cat /etc/hosts   # for debugging
  - ...              # install app dependencies
specs:
  script:
    - rspec          # <- a test in here fails without the /etc/hosts entry 

All my tests pass, except for the one that requires that /etc/hosts entry.

Let's say I'm trying to have the hostname myhost.local resolve to the IPv4 address XX.XX.XX.XX...

I tried using extra_hosts on the runner config, but it didn't seem to have any effect (got idea from here):

/etc/gitlab-runner/config.toml:

concurrent = 1
check_interval = 0

[[runners]]
  name = "shell"
  url = "https://mygitlabinstance.com/"
  token = "THETOKEN"
  executor = "shell"
  [runners.cache]

[[runners]]
  name = "docker-ruby-2.5"
  url = "https://mygitlabinstance.com/"
  token = "THETOKEN"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "ruby:2.5"
    privileged = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
    extra_hosts = ["myhost.local:XX.XX.XX.XX"]
  [runners.cache]

But the test still failed. The cat /etc/hosts shows that it's unchanged:

# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
#     /etc/cloud/cloud.cfg or cloud-config from user-data
#
127.0.1.1 ip-172-31-2-54.ec2.internal ip-172-31-2-54
127.0.0.1 localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

I figured I could just add the entry myself in a before_script line, but I don't seem to be able to execute anything with root privileges in the container:

before_script:
  - echo 'XX.XX.XX.XX myhost.local' >> /etc/hosts
  ...

But that just fails because the gitlab-runner user doesn't have permissions to write to that file. I tried to use sudo, but gitlab-runner can't do that either (echo 'XX.XX.XX.XX myhost.local' | sudo tee --non-interactive --append /etc/hosts --> sudo: a password is required)

So in summary, how can I get my container to have the host entry I need (or how can I execute a before_script command as root)?

Prophet

Answer 1

The following statement is incorrect:

"But that just fails because the gitlab-runner user doesn't have permissions to write to that file."

The gitlab-runner is not the user executing your before_script, it is the user that runs the container in which your job is executed.

You are using the ruby:2.5 Docker image as far as I can tell and that does not contain any USER reference in its or its parents Dockerfile.

Try adding a whoami command right before your echo 'XX.XX.XX.XX myhost.local' >> /etc/hosts command to verify you are root.

Update

If gitlab-runner is shown as the result of whoamithe docker-executor is not used and instead a shell-executor has picked up the job.

Itagaki

Yeah, I did that actually -- to the before_script I added whoami and got gitlab-runner. I did groups too and got gitlab-runner docker rvm, in case that's relevant. The GitLab runner must be setting up a user to execute the builds, right? – Prophet

Eureka! Your answer helped me figure it out... Indeed I was trying to use the ruby:2.5 docker image, but as you correctly pointed out, that image doesn't have the gitlab-runner user. So why was whoami showing that? Because the shell runner was running the jobs, not the docker runner! I missed the fact that the project would default to the shell runner, despite having both available. I configured it to use the Docker runner and added the appropriate image: and the now-necessary services: - postgres:9.4 to gitlab-ci.yml and it works! If you update your answer, I'll give you credit :) – Prophet

Answer 2

In your config.toml on your Gitlab CI runner, you can add a setting to the config.toml so you can achieve this without touching /etc/hosts.

[runners.docker]
# ... other settings ...
  extra_hosts = ["myhost.local:xx.xx.xx.xx"]

You can read more about the extra_hosts configuration here: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnersdocker-section

Naphthol

The user already has this line in the config.toml as he shows in the answer. So this answer does not add anything new. – Humdrum