Runner is not healthy and will be disabled

Question

I hosts my repository with gitlab.com and I install runner in the DigitalOcean. It ran fine until today 16March2019 14:24 Thailand time.

# gitlab-runner status
Runtime platform                                    arch=amd64 os=linux pid=16937 revision=4745a6f3 version=11.8.0
gitlab-runner: Service is running!
# gitlab-runner unregister --all-runners
Runtime platform                                    arch=amd64 os=linux pid=16299 revision=4745a6f3 version=11.8.0
Running in system-mode.

WARNING: Unregistering all runners
ERROR: Unregistering runner from GitLab forbidden   runner=2bcd7af4
ERROR: Failed to unregister runner HerrRunner
# gitlab-runner list
Runtime platform                                    arch=amd64 os=linux pid=16346 revision=4745a6f3 version=11.8.0
Listing configured runners                          ConfigFile=/etc/gitlab-runner/config.toml
HerrRunner                                          Executor=shell Token=2bcd7af455f866ede7991992a68780 URL=https://gitlab.com/
# gitlab-runner --debug run
Runtime platform                                    arch=amd64 os=linux pid=16395 revision=4745a6f3 version=11.8.0
Starting multi-runner from /etc/gitlab-runner/config.toml ...  builds=0
Checking runtime mode                               GOOS=linux uid=0
Running in system-mode.

Configuration loaded                                builds=0
listenaddress: ""
sessionserver:
  listenaddress: ""
  advertiseaddress: ""
  sessiontimeout: 1800
metricsserveraddress: ""
concurrent: 1
checkinterval: 0
loglevel: null
logformat: null
user: ""
runners:
- name: HerrRunner
  limit: 0
  outputlimit: 0
  requestconcurrency: 0
  runnercredentials:
    url: https://gitlab.com/
    token: 2bcd7af455f866ede7991992a68780
    tlscafile: ""
    tlscertfile: ""
    tlskeyfile: ""
  runnersettings:
    executor: shell
    buildsdir: ""
    cachedir: ""
    cloneurl: ""
    environment: []
    preclonescript: ""
    prebuildscript: ""
    postbuildscript: ""
    shell: ""
    ssh: null
    docker: null
    parallels: null
    virtualbox: null
    cache:
      type: ""
      path: ""
      shared: false
      s3: null
      gcs: null
      s3cachepath: ""
      cacheshared: false
      serveraddress: ""
      accesskey: ""
      secretkey: ""
      bucketname: ""
      bucketlocation: ""
      insecure: false
    machine: null
    kubernetes: null
sentrydsn: null
modtime: 2018-08-12T18:07:07.963445119Z
loaded: true
  builds=0
Waiting for stop signal                             builds=0
Listen address not defined, metrics server disabled  builds=0
Listen address not defined, session server disabled  builds=0
Starting worker                                     builds=0 worker=0
Feeding runners to channel                          builds=0
Dialing: tcp gitlab.com:443 ...
ERROR: Checking for jobs... forbidden               runner=2bcd7af4
Feeding runners to channel                          builds=0
ERROR: Checking for jobs... forbidden               runner=2bcd7af4
Feeding runners to channel                          builds=0
ERROR: Checking for jobs... forbidden               runner=2bcd7af4
ERROR: Runner https://gitlab.com/2bcd7af455f866ede7991992a68780 is not healthy and will be disabled!
Feeding runners to channel                          builds=0
Feeding runners to channel                          builds=0
Feeding runners to channel                          builds=0
^CWARNING: Requested service stop: interrupt          builds=0
All workers stopped. Can exit now                   builds=0

Ultimate Goal

Get my runner up and run again

Question:

1. What does not healthy means?

2. I can't unregister my runner. How to fix this?

Elate

Answer 1

I was integrating my runner for the first time, and anyone familiar with gitlab will know this was never going to be a walk in the park. I spent many hours chasing down this error message only to find I was completely looking in the wrong direction.

ERROR: Checking for jobs... forbidden               runner=s3xBVnW8
ERROR: Checking for jobs... forbidden               runner=s3xBVnW8
ERROR: Checking for jobs... forbidden               runner=s3xBVnW8
ERROR: Runner https://gitlab.com/s3xBVnW8JZPTaocALN3i is not healthy and will be disabled!

I now believe my runner was telling me that the "runner" at gitlab.com is not healthy and has been rejected (by my runner). When I left it running and put a pipeline job in, it processed fine, there was no issue at my end, I had spent hours looking for nothing!

Freeboard

I was having same issue, any answer found ? – Landis

So did I. when I was trying locally in a docker container, it kept saying gitlab-runner: "service" failed: exec: "service": executable file not found in $PATH and I tried to figure it out for hours. From alpine to ubuntu, from the latest to tagged versions, also found a few different issues. Later running the job worked fine! 🤦 – Subclavius

Answer 2

No idea. But seems like gitlab.com remove my runner token. Therefore I have to remove my runner, register, and run it again.

Elate

Answer 3

I faced with the same issue. Register you runner as described in the installation instructions. Go to admin/runners and click Show runner installation instructions button. The most important is Register runner session where you have to run register command that resolves the issue described. For instance, linux:

sudo gitlab-runner register --url YOUR_GITLAB_SERVER_URL --registration-token TOKEN

# YOUR_GITLAB_SERVER_URL like http://localhost:8100/ or https://mygitlabserver.com/

Follow instructions and you are good to go.

Once your runner is setup and running you can add modifications to the config file, stop and start it again after config changes.

You may find duplicate [[runners]] sections, that happens if you tried to configure it yourself and then with register command. That's fine, leave only one latest.

Rupiah

Answer 4

When I ran into this (running in user mode because of lack of sudo access), removing and reregistering the runner didn't seem to help. The extra step that was required was removing the config.toml file (located at ~/.gitlab-runner/config.toml for me). After deleting that file, I was able to register and run a new runner (using the exact same commands as before) and everything seems to work fine.

Note:

When you register a runner, it should say something like Configuration (with the authentication token) was saved in "<some/path/config.toml>". That's the file that needed deleting.

Decaffeinate

This worked for me. My config file was located at /etc/gitlab-runner/config.toml on Ubuntu 20.04 – Divergency

Thank you. Had the same issue. Runner was configured on one debian user and I was trying to use it from another user which is not sudo. Deleting the file and registering again helped. – Sanjiv

Answer 5

I also had this problem, which was solved by changing the service as a sub-problem From: ExecStart=/usr/bin/gitlab-runner "run" "--working-directory" "/var/lib/gitlab-runner" "--config" "/etc/gitlab-runner/config.toml" "--service" "gitlab-runner" "--user" "gitlab-runner" ExecStart=/usr/bin/gitlab-runner "run"

To: ExecStart=/usr/bin/gitlab-runner "run"

Photoengrave

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Spinster

Answer 6

You can try to run the command with sudo like this "sudo gitlab-runner run". I was encountering the same error until I used sudo.

Huffman

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Spinster