gitlab 500 errors in the admin area

Question

How to reproduce:

1. login as admin
2. go into the admin menu
3. click on overview - runners
4. or click save in the settings

Result: 500 Error: Whoops, something went wrong on our end.

Which system is running?

RHEL 7, outbound gitlab with Nginx and passengers

Errors:

OpenSSL::Cipher::CipherError ():

lib/gitlab/crypto_helper.rb:27:in `aes256_gcm_decrypt'
app/models/concerns/token_authenticatable_strategies/encrypted.rb:55:in `get_token'
app/models/concerns/token_authenticatable_strategies/base.rb:27:in `ensure_token'
app/models/concerns/token_authenticatable_strategies/encrypted.rb:42:in `ensure_token'


ActionView::Template::Error ():
    37: 
    38:     .col-sm-6
    39:       .bs-callout
    40:         = render partial: 'ci/runner/how_to_setup_runner',
    41:                  locals: { registration_token: Gitlab::CurrentSettings.runners_registration_token,
    42:                            type: 'shared',
    43:                            reset_token_url: reset_registration_token_admin_application_settings_path }

Naseby

So, whats your question? Did you file an issue? – Allieallied

The issue is that I cannot use Gitlab because I can't change settings. I want to know how I can fix this error – Naseby

Did you migrate this installation by any chance? You need to copy the secrets.json of the old instance, that contains keys that are used to decrypt information – Tripterous

Answer 1

I am almost sure it's a wrong decryption key that gitlab reads from /etc/gitlab/gitlab-secrets.json (omnibus install) or $GITLAB_HOME/config/secrets.yml(source based install). If you don't care about your runner_registration_token you can reset it like this from the console:

root@gitlab:/# gitlab-rails console
-------------------------------------------------------------------------------------
 GitLab:       11.5.1 (c90ae59)
 GitLab Shell: 8.4.1
 postgresql:   9.6.8
-------------------------------------------------------------------------------------
Both Deployment and its :status machine have defined a different default for "status". Use only one or the other for defining defaults to avoid unexpected behaviors.
Loading production environment (Rails 4.2.10)
irb(main):001:0> ApplicationSetting.current.reset_runners_registration_token!
=> true
irb(main):002:0> 

Tripterous

GitLab: 13.12.0-ee EE Loading production environment (Rails 6.0.3.6) irb(main):001:0> ApplicationSetting.current.reset_runners_registration_token! Traceback (most recent call last): 5: from (irb):1 4: from app/models/concerns/token_authenticatable.rb:53:in block in add_authentication_token_field' 3: from app/models/concerns/token_authenticatable_strategies/base.rb:44:in reset_token!' 2: from lib/gitlab/database.rb:370:in transaction' 1: from lib/gitlab/database.rb:371:in block in transaction' OpenSSL::Cipher::CipherError () – Tatman

Answer 2

In the GitLab 13.0 version, it has been confirmed that the command ApplicationSetting.current.reset_runners_registration_token! is no longer supported. Previously, this command could be used to reset the registration token of GitLab CI/CD runners, but this functionality has been removed in later versions. Instead, GitLab administrators can reset the registration token of runners in the administration area of GitLab.

In GitLab versions 13.0 and later, if you attempt to use the command ApplicationSetting.current.reset_runners_registration_token!, an error message will be displayed. This is because this command is no longer supported.

Source: GitLab 13.0 release notes

To resolve this issue in GitLab versions 13.0 and later, you can follow the steps below:

gitlab-rails console
> ApplicationSetting.first.delete
> ApplicationSetting.first
=> nil

Source: GitLab issue #57038

Mccay