cURL not working (Error #77) for SSL connections on CentOS for non-root users

Question

Just recently my server has stopped working for curl requests to https:// addresses for my web server. Having dug around a little it appears that it's a problem with the user the webserver is running.

If I SSH onto the server as root & call

curl -I -v https://google.com

... I get the following response...

* About to connect() to google.com port 443 (#0)
*   Trying 173.194.67.113... connected
* Connected to google.com (173.194.67.113) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using SSL_RSA_WITH_RC4_128_SHA
* Server certificate:
*       subject: CN=*.google.com,O=Google Inc,L=Mountain View,ST=California,C=US
*       start date: May 22 15:50:20 2013 GMT
*       expire date: Oct 31 23:59:59 2013 GMT
*       common name: *.google.com
*       issuer: CN=Google Internet Authority,O=Google Inc,C=US
> HEAD / HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: google.com
> Accept: */*

However, if I log in as any of the cPanel accounts (also used when running via the web server) I get the following...

* About to connect() to google.com port 443 (#0)
*   Trying 173.194.67.101... connected
* Connected to google.com (173.194.67.101) port 443 (#0)
* Initializing NSS with certpath: none
* NSS error -5978
* Closing connection #0
* Problem with the SSL CA cert (path? access rights?)
curl: (77) Problem with the SSL CA cert (path? access rights?)

I've not been able to find a definitive answer to the problem, & my hosting company are refusing to help as it's "Out of support" even though it was working fine last week!

I did find mention on http://curl.haxx.se/docs/sslcerts.html that

"If libcurl was built with NSS support, then depending on the OS distribution, it is probably required to take some additional steps to use the system-wide CA cert db. RedHat ships with an additional module, libnsspem.so, which enables NSS to read the OpenSSL PEM CA bundle. This library is missing in OpenSuSE, and without it, NSS can only work with its own internal formats. NSS also has a new database format: https://wiki.mozilla.org/NSS_Shared_DB"

... but I can find no information on how I get this working system-wide on my CentOS server.

Info

curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp 
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz 

Can anyone shed some light on why this might have suddenly changed, or better still how to fix it?

Thanks

Onehorse

Answer 1

Turns out that the problem was with face that the script was running from a cPanel "email piped to script", so was running as the user, so is was a user problem, but was not affecting the web server at all.

The cause for the user not being able to access the /etc/pki directory was due to them only having jailed ssh access. Once I granted full access, it all worked fine.

Thanks for the info though, Remi.

Onehorse

Answer 2

I just had a similar problem with Error#77 on CentOS7. I was missing the softlink /etc/pki/tls/certs/ca-bundle.crt that is installed with the ca-certificates RPM.

'curl' was attempting to open this path to get the Certificate Authorities. I discovered with:

strace curl https://example.com

and saw clearly that the open failed on that link.

My fix was:

yum reinstall ca-certificates

That should setup everything again. If you have private CAs for Corporate or self-signed use make sure they are in /etc/pki/ca-trust/source/anchors so that they are re-added.

Peony

Sweet! Reinstalling fixed my issue too. – Dobson

Thank You Perfect solution after searching a lot and got this solution after 2 months. – Broadcasting

Answer 3

If you recently reached here as I did when searching for the same error in vain you may find it to be an update to NSS causing failure on CentOS. Test by running yum update and see if you get errors, curl also creates this error. Solution is simple enough just install NSS manually.

Read on...

If you're like me it threw up an error similar to this:

curl: (77) Problem with the SSL CA cert (path? access rights?)

This took some time to solve but found that it wasn't the CA cert because by recreating them and checking all the configuration I had ruled it out. It could have been libcurl so I went in search of updates.

As mentioned I recreated CA certs. You can do this also but it may be a waste of time. http://wiki.centos.org/HowTos/Https

The next step (probably should of been my first) was to check that everything was up-to-date by simply running yum.

$ yum update
$ yum upgrade

This gave me an affirmative answer that there was a bigger problem at play: Downloading Packages: error: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: BAD Problem opening package nss-softokn-freebl-3.14.3–19.el6_6.x86_64.rpm I started reading about Certificate Verification with NSS and how this new update may be related to my problems. So yum is broken. This is because nss-softokn-* needs nss-softokn-freebl-* need each other to function. The problem is they don't check each others version for compatibility and in some cases it ends up breaking yum. Lets go fix things:

$ wget http://mirrors.linode.com/centos/6.6/updates/x86_64/Packages/nsssoftokn-freebl-3.14.3-19.el6_6.x86_64.rpm
$ rpm -Uvh nss-softokn-freebl-3.14.3–19.el6_6.x86_64.rpm
$ yum update

You should of course download from your nearest mirror and check for the correct version / OS etc. We basically download and install the update from the rpm to fix yum. As @grumpysysadmin pointed out you can shorten the commands down. @cwgtex contributed that you should install the upgrade using the RPM command making the process even simplier.

To fix things with wordpress you need to restart your http server.

$ service httpd restart

Try again and success!

Ake

In my case after installing Java I just had to call service httpd restart and this simple command solved the issue with curl error #77. – Arvizu

This issue came up after upgrading CentOS from 7.3 to 7.9. I followed both the above answers .. reinstalled ca certificates and restarted httpd. It worked. Thanks. – Catherinacatherine

Answer 4

For Ubuntu:

sudo apt-get install ca-certificates

Hit this problem trying to curl things as ROOT inside of Dockerfile

Palaeontography

Thanks. Short and to the point. I had the same problem with a Dockerfile. – Soccer

Answer 5

I had the same issue on CentOS-7. It's worked good on me.

Creating empty file.

touch /etc/sysconfig/64bit_strstr_via_64bit_strstr_sse2_unaligned

There was a curl bug in the distribution image of CentOS-7.

https://bugs.centos.org/view.php?id=16282

Putrefaction

Of all of the solutions I came across, this solution solves it. Damn, Many thanks! – Dictionary

Here's a link to the Wayback Machine's last version since the CentOS infra team decided to just straight up delete bugs.centos.org without a redirect or landing page: web.archive.org/web/20221002190726/https://bugs.centos.org/… – Warfield

Answer 6

Check that you have the correct rights set on CA certificates bundle. Usually, that means read access for everyone to CA files in the /etc/ssl/certs directory, for instance /etc/ssl/certs/ca-certificates.crt.

You can see what files have been configured for you curl version with the

curl-config --configure

command :

$ curl-config --configure
 '--prefix=/usr' 
 '--mandir=/usr/share/man' 
 '--disable-dependency-tracking' 
 '--disable-ldap' 
 '--disable-ldaps' 
 '--enable-ipv6' 
 '--enable-manual' 
 '--enable-versioned-symbols' 
 '--enable-threaded-resolver' 
 '--without-libidn' 
 '--with-random=/dev/urandom' 
 '--with-ca-bundle=/etc/ssl/certs/ca-certificates.crt' 
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector --param=ssp-buffer-size=4' 'LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro' 
 'CPPFLAGS=-D_FORTIFY_SOURCE=2'

Here you need read access to /etc/ssl/certs/ca-certificates.crt

$ curl-config --configure
 '--build' 'i486-linux-gnu' 
 '--prefix=/usr' 
 '--mandir=/usr/share/man' 
 '--disable-dependency-tracking' 
 '--enable-ipv6' 
 '--with-lber-lib=lber' 
 '--enable-manual' 
 '--enable-versioned-symbols' 
 '--with-gssapi=/usr' 
 '--with-ca-path=/etc/ssl/certs' 
 'build_alias=i486-linux-gnu' 
 'CFLAGS=-g -O2' 
 'LDFLAGS=' 
 'CPPFLAGS='

And the same here.

Geostatics

Answer 7

I had faced same issue whenever I tried executing curl on my https server.

About to connect() to localhost port 443 (#0)
Trying ::1...
Connected to localhost (::1) port 443 (#0)
Initializing NSS with certpath: sql:/etc/pki/nssdb

Observed this issue when I configured keystore path incorrectly. After correcting keystore path it worked.

Coverdale

how do you configure it ? – Connors

Answer 8

The error is due to corrupt or missing SSL chain certificate files in the PKI directory. You’ll need to make sure the files ca-bundle, following steps: In your console/terminal:

mkdir /usr/src/ca-certificates && cd /usr/src/ca-certificates

Enter this site: https://rpmfind.net/linux/rpm2html/search.php?query=ca-certificates , get your ca-certificate, for yout SO, for example: ftp://rpmfind.net/linux/fedora/linux/updates/24/x86_64/c/ca-certificates-2016.2.8-1.0.fc24.noarch.rpm << CentOS. Copy url of download and paste in url: wget your_url_donwload_ca-ceritificated.rpm now, install yout rpm:

rpm2cpio your_url_donwload_ca-ceritificated.rpm | cpio -idmv

now restart your service: my example this command:

sudo service2 httpd restart

very great good look

Delogu

Answer 9

Please check the permissions on the the ca certificates installed on server.

Place

Answer 10

for me it's fixed by the hint of @DavidG above. And practically i run:

ln -s /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem /etc/pki/tls/certs/ca-bundle.crt

Fixed after that!

Itinerate

Answer 11

The php-curl error #77 was preventing our payment gateways from working on our magento site.

This error #77 is related to certificated however we never had to configure a certificate for this to work. And the error began suddently and effected every customer.

After trying many of the solutions listed here and elsewhere nothing worked for us.
A lot of people suggested restarting the httpd service but it turns out the solution for us which I haven't seen mentioned anywhere was restarting the php service.

sudo systemctl restart php-fpm

Hope this can help you!

Hoon

Answer 12

Windows users, add this to PHP.ini:

curl.cainfo = "C:/cacert.pem";

Path needs to be changed to your own and you can download cacert.pem from a google search

(yes I know its a CentOS question)

Crump