cURL randomly throws "curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL" after some failed requests
Asked Answered
O

1

9

I have a PHP script that quickly sends a bunch of requests to the Apple API (APNs). Sometimes 10k requests are sent totally fine (just for the record, it takes ~30sec). However, when the API returns some non-200 codes, establishing new connections to this API throws the following error:

curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to api.push.apple.com:443

With the debug mode enabled I get strange results:

When everything is fine:

* Found bundle for host api.push.apple.com: 0x55eba8b11660 [serially]
* Found bundle for host api.push.apple.com: 0x55eba8b11660 [serially]
*   Trying 17.188.140.151...
* TCP_NODELAY set
* Hostname 'api.push.apple.com' was found in DNS cache
*   Trying 17.188.140.151...
...
...

(full output is here).

After the issue happened:

* Found bundle for host api.push.apple.com: 0x561d83bb1f00 [serially]
* Server doesn't support multiplex (yet)
* Connection #0 is still name resolving, can't reuse
* Found bundle for host api.push.apple.com: 0x561d83bb1f00 [serially]
* Server doesn't support multiplex (yet)
* Connection #0 is still name resolving, can't reuse
* Connection #1 is still name resolving, can't reuse
*   Trying 17.188.156.30:443...
* TCP_NODELAY set
* Hostname 'api.push.apple.com' was found in DNS cache
*   Trying 17.188.156.30:443...
...
...

(full output is here).

So the IP is changed and something about multiplexing.

After it happens, sending requests to this API using the cli curl also stops working:

$ curl -v -I https://api.push.apple.com
*   Trying 17.188.156.30:443...
* TCP_NODELAY set
* Connected to api.push.apple.com (17.188.156.30) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to api.push.apple.com:443 
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to api.push.apple.com:443 

(full output is here).

The error may disappear after a few minutes, may not. The code runs in a docker container (php:7.3-cli-alpine), restarting the container usually resets the issue until some future requests get non-200 codes. Performing restarts isn't an option.

Presumably, curl somehow stores opened connections and tries to re-use them, but for some reason something is broken inside the curl and it doesn't allow curl to correctly re-use the connections.

While curl stops working, openssl works fine:

$ openssl s_client -connect api.push.apple.com:443
CONNECTED(00000003)
write:errno=0
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 320 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

Software versions:

$ curl --version
curl 7.67.0 (x86_64-alpine-linux-musl) libcurl/7.67.0 OpenSSL/1.1.1d zlib/1.2.11 nghttp2/1.40.0
Release-Date: 2019-11-06
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS HTTP2 HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets

$ openssl version
OpenSSL 1.1.1d  10 Sep 2019

Looks like a curl issue, but how to fix it?

Any help will be appreciated

Official answered 19/3, 2020 at 14:32 Comment(4)
Did you manage to get it solved ?Viscacha
@Viscacha unfortunately, no. This happened on my dev machine, the production environment works fine. So it was decided to keep it "as is" because no one knows how to fixOfficial
Ok thank you. It happened to me also and don't know how to solve this. It's such a pain. It works on dev but not on prodViscacha
@Viscacha maybe upgrade or even downgrade curl/openssl will help. It's the main difference between our dev and prodOfficial
P
4

Just in case it's helpful to you or someone else, the "one error breaks things for a long time" nature of this (and the re-use of Curl connections) reminded me a bit of the Curl bug: https://github.com/curl/curl/issues/3966

Not sure whether the fix for that is in your Curl version though - from the date it might well be.

Pterodactyl answered 16/7, 2020 at 10:28 Comment(1)
Hi @josie-h, welcome to Stack Overflow :) I see that you are trying to provide some information that might be helpful to OP or people in general, but it actually is not an answer to the question. When you earn enough reputation, you can provide such inputs as comments to the question rather than answer... :)Rheumatoid

© 2022 - 2024 — McMap. All rights reserved.