Login/Register
"Could not resolve host" error with curl_multi on cURL version 7.64
Asked Answered
Solved phpcurldnsphp-curlcurl-multi
C

1

2

I'm getting "Could not resolve host" when I call curl_error($ch) in curl_multi context, the errno is 0 though, and the request is done successfully also(I'm getting the expected response from the remote server).

I'm getting this error for all the requests in the curl_multi group, except for only the first request of them which is done without any error messages!

(For example if the curl_multi has 10 sub requests attached to it, only the first request will not have the error and the other 9 will have the error)

The code to reproduce the problem

<?php
$url = "https://www.example.com";

$options = array(
    CURLOPT_RETURNTRANSFER => true,     // don't echo page
    CURLOPT_VERBOSE =>true,
);

$cm = curl_multi_init();


// step 1 of 3 (adding requests)

//request 1
$ch1 = curl_init( $url );
$verboseH1 = fopen('php://temp', 'w+');
$options[CURLOPT_STDERR] = $verboseH1;
curl_setopt_array( $ch1, $options);
curl_multi_add_handle($cm, $ch1);

//request 2
$ch2 = curl_init( $url );
$verboseH2 = fopen('php://temp', 'w+');
$options[CURLOPT_STDERR] = $verboseH2;
curl_setopt_array( $ch2, $options);
curl_multi_add_handle($cm, $ch2);

//request 3
$ch3 = curl_init( $url );
$verboseH3 = fopen('php://temp', 'w+');
$options[CURLOPT_STDERR] = $verboseH3;
curl_setopt_array( $ch3, $options);
curl_multi_add_handle($cm, $ch3);

// step 2 of 3 (executing requests in parallel)
// for more information about these strange loops , check http://php.net/manual/en/function.curl-multi-init.php#115055
$active = null;    
do {
    $mrc = curl_multi_exec($cm, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);

while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($cm) == -1) {
        usleep(100);
    }
    do {
        $mrc = curl_multi_exec($cm, $active);
    } while ($mrc == CURLM_CALL_MULTI_PERFORM);
}

// step 3 of 3 (getting results)
echo "<h1>Requet 1</h1>\n";
echo "errmsg1: '" .  curl_error($ch1) . "'\n";
rewind($verboseH1);
echo "verbose1:\n" .  stream_get_contents( $verboseH1) . "\n";

echo "===============================================================\n";
echo "<h1>Requet 2</h1>\n";
echo "errmsg2: '" .  curl_error($ch2) . "'\n";
rewind($verboseH2);
echo "verbose2:\n" .  stream_get_contents( $verboseH2) . "\n";

echo "===============================================================\n";
echo "<h1>Requet 3</h1>\n";
echo "errmsg3: '" .  curl_error($ch3) . "'\n";
rewind($verboseH3);
echo "verbose3:\n" .  stream_get_contents( $verboseH3) . "\n";

exit;

The output of this code

<h1>Requet 1</h1>
errmsg1: ''
verbose1:
* Expire in 0 ms for 6 (transfer 0x2b750f0)
* Expire in 1 ms for 1 (transfer 0x2b750f0)
* Expire in 0 ms for 1 (transfer 0x2b750f0)
* Expire in 0 ms for 1 (transfer 0x2b750f0)
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Expire in 149999 ms for 3 (transfer 0x2b750f0)
* Expire in 200 ms for 4 (transfer 0x2b750f0)
* Connected to www.example.com (93.184.216.34) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; OU=Technology; CN=www.example.org
*  start date: Nov 28 00:00:00 2018 GMT
*  expire date: Dec  2 12:00:00 2020 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2b750f0)
> GET / HTTP/2
Host: www.example.com
Accept: */*

* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 19 Feb 2019 17:01:16 GMT
< etag: "1541025663+ident"
< expires: Tue, 26 Feb 2019 17:01:16 GMT
< last-modified: Fri, 09 Aug 2013 23:54:35 GMT
< server: ECS (dfw/27B2)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1270
< 
* Connection #0 to host www.example.com left intact

===============================================================
<h1>Requet 2</h1>
errmsg2: 'Could not resolve host: www.example.com'
verbose2:
* Expire in 0 ms for 6 (transfer 0x2b7a9a0)
* Found bundle for host www.example.com: 0x2b86800 [serially]
* Server doesn't support multi-use (yet)
* Expire in 1 ms for 1 (transfer 0x2b7a9a0)
* Expire in 0 ms for 1 (transfer 0x2b7a9a0)
* Hostname 'www.example.com' was found in DNS cache
* Expire in 0 ms for 1 (transfer 0x2b7a9a0)
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Expire in 149999 ms for 3 (transfer 0x2b7a9a0)
* Expire in 200 ms for 4 (transfer 0x2b7a9a0)
* Connected to www.example.com (93.184.216.34) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; OU=Technology; CN=www.example.org
*  start date: Nov 28 00:00:00 2018 GMT
*  expire date: Dec  2 12:00:00 2020 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2b7a9a0)
> GET / HTTP/2
Host: www.example.com
Accept: */*

* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 19 Feb 2019 17:01:16 GMT
< etag: "1541025663+ident"
< expires: Tue, 26 Feb 2019 17:01:16 GMT
< last-modified: Fri, 09 Aug 2013 23:54:35 GMT
< server: ECS (dfw/562D)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1270
< 
* Could not resolve host: www.example.com
* Closing connection 1

===============================================================
<h1>Requet 3</h1>
errmsg3: 'Could not resolve host: www.example.com'
verbose3:
* Expire in 0 ms for 6 (transfer 0x2b80250)
* Found bundle for host www.example.com: 0x2b86800 [serially]
* Server doesn't support multi-use (yet)
* Expire in 1 ms for 1 (transfer 0x2b80250)
* Expire in 0 ms for 1 (transfer 0x2b80250)
* Hostname 'www.example.com' was found in DNS cache
* Expire in 0 ms for 1 (transfer 0x2b80250)
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Expire in 150000 ms for 3 (transfer 0x2b80250)
* Expire in 200 ms for 4 (transfer 0x2b80250)
* Connected to www.example.com (93.184.216.34) port 443 (#2)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; OU=Technology; CN=www.example.org
*  start date: Nov 28 00:00:00 2018 GMT
*  expire date: Dec  2 12:00:00 2020 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2b80250)
> GET / HTTP/2
Host: www.example.com
Accept: */*

* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< accept-ranges: bytes
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 19 Feb 2019 17:01:16 GMT
< etag: "1541025663"
< expires: Tue, 26 Feb 2019 17:01:16 GMT
< last-modified: Fri, 09 Aug 2013 23:54:35 GMT
< server: ECS (dfw/F4AA)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1270
< 
* Could not resolve host: www.example.com
* Closing connection 2

As you can see only the first request does not have the error, requests 2 and 3 has error "Could not resolve host: www.example.com" and have different verbose than the first request.

Additional info

  1. This problem happens on CentOS + PHP 7.0.33 + cURL 7.64.0 (released : 6th of February 2019)
  2. This problem does NOT happen on fedora28 + PHP 7.2.14 + cURL 7.59.0
  3. This problem does NOT happen if the curl_multi has only 1 request attached to it

I would like to understand this behavior. How can I make all the requests do like the first request and not produce this error?

Update

Adding this line to the $options array fixes the particular example in the question, however this is not a practical solution and does not explain why cURL can't resolve the name by itself.

CURLOPT_RESOLVE => ["www.example.com:443:93.184.216.34"]
Crawl answered 19/2, 2019 at 17:32 Comment(3)
This has been confirmed a bug in curl and the fix is here – Zetana
@DanielStenberg I understand you don't have much time to debug issue 3602 again as it's unreliably reproducible and could be because of many reasons other than curl, but I hope that one day if you get some time, to give it another deeper look, because you are the only one I know who can dig into this area :) – Camembert
Meanwhile this was also fixed on the PHP side and it's included in version 7.2+: github.com/php/php-src/pull/3901#issuecomment-468673948 – Sapowith
T
3

I have the exact same issue: https://github.com/amazeeio/lagoon/issues/923

This is a curl regression: https://github.com/curl/curl/issues/3629 - which has been solved since opening the bug.

Thanks for the awesome test scripts :)

Tjader answered 28/2, 2019 at 15:39 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.