PHP CURL & HTTPS

Question

I found this function that does an AWESOME job (IMHO): http://nadeausoftware.com/articles/2007/06/php_tip_how_get_web_page_using_curl

/**
 * Get a web file (HTML, XHTML, XML, image, etc.) from a URL.  Return an
 * array containing the HTTP server response header fields and content.
 */
function get_web_page( $url )
{
    $options = array(
        CURLOPT_RETURNTRANSFER => true,     // return web page
        CURLOPT_HEADER         => false,    // don't return headers
        CURLOPT_FOLLOWLOCATION => true,     // follow redirects
        CURLOPT_ENCODING       => "",       // handle all encodings
        CURLOPT_USERAGENT      => "spider", // who am i
        CURLOPT_AUTOREFERER    => true,     // set referer on redirect
        CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
        CURLOPT_TIMEOUT        => 120,      // timeout on response
        CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
    );

    $ch      = curl_init( $url );
    curl_setopt_array( $ch, $options );
    $content = curl_exec( $ch );
    $err     = curl_errno( $ch );
    $errmsg  = curl_error( $ch );
    $header  = curl_getinfo( $ch );
    curl_close( $ch );

    $header['errno']   = $err;
    $header['errmsg']  = $errmsg;
    $header['content'] = $content;
    return $header;
}

The only problem I have is that it doesn't work for https://. Anny ideas what I need to do to make this work for https? Thanks!

Aircool

define "doesn't work" please. – Dripps

curl by default check if the SSL certificate is valid ... you might want to disable that behaviour if you self signed the certificate in question – Treadwell

@RegeZ - how to do your suggestion? – Aircool

related: #316599 – Betony

the answer which is marked correct should not be the correct answer. – Stodder

https://mcmap.net/q/246713/-can-39-t-connect-to-https-site-using-curl-returns-0-length-content-instead-what-can-i-do – Calderon

Possible duplicate of Configuring cURL for SSL – Farquhar

Answer 1

Quick fix, add this in your options:

curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false)

In this situation you have no idea what host you're actually connecting to, because cURL will not verify the certificate in any way. This is not recommended and will leave you open to man-in-the-middle attacks!

Or just add it to your current function:

/**
 * Get a web file (HTML, XHTML, XML, image, etc.) from a URL.  Return an
 * array containing the HTTP server response header fields and content.
 */
function get_web_page( $url )
{
    $options = array(
        CURLOPT_RETURNTRANSFER => true,     // return web page
        CURLOPT_HEADER         => false,    // don't return headers
        CURLOPT_FOLLOWLOCATION => true,     // follow redirects
        CURLOPT_ENCODING       => "",       // handle all encodings
        CURLOPT_USERAGENT      => "spider", // who am i
        CURLOPT_AUTOREFERER    => true,     // set referer on redirect
        CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
        CURLOPT_TIMEOUT        => 120,      // timeout on response
        CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
        CURLOPT_SSL_VERIFYPEER => false     // Disabled SSL Cert checks
    );

    $ch      = curl_init( $url );
    curl_setopt_array( $ch, $options );
    $content = curl_exec( $ch );
    $err     = curl_errno( $ch );
    $errmsg  = curl_error( $ch );
    $header  = curl_getinfo( $ch );
    curl_close( $ch );

    $header['errno']   = $err;
    $header['errmsg']  = $errmsg;
    $header['content'] = $content;
    return $header;
}

Early

There is more information on this here: unitstep.net/blog/2009/05/05/… – Early

that's ok - i am only new here since yesterday. thanks for making that code better for the person asking - ur awesome. – Early

Setting CURLOPT_SSL_VERIFYPEER to false allows for man-in-the-middle-attacks. rmckay at webaware dot com dot au warns for this on nl3.php.net/manual/en/function.curl-setopt.php and gives an alternative solution that works: downloading a CA root certificate bundle at the curl website and saving it on your server. See at the given site, scroll down to the user comments. – Natal

Thank you. Lots of research and this worked for me. You are so awesome. – Bevbevan

Answer 2

I was trying to use CURL to do some https API calls with php and ran into this problem. I noticed a recommendation on the php site which got me up and running: http://php.net/manual/en/function.curl-setopt.php#110457

Please everyone, stop setting CURLOPT_SSL_VERIFYPEER to false or 0. If your PHP installation doesn't have an up-to-date CA root certificate bundle, download the one at the curl website and save it on your server:

http://curl.haxx.se/docs/caextract.html

Then set a path to it in your php.ini file, e.g. on Windows:

curl.cainfo=c:\php\cacert.pem

Turning off CURLOPT_SSL_VERIFYPEER allows man in the middle (MITM) attacks, which you don't want!

Stodder

I did this but now I only get 35 error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure – Cornstalk

Answer 3

Another option like Gavin Palmer answer is to use the .pem file but with a curl option

1. download the last updated .pem file from https://curl.haxx.se/docs/caextract.html and save it somewhere on your server(outside the public folder)

2. set the option in your code instead of the php.ini file.

In your code

curl_setopt($ch, CURLOPT_CAINFO, $_SERVER['DOCUMENT_ROOT'] .  "/../cacert-2017-09-20.pem");

NOTE: setting the cainfo in the php.ini like @Gavin Palmer did is better than setting it in your code like I did, because it will save a disk IO every time the function is called, I just make it like this in case you want to test the cainfo file on the fly instead of changing the php.ini while testing your function.

Elsy

Answer 4

One important note, the solution mentioned above will not work on local host, you have to upload your code to server and then it will work. I was getting no error, than bad request, the problem was I was using localhost (test.dev,myproject.git). Both solution above work, the solution that uses SSL cert is recommended.

1. Go to https://curl.haxx.se/docs/caextract.html, download the latest cacert.pem. Store is somewhere (not in public folder - but will work regardless)

2. Use this code

".$result; //echo "
Path:".$_SERVER['DOCUMENT_ROOT'] . "/ssl/cacert.pem"; // this is for troubleshooting only ?>

3. Upload the code to live server and test.

Nones