Make cURL output STDERR to file (or string)

Question

We're trying to debug some cURL errors on the server, and I would like to see the STDERR log. Currently, all we can see for our error is "error code: 7" and that we can't connect to target server. We have contacted the host and made special rule to open the port we need and we're even ignoring the certificate for the time being.

Still, we can't connect. I need to debug this, but I can't see any pertinent information on my end.

The lines mentioning "VERBOSE" and "STDERR" are the most important, I think. Nothing is written to $curl_log. What am I doing wrong? Following the manuals logic, this should be correct...

PHP in use:

<?php
$curl = curl_init();
$curl_log = fopen("curl.txt", 'w');
$url = "http://www.google.com";

curl_setopt_array($curl, array(
    CURLOPT_URL             => $url,        // Our destination URL
    CURLOPT_VERBOSE         => 1,           // Logs verbose output to STDERR
    CURLOPT_STDERR          => $curl_log,   // Output STDERR log to file
    CURLOPT_SSL_VERIFYPEER  => 0,           // Do not verify certificate
    CURLOPT_FAILONERROR     => 0,           // true to fail silently for http requests > 400
    CURLOPT_RETURNTRANSFER  => 1            // Return data received from server
));

$output = fread($curl_log, 2048);
echo $output; // This returns nothing!
fclose($curl_log);

$response = curl_exec($curl);
//...restofscript...
?>

From PHP manual: http://php.net/manual/en/function.curl-setopt.php

CURLOPT_VERBOSE TRUE to output verbose information. Writes output to STDERR CURLOPT_STDERR An alternative location to output errors to instead of STDERR.

It is not a permission issue either, I have set file and script permissions to 777 on server side and my local client is windows and has never cared about permission settings (it's only for dev anyway).

Coble

Is the script complete? I ask because a curl_exec() is missing. And what is the output you expect (and why)? – Sloat

Oh, yeah I took that out in my copy/paste. Put curl_exec back in. I hope to get some sort of debugging information, although really I have no idea what to expect from it. I have no idea what is causing the connection to fail on the server end. – Coble

You can find detailed code-examples in a related answer to a different question including doing this in memory and not on disk – Solfeggio

this worked for me to find where the actual problem is description here – Beckham

Answer 1

You are making couple mistakes in your example:

1) you have to call curl_exec() prior to reading from the "verbose log", because curl_setopt() doesn't perform any action, so nothing can be logged prior to the curl_exec().

2) you are opening $curl_log = fopen("curl.txt", 'w'); only for write, so nothing could be read, even after you write to the file and rewind the internal file pointer.

So the correct shortened code should look like:

<?php
$curl = curl_init();
$curl_log = fopen("curl.txt", 'w+'); // open file for read and write
$url = "http://www.google.com";

curl_setopt_array($curl, array(
    CURLOPT_URL             => $url,
    CURLOPT_VERBOSE         => 1,
    CURLOPT_STDERR          => $curl_log,
    CURLOPT_RETURNTRANSFER  => 1
));

$response = curl_exec($curl);

rewind($curl_log);
$output= fread($curl_log, 2048);
echo "<pre>". print_r($output, 1). "</pre>";
fclose($curl_log);

// ...

?>

NOTE: verbose log could be longer than 2048 bytes, so you could "fclose" the $curl_log after curl_exec() and then read the whole file with for example file_get_contents(). In that case, the point 2) should not be considered as mistake :-)

Myocarditis

Instead of fopen("curl.txt", 'rw') use fopen("curl.txt", 'r+'). The w option is ignored after r. – Halfmoon

@Halfmoon you are right. In this case rather with w+ option - you want to write first and then read. If the file does not exist, fopen will return false instead of file handle. I've updated the answer. – Myocarditis

Answer 2

A bit late to the party, but this page still pops up high in Google, so let's go.

It seems that CURLOPT_VERBOSE doesn't log anything if CURLINFO_HEADER_OUT is also set to TRUE.

This is a know bug in PHP (#65348), and due to reasons they decided not to fix it.

Davina

Answer 3

Putting al above answers together, I use this function to make a Curl Post Request with loggin to a file option:

function CURLPostRequest($url, array $post = NULL, array $options = array(), $log_file = NULL){
    $defaults = array(
            CURLOPT_POST => 1,
            CURLOPT_HEADER => 0,
            CURLOPT_URL => $url,
            CURLOPT_FRESH_CONNECT => 1,
            CURLOPT_RETURNTRANSFER => 1,
            CURLOPT_FORBID_REUSE => 1,
            CURLOPT_TIMEOUT => 4,
            CURLOPT_POSTFIELDS => http_build_query($post)
    );

    if (is_resource($log_file)){
        $defaults[CURLOPT_VERBOSE]=1;
        $defaults[CURLOPT_STDERR]=$log_file;
        $defaults[CURLINFO_HEADER_OUT]=1;
    }

    $ch = curl_init();
    curl_setopt_array($ch, ($options + $defaults));
    if( ! $result = curl_exec($ch)){
        throw new Exception(curl_error($ch));
    }

    if (is_resource($log_file)){

        $info = curl_getinfo($ch);

        if (isset($info['request_header'])){
            fwrite($log_file, PHP_EOL.PHP_EOL.'* POST Content'.PHP_EOL.PHP_EOL);
            fwrite($log_file, print_r($info['request_header'],true));
            fwrite($log_file, http_build_query($post));
        }

        fwrite($log_file, PHP_EOL.PHP_EOL.'* Response Content'.PHP_EOL.PHP_EOL);
        fwrite($log_file, $result.PHP_EOL.PHP_EOL);
    }

    curl_close($ch);
    return $result;
}

Hope this help to someone.

Begrudge

should be $defaults[CURLINFO_HEADER_OUT]=1; verbose doesn't get written when CURLINFO_HEADER_OUT is enabled – Stansberry

Answer 4

I needed to close the file before being able to read it, this worked for me:

$filename = 'curl.txt';
$curl_log = fopen($filename, 'w'); // open file for write (rw, a, etc didn't help)
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_STDERR, $curl_log);        

$result = curl_exec($ch);

fclose($curl_log);      
$curl_log = fopen($filename, 'r'); // open file for read
$output= fread($curl_log, filesize($filename));
echo $output;

(PHP 5.6.0, Apache/2.2.15)

Titivate

Answer 5

From php manual for function curl_setopt:

CURLOPT_FILE The file that the transfer should be written to. The default is STDOUT (the browser window).  

Airedale

CURLOPT_VERBOSE claims to write to STDERR, when would CURLOPT_FILE output data? CURLOPT_FILE => $curl_log ...still does not write to the file. – Coble

Answer 6

You should put

$output = fread($curl_log, 2048);
echo $output; // This returns nothing!
fclose($curl_log);

after $response = curl_exec($curl); otherwise, file is closed during curl is executing.

Sabinesabino