cURL download progress in PHP

Question

I'm pretty new to cURL so I've been struggling with this one for hours. I'm trying to download the source of a website in an iframe using cURL and while it's loading to show how much of it is loaded. So far I have successfully downloaded the source without showing the loading progress. Can you explain how to show the download progress? Without cURL I would read the file byte by byte and divide the total amount of downloaded bytes with the total size of the file. How can this be done in cURL since it reads the source as a whole? (at least I think that this is the only way, not sure) Here's what I've got so far:

/* Download source */
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $adress);  
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, 0); 
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
$html = curl_exec($ch);
curl_close($ch); 

Uncalledfor

Are you executing the script in terminal or a web browser? For web browser you will need javascript. – Cheeky

In a web browser - printing the webpage source in an iframe – Uncalledfor

I belive this PHP based download accelerator is what you want. – Chaulmoogra

Answer 1

What you need is

<?php
ob_start();

echo "<pre>";
echo "Loading ...";

ob_flush();
flush();

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://stackoverflow.com");
//curl_setopt($ch, CURLOPT_BUFFERSIZE,128);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, 'progress');
curl_setopt($ch, CURLOPT_NOPROGRESS, false); // needed to make progress function work
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
$html = curl_exec($ch);
curl_close($ch);


function progress($resource,$download_size, $downloaded, $upload_size, $uploaded)
{
    if($download_size > 0)
         echo $downloaded / $download_size  * 100;
    ob_flush();
    flush();
    sleep(1); // just to see effect
}

echo "Done";
ob_flush();
flush();

?>

Melan

It works fine but there are a bunch of errors that there is a division by zero in the progress function. Maybe when the downloaded size is 0? – Uncalledfor

Notice: ob_flush() [ref.outcontrol]: failed to flush buffer. No buffer to flush. in C:\Users\mr.v\my sites_vlab\test-php-progress-bar\test1.php on line 7 Notice: ob_flush() [ref.outcontrol]: failed to flush buffer. No buffer to flush. in C:\Users\mr.v\my sites_vlab\test-php-progress-bar\test1.php on line 26 does anything wrong? – Corset

Answer 2

This is how the callback looks in C:

typedef int (*curl_progress_callback)(void *clientp,
                                      double dltotal,
                                      double dlnow,
                                      double ultotal,
                                      double ulnow);

Probably in PHP it should look like

curl_progress_callback($clientp, $dltotal, $dlnow, $ultotal, $ulnow)

So, assuming you have page.html which loads a .php file in an iframe.

In your php script, you will require the following functions:

curl_setopt($curl, CURLOPT_PROGRESSFUNCTION, 'curl_progress_callback');    
curl_setopt($curl, CURLOPT_BUFFERSIZE,64000);    
curl_setopt($curl, CURLOPT_FOLLOWLOCATION, true);

which should produce an output similar to the following:

0
0.1
0.2
0.2
0.3
0.4
...

then on the iframe page, you will have a progress bar

<div id="progress-bar">
    <div id="progress">0%</div>
</div>

CSS would be something like this

#progress-bar {
    width: 200px;
    padding: 2px;
    border: 2px solid #aaa;
    background: #fff;
}

#progress {
    background: #000;
    color: #fff;
    overflow: hidden;
    white-space: nowrap;
    padding: 5px 0;
    text-indent: 5px;
    width: 0%;
}

The javascript

var progressElement = document.getElementById('progress')

function updateProgress(percentage) {
    progressElement.style.width = percentage + '%';
    progressElement.innerHTML = percentage + '%';
}

You can have it output JavaScript and have it update the progress bar for you, for example:

<script>updateProgress(0);</script>
<script>updateProgress(0.1);</script>
<script>updateProgress(0.2);</script>

You might be interested in some more example code

Cheeky

why does the callback function have different names? i assume that curl_setopt($curl, CURLOPT_PROGRESSFUNCTION, 'callback'); should be curl_setopt($curl, CURLOPT_PROGRESSFUNCTION, 'curl_progress_callback'); right? – Uncalledfor

Answer 3

To use the callback inside a class, you must do it like this:

curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, array($this, 'progress'));

or if using static functions, like this:

curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, array('self', 'progress'));

... to a callback function to do whatever you need:

private static function progress($resource, $downloadSize, $downloaded, $uploadSize, $uploaded)
{
    // emit the progress
    Cache::put('download_status', [
        'resource' => $resource,
        'download_size' => $downloadSize,
        'downloaded' => $downloaded,
        'upload_size' => $uploadSize,
        'uploaded' => $uploaded
    ], 10);
}

Pharisee

Answer 4

I had problems with the flush on my web server. I solved it by adding:

    for ($j = 0; $j < 128; $j++) {
        print "<!---------------------------->\n";
    }

before the flush(), ob_flush().

Camara