How can I add a progress bar to WWW::Mechanize?

Question

I have the following code:

$mech->get($someurl, ":content_file" => "$i.flv");

So I'm getting the contents of a url and saving it as an flv file. I'd like to print out every second or so how much of the download is remaining. Is there any way to accomplish this in WWW::Mechanize?

Rhinal

Answer 1

Many thanks to Peter Kovacs' answer for leading me to the correct answer. It turned out to be a bit more elaborate than I'd expected though so I decided to (horror) answer my own question.

As Peter showed, I can set a callback like so:

$m->get($u, ":content_cb" => \&callback);

But now I can't save the content using the :content_file value, because I can only choose one of the two. The callback function gets passed the data, and I ended up writing that to a file instead.

I also get a response object which contains the total size of the content as friedo pointed out. So by keeping a running total of content received so far and dividing it by the total content I can find out what percent of the content has been downloaded. Here's the full callback function:

open (VID,">$i.flv") or die "$!";
$total = 0;
sub callback
{
    my( $data, $response, $proto ) = @_;
    print VID "$data"; # write data to file
    $total+= length($data);
    $size = $response->header('Content-Length');
    print floor(($total/$size)*100),"% downloaded\n"; # print percent downloaded
}

I hope that helps someone.

Rhinal

This works for me. I was able to use the file handle as a temp file just for stats. I use \r to backspace and keep the terminal at one line as the downloaded percentage goes up and all I did after $m->get($u, ":content_cb" => \&callback); is call $m->save_content($newLocation); This does not download the content twice. Tested on a 250M file. – Hist

Answer 2

WWW::Mechanize says that the get method is a "well-behaved" overload of LWP::UserAgent get. Looking at the docs for LWP::UserAgent, you can provide a content_cb key which is called with each chunk of the downloaded file:

$mech->get( $someurl, ":content_cb" => \&callback );
sub callback
{ 
  my( $data, $response, $proto ) = @_;
  # save $data to $i.flv
  # print download notification
 }

Thitherto

You'd also need something to get the total size (if available from the http response headers) – Serviceberry

You can get the size easily enough by looking at $mech->response->header( 'Content-Length' ), assuming the server sends that header. (For dynamic content, you might be out of luck.) – Bunyabunya