Can I get the progress of a git clone command, issued in grit, output to stdout?

Question

I'm fairly confident this is either not possible or I'm missing an obvious option, but after consulting grit's Git class, the gist linked in this SO post, and the other grit tagged questions on SO, I'm coming up blank.

I'm using grit for a series of rake tasks that install my application. One of these tasks clones a few repositories.

Using the code in the linked gist as an example, this is the output of git cloning in grit (should work out of the box after a gem install grit in irb, ruby 1.9.2):

> require 'grit'
> gritty = Grit::Git.new('/tmp/filling-in')
=> #<Grit::Git:0x007f93ae105df8 @git_dir="/tmp/filling-in", @work_tree="/tmp/filling-in", @bytes_read=0>
> gritty.clone({:quiet => false, :verbose => true, :progress => true, :branch => '37s', :timeout => false}, "git://github.com/cookbooks/aws.git", "/tmp/aws") 
=> "Cloning into /tmp/aws...\n" 

My question is this: Can I recover the rest of the clone command's stdout, preferably while the clone is actually happening? The "Cloning into /tmp/aws...\n" is the first line of output, and is only returned once the clone completes.

A secondary question would be: If it's not possible to recover the clone's progress while it's happening with grit, is there another way to show progress of the command while it happens? My concern is a few of our repositories are quite large, and I'd like to give my rakefile's users something so they don't conclude the script is simply hanging while trying to communicate with the remote.

For reference, the "normal" out put of the git clone command would be:

$ git clone git://github.com/cookbooks/aws.git /tmp/test-aws
Cloning into /tmp/test-aws...
remote: Counting objects: 12364, done.
remote: Compressing objects: 100% (3724/3724), done.
remote: Total 12364 (delta 7220), reused 12330 (delta 7203)
Receiving objects: 100% (12364/12364), 5.92 MiB | 70 KiB/s, done.
Resolving deltas: 100% (7220/7220), done.

Quick note: Some of the functionality described here, namely :process_info requires an up-to-date version of the gem, which hasn't been updated since 23 Jan 2011, as of today, 26 Sept 2011 (many thanks to Daniel Brockman for pointing that out, below). You'll have to clone it from github and build it manually.

SOLUTION

Clone passes the correct information to stderr if you give it :process_info and :progress (my original example failed because I had an outdated gem). The following is working code. You need to build the gem manually for reasons described above, at least at this time.

> require 'grit'
> repo = Grit::Git.new('/tmp/throw-away')
> process = repo.clone({:process_info => true, :progress => true, :timeout => false}, 'git://github.com/cookbooks/aws.git', '/tmp/testing-aws-again')  # output supressed
> print process[2]   # i.e. the stderr string of the output
remote: Counting objects: 12364, done.
remote: Compressing objects: 100% (3724/3724), done.
remote: Total 12364 (delta 7220), reused 12330 (delta 7203)
Receiving objects: 100% (12364/12364), 5.92 MiB | 801 KiB/s, done.
Resolving deltas: 100% (7220/7220), done.

Dorri

Try running git clone git://github.com/cookbooks/aws.git /tmp/test-aws 2>/dev/null and you’ll see which parts of the output are going to stderr. – Quaquaversal

No dice. That command me no output at all. Also ran repo.fetch({:process_info => true}) (forgot to run it earlier) and got [0, "", ""] as a result. – Dorri

Well, that’s what should happen when git fetch has nothing to do. – Quaquaversal

Ah there we go. Yeah, the fetch with :process_info returns the last line of the fetch into the stderr string. Still can't get it to work with clone but I'm definitely wrong above about fetch. – Dorri

There we go. You were right all along. My combo of an outdated gem + not passing :progress on clone caused the issue. – Dorri

I’m so glad you solved it! Horray! :D – Quaquaversal

Answer 1

The reason you’re only getting the first line of output is that the rest of it is sent to stderr, not stdout. There is a :process_info option you can pass to get both exit status, stdout and stderr. See https://github.com/mojombo/grit/blob/master/lib/grit/git.rb#L290.

Quaquaversal

Sorry, should have added that info to the original post. I did try the :process_info option (I might be using it incorrectly). When running the same command above, but with slimmed down options that I know work without :process_info, it won't clone at all. It immediately spits out an empty string: > gritty.clone({:branch => '37s', :timeout => false, :process_info => true}, "git://github.com/cookbooks/aws.git", "/tmp/aws-new") => "" – Dorri

By setting Grit.debug = true, you will see what is going on. Furthermore, by using blame, you find that :process_info was introduced in this commit, from 2011-02-06; however, the latest version of Grit is from 2011-01-13. – Quaquaversal

Got the most recent version of the gem (and hence :procces_info) working, but the [exitstatus, out, err] tuple still returns the first line of the clone and nothing more. Output from the my first comment's irb command: => [0, "Cloning into /tmp/aws-new...\n", ""] – Dorri

Added an update to the question with a working theory, but I'm really not 100% confident I'm right. – Dorri