Git: fetch a specific object from a remote

Question

At work we have to cope with a very huge git repository (90Go). After having tried to tweak some things on my local copy (I had errors with a pack file, tried to unpack it), I might have deleted some object files. For instance:

> git gc

error: Could not read af9ed8[:snip:]

fatal: bad tree object af9ed8[:snip:]

error: failed to run repack

How can I fetch a specific object (in that case the one with the hash af9ed8...) from a remote repository?

Conventionality

I know this doesn't answer your question, but you might consider suggesting splitting up that repo! If it's that big now, it's only going to get bigger, and wow, that must take a long time to clone. – Merriment

What happens if you run git fetch origin? Does it try to fetch too many objects? – Merriment

@Jefromi That's basically unclonable ^^. The procedure is just co copy rawly the repo. I'd love to get it splitted up, but that would be a hell of a work, paralyzing the team for the whole time and that's not what the guys in charge are gonna do. I'm not responsible for it (I'm kinda new...). They're just used to cope with it :sigh: ... – Conventionality

Never mind, it looks like fetch/fetch-pack don't actually check to see if objects are intact; they just see that you have the commit and assume you have everything else too. Do you happen to have filesystem access to the remote? – Merriment

@Jefromi Yes, fetch doesn't check. I might have access, but as it is the central repository I wish to minimize the risks ;). A git command for that purpose is IMO safer that tweaking with the objects everybody's using... – Conventionality

Answer 1

If you have filesystem access to the remote or any other intact repository containing this object, you should be able to go there and run:

git cat-file tree af9ed8 > 9ed8...

Note that this only requires read-only access to the repo (you can write the file elsewhere) so it should be very safe, and you could even do it as a user who doesn't have write permissions to the filesystem. You can then transfer/copy it into the appropriate place in your repo:

cp path/to/9ed8... .git/objects/af/9ed8...

Loose objects are stored in directories with the first two digits of the hash; the filenames are the rest of the hash.

Edit: If the object is loose on the remote end you could also just directly copy it out of .git/objects, but if it's packed, you'd have to unpack it with git unpack-objects, and I imagine the packfiles in that repo are prohibitively large. The nicest way to do that would be to copy the packfile into your corrupted repo, delete any corrupted objects, and then use git unpack-objects < packfile, which will not unpack any objects which already exist.

I'm not sure how to do it via normal remote commands; even the lower-level git fetch-pack still operates at a ref level, fetching a pack with the necessary commits to complete the ref. I suspect that you might be able to do something sneaky like deleting all of the commit objects which reference that tree (git fsck might help you find them). I was really hoping you might be able to create a tag pointing to that object, push it to the remote, then try to fetch the tag from the remote (coaxing it into fetching the object) but it looks like Git is quite careful not to let you make or manipulate a tag pointing to a nonexistent object.

Merriment

It tells me when I try to prune again that the object af9ed8... of my local repo is corrupted. – Conventionality

Prune? You mean git gc? Maybe you should be starting with just git fsck. And it says that object is corrupt after you've copied an intact one in from another repo? (I'm not sure I'd want to run git gc on a repo this big, either. That's going to take a loooooong time.) – Merriment

I don't even know what you've tried. I can try to help you, if you'll give me information. I've told you the basic way you recover an object, if you have filesystem access to a repo with an intact version, which sounds like it should be good enough in your case. – Merriment

Sorry, I tried what you specifed: log on to the remote pc, cat-file the object, and copy it in my local gît database. It seemed to work, yet git prune/gc tells me the object is corrupted. It must be the remote that has a problem. – Conventionality

Answer 2

The following should work:

git fetch-pack --thin --keep "$remote" <hashe(s)>

Run git fsck --full | grep -v dangling before and after to get a list of all hashes that are missing / bad.

Seemaseeming

--keep may not be required. Try it without first. Let me know and I'll update the answer. – Seemaseeming

Answer 3

Try

git fetch-pack --refetch "/path/to/remote" <hash(es)>

to fetch any refs that are labeled as bad object etc. This (--refetch) forces refetch of any object even if they are logged as existent by git, which - at least for me - was the problem.

Logorrhea