How do I properly git stash/pop in pre-commit hooks to get a clean working tree for tests?
Asked Answered
G

4

28

I'm trying to do a pre-commit hook with a bare run of unit tests and I want to make sure my working directory is clean. Compiling takes a long time so I want to take advantage of reusing compiled binaries whenever possible. My script follows examples I've seen online:

# Stash changes
git stash -q --keep-index

# Run tests
...

# Restore changes
git stash pop -q

This causes problems though. Here's the repro:

  1. Add // Step 1 to a.java
  2. git add .
  3. Add // Step 2 to a.java
  4. git commit
    1. git stash -q --keep-index # Stash changes
    2. Run tests
    3. git stash pop -q # Restore changes

At this point I hit the problem. The git stash pop -q apparently has a conflict and in a.java I have

// Step 1
<<<<<<< Updated upstream
=======
// Step 2
>>>>>>> Stashed changes

Is there a way to get this to pop cleanly?

Garpike answered 9/12, 2013 at 20:13 Comment(0)
C
39

There is—but let's get there in a slightly roundabout fashion. (Also, see warning below: there's a bug in the stash code which I thought was very rare, but apparently more people are running into. New warning, added in Dec 2021: git stash has been rewritten in C and has a whole new crop of bugs. I used to suggest mildly that git stash be avoided; now I urge everyone to avoid it if at all possible.)

git stash push (the default action for git stash; note that this was spelled git stash save in 2015, when I wrote the first version of this answer) makes a commit that has at least two parents (see this answer to a more basic question about stashes). The stash commit is the work-tree state, and the second parent commit stash^2 is the index-state at the time of the stash.

After the stash is made (and assuming no -p option), the script—git stash is a shell script—uses git reset --hard to clean out the changes.

When you use --keep-index, the script does not change the saved stash in any way. Instead, after the git reset --hard operation, the script uses an extra git read-tree --reset -u to wipe out the work-directory changes, replacing them with the "index" part of the stash.

In other words, it's almost like doing:

git reset --hard stash^2

except that git reset would also move the branch—not at all what you want, hence the read-tree method instead.

This is where your code comes back in. You now # Run tests on the contents of the index commit.

Assuming all goes well, I presume you want to get the index back into the state it had when you did the git stash, and get the work-tree back into its state as well.

With git stash apply or git stash pop, the way to do that is to use --index (not --keep-index, that's just for stash-creation time, to tell the stash script "whack on the work directory").

Just using --index will still fail though, because --keep-index re-applied the index changes to the work directory. So you must first get rid of all of those changes ... and to do that, you simply need to (re)run git reset --hard, just like the stash script itself did earlier. (Probably you also want -q.)

So, this gives as the last # Restore changes step:

# Restore changes
git reset --hard -q
git stash pop --index -q

(I'd separate them out as:

git stash apply --index -q && git stash drop -q

myself, just for clarity, but the pop will do the same thing).


As noted in a comment below, the final git stash pop --index -q complains a bit (or, worse, restores an old stash) if the initial git stash push step finds no changes to save. You should therefore protect the "restore" step with a test to see if the "save" step actually stashed anything.

The initial git stash --keep-index -q simply exits quietly (with status 0) when it does nothing, so we need to handle two cases: no stash exists either before or after the save; and, some stash existed before the save, and the save did nothing so the old existing stash is still the top of the stash stack.

I think the simplest method is to use git rev-parse to find out what refs/stash names, if anything. So we should have the script read something more like this:

#! /bin/sh
# script to run tests on what is to be committed

# First, stash index and work dir, keeping only the
# to-be-committed changes in the working directory.
old_stash=$(git rev-parse -q --verify refs/stash)
git stash push -q --keep-index
new_stash=$(git rev-parse -q --verify refs/stash)

# If there were no changes (e.g., `--amend` or `--allow-empty`)
# then nothing was stashed, and we should skip everything,
# including the tests themselves.  (Presumably the tests passed
# on the previous commit, so there is no need to re-run them.)
if [ "$old_stash" = "$new_stash" ]; then
    echo "pre-commit script: no changes to test"
    sleep 1 # XXX hack, editor may erase message
    exit 0
fi

# Run tests
status=...

# Restore changes
git reset --hard -q && git stash apply --index -q && git stash drop -q

# Exit with status from test-run: nonzero prevents commit
exit $status

warning: small bug in git stash

(Note: I believe this bug was fixed in the conversion to C. Instead, there are numerous other bugs now. They will no doubt eventually be fixed, but depending on which version of Git you are using, git stash may have various bugs of varying seriousness.)

There's a minor bug in the way git stash writes its "stash bag". The index-state stash is correct, but suppose you do something like this:

cp foo.txt /tmp/save                    # save original version
sed -i '' -e '1s/^/inserted/' foo.txt   # insert a change
git add foo.txt                         # record it in the index
cp /tmp/save foo.txt                    # then undo the change

When you run git stash push after this, the index-commit (refs/stash^2) has the inserted text in foo.txt. The work-tree commit (refs/stash) should have the version of foo.txt without the extra inserted stuff. If you look at it, though, you'll see it has the wrong (index-modified) version.

The script above uses --keep-index to get the working tree set up as the index was, which is all perfectly fine and does the right thing for running the tests. After running the tests, it uses git reset --hard to go back to the HEAD commit state (which is still perfectly fine) ... and then it uses git stash apply --index to restore the index (which works) and the work directory.

This is where it goes wrong. The index is (correctly) restored from the stash index commit, but the work-directory is restored from the stash work-directory commit. This work-directory commit has the version of foo.txt that's in the index. In other words, that last step—cp /tmp/save foo.txt—that undid the change, has been un-un-done!

(The bug in the stash script occurs because the script compares the work-tree state against the HEAD commit in order to compute the set of files to record in the special temporary index before making the special work-dir commit part of the stash-bag. Since foo.txt is unchanged with respect to HEAD, it fails to git add it to the special temporary index. The special work-tree commit is then made with the index-commit's version of foo.txt. The fix is very simple but no one has put it into official git [yet?].

Not that I want to encourage people to modify their versions of git, but here's the fix.)

Cherice answered 9/12, 2013 at 20:56 Comment(18)
Your answer makes me wish there was a nice infographic/flowchart for git so that I could understand what's actually going on. Still reading and trying to grok.Garpike
There are some, but none are complete because it's so ridiculously complicated. As an alternative to using git stash though, you could consider creating two commits of your own, on a branch of your own creation, or even on the current branch. (It will work out the same in the end no matter which way you go. :-) )Cherice
That's what I feel like. I'd love to take a whack at it myself, but like you said: so ridiculously complicated. The multiple commit thing is a no-go for me though. In our environment each commit corresponds to a separately code-reviewed change. Multiple changes would lead to an explosion of code reviews, particularly in a hook. In general we have to constantly amend our last change whenever you want to review a minor edit to a change.Garpike
Works like charm! Thks. I have one problem, though. If you commit with --amend to fix the commit message (no files changed or modified or anything), you get some errors, I think with pop. Any thoughts?Unwisdom
@Nico: when you git commit --amend you are actually adding a new, different commit to the commit-graph. The "stash bag" is still hung off the old commit. If the trees of the old and new commits match, the stash should apply cleanly, even with --index. I'd have to see the errors to say more, but you can compare the old and new commit trees with git diff. Just give it both commit specifiers (e.g., HEAD and stash^ before attempting the git stash apply).Cherice
@Cherice follow this steps: Make a change on a file, add and commit. git status shows "nothing to commit, working dir is clean". Now run git commit --amend -m "new commit msg". When the pre-commit hook runs, it fails with "No stash found."Unwisdom
@Nico: Ah, I see now. The problem is that when this whole thing is dumped into a pre-commit hook, the initial git stash save -q --keep-index is just an expensive no-op as git stash save decides there are no local changes to save (run it without -q to see the message). One must detect this case and avoid attempting to restore the non-existent saved stash. I'll add a bit of code to the answer.Cherice
@Cherice Awesome, works great. Thanks and thanks for the example. I found it very hard to find good and complete examples of pre-commit hooks for git.Unwisdom
This doesn't work for me in this scenario: 1) edit line 5 in file x.txt 2) add this change to commit, 3) edit line 5 in file x.txt again, 4) at this point developer wants to commit. When I run this script then the end state is - I have lost the unstaged change to line 5. :(Pierre
@zilupe: there's a long standing bug in git stash that you could be hitting here. Are you changing line 5 back to the way it was before git adding? If so, you're hitting the bug. I posted a fix to the mailing list but it was apparently ignored...Cherice
@Cherice yes, you are absolutely right! That explains it. Thank you.Pierre
Great solution and helped me a lot in the other thread. It's a bit maddening that there's no way to get git stash to return a different exit code if it actually created a new stash (along the linesgit diff --exit-code)... that would simplify the rev-parse ugliness.Duisburg
If the purpose of git rev-parse -q --verify refs/stash is to check whether or not anything was stashed, what’s wrong with using git stash list | wc -l? The latter is much easier for me to wrap my tiny brain around.Detoxicate
@JezenThomas: That should work too: if the count goes up, you got a new stash. However, that's substantially more processes and code to run (git rev-parse is one binary doing, in this case, just a few simple things; git stash list is a script that runs half a gazillion sub-commands—well, ok, probably just 3 or 4 here :-)—and then wc -l has to read and count all the output lines written over the pipe).Cherice
Had to downvote, as git stash save is deprecated in favor of git stash push. I don't know what from this whole thread is still relevant.Talesman
@DevinRhode: git stash push did not even exist when I wrote the answer above. Also, git stash has been rewritten in C, and a whole new crop of bugs added: now you really should avoid git stash.Cherice
Yeah I've found new/untracked files are particularly weird...Talesman
FYI @Cherice - I search official git release notes for stash and have uploaded results here - gist.github.com/devinrhode2/1284e8ef0052a5f086b2055e72637d8a - Maybe the new C bugs are getting fixed, idk. Also, I removed my downvote :)Talesman
E
2

Thanks to the answer of @torek I was able to put together a script that also deals with untracked files. (Note: I don't want to use git stash -u due to a unwanted behaviour of git stash -u)

The mentioned git stash bug remains unchanged and I am not yet sure, whether this method might run into issues when a .gitignore is among the changed files. (same applies to @torek's answer)

#! /bin/sh
# script to run tests on what is to be committed
# Based on https://mcmap.net/q/12277/-how-do-i-properly-git-stash-pop-in-pre-commit-hooks-to-get-a-clean-working-tree-for-tests

# Remember old stash
old_stash=$(git rev-parse -q --verify refs/stash)

# First, stash index and work dir, keeping only the
# to-be-committed changes in the working directory.
git stash save -q --keep-index
changes_stash=$(git rev-parse -q --verify refs/stash)
if [ "$old_stash" = "$changes_stash" ]
then
    echo "pre-commit script: no changes to test"
    sleep 1 # XXX hack, editor may erase message
    exit 0
fi

#now let's stash the staged changes
git stash save -q
staged_stash=$(git rev-parse -q --verify refs/stash)
if [ "$changes_stash" = "$staged_stash" ]
then
    echo "pre-commit script: no staged changes to test"
    # re-apply changes_stash
    git reset --hard -q && git stash pop --index -q
    sleep 1 # XXX hack, editor may erase message
    exit 0
fi

# Add all untracked files and stash those as well
# We don't want to use -u due to
# http://blog.icefusion.co.uk/git-stash-can-delete-ignored-files-git-stash-u/
git add .
git stash save -q
untracked_stash=$(git rev-parse -q --verify refs/stash)

#Re-apply the staged changes
if [ "$staged_stash" = "$untracked_stash" ]
then
    git reset --hard -q && git stash apply --index -q stash@{0}
else
    git reset --hard -q && git stash apply --index -q stash@{1}
fi

# Run tests
status=...

# Restore changes

# Restore untracked if any
if [ "$staged_stash" != "$untracked_stash" ]
then
    git reset --hard -q && git stash pop --index -q
    git reset HEAD -- . -q
fi

# Restore staged changes
git reset --hard -q && git stash pop --index -q

# Restore unstaged changes
git reset --hard -q && git stash pop --index -q

# Exit with status from test-run: nonzero prevents commit
exit $status
Embranchment answered 18/2, 2015 at 10:31 Comment(3)
I created a more sophisticated hook framework using this script as a base for anyone who's interested: github.com/schamp/capn-hookKwabena
> I am recommending capn-hook be deprecated, since the introduction of the most excellent pre-commit. It does everything capn-hook was meant to do, only better. Go there instead, but I will keep this around for historical purposes.Talesman
I'm looking through git stash release notes... gist.github.com/devinrhode2/1284e8ef0052a5f086b2055e72637d8a I'm thinking this has probably been fixed in the last ~7 years, so I'm going to try and do what I want from scratch reading the docs, and then post my answer back here.Talesman
T
1

It seems that most answers here, at this time, are at least 5 years old. git stash has been re-written in C, there were some new bugs, and I don't really know how reliable it is.

I've used this a few times and it seems to be working - I wrote it from scratch, just using the docs as a guide. Has not been battle tested.

# We stash and un-stash changes ourselves.
#  - If any pre-commit/lint-staged checks fail, any auto-fixes will be lost.

# Create stash
#   index is the "staging area", so --keep-index means that anything you have already staged will be un-touched.
# NOTE: we always create a stash - possibly even a totally empty one.
git stash --keep-index --include-untracked --message="pre-commit auto-stash"
uncoloredStashedStat=$(git stash show --include-untracked stash@{0})
[[ $uncoloredStashedStat ]] && {
  echo "Stashed:"
  git diff --stat --staged stash@{0}
}

lintStagedStatus='failed'

yarn lint-staged --no-stash --concurrent $pre_commit_concurrency --shell "/bin/bash"  && {
  lintStagedStatus='passed'
}

outputSuppressed=$(git add --intent-to-add "**/*.snap")
diff=$(git diff)
[[ $diff ]] && {
  echo "staging modifications from pre-commit scripts:"
  git diff
  git add .
}

# Pop stash
#   We always create a stash - so we will always pop it.
#   Popped stash should generally not cause merge conflicts,
#   if your editor is formatting+autofixing code on save.
[[ $uncoloredStashedStat ]] && echo "restoring stash..."
git stash pop

if test "$lintStagedStatus" != 'passed'; then
  exit 1;
fi
Talesman answered 10/12, 2021 at 20:12 Comment(3)
Thanks for this! I got some good help from this, but what does this line do: outputSuppressed=$(git add --intent-to-add "**/*.snap")?Laktasic
haha, I BARELY know bash, just enough to accidentally "take down prod" if you will.Talesman
The git add command here is wrapped in outputSuppressed=$( so that the output of this command does not appear to users of this script (i.e. other developers)Talesman
L
0

based off torek's answer I came up with a method to ensure the right behavior of stashing changes without using git rev-parse, instead I used git stash create and git stash store (although using git stash store is not strictly necessary) Note due to the environment I am working in my script is written in php instead of bash

#!/php/php
<?php
$files = array();
$stash = array();
exec('git stash create -q', $stash);
$do_stash = !(empty($stash) || empty($stash[0]));
if($do_stash) {
    exec('git stash store '.$stash[0]); //store the stash (does not tree state like git stash save does)
    exec('git stash show -p | git apply --reverse'); //remove working tree changes
    exec('git diff --cached | git apply'); //re-add indexed (ready to commit) changes to working tree
}
//exec('git stash save -q --keep-index', $stash);
exec('git diff-index --cached --name-only HEAD', $files );

// dont redirect stderr to stdin, we will get the errors twice, redirect it to dev/null
if ( PHP_OS == 'WINNT' )
  $redirect = ' 2> NUL';
else
  $redirect = ' 2> /dev/null';
$exitcode = 0;

foreach( $files as $file ) {

  if ( !preg_match('/\.php$/i', $file ) )
    continue;

  exec('php -l ' . escapeshellarg( $file ) . $redirect, $output, $return );
  if ( !$return ) // php -l gives a 0 error code if everything went well
    continue;

  $exitcode = 1; // abort the commit
  array_shift( $output ); // first line is always blank
  array_pop( $output ); // the last line is always "Errors parsing httpdocs/test.php"

  echo implode("\n", $output ), "\n"; // an extra newline to make it look good
}
if($do_stash) {
    exec('git reset --hard -q');
    exec('git stash apply --index -q');
    exec('git stash drop -q');
}
exit( $exitcode );

?>

php script adapted from here http://blog.dotsamazing.com/2010/04/ask-git-to-check-if-your-codes-are-error-free/

Libertylibia answered 14/9, 2016 at 6:37 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.