git push Out of memory, malloc failed

Question

I am trying to push my changes remotely to GitHub, every so often git fails due to

C:\dev\projects>git push -v
Pushing to https://[email protected]/mycompany/My-Project.git
Password for 'github.com':
fatal: Out of memory, malloc failed (tried to allocate 524288000 bytes)
fatal: write error: Invalid argument

This is very, very aggravating. I have run the following commands, upgraded git (which wiped out my settings and caused lots of pain, but I digress)

git gc --auto --prune=today --aggressive
git repack

I have even bumped the value of

http.postbuffer

but eventually it will fail again.

This is a typical Rails 3.1 application, total project size on disk is 9.69 MB.

Karinekariotta

It's a little scary that it's trying to allocate 500MB - are you pushing something enormous? – Tiruchirapalli

No, that's the thing, I'm not, the total directory size is 55.7MB, and the log file which is ignored is 46MB, removed my log file, see edit – Karinekariotta

Are you including the .git directory in that? (That is, did you possibly commit then remove a lot of stuff?) – Tiruchirapalli

No, I ran a couple git commands, but I killed the rails server when I was removing my log file. I think my machine may have run out of memory from JRuby... hard to think 8GB Ram runs out so quickly. – Karinekariotta

Well, of course it's nice if your machine isn't already dying for other reasons, but Git did ask for 500MB for a reason, so if you want to try to track that down, checking the size of the .git directory is a start; if it's much larger than you'd expect, you can then look back through recent history and see if there are any enormous diffs (try git log --shortstat for example). – Tiruchirapalli

You can simply pause other unnecessary processes for some time and push because the issue is related to the free RAM memory of your PC. If still not working then you can do as mentioned in above comments. – Looney

I had this after terminating visual studio in reboot. Looks like the git is not failsafe anymore. I guess this bug should be reported to git team so they could fix it. – Inbeing

Answer 1

My advice is to try several git parameters related with pack:

[pack]
   threads = 1
   deltaCacheSize = 128m
   windowMemory = 50m

What it got better results for me was setting git config pack.threads 1 and git config pack.windowMemory 50m (default is 10m).

Still, my host didn't have enough RAM memory (2GB) and kept failing. I hard copied the repo and moved it to another machine with more RAM (8GB). It got better but still failed.

Finally, I downloaded the latest version of git (https://github.com/git/git), compile it and install it. That fixed the problem just by running git repack -adf with the same parameters. After that I run git gc --aggressive --prune=now

Once I got the repo fixed in my local machine I pushed it to master, overwriting the remote repo, git push -f origin master.

To prevent similar errors in the future try not to add unnecessary large files to the repo (in my case I got a SQL dump of 3.5GB :)) and disable delta compression for large files (such as images, PDFs, videos). Add the following lines to .gitattributes:

*.pdf -delta
*.jpg -delta

Jordonjorey

Hi Diego...Can you please tell us you upgraded which git version to which one? – Mulhouse

Hello @Ahmad, this was a long time ago. I built git from trunk. I remember I posted this answer right after I fixed the problem, so taking a look at the "git tags" v1.8.0 is the closest you can get. – Jordonjorey

Had to go up to 1024m in my case. I had some dozens of 200m tar files that had been checkout out in multiple revisions, so maybe I'll go with the .gitattributes idea. – Yeorgi

Answer 2

Use this:

git gc --auto --prune=today --aggressive 
git repack 
git config --global http.postbuffer 524288000 
git config --global pack.windowMemory 256m

Its fixes for me.

Jeannettejeannie

Changing the post buffer fixed it for me when I was trying to clone a repository. – Gamboa

I added configs 'pack.threads=1 , pack.windowmemory=256m, pack.packsizelimit=20m` 1 by 1 and still fail in pushing. It works until I used the git repack and push again. – Altruist

Answer 3

You could try changing the config for repack with

git config --global pack.windowMemory 256m

Parasynthesis

@ashishsony probably varies between machines. 256m was just an example. – Parasynthesis

PS omit the global flag to limit it to just the current repo. And for me 32m seemed to be enough. – Slunk

Answer 4

I had the same issue and after changing some parameters to 1024m the problem persisted:

[pack]
     threads = 1
     deltaCacheSize = 1024m
     packSizeLimit = 1024m
     windowMemory = 1024m
[core]
     packedGitLimit = 1024m
     packedGitWindowSize = 1024m

I think the issue is related to the free RAM memory of your PC.

Mine was quite busy, and after rebooting it I could finally push the changes.

Hope it helps.

Vacillation

I'm not sure why you're posting all those values if they did not help. This is confusing – Aflutter

"I think the issue is related to the free RAM memory of your PC" was the right answer for me – Condemn

128m for [core] and [pack] values in .git/config + A new value under [http]: postbuffer = 5m fixed the out of memory error for me. – Oracle

Answer 5

for someone who use gitlab and see this error

find gitlab config (/etc/gitlab/gitlab.rb)

change the value of gitlab_rails['git_max_size'] (to a bigger value)

then: gitlab-ctl reconfigure to refresh

Seabrooke

Answer 6

git repack 
git config --global http.postbuffer 524288000 
git config --global pack.windowMemory 256m

It fixed for me. and do git push after it.

Nutting

Answer 7

i deleted these settings in my .gitconfig file:

[http]
postbuffer = 524288000
[pack]
windowMemory = 1024m
deltaCacheSize = 1024m
packSizeLimit = 1024m

and the push worked again

Thelen

Answer 8

If you are on Gitlab find the repo that's causing the problem on the remote server.

Gitlab stores the repositories in this location

/var/opt/gitlab/git-data/repositories

Find the directory and run this command.

git repack -a -f -d

Done.

Aaron

Answer 9

I had the same problem on an AWS t2.small. Also running sonar. I shutdown Sonar to free memory, checked out and restarted Sonar. I will increase the instance size.

Flannelette

Actually no need to increase the Memory. You could have just simply enabled the SWAP file instead. – Thrombo

Answer 10

In my case it had nothing to do with the client machine. This happened to me because the memory of the server where GitLab is running went full. I increased the memory of that virtual machine and the problem was solved.

Novah

Actually no need to increase the Memory. You could have just simply enabled the SWAP file instead. – Thrombo

In order just to solve the specific problem above, swap alone is the best imo. Because you just need it for the first push as a whole anyway. Follow up pushes will be down to normal (few KBs) again. – Thrombo

Answer 11

Check if swap is enabled in your box.

$free -m
             total       used       free     shared    buffers     cached
Mem:           494        339        154         33          0         60
-/+ buffers/cache:        278        216
Swap:         2047         40       2007

If not, you could create one. I tried this guide for ubuntu that works for Debian also. But there should be plenty of tutorials about this subject.

Dufresne

Answer 12

For me, the problem was also with the server not having enough memory as opposed to the client. I have a feeling the client's the problem if this happens while checking out and the server may be the problem if this happens when pushing up.

Dippy

Answer 13

I tried all listed answers; however, my case was the file size in my commit. When I checked what was in my commit, I then realize it was due to the size of one of the files.

I first logged my commit to show the commit id, then listed the files with git show:

git log
git show <COMMIT ID> --name-only

Determine if you have a large file. If so, and you don't need to have it in your commit, start by resetting most recent commit:

git reset --soft HEAD~1

Next, unstage file:

git reset HEAD <file>

(Optional) If you want to delete the file from index, run:

git rm --cached <file>

Then commit once more with --amend:

git commit --amend

Try pushing again:

git push

Oligarchy

Answer 14

I was facing the same problem, I try all the suggestions, but what solve the situation was restarting the computer... And then, I could push.

Flameproof

In fact, I was facing exactly the same message, and when I tryed to solve it like all the other answers didn't solve my situation, reading and understanding better the problem, I restart the PC, probably because it clear some free space memory, so I think is good to share with others that have the same problem, you think is better remove my answer? – Flameproof

Answer 15

You may just have one or more really big files. Check for files greater than 50MB, which is the normal github limit:

find . -type f -size +50M

Croteau

Answer 16

In my case Git.exe just needed a little bit more then 32bit process could get.

As you can see on the picture 64bit git.exe did it well.

Inbeing