How can I call 'git pull' from within Python?

Question

Using the github webhooks, I would like to be able to pull any changes to a remote development server. At the moment, when in the appropriate directory, git pull gets any changes that need to be made. However, I can't figure out how to call that function from within Python. I have tried the following:

import subprocess
process = subprocess.Popen("git pull", stdout=subprocess.PIPE)
output = process.communicate()[0]

But this results in the following error

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Is there a way that I can call this bash command from within Python?

Anting

This is a duplicate of #4256607 – Cycloplegia

@Brandon that's not true, there are many other solutions, most better. – Mathilda

Is the git executable in the PATH? – Matriculation

@Mathilda perhaps, from what I understand celecnius is effectively asking "how do I run a bash command". This has been asked and answered many times. – Cycloplegia

This is an old question, but I think subprocess.Popen now has a cwd keyword argument that will execute the command in a specified directory. Ex: subprocess.Popen(['git', 'pull', '-v', 'origin', 'master'], cwd='/path/to/git/repo', ...) – Extravascular

Does this answer your question? Use Git commands within Python code – Alcoholic

Answer 1

Have you considered using GitPython? It's designed to handle all this nonsense for you.

import git  # pip install gitpython


g = git.cmd.Git(git_dir)
g.pull()

To install the module use pip install GitPython

Project can be found here, GitPython

Mathilda

git_dir is any source controlled directory you have (which contains the .git folder) – Eskridge

It is prompting for password.. How can I pass the password for that?? – Quadri

You can use ssh to establish keys, so no password authentication is requied – Mekong

msg = g.pull() tells you what happened. msg is '' or 'Updating ... Fast-forward README.md | 1 + 1 file changed, 1 insertion(+)' if you pulled successfully and 'Already up-to-date.' if you're... already up to date. – Lemal

This is a really bad way to use GitPython - it handles barely any of the garbage, and just translates to subprocess.check_output(['git', 'pull'], cwd=git_dir), leaving you on the own to parse the output of a "git porcelain" command, which you're not supposed to do – Chiastic

No module named 'git' – Shushan

@EightRice sudo pip install GitPython – Mathilda

gracias @Mathilda – Shushan

Wasn't using git known to have memory leaks though? – Valorous

Answer 2

subprocess.Popen expects a list of the program name and arguments. You're passing it a single string, which is (with the default shell=False) equivalent to:

['git pull']

That means that subprocess tries to find a program named literally git pull, and fails to do so: In Python 3.3, your code raises the exception FileNotFoundError: [Errno 2] No such file or directory: 'git pull'. Instead, pass in a list, like this:

import subprocess
process = subprocess.Popen(["git", "pull"], stdout=subprocess.PIPE)
output = process.communicate()[0]

By the way, in Python 2.7+, you can simplify this code with the check_output convenience function:

import subprocess
output = subprocess.check_output(["git", "pull"])

Also, to use git functionality, it's by no way necessary (albeit simple and portable) to call the git binary. Consider using git-python or Dulwich.

Tatty

Per documentation: “args should be a sequence of program arguments or else a single string.” (emphasis mine). Are you sure about this, because it works for me? – Matriculation

@Matriculation Yes, the documentation is a little bit unclear. If you want to pass in a string, you must pass in shell=True. – Tatty

Well, as I said, it works for me doing just subprocess.Popen("git pull", stdout=subprocess.PIPE).communicate() (3.3 and 2.7 on Windows). – Well, reading a bit further, the docs explain that this will only work on Unix when not passing any arguments.. Nevermind then ^^ – Matriculation

@Matriculation You were right about the single string case though, my initial explanation was wrong. Turns out, you can pass in a string, but (quoting): If args is a string, the interpretation is platform-dependent. Added more details. In Python 3.3, the error message is much nicer. – Tatty

This does not work on windows when autocrlf is needed. Calling git from cmd is not the same like calling git from sh.exe. The automatic crfl replacement is not taken into acoount and that is why git status does not work – Personate

Answer 3

The accepted answer using GitPython is little better than just using subprocess directly.

The problem with this approach is that if you want to parse the output, you end up looking at the result of a "porcelain" command, which is a bad idea

Using GitPython in this way is like getting a shiny new toolbox, and then using it for the pile of screws that hold it together instead of the tools inside. Here's how the API was designed to be used:

import git
repo = git.Repo('Path/to/repo')
repo.remotes.origin.pull()

If you want to check if something changed, you can use

current = repo.head.commit
repo.remotes.origin.pull()
if current != repo.head.commit:
    print("It changed")

Chiastic

Answer 4

If you're using Python 3.5+ prefer subprocess.run to subprocess.Popen for scenarios it can handle. For example:

import subprocess
subprocess.run(["git", "pull"], check=True, stdout=subprocess.PIPE).stdout

Octofoil

Answer 5

This is a sample recipe, I've been using in one of my projects. Agreed that there are multiple ways to do this though. :)

>>> import subprocess, shlex
>>> git_cmd = 'git status'
>>> kwargs = {}
>>> kwargs['stdout'] = subprocess.PIPE
>>> kwargs['stderr'] = subprocess.PIPE
>>> proc = subprocess.Popen(shlex.split(git_cmd), **kwargs)
>>> (stdout_str, stderr_str) = proc.communicate()
>>> return_code = proc.wait()

>>> print return_code
0

>>> print stdout_str
# On branch dev
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#   file1
#   file2
nothing added to commit but untracked files present (use "git add" to track)

>>> print stderr_str

The problem with your code was, you were not passing an array for subprocess.Popen() and hence was trying to run a single binary called git pull. Instead it needs to execute the binary git with the first argument being pull and so on.

Hussey

Not that split should really be shlex.split, and that code is needlessly complicated - you don't need wait after communicate, and for most usage cases, check_output does the right string already. – Tatty

I use the wait() to gather the return code of the process, for my needs. It is optional, if the OP is not interested in return code checking. Agreed about using shlex.split though. And yes if you're not interested in grabbing stdout or stderr you can skip those. Nothing wrong in providing alternate solutions which lets you do something more instead of the standard python documentation examples ;) – Hussey

And as you mentioned, check_output has been added only since 2.7, I have had the need to run my script in environments with python versions older than that too. – Hussey

While wait works, proc.pid is way easier to understand. And since check_output is pure Python, one can simply copy it to have it available on older Python versions as well. – Tatty

Oh sweet! yes check_output seems more robust too :) – Hussey

For "git rev-parse --short=9 HEAD", to get the latest commit's SHA into a variable, this worked for me where the simpler command didn't on python 2.7.10, mac osx 10.10.5. – Astro

Answer 6

Try:

import subprocess
cwd = '/path/to/relevant/dir'
command = 'git pull'
process = subprocess.Popen(command.split(), stdout=subprocess.PIPE, cwd=cwd)
output, unused_err = process.communicate()
print(output)

Stomodaeum

Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Garboard

Answer 7

Try:

subprocess.Popen("git pull", stdout=subprocess.PIPE, shell=True)

Avast

This would needlessly spawn a shell though, wouldn't it? – Tatty