How to write unit tests for GitPython clone/pull functions?

Question

I have a python project that is using GitPython to perform clone and pull functions against a remote Git repository.

As a simple example:

import git
from git import Git
from git import Repo


def clone_and_checkout(full_dir, git_url, repo_ver):

    repo = Repo.clone_from(
        url=git_url,
        to_path=full_dir
    )

    # Trigger re-create if repository is bare
    if repo.bare:
        raise git.exc.InvalidGitRepositoryError

    # Set origin and pull
    origin = repo.remotes.origin
    origin.pull()

    # Check out desired version of repository
    g = Git(full_dir)
    g.checkout(repo_ver)

I want to be able to write a unit test for this function, but obviously this needs to reach out to an external system as it stands currently.

I am curious if anyone has experience mocking up this external interaction, in a manner similar to using Mock to mock up HTTP calls. I'd like to be able to perform these tasks in a way that can be mocked at test time without needing to call an actual Git remote.

How should I go about writing tests for this?

EDIT: To be clearer about what I'm asking, I should mention I'm new to Mock and was struggling to understand how to Mock instances of these classes rather than the classes themselves. My question should have been phrased better - something along the lines of "how do I use Mock to set instance-specific properties like bare?"

I have since learned much about Mock and have figured out how to do this, so I will provide an answer to my own question.

Aggappe

You refer to mocking and Mock - have you tried that? – Greensboro

That's actually what I'm using for mocking the GitPython methods, but I think what I'm struggling with most is mocking up the Repo and Git objects. – Aggappe

You'll need to be more specific than "struggling" - provide a minimal reproducible example of your attempt and a precise description of the issue with it. – Greensboro

@Greensboro - thanks for the tip. Admittedly I wrote this in a bit of a rush. I will improve the question with more detail. – Aggappe

Answer 1

This seems to be a common result of an incomplete understanding of Mock, or the use of the Patch method.

The first thing to do is read the "where to patch" section located on the Mock documentation. Armed with that information, you should be able to use the patch function to mock the GitPython objects used in the function above. These decorators would appear above your unit test function.

@mock.patch('gitter.Repo')
@mock.patch('gitter.Git')

In order to provide a return value for an instance of one of these mocked objects, you can use PropertyMock. Here's a full example of a unit test which leverages this:

import gitter  # file containing our clone function
import mock
import unittest


class test_gitter(unittest.TestCase):

    @mock.patch('gitter.Repo')
    @mock.patch('gitter.Git')
    def runTest(self, mock_git, mock_repo):

        # Set the "bare" attribute of the Repo instance to be False
        p = mock.PropertyMock(return_value=False)
        type(mock_repo.clone_from.return_value).bare = p

        gitter.clone_and_checkout(
            '/tmp/docker',
            '[email protected]:docker/docker.git',
            'master'
        )
        mock_git.checkout.called_once_with('master')

Aggappe

hey @mierdin this is an old (2015!) but useful discussion. i'm just curious if you're still maintaining this project and if so could you link to it? i need to patch some git clone operations as well but i'm using github3.py and pytest. i'm trying to understand how to do this with pytest and unittest if i need objects from it too. many thanks. – Surely

Unfortunately no, this was an internal project for an employer. However, the same principles described in the "where to patch" document should apply equally to any library, not just the ones I've used above. – Aggappe

Answer 2

Help comes from knowledge of the git protocols

Actually, you don't need to mock anything. You can test all the functionality, i.e. local git operations, like: add, commit, checkout, rebase or cherry-pick and remote operations, like fetch, push or pull without setting up a git server.

Local and remote repositories exchange data using one of those protocols:

• local protocol (no network used, in unixes it's denoted as file://)
• http(s):// protocol
• ssh:// protocol
• git:// protocol

More on this in git protocols explained

Creating test environment for git operations

The local protocol is being used - when you specify path to other repository in the local file system. So the only thing you have to do in order to perform tests in a clean and isolated environment - is to arrange two repositories. The convention is to set up the "remote" as bare repository. Then the other should have upstream set by path to the first one, and voilà! Since now, you have fully functional test setup. Say thanks to Linus Torvalds.

Pytest implementation of fake git repository fixture

This uses a built-in tmp_path fixture that takes care of creating (and cleaning up) the repositories in a temporary folder, in my case it's in:

/tmp/pytest-of-mikaelblomkvistsson/pytest-current/test_preparing_repo0

import datetime
from pathlib import Path

import pytest
from git import Actor, Remote, Repo


@pytest.fixture
def fake_repo(tmp_path) -> "Helper":
    return Helper(tmp_path)


class Helper:
    """The main purpose of defining it as a class is to gather all the variables
    under one namespace, so that we don't need to define 6 separate pytest fixtures.

    You don't need git server to test pull/push operations. Since git offers
    "local protocol" - plain bare repository in your filesystem is fully
    compatible with http(s), ssh and git protocol (Neglecting authentication functionality).
    """

    def __init__(self, tmp_path_fixture: Path):
        self.local_repo_path: Path = tmp_path_fixture / "local-repo"
        remote_repo_path: Path = tmp_path_fixture / "remote-repo"
        remote_repo_path.mkdir()

        self.remote_repo: Repo = Repo.init(str(remote_repo_path), bare=True)

        self.repo: Repo = Repo.init(str(self.local_repo_path))
        self.remote_obj: Remote = self.repo.create_remote("origin", str(remote_repo_path))

        # do initial commit on origin
        commit_date = self.tz_datetime(2023, 10, 1, 11, 12, 13)
        self.repo.index.commit("Initial commit", author_date=commit_date, commit_date=commit_date)
        self.remote_obj.push("master")

    def local_graph(self) -> str:
        return self.repo.git.log("--graph --decorate --pretty=oneline --abbrev-commit".split())

    @classmethod
    def tz_datetime(cls, *args, **kwargs):
        tz_info = datetime.datetime.utcnow().astimezone().tzinfo
        return datetime.datetime(*args, **kwargs, tzinfo=tz_info)

    def do_commit(self, *files_to_add, msg: str = "Sample commit message.", author: str = "author") -> None:
        author = Actor(author, f"{author}@example.com")
        # constant date helps to make git hashes reproducible, since the date affects commit sha value
        date = self.tz_datetime(2023, 10, 4, 15, 45, 13)

        self.repo.index.add([str(file_) for file_ in files_to_add])
        self.repo.index.commit(msg, author=author, committer=author, author_date=date, commit_date=date)


def test_preparing_repo(fake_repo):
    file_1 = fake_repo.local_repo_path / "file_1.txt"

    file_1.write_text("Initial file contents")
    fake_repo.do_commit(file_1, msg="First commit.")

    fake_repo.repo.git.checkout("-b", "new_branch")
    file_1.write_text("Changed file contents")
    fake_repo.do_commit(file_1, msg="Second commit.")

    fake_repo.repo.git.checkout("-b", "other_branch")
    file_1.write_text("Another change")
    fake_repo.do_commit(file_1, msg="Change on other_branch.")

    assert (
        fake_repo.repo.git.branch("-a")
        == """\
  master
  new_branch
* other_branch
  remotes/origin/master"""
    )

    assert (
        fake_repo.local_graph()
        == """\
* 1743bd6 (HEAD -> other_branch) Change on other_branch.
* 2696781 (new_branch) Second commit.
* 5ea439d (master) First commit.
* 04fc02f (origin/master) Initial commit"""
    )

Genic