Why test in continuous integration if you can test on pre-commit and pre-push git hooks?

Question

What is the point of using a Continuous Integration system to test your code if you already have a system like Husky running that allows you to test you code before pre-commit and pre-push?

Hornswoggle

Answer 1

Pre-commit and pre-push hooks are great for quick operations and tests. Sometimes you can even setup a hook in your IDE that will run quick unit tests every time you save a file. But usually you have multiple suites of tests and unlike unit tests functional, integration and performance tests often take longer time to run, which is not feasible for hooks.

Also, you want to run your tests in the same environment where you build your deliverables, which is usually not your local machine.

Another reason to use CI system is to run post-merge tests to verify that there are no issues introduced by multiple parallel merges.

All-in-all, the more tests you run, the better and a CI system allows you to run both pre-merge tests usually triggered by some sort of pull request hook and post-merge tests. And all of that in a controlled reliable environment.

Ickes

Thanks for such an awesome answer! – Hornswoggle

Answer 2

I'm not really interested about whether it passes in your local environment, where you may have a different version of some dependent library on your environment path. I want to know for sure that anyone's contributions don't break the software when linked against the specific library versions that we ship with.

Unplug

Answer 3

One reason to test using a Continuous Integration platform like Travis would be to assure developers haven't circumvented their own local development environment's testing git hooks.

Hornswoggle

Answer 4

CI is not only tests, it's a lot more, but the test stage is of course a very important part of the flow.
As you said in your own answer, local environments could be changed, the tests on the CI could have stricter settings, the environment you test on could be more like the environment that the end-user uses (say, set versions of software or even hardware).

Say for example that you develop a PHP package. The package have support for everything between php 5.6 and 7.2, it should also support multiple types of operating systems and should behave differently if ext/open_ssl is installed or not. A local test suite would rarely have a setup allowing the developer to test each of the possible versions on each of the required platforms, but a test suite set up in a CI pipeline could.

And honestly, it's always a good idea to test one more time, just to be safe! ;)

Nola

Answer 5

In certain useful and reasonable workflows, it is acceptable to commit and push broken commits (though not to the master branch). Preventing such workflows with git hooks is annoying.

Rebasing or merging as an example does not run hooks again, even though files are changed.

Hooks are also very difficult to get right. They check a local state which might not be what gets pushed (if certain files are present that are not in git).

CI servers also provide a stable predictable environment. E.g. consider a CI server having Linux and developers using MacOs laptops. The git hooks run on MacOs, which has case insensitive file system, allowing tests to pass even if filenames are wrong.

Hooks also punish diligent developers who run checks manually before committing, because tests are just run again one more time.

Each professional project should have CI. The real question is why any project should maintain annoying slow fragile broken local hooks when you already have CI.

Use hooks only for private toy projects.

Neumark