Abort ExUnit on the first test that does not pass

Question

In Ruby, specifically RSpec, you can tell the test runner to abort on the first test that does not pass by the command-line flag --fail-fast. This helps a lot to not waste time or lose focus when fixing a lot of test in a row, for example when doing test-driven or behavior-driven development.

Now on Elixir with ExUnit I am looking for a way to do exactly that. Is there a way to do this?

Fontanez

It looks like there is something in the making: github.com/elixir-lang/elixir/pull/7480 – Hower

Unfortunately the attempt mentioned by @Hower has been aborted and the feature is no longer in the works. PRs welcome! – Wingback

Answer 1

There is such an option since Elixir 1.8.

Use the --max-failures switch to limit the number of tests evaluated with failure. To halt the test suite after the first failure, run this:

mix test --max-failures 1

Merridie

Answer 2

Unfortunately there is (to my knowledge) no such flag implemented.

However, you can run a single test by

mix test path/to/testfile.exs:12

where 12 is the line number of the test.

Hope that helps!

Rhamnaceous

Answer 3

That makes not much sense since tests in Elixir are a) to be run blazingly fast and b) in most cases are to be run asynchronously. Immediate termination of the test suite on the failed test is an anti-pattern and that’s why it’s not allowed by ExUnit authors.

One still has an option to shoot their own leg: just implement a custom handler for the EventManager and kill the whole application on “test failed” event.

---

For BDD, one preferably uses tags, running the test suite with only this feature included. That way you’ll get an ability to run tests per feature at any time in the future.

Also, as a last resort one might run a specific case only by passing the file name to mix test and/or a specific test only by passing the file name followed by a colon and a line number.

Cytochemistry

In my case, the tests are not blazingly fast and they test a stateful system's HTTP interface, therefore they aren't running asynchronous. To be specific, I practice BDD with Gherkin scenarios through Elixir's Cabbage and Wallaby. – Fontanez

Then you probably need to reorganize your tests amongst cases, since tests within one case are always run synchronously. – Cytochemistry