Continuous testing with IntelliJ Idea

Question

I'm looking for an IntelliJ IDEA plugin that would run my tests each time I change my code. I looked for such a solution and I found:

• Infinitest, which works, but is inconvenient because I need to add the facet to each module, and it opens a new tool window for each module (which means 15 tool windows for me).
• Fireworks - didn't work for me, maybe it just doesn't work with IDEA 14 (in its repo I can see that last changes were made in 2009). IntelliJ also reports that it throws exceptions.

There are lots of ways I could run all my tests (including writing a simple script for this), but I'm looking for a tool that would be smart enough to rerun failed tests first, and that would understand module dependencies (so after a change in some module it would run only tests of dependent modules).

I prefer free options, but if there's something paid for a reasonable price, I would accept it as well.

Ingot

This is kinda frustrating, because I know for a fact that pycharm has this enabled by default, but IDEA doesn't :( – Lishalishe

The latest version of Infinitest no longer opens windows and it is no longer a facet. Just install the plugin, build the project and it should run your tests! – Myungmyxedema

Answer 1

IntelliJ now actually has a Toggle auto-testin the run dialog. Just run a run-configuration and then select Toggle auto-testin the run dialog.

It's not as intelligent as you would have hoped. It just reruns when it detects changes.

Treed

Heads up you have to keep the test window tap open or the test will not rerun – Vesta

Can you configure it to auto-test all tests in your project, or will it only auto-test the test in the current run-configuration? – Lactoscope

Unfortunately the auto-test feature doesn't work if compilation is delegated to gradle (the default nowadays) – Zinc

To add to other's experiences, mine was that tests are run ONLY after you build the project, so I had to also enable autobuild ("Build project automatically"), which in it's turn is run with some delay. So, to say at least, this feature is not instant auto-test :/ – Intwine

Answer 2

I found out a way to enable SBT style auto test execution in IntelliJ studio.

We need to do two things to enable auto test execution.

1. Enabling auto project build - This can be done in settings by going into File -> Settings -> Build, Execution... -> Compiler and selecting "Build Project automatically".

2. Enabling "Toggle auto-test" in run dialog box

This will start auto testing. Although this works fine, it takes time to build the project even when my project is tiny so for larger projects it will certainly take very long time to complete the build and execute tests.

Reference: Original article which explains these steps

Kerr

This works for me! It's pretty slow to re-execute the tests but I don't have to click or use any shortcuts anyway. Guess that's a win :) – Inn

Unfortunately the auto-test feature doesn't work if compilation is delegated to gradle (the default nowadays) – Zinc

Answer 3

If you are OK running tests which cover a single method chosen by you, you can try this plugin (it is continuos in the sense that you make changes to a method, then click on the method and the plugin will run all the unit tests automatically which cover that method): https://plugins.jetbrains.com/plugin/15063-unit-test-coverage-history-runner

Byers

Answer 4

You can use the Intellij Teamcity plugin. Teamcity is a paid product but there is a free version which gives you 20 projects and 3 agents for free .

1. It has a remote run feature using which you can send in unchecked code to run tests before committing.
2. It also has options to run failed tests first

Usage instructions for Remote Run

Mansard

this is not an answer to his question - he is looking for a continuous testing mode, simliar to what Eclipse has. – Affairs

Eclipse does not have a continuous testing mode. You're most likely using some third-party plugin – Garofalo