Is edit-triggered testing the next step?

Question

I have just read Michael C. Feathers great book Working Effectively with Legacy Code, the bible of introducing tests to legacy code. In this book he describes something called Edit-triggered testing:

If it isn't out by the time this book is released, I suspect that someone will soon develop an IDE that allows you to specify a set of tests that will run at every keystroke. It would be an incredible way of closing the feedback loop.

It has to happen. It just seems inevitable. There are already IDEs that check syntax on each keystroke and change the color of code when there are erros. Edit-triggered testing is the next step.

When I read this I hadn't heard about any IDEs or tools that support this. However, I just found a project called Infinitest that supports this for Java.

My questions are:

1. Are there any other tools/framework that support this (hopefully also for Visual Studio)?
2. What are your experiences with this kind of testing (efficient, slows down the IDE, etc)?
3. Is this the next step of TDD?

Updates:

• Take a look a look at this screencast to see Infinitest in action for Eclipse.
• Just found a blog where Brett Schuchert shares some experiences with Infinitest.

Behling

Interesting concept! I think it will be a while before this is reliable and performant though. – Angleaangler

Answer 1

There's Autotest for ruby. It continuously runs unit tests as you edit files, using some simple conventions to map source files to test files. There's also a plugin for it I've seen that colors the test code red or green in emacs interactively. I think this idea of immediate visual feedback of passing/failing in the code editor itself is great.

Cantu

Answer 2

It's been a few years since the original question, and there are a few tools available for .NET (I assume that this space has grown for other languages as well).

There is a .NET version of Autotest (mentioned previously in this thread) called AutoTest.NET, which has been forked and updated into a more fully-featured tool called ContinuousTests. I tried it about a year ago, and it looked to have a lot of potential at that point, but unfortunately, it sounds like they've decided to stop development. On the up-side, however, it's now free!

I haven't used NCrunch, but it's a commercial product that appears to be actively developed.

There are script-based tools that might also fit the bill for what you're asking about

See also this question: How do I do continuous testing in .NET?

Chery

Answer 3

Well, it essentially is Test-driven development, with tests being run continuously, as you type? So I wouldn't say "edit-triggered testing" is a new development technique, but merely a feature of your IDE.

Regarding the speed of this feature, I guess it really depends on how complex your project is, what compiler and language you are using, and what changes you are making. If you are modifying some base library which is used throughout the project, your compiler will have to recompile a large part of your program to be able to run tests in other libraries using your code.

Maldon

Agree that it is not a new development technique, it is more about shortening the feedback from the tests. – Behling