Are there any good mutation testing tools for ruby 1.9 and RSpec2?

Question

I used to use Heckle, but it is incompatible with ruby 1.9 because of issues with ParseTree. I've looked for alternatives, but the only thing that looked promising was Chaser, and that did not have any clear documentation that I could use to see if I could make it work with RSpec. It seems to have Test::Unit dependencies.

So - is anyone out there using any cool tools to really check the quality of your tests?

Alternatively - are there any coverage tools that provide better than c0 coverage? This would kind of help solve the same problem. I'm using cover_me at the moment, but it is c0, like rcov.

Decant

Answer 1

Have you looked into the Mutant gem?

It works with Rspec.

There is a nice tutorial here

$ gem install mutant-rspec

$ mutant -I lib -r your_library --use rspec "YourClass#method_to_mess_with"

Looming

The current release of mutant does not support that syntax anymore. See solnic.eu/2013/01/23/mutation-testing-with-mutant.html for current interface. I'll readd a expicit spec file selection option soon. -- Disclaimer, I'm the author of mutant. – Attar

Just popping back here for an update - seems like both mutant & heckle have problems with Rails, or at least the stack I am using. They've come a bit farther, it seems - ParseTree is no longer the issue. I wish I knew enough to help out. :( – Decant

IMHO it is the other way round. Rails and its strange autoload / environment setup is causing problems with mutant ;) Here is the rails support ticket: github.com/mbj/mutant/issues/21 Hope to have more time for opensource in the future. – Attar

Answer 2

Chaser also works with a mutation testing framework exemplor: exemplor-chaser.

Unfortunately, the creator of chaser has the attention span of a goldfish and isn't maintaining it any more. He's kind of interested in mutant, and also in seeing if heckle can be modified to use ruby_parser rather than ParseTree.

Dowie