Getting QUnit to run tests in order

Question

I've used qunit to write a series of tests for javascript code I have. Right now for some reason, the first test in my list will run, and then the LAST test in the list runs, followed by the 2nd to last, 3rd to last, 4th to last, etc... It's crucial for my tests that things run in the order that I have them in. I tried turning off that option where qunit runs tests that failed last time first, but it's still doing this. Is there any way to fix this?

Queen

Answer 1

First, figure out why your tests MUST run in a specific order. The whole point of unit testing is that the tests are atomic and it should be possible to run them in any order - if your test suite isn't capable of this, you need to figure out why as it may represent a larger problem.

If you can't figure it out, then you may need to break your test suite up into smaller groups of tests until you find the one(s) causing the issue.

edit: Found this reference at http://www.educatedguesswork.org/2011/06/curse_you_qunit_1.html. Apparently, adding this to your test suite will help QUnit.config.reorder = false;

Lotetgaronne

No, I know why. To increase efficiency. (I don't want to get into details but as of right now each test takes a very long time to run and I need to make things as streamlined as possible). Some tests rely on the results of a prior test. I can make everything atomic but then everything takes even longer than it already does. Once I'm put in a position where time does not matter I may revert to an atomic model, but for now I want my tests to run in the order I wrote them in for efficiency. – Queen

@Queen "Some tests rely on the results of a prior test" then you don't have atomicity - you should prepare what is needed for the test in the test itself. – Qualification

I was doing that yes. But at this time I need efficiency, and this was the best way to do things. I don't want to explain everything, but suffice to say that at this time I need tests to run in the order I have them in. – Queen

As for the QUnit.config.reorder = false line, I already set reorder to false in my qunit file and this is still happening. – Queen

I just redownloaded the qunit file (there were a couple of updates) and swapped the one I had with the new one, and it still does this. I don't get it, I'm assuming it's doing it because of some kind of memory thing, but when I do things like that shouldn't it forget? – Queen

I'm not actually sure what's going on here. I've just checked the latest code, from 1.5.0pre - the reorder option is definitely present in the code, and should be doing what you expect. – Lotetgaronne

It's really weird. One thing I've noticed is that I have reorder active, it runs the first test first, then runs the failed tests in backwards order, then goes backwards from the last test that failed last time. Really confusing! – Queen

If you can't have atomic tests you could consider writing just one test, which calls your own function. That function can do everything in the order you want, sharing state, and return true or false if everything passes or not. Output diagnoses to the console. Only group those things together that must share state. It's clumsier but more honest - if you are relying on test order your code shouldn't be in individual test blocks as they really should be atomic. – Grogram

The only case I know of where you want an ordered test would be in a koans project. The shuso would be overwhelmed with all the tests failing and should only see the first one and they are resolved in order. Each red,green,refactor step leading to enlightenment. – Biconvex

Answer 2

Maybe you could consider placing the code that does each major computation in a function that has a test at the beginning that checks if the computation was already done. If the computation is not done, do the computation and save the result somewhere. If the computation has already been done then just return the results. In this way you can have a single computation for all the tests but still be autonomous to the order the tests are done.

I can relate to the problems of time consuming computations in unit testing, but it is imperative to the test group to be able to take any unit test and execute as an independent autonomous test. This is especially true when a critical problem comes up and has to be addressed specifically.

Dialytic