QUnit fails tests inconsistently/alternately

Question

I have a simplified QUnit test which consists of 2 simple tests that fails randomly/alternately for no good reason (They are both atomic, meaning that one test doesn't change anything of the other element)

Please see this jsFiddle try to run multiple times

module("Basic actionBind");
//two simple tests
test("action1", function() {
    ok(ele2.trigger("click").hasClass("clicked"), "basic click action");
});

test("action2", function() {
    ok(ele1.click().hasClass("clicked"), "basic click action");
});

Hydrophyte

What does the second test do? I'm not sure what the click function does without any arguments. – Suzette

the click function itself without arguments trigger's a click – Hydrophyte

Very strange. I've updated to include the two tests twice, and made them async with random pauses, but only one test runs. It's like click/trigger gives up after the first. jsfiddle.net/bAbNd/1. – Suzette

Same deal with dispatchEvent? jsfiddle.net/bAbNd/2 – Suzette

That happens because the first failed test will run first the next time. – Yevette

Thaks! what should that mater? Besides that, I also tried QUnit.config.reorder = false; and sessionStorage.clear() which opts-outs of this failing test running first feature (feature ?) – Hydrophyte

Answer 1

The problem is in the caching of the jQuery objects in the global context at run-time. Here's two snippets from your code _{(from the link in your question: http://jsfiddle.net/adardesign/aZRK7/12/):}

HTML:

<div id="qunit-fixture">
    <span id="ele1" class="action" data-action="action1"></span>
    <span id="ele2" class="action" data-action="action2" ></span>
</div>

JS:

// caching elements
var $ele1 = $('#ele1'),
    $ele2 = $('#ele2');

test('action1', function() {
    ok($ele2.trigger('click') ....


test('action2', function () {
    ok($ele1.trigger('click') ....

The actual elements are inside the #qunit-fixture which is reset and re-parsed for each test, thus the click events you fired from inside the tests were triggered on the elements that are now no longer in the document. Instead those elements have been detached and #qunit-fixture has been re-parsed from the original html text, thus creating new DOM elements.

I've forked your jsFiddle and revised it accordingly: http://jsfiddle.net/aZRK7/15/

Though I cleaned up various parts before I noticed it, the essential change is moving the query for the element to inside the test:

test('foo', function() {
    var $foo = $('#ele1');
    ok($foo.trigger('click') ....


test('bar', function () {
    var $bar = $('#bar');
    ok($bar.trigger('click') ....

Pegeen

+1 thanks much! I wish this should have been in the documentation before spending hours... thanks again. – Hydrophyte

@adardesign: "QUnit will reset the elements inside the #qunit-fixture element after each test, removing any events that may have existed. As long as you use elements only within this fixture, you don't have to manually clean up after your tests to keep them atomic." qunitjs.com/cookbook – Vladimir

The problem can not only be caching of elements in the global scope, but also wiring an event via script elsewhere. I've got a button, to which I attached a click handler in a script ($("#equalsButton").click(...);). Due to the cleanup of the fixture, however without re-running my event-wiring in default.js, the handler alternatingly was or wasn't attached to the button, and therefore causing the test to fail or succeed accordingly. What fixed the problem was wiring the click event before every test run (or in an test setup routine). – Vladamar

Answer 2

EDIT: Looks like OP took out the async part of the code, leaving the answer in case any stumbles onto this, but it does not answer the question now.

Well, I'm not sure what issues you might be having the the test itself, but any AsyncTest must have a start(); call somewhere. So from your fiddle:

asyncTest("action1", delay.bind(null, function() {
  console.log("test1");
  ok(ele2.trigger("click").hasClass("clicked"), "basic click action");

  start();

}, 2000*Math.random()));

From the QUnit asyncTest documentation, see their examples.

Spirketing

Thanks, This is not a async operation, is it? Besides that I tried your code as well and still fails jsfiddle.net/adardesign/bAbNd/3 – Hydrophyte

Well, in your fiddle you're using asynTest(), and doing so requires that you tell QUnit when the async portion of the test is done and to continue (that the call to start();). As I mentioned, I'm not 100% sure why you're getting failures, was more answering the other part. I'll try to see if I can figure anything else out. – Spirketing