chai-as-promised erroneously passes tests

Question

I'm trying to write tests for a method that returns an angular promise ($q library).

I'm at a loss. I'm running tests using Karma, and I need to figure out how to confirm that the AccountSearchResult.validate() function returns a promise, confirm whether the promise was rejected or not, and inspect the object that is returned with the promise.

For example, the method being tested has the following (simplified):

.factory('AccountSearchResult', ['$q',
  function($q) {

  return {
    validate: function(result) {
      if (!result.accountFound) {
        return $q.reject({
          message: "That account or userID was not found"
        });
      }
      else {
        return $q.when(result);
      }
    }
  };
}]);

I thought I could write a test like this:

    it("it should return an object with a message property", function () {
      promise = AccountSearchResult.validate({accountFound:false});
      expect(promise).to.eventually.have.property("message"); // PASSES
    });

That passes, but so does this (erroneously):

    it("it should return an object with a message property", function () {
      promise = AccountSearchResult.validate({accountFound:false});
      expect(promise).to.eventually.have.property("I_DONT_EXIST"); // PASSES, should fail
    });

I am trying to use the chai-as-promised 'eventually', but all my tests pass with false positives:

    it("it should return an object", function () {
      promise = AccountSearchResult.validate();
      expect(promise).to.eventually.be.an('astronaut');
    });

will pass. In looking at docs and SO questions, I have seen examples such as:

expect(promise).to.eventually.to.equal('something');
return promise.should.eventually.equal('something');
expect(promise).to.eventually.to.equal('something', "some message about expectation.");
expect(promise).to.eventually.to.equal('something').notify(done);
return assert.becomes(promise, "something", "message about assertion");
wrapping expectation in runs() block
wrapping expectation in setTimeout()

Using .should gives me Cannot read property 'eventually' of undefined. What am I missing?

Gutow

Not sure if the chai-as-promised pluging support the AngularJS $q or not. – Hoffarth

@Hoffarth I'm assuming it does, since the docs give an example of Q.all([]), which angular's $q is taken from. – Gutow

Just a guess, you might have to call $rootScope.$apply() after the expectation. – Hoffarth

@Hoffarth Because I'm testing the service itself, (not the controller that is using the service), there is no $scope (or $rootScope). – Gutow

The $rootScope is always available, you could inject it anywhere in your test. – Hoffarth

@Hoffarth I figured it out. I needed to run a $digest with run blocks. If you want to post your comment as a question, I'll accept it and edit it with syntax, since your answer pushed me in the right direction. – Gutow

I don't mind, lets post your findings as an answer right away. I'm sure I will learn something from you. – Hoffarth

Answer 1

@runTarm 's suggestions were both spot on, as it turns out. I believe that the root of the issue is that angular's $q library is tied up with angular's $digest cycle. So while calling $apply works, I believe that the reason it works is because $apply ends up calling $digest anyway. Typically I've thought of $apply() as a way to let angular know about something happening outside its world, and it didn't occur to me that in the context of testing, resolving a $q promise's .then()/.catch() might need to be pushed along before running the expectation, since $q is baked into angular directly. Alas.

I was able to get it working in 3 different ways, one with runs() blocks (and $digest/$apply), and 2 without runs() blocks (and $digest/$apply).

Providing an entire test is probably overkill, but in looking for the answer to this I found myself wishing people had posted how they injected / stubbed / setup services, and different expect syntaxes, so I'll post my entire test.

describe("AppAccountSearchService", function () {
  var expect = chai.expect;

  var $q,
    authorization,
    AccountSearchResult,
    result,
    promise,
    authObj,
    reasonObj,
    $rootScope,
    message;

  beforeEach(module(
    'authorization.services',     // a dependency service I need to stub out
    'app.account.search.services' // the service module I'm testing
  ));

  beforeEach(inject(function (_$q_, _$rootScope_) {
    $q = _$q_;                  // native angular service
    $rootScope = _$rootScope_;  // native angular service
  }));

  beforeEach(inject(function ($injector) {
    // found in authorization.services
    authObj = $injector.get('authObj'); 
    authorization = $injector.get('authorization');

    // found in app.account.search.services
    AccountSearchResult = $injector.get('AccountSearchResult'); 
  }));

  // authObj set up
  beforeEach(inject(function($injector) {
    authObj.empAccess = false; // mocking out a specific value on this object
  }));

  // set up spies/stubs
  beforeEach(function () {
    sinon.stub(authorization, "isEmployeeAccount").returns(true);
  });

  describe("AccountSearchResult", function () {

    describe("validate", function () {

      describe("when the service says the account was not found", function() {

        beforeEach(function () {
          result = {
            accountFound: false,
            accountId: null
          };

          AccountSearchResult.validate(result)
            .then(function() {
              message = "PROMISE RESOLVED"; 
            })
            .catch(function(arg) {
              message = "PROMISE REJECTED";
              reasonObj = arg;
            });
          // USING APPLY... this was the 'magic' I needed
          $rootScope.$apply();
        });

        it("should return an object", function () {
          expect(reasonObj).to.be.an.object;
        });

        it("should have entered the 'catch' function", function () {  
          expect(message).to.equal("PROMISE REJECTED");
        });

        it("should return an object with a message property", function () {
          expect(reasonObj).to.have.property("message");
        });
        // other tests...
      });


      describe("when the account ID was falsey", function() {
        // example of using runs() blocks.
        //Note that the first runs() content could be done in a beforeEach(), like above
        it("should not have entered the 'then' function", function () {  
          // executes everything in this block first.
          // $rootScope.apply() pushes promise resolution to the .then/.catch functions
          runs(function() {
            result = {
              accountFound: true,
              accountId: null
            };

            AccountSearchResult.validate(result)
              .then(function() {
                message = "PROMISE RESOLVED"; 
              })
              .catch(function(arg) {
                reasonObj = arg;
                message = "PROMISE REJECTED"; 
              });

            $rootScope.$apply();
          });

          // now that reasonObj has been populated in prior runs() bock, we can test it in this runs() block.
          runs(function() {
            expect(reasonObj).to.not.equal("PROMISE RESOLVED");
          });

        });
        // more tests.....
      });

      describe("when the account is an employee account", function() {
        describe("and the user does not have EmployeeAccess", function() {

          beforeEach(function () {
            result = {
              accountFound: true,
              accountId: "160515151"
            };

            AccountSearchResult.validate(result)
              .then(function() {
                message = "PROMISE RESOLVED"; 
              })
              .catch(function(arg) {
                message = "PROMISE REJECTED";
                reasonObj = arg;
              });
            // digest also works
            $rootScope.$digest();
          });

          it("should return an object", function () {
            expect(reasonObj).to.be.an.object;
          });

          // more tests ...
        });
      });
    });
  });
});

Now that I know the fix, it is obvious from reading the $q docs under the testing section, where it specifically says to call $rootScope.apply(). Since I was able to get it working with both $apply() and $digest(), I suspect that $digest is really what needs to be called, but in keeping with the docs, $apply() is probably 'best practice'.

Decent breakdown on $apply vs $digest.

Finally, the only mystery remaining to me is why the tests were passing by default. I know I was getting to the expectations (they were being run). So why would expect(promise).to.eventually.be.an('astronaut'); succeed? /shrug

Hope that helps. Thanks for the push in the right direction.

Gutow