Google Mock: why is a partial ordering of expectations harder to satisfy than a total ordering?

Question

I'm mostly using ordered expectations with GoogleMock, so all EXPECT_CALLs were written inside the scope of a testing::InSequence object.

Now I want to relax the ordering so I split the expectations in 2 sequences. You would say the test should pass, but no - it fails, complaining about unmet preconditions. How should I reason about this?

Edit: Reduced version of my code:

//InSequence s;                                     // uncomment this and it works
for (int i = 1; i <= 2; ++i)
{
    {
        //InSequence s;                             // uncomment this and it doesn't work

        EXPECT_CALL(mock1, produceMessage(_))
            .WillOnce(DoAll(SetArgReferee<0>(val1), Return(false)))
            .WillOnce(DoAll(SetArgReferee<0>(val2), Return(false)))
            .WillOnce(DoAll(SetArgReferee<0>(val2), Return(false)));

        EXPECT_CALL(mock2, handleEvent(A<MyType>()));
        EXPECT_CALL(mock2, handleMessage(NotNull()));
    }
}

So if the InSequence is nested inside the for loop, I should have a partial order, which is a relaxed requirement, compared to the case when the InSequence is on the outside.

Error I'm getting:

Mock function called more times than expected - returning default value.
    Function call: handleMessage(0xd7e708)
          Returns: false
         Expected: to be called once
           Actual: called twice - over-saturated and active

And then, at the end of the test:

Actual function call count doesn't match EXPECT_CALL(mock2, handleMessage(NotNull()))...
         Expected: to be called once
           Actual: never called - unsatisfied and active

Studner

can you write some example code? – Paraformaldehyde

just added the code. – Studner

Additional remark: Appending .RetiresOnSaturation() on the 3rd expectation fixes the partial order case. But not my real-life case, which is more complex. – Studner

how produceMessage looks like? – Paraformaldehyde

Answer 1

After some more progress on the GoogleMock learning curve, I'll try to answer my own question in a way that is general enough to be helpful.

Let's consider the following example of totally ordered expectations:

{
    InSequence s;

    EXPECT_CALL(mock1, methodA(_));     // expectation #1
    EXPECT_CALL(mock2, methodX(_));     // expectation #2

    EXPECT_CALL(mock1, methodA(_));     // expectation #3
    EXPECT_CALL(mock2, methodY(_));     // expectation #4
}

Now, let's slice the ordering in two.

{
    InSequence s;

    EXPECT_CALL(mock1, methodA(_));     // expectation #1
    EXPECT_CALL(mock2, methodX(_));     // expectation #2
}

{
    InSequence s;

    EXPECT_CALL(mock1, methodA(_));     // expectation #3
    EXPECT_CALL(mock2, methodY(_));     // expectation #4
}

The intention is to allow expectations from the two sequences to "merge", i.e. to have expectation #1 as precondition for #2 and #3 for #4 but not more than that.

However, the following sequence of calls will satisfy the totally ordered expectations but not the "partially ordered" ones:

mock1.methodA();   // call #1
mock2.methodX();   // call #2
mock1.methodA();   // call #3
mock2.methodY();   // call #4

Reason: it is obvious why the totally ordered expectations are satisfied: the example just satisfies them in the order they are written. Being InSequence, they retire as soon as they are satisfied.

However, the "partially ordered" scenario doesn't work because call #1 will satisfy expectation #3, then call #2 will match against expectation #2, which can't be met because it has expectation #1 as precondition. Even though technically, expectations #1 and #3 are identical, they are satisfied in reverse order of writing, since they don't belong to the same ordering, hence the failure.

I have the feeling that this phenomenon is not well enough documented by Google Mock. I'm still looking for a better formalization. I suspect that there's something wrong with the "partial order" concept as used here.

Studner

did you find any solution to this problem? – Fidelfidela

no. Not a generic one. I believe a general solution would require some form of backtracking, but I haven't put more thought into it. – Studner

thanks. I was able to workaround my variant of this problem because I could differentiate the methodA in either block by the value of one of the parameters. Then have the methodA in the first block match just that, and in the second match everything but that value. Obviously that's not going to be applicable everywhere, it just happened to fit my problem. – Fidelfidela

Answer 2

Looking at your question and your answer, I think your case is a typical example of DAG (directed acyclic graph) which can be solved by InSequence clause to EXPECT_CALL (not the InSeqeunce class from ::testing:: namespace). See gmock Cookbook explanation here. Example test provided in your answer can be solved this way:

Sequence s1, s2;
EXPECT_CALL(mock1, methodA(_)).InSequence(s1, s2);
EXPECT_CALL(mock2, methodX(_)).InSequence(s1);
EXPECT_CALL(mock2, methodY(_)).InSequence(s2);

The above test code will make sure methodA is called before methodX and methodY. Also, it doesn't care in which order are methodX and methodY called.

Quadrisect