Client - server integration testing: mock or not?

Question

I'm working on project with two applications: android app (client) and rest service (server). My android app consumes my rest service.

Both applications are tested separately to ensure they're doing their business as expected. During server tests I prepare requests and check server responses. During client tests I set up a simple http mock server and test client's requests against different mocked responses.

Now, this technique works pretty well. It gives me a flexibility I like. I can use different test frameworks and continuous integration environments. But there is one weak point. In both (client and server) test cases I specify the same api. I assume that e.g.

GET /foo-list.json

will return HTTP 200 with json

[{
    id: 1,
    name: foo1,
}, {
    id: 2,
    name: foo2
}]

So I repeat myself. If I change a response format my client tests won't fail.

My question is about good practices in testing this kind of scenario. How to make true integration tests without sacrificing flexibility of independent tests. Should I test client with mocked server or with a real instance of my rest service?

Please share your professional experience.

Wilkinson

Answer 1

In your scenario you should continue to write unit tests to test individual classes, and integration tests to test the inter-operation between multiple application layers (e.g. business and database layers).

You ask:

"How to make true integration tests without sacrificing flexibility of independent tests"

All of your code should should use abstractions, so that you can use dependency injection to unit test classes in complete isolation using mock dependencies. The use of mocks will ensure that these tests will remain independent i.e. not coupled to any other classes. Hence taking this approach, the integration tests, which would use your final concrete classes, would not affect the unit tests which use the mocked classes.

Also:

"Should I test client with mocked server or with a real instance of my rest service?"

In addition to unit and integration tests you should also perform client-server integration testing; I use automated acceptance testing for doing this. Using a test framework such as Cucumber (also check out calabash-android, which is written specifically to test mobile applications) you can write tests which would test specific features and scenarios which would interact with both the client (your Android application) and server (your RESTful service). These client-server integration tests would start-up and stop concrete instances of the client and server.

Bracing

Answer 2

Mocks are for unit testing. Your description of the tests with the mocks describes exactly that. You test the client and server as separate units.

Integration testing tests if the units work well together. Since the interface is a REST interface, mocking makes no sense then, you have to test the real thing over HTTP.

See also What is the difference between integration and unit tests?

Persinger

So you recommend to test a client with a real instance of rest service. Correct? – Wilkinson

Correct! For integration tests, hook the units up as they will be in production. – Persinger

What if my real instance is down and I depend on this to have a green build on my CI pipeline? What are the possible workarounds for this downside? – Psychoanalysis

@Psychoanalysis you can spin up a test instance for the duration of the test – Persinger

Answer 3

If your service is based in Java, I'd strongly recommend looking into the Spock framework, for mocking any sort of calls that might be coming from the client. Since Spock is just an extension of jUnit, you might also be able to use it for Android (though, to be fair I've never done Android development)

I'd say you want to do two things. Integration testing and Unit testing. Integration testing would attempt to bring up the android application and cause it to make service calls, ensuring the contexts interact with each other kindly.

However, in your regular commits, I'd suggest unit testing that mocks away everything but the class under test. Spock makes this pretty easy to do, and since it's built on top of jUnit all it takes a jar.

Incept

Let me know if you want me to elaborate - I realize this might be a little bit too generic of an answer for you. – Incept

You're right about necessity of both integration and unit testing. My question is about integration testing. When I test the integration of client with service, should I mock the service or should I run a real instance of the service? – Wilkinson

Well, the purpose of integration testing would be to test across boundaries, in some cases. In our environment, I'd be likely to do something like using the test version of the service for integration testing to make this work. You might also be able to get something like a standalone test server running as part of your integration testing context. Short answer - yes, to do integration testing I would recommend finding a way to get the app to make service calls, instead of mocking them away. – Incept

Answer 4

There is no reason you can't run automated end to end tests with a real service instance. You can run a real service instance on the same test machine you are using to run the unit tests, perhaps in the same container. You can set up configuration to use a different URL for the server instance for running automated end to end testing.

Why would you want to do the extra work of creating the mock service if you can run them against the real service?

I would only create a mock service, if the service was an external service over which I had no control!

Freelance