How to unit-test an internet protocol implementation?

Question

I decided to add unit tests to my project and continue development in a test-driven kind of way. I’m currently working on implementing unit tests for my ManageSieve client object and I’m not sure what’s the best way to test that beast.

My SieveClient object relies on two other objects for the network communication: CocoaAsyncSocket and my own SaslConn object, which is my wrapper around the Cyrus SASL library to handle the authentication methods. For testing I need to replace those with mock objects. I’m going to use the OCMock framework for this. I’m not quite sure how to do this, since the SieveClient object needs to create those objects itself. Right now I overwrite the (private) setters for that object to always install my mock objects using OCMocks partialMockForObject: method. But this feels not right to me. Any ideas how this could be solved better?

The other part I have trouble with is the socket itself. To be able to test the protocol details I’d need a way to return predefined test data from the socket. I suppose I could just use OCMock mechanisms to fake the return values from the socket. But since CocoaAsyncSocket provides many different methods to read data from the socket I have to know exactly which are being used by the protocol object in which order. I don’t want my unit test to be that dependent on implementation details of my protocol object. So what should I do here? Implement a mock object for the socket class by hand? This seems non-trivial, so I’d probably need unit tests for that too. Is that a good idea?

I’ve read that if something is hard to test it’s probably not very well designed either. But I don’t see how I could do better, since the hard part lies in interacting with the socket which I have to do.

If you’d like to see code you can find it at Bitbucket: SieveClient.m and SieveClient.h

Edit: Dependency Injection

So I read about Dependency Injection, and I think I’m going to use this to get the AsyncSocket and SaslConn objects into my SieveClient object. I’ll change my constructor to accept those objects and use them. Since the user of this class usually doesn’t care about the socket and the SASL object I’ll add a factory method (in the form of a convenience constructor) that just creates those objects and passes them to the constructor.

But this solves only the first (and easier) part of my testing problem.

Singularity

Answer 1

Can you split what you're doing into two parts, one of which is the abstract protocol and the other of which is the binding to sockets? Then you can test the abstract protocol more easily, and focus the testing of the binding on whether it invokes methods/operations of any connected abstract protocol correctly.

Abstractly, you'd be decreasing the coupling between the parts of your code. That increases testability, at a cost of some increase in overall complexity (though not too bad because you're getting better tools to manage it through Separation of Concerns) and some potential decrease in performance (not much of an issue with most systems though; your computer is far faster than its I/O subsystems).

Darb

I was thinking about that too. But then I rejected the idea, because it wouldn’t help very much. I could test the SieveClient object more easily, true. But then I’d have the same problems with testing the new object. Seems to me that this is just putting up the trouble for later. Especially since I have nothing I could re-use the new class for. – Singularity

@Sven: I know, but it's damn hard to test the layer of code that actually couples to the network precisely because it is so coupled to the network. The only way I know of making things better is to get some decoupling in there. (For the record, this is why I try to avoid writing low-level networking code, and instead use someone else's library.) – Darb

Answer 2

But then I rejected the idea, because it wouldn’t help very much. I could test the SieveClient object more easily, true. But then I’d have the same problems with testing the new object. Seems to me that this is just putting up the trouble for later. Especially since I have nothing I could re-use the new class for.

It wouldn't be the same problem.

I'll assume you need SieveClient to control the instantiation of the other objects internally because its part of an API you don't want to expose. If that's the reason, by separating them, you no longer have the same need, since you can have the SieveClient control the bindings and the other part that does the protocol receives the instances it'll work with.

By doing the above, you can hand over the mocked objects to your protocol implementation. Those mocks would then have any expectations you may need. If you find it you end up with it being too involved, then you probably need to refocus responsibilities, which usually results in a cleaner/simpler protocol implementation anyway (if you find that you need to from doing those unit tests).

Above said, you also need to consider if the code you are trying to test is as focused as possible on the protocol and doesn't have any extra elements. If that's the case, it wouldn't be a good candidate to unit test it, as its sole responsibility is the interaction with an external system. I'd decide how important is the protocol spec in this system, and if its all about integration with an external system I'd treat it like a focused integration test instead that hits the real external system and is kept separated from the unit tests (so it doesn't affects the speed needed to run unit tests of the rest of the system).

---

After re-reading the question because of the edit, I have to stress out what I said about focused integration tests above. You ask:

But since CocoaAsyncSocket provides many different methods to read data from the socket I have to know exactly which are being used by the protocol object in which order. I don’t want my unit test to be that dependent on implementation details of my protocol object. So what should I do here? Implement a mock object for the socket class by hand? This seems non-trivial, so I’d probably need unit tests for that too. Is that a good idea?

If you are dealing with a very complex object and that object is all about integration beyond a boundary, you usually are best avoiding it as part of unit tests. In that scenario you want a focused integration test / to hit the real external system. This doesn't mean all the unit tests of the rest of your code, hit the external system, just the very simple unit of code/class that uses that object.

It may very well be the case that such object is SieveClient in your scenario, in that case forget about unit tests of that piece of the code. What you want to do instead is mock the SieveClient when testing code that uses it. On the other hand, if you find that SieveClient is much more than that, you want to add a class that simplifies these communication aspects, and that'd be what you mock when testing SieveClient and also what you do a focused integration test against.

This type of tests are a very effective way to make sure that the code that interacts with the external is working as expected, since that's the focus of both the class and the tests involved. In case something on the external system starts working differently, you notice it clearly - as opposed to having it mixed with your application logic or worst not tested at all.

Judoka

Answer 3

Don't listen to dogmas too much. Go for the simplest thing that could possibly work, also for tests. (Disclaimer: I do know TDD, but I don't know Objective C).

To let SieveClient create its SaslConn's in production code, but use mock ones in tests, you can use dependency injection. Add a setter method to SieveClient to pass in a factory (as an object or a function, depending on what Objective C permits), which SieveClient will use to make its SaslConn's, instead of making them by itself. The test code provides a test factory that dishes out mocks. The production-case code for making SaslConn's either moves to another factory to be unit-tested independently, or if it's too simple to break, remains as the default behavior inside SieveClient when the factory setter is not called.

The simplest way to test network client code is by far to implement or re-use a mock server. Don't mock out the gory socket details in SaslConn; instead, write an SASL server in your tests. The fact that your SaslConn can talk to it goes a long way towards providing testing for that mock server; in other words, SaslConn and the mock server are each other's unit tests. (Yeah, not "unit" in the purist sense, but nobody cares.)

Finally, I have mixed feelings about the precept that hard to test code is badly designed. It depends. You should design your code so that it's easy to use (in caller code) and easy to modify. Unit tests are but a means to these ends: they are the first caller code that you will write, and they give you confidence that you don't screw up when making changes. Don't let a particular framework or methodology twist and maim your design to the point of outweighing the benefits of TDD. In particular, expectation-based mocking frameworks such as OCMock make it way too easy to write brittle tests that go like "I expect method foo to be called 3 times, and only then method bar to be called with exactly such and such arguments". Rather than using the wrong tools for the job, write your own!

Exedra