100% unit test coverage in Java: The smells of perfection

Unit testing is defined as testing one class in complete isolation. One approach to isolate your test is to use mocks and stubs, where those mocks and stubs act in a predictable way. Over time, we have gone from rolling our own to using cool libraries like EasyMock and Mockito. With these tools, and applying the now-ubiquitous IoC pattern, it is not unreasonable to expect test coverage of 100%.

Despite these tools being invaluable to unit testing, they have limitations that can lead to smells. Three of the biggest limitations have led to some smells that have persisted over several years. Here are my three biggest frustrations caused by today's mocking frameworks:

Cannot mock static methods
Proxy-based mocking tools generally do not support the mocking out of static methods. These tools mock classes by extending them with dynamic proxies, and since static methods cannot be extended, they can't be mocked either.

Solution: To test statics, I've used the Boundary pattern (sometimes called Repository) to defeat static cling. How else are you going to mock out a call to Calendar.getInstance()?

Every class that has a static method we want to mock out is wrapped in a boundary. So for our Calendar class, we will write a CalendarBoundary that is injected in to the client The CalendarBoundary contains non-static methods that delegate to the static equivalents. So, now anyone wanting to create a Calendar can do so by using the CalendarBoundary, and that can easily be mocked.

Smell: These boundary classes are somewhat contrived and unnatural. I have to declare them as explicit dependencies to my class - thats not always a bad thing but it can lead to strange-looking constructors that take a lot of boundaries. It's also more code to maintain and test. I could generate them, but it's easier to just call Calendar.getInstance() where I need it!

Cannot mock final classes or final methods
In a similar vein to the above, proxy-based mocking tools cannot extend final classes (like java.lang.Class) or final methods (like Calendar.getTime()).

Solution: The boundary pattern, or a class wrapper, can be used in these cases. For Calendar, I can write a class called CalendarWrapper that decorates a Calendar instance, provides the same method signatures, but the methods are not final. I need to provide delegation methods for every method I plan to use.

Smell: First of all, writing such boundaries manually is arduous. Calendar is a big API, and I really need to implement all of the methods and delegate to the real Calendar underneath. Sure, I could be smart and generate this code, but my API is still strange to look at and use. Some would even call it broken.

If my API uses CalendarWrapper instead of Calendar, how will users of that API react?

Dev A: "Why does this Widget class return a CalendarBoundary and not a Calendar?"
Dev B: "Oh, it's because we needed to test some final method on Calendar."
Dev C: "Right.... but I need a real Calendar so my JSP tag can render it."
Dev B: "Umm...."

You could write methods to get the real object and set the real object, but that's even more code. And what value is all this code really adding here? Isn't one aspect of well tested code a well designed API? I'd suggest this is not what they had in mind.

Cannot mock new instance creation
When we need to create a new instance of an object, usually it is so we can interact with it in some way. Imagine I am creating a new Widget, and I want to assert that this widget gets passed to a collaborator after being initialised in some way.

Solution: One approach is a mix of stateful testing and interaction testing. If it's a simple case, I can use a stateful test to see if the Widget that gets returned is in some state that I expect it to be in. Maybe the method I'm testing creates a new Widget and passes it to another collaborator, in which case I can use something like Mockito's ArgumentCaptor to test the state of the object in-flight.

Another solution is to avoid the "new". We could have some sort of generic Object Factory that creates new instances, so

Widget widget = new Widget();

becomes

Widget widget = objectFactory.newInstance(Widget.class);

I can then get a mock objectFactory to return a mock Widget instance.

Smell: The first approach can lead to very lengthy and awkward test cases. I've rarely seen a mxture of interaction (mock/verify) testing work well with stateful (assertEquals) testing without being horribly confusing. And a confusing test is not helping anyone understand what's going on!

The second approach looks contrived and confuses developers not used to seeing it, or understanding it must be done in the first place. Secondly, the ObjectFactory gets uglier when the constructor takes parameters. And last but not least, the code becomes fragile to refactoring - how would your IDE add/remove/change a constructor param if I am using reflection to create the new instance?

Summary
Until recently I thought these limitations were unavoidable due to the nature of the Java language. Happily, I might be wrong. Powermock is an addon to EasyMock and Mockito. It promises to fill the gaps - mocking out static methods, final methods, new instance, as well as a raft of other things previously not possible using EasyMock or Mockito.

I plan to have a look at PowerMock soon and write soon about my experiences, and hopefully eliminating these very annoying smells.

If you've used PowerMock I'd like to hear about it! If not, come back soon and I'll hopefully have something up about it.