Are Unit Tests Over-Rated?

Having worked in software development for more than 12 years now, I can safely say that about 20% of the projects I worked on, had unit tests. And none, had unit tests that were actually effective. Yet, I wouldn’t call even one of those applications as poorly written or of low quality. In fact, most of them were quite good – even in terms of maintainability and code quality.

One reason why I think unit test presence was low, is that all software I worked on were customer facing web applications. The emphasis was more on integration tests. I can’t help but think – is it really bad? I think it’s not. While I do recognise a lot of situations in which unit tests might help save the day, I don’t believe in having mandatory requirements of meeting a certain percentage in code coverage. The project I’m working on currently for a multinational corporation, has a 90% code coverage requirement.

Are your unit tests actually testing your code? Or are they just helping you achieve a code coverage metric?

Reasons against Writing Unit Tests

1. Code is not Reused Anyway

In a web application, except for a very little part, methods are always single use. When the code is divided into methods, its mostly for reducing complexity, making it readable and in general having modularity. Think of your typical method – validateTableCreationPayload(), createUser(User user), deleteBook(int bookId) and so on. These methods are going to be used in only one other place in your application right?

It’s quite easy to ensure changing the method doesn’t break the one, single usage of it. And when it does break, almost always, the unit test is what gets rewritten. That’s the thing with application code – you’re not going to deny your client’s requirement just because your unit test fails right? And there is nothing dependent to be changed – remember there’s only one usage. Considering that none of the functionality is breaking – and your integration tests ensure that, unit tests lose much of their worth anyway. And it becomes a ritual to just change the unit test whenever we change code.

2. Sacrificing Development Speed

Unit tests for application code will take as much time developing the actual code. Often, the unit tests take more time to develop than the actual functionality. An architectural purist might not even consider this as an issue. But a business manager will be – Are you saying it actually only needs half the time to develop!?

And unit tests tend to become very complex very fast. In a regular web application, you end up having to mock several components in almost every unit test. As far as I’ve seen, going beyond a certain coverage percentage, makes unit tests unreadable and off-putting to modify.

3. Misleading Sense of Security

You should always do proper diligence before modifying existing functionality. Do not ever rely on unit tests to catch these. Because unit tests are almost always insufficient. Take this example –

public void createUser(User user) {
    addMetaDataToUser(user);
    // new statement inserted here
    repository.createUser(user);
}

@Test
public void createUserShouldCallRepository() {
    createUser(dummyUser);
    verify(mockRepository).createUser(any());   
}

The above test has 100% coverage for the createUser method. But what happens if we insert another statement in the createUser method? The test still would show 100% coverage. The developer who inserts that statement can get away with not even writing unit tests for their change. Such things can only be caught by careful manual review. And unit tests are often ignored when code reviews are done. Especially in teams where an automated process reports code coverage metrics, and (wrongly) says everything is fine.

Worse, the above unit test is inadequate. It only checks if the repository.createUser() method is called. Not that it’s called with the correct argument! If the new inserted statement modifies the user argument, the unit test will still pass, code coverage will still be 100%. Bad, no?

Reasons for Writing Unit Tests

1. Documents Code

If unit tests are well written, they will act as documentation for how and why code is written in a certain way. It’s not uncommon to see a comment in code that says ‘ugly hack .. do not change this’ or something similar to that. The programmer who wrote it understands that someone in future might modify it and the fallout might not be immediate. But this assumes that nobody would have ‘cleaned up’ that comment. Or a future programmer would simply miss reading it. But if there is a unit test that looks out for this situation, that’s a way better safeguard.

Also, well written unit tests have great names that set expectations about the method being tested – shouldThrowExceptionForBadInput(), shouldReturnAValidUuid() etc. Also, these are arranged in neat groups in your test classes. So if your developers follow discipline you can take a look at the unit tests and learn a lot about the code.

2. Protects Sensitive Methods

General purpose methods, which are used all over the application need to be protected from change. For example, if you have a validation method that checks if a string has a particular pattern (maybe your company has a certain pattern for unique IDs), it is very risky if someone modifies this method. Every method you have used twice or more in your application, must have a unit test. The method serves for two tasks. What if you change it for one task and the method no longer works correctly for the second task? Preventing this, is the core purpose of unit tests. Unit tests serve as a guarantee to what the method will do and will not do.

3. Improves Code Quality

In my opinion, this is the number one reason to enforce unit tests. Especially if there are inexperienced developers in your team. If you write bad code, it becomes more difficult to write unit tests. In my initial days as a developer every time I struggled to write a unit tests, it showed me how I could have written my code to be more loosely-coupled, modular and simple.

Your function might have to be broken into pieces, if you are writing too many unit tests for it. Your function is too tightly coupled, if you are writing too many mocks. Are you finding you need to inject ‘private’ dependencies using Reflection? Are you unable to write a unit test because one of your private fields is initialised indirectly (like Spring’s @Value annotation)? Just about every time you find yourself writing unit tests that are too complicated, you can see opportunities to simplify your original code.

Conclusion

Now, if I propose we write only minimum required unit tests, I would be no different from the people who tell me to write the maximum possible unit tests. But one thing is obvious to me, having a code coverage target is pointless. Only the developer who is writing code can decide which part of it has to be registered in unit tests and which parts don’t quite benefit from unit testing. If you don’t give importance to unit tests in code reviews, don’t even bother requiring them – at least your development will be faster and your code base will be cleaner.

Author: heppydepe

I'm Heppy Depe. (Read "Heppy" like "Peppy" and "Depe" like "DayPay"). Writing under a pseudonym because .. I dunno I just felt like writing under a pseudonym. I'm still trying to find the perfect picture of a fish to set as my profile photo.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s