August 2021 – Page 2

Are Unit Tests Over-Rated?

Having worked in software development for more than 12 years now, I can safely say that about 20% of the projects I worked on, had unit tests. And none, had unit tests that were actually effective. Yet, I wouldn’t call even one of those applications as poorly written or of low quality. In fact, most of them were quite good – even in terms of maintainability and code quality.

One reason why I think unit test presence was low, is that all software I worked on were customer facing web applications. The emphasis was more on integration tests. I can’t help but think – is it really bad? I think it’s not. While I do recognise a lot of situations in which unit tests might help save the day, I don’t believe in having mandatory requirements of meeting a certain percentage in code coverage. The project I’m working on currently for a multinational corporation, has a 90% code coverage requirement.

Are your unit tests actually testing your code? Or are they just helping you achieve a code coverage metric?

Reasons against Writing Unit Tests

1. Code is not Reused Anyway

In a web application, except for a very little part, methods are always single use. When the code is divided into methods, its mostly for reducing complexity, making it readable and in general having modularity. Think of your typical method – validateTableCreationPayload(), createUser(User user), deleteBook(int bookId) and so on. These methods are going to be used in only one other place in your application right?

It’s quite easy to ensure changing the method doesn’t break the one, single usage of it. And when it does break, almost always, the unit test is what gets rewritten. That’s the thing with application code – you’re not going to deny your client’s requirement just because your unit test fails right? And there is nothing dependent to be changed – remember there’s only one usage. Considering that none of the functionality is breaking – and your integration tests ensure that, unit tests lose much of their worth anyway. And it becomes a ritual to just change the unit test whenever we change code.

2. Sacrificing Development Speed

Unit tests for application code will take as much time developing the actual code. Often, the unit tests take more time to develop than the actual functionality. An architectural purist might not even consider this as an issue. But a business manager will be – Are you saying it actually only needs half the time to develop!?

And unit tests tend to become very complex very fast. In a regular web application, you end up having to mock several components in almost every unit test. As far as I’ve seen, going beyond a certain coverage percentage, makes unit tests unreadable and off-putting to modify.

3. Misleading Sense of Security

You should always do proper diligence before modifying existing functionality. Do not ever rely on unit tests to catch these. Because unit tests are almost always insufficient. Take this example –

public void createUser(User user) {
    addMetaDataToUser(user);
    // new statement inserted here
    repository.createUser(user);
}

@Test
public void createUserShouldCallRepository() {
    createUser(dummyUser);
    verify(mockRepository).createUser(any());   
}

The above test has 100% coverage for the createUser method. But what happens if we insert another statement in the createUser method? The test still would show 100% coverage. The developer who inserts that statement can get away with not even writing unit tests for their change. Such things can only be caught by careful manual review. And unit tests are often ignored when code reviews are done. Especially in teams where an automated process reports code coverage metrics, and (wrongly) says everything is fine.

Worse, the above unit test is inadequate. It only checks if the repository.createUser() method is called. Not that it’s called with the correct argument! If the new inserted statement modifies the user argument, the unit test will still pass, code coverage will still be 100%. Bad, no?

Reasons for Writing Unit Tests

1. Documents Code

If unit tests are well written, they will act as documentation for how and why code is written in a certain way. It’s not uncommon to see a comment in code that says ‘ugly hack .. do not change this’ or something similar to that. The programmer who wrote it understands that someone in future might modify it and the fallout might not be immediate. But this assumes that nobody would have ‘cleaned up’ that comment. Or a future programmer would simply miss reading it. But if there is a unit test that looks out for this situation, that’s a way better safeguard.

Also, well written unit tests have great names that set expectations about the method being tested – shouldThrowExceptionForBadInput(), shouldReturnAValidUuid() etc. Also, these are arranged in neat groups in your test classes. So if your developers follow discipline you can take a look at the unit tests and learn a lot about the code.

2. Protects Sensitive Methods

General purpose methods, which are used all over the application need to be protected from change. For example, if you have a validation method that checks if a string has a particular pattern (maybe your company has a certain pattern for unique IDs), it is very risky if someone modifies this method. Every method you have used twice or more in your application, must have a unit test. The method serves for two tasks. What if you change it for one task and the method no longer works correctly for the second task? Preventing this, is the core purpose of unit tests. Unit tests serve as a guarantee to what the method will do and will not do.

3. Improves Code Quality

In my opinion, this is the number one reason to enforce unit tests. Especially if there are inexperienced developers in your team. If you write bad code, it becomes more difficult to write unit tests. In my initial days as a developer every time I struggled to write a unit tests, it showed me how I could have written my code to be more loosely-coupled, modular and simple.

Your function might have to be broken into pieces, if you are writing too many unit tests for it. Your function is too tightly coupled, if you are writing too many mocks. Are you finding you need to inject ‘private’ dependencies using Reflection? Are you unable to write a unit test because one of your private fields is initialised indirectly (like Spring’s @Value annotation)? Just about every time you find yourself writing unit tests that are too complicated, you can see opportunities to simplify your original code.

Conclusion

Now, if I propose we write only minimum required unit tests, I would be no different from the people who tell me to write the maximum possible unit tests. But one thing is obvious to me, having a code coverage target is pointless. Only the developer who is writing code can decide which part of it has to be registered in unit tests and which parts don’t quite benefit from unit testing. If you don’t give importance to unit tests in code reviews, don’t even bother requiring them – at least your development will be faster and your code base will be cleaner.

What Slows Down Software Development

Delays in delivery is so common in the software development world, that it has become a common practice to include a ‘buffer’ time when giving commitments. And the sad thing is, we face delays even after including such a buffer. There are two sides to this problem – either the task has not been sized correctly, or the developers were slow in delivering. I thought of things that causes slow development and could find these major factors slowing down development for me –

Starting Without Clarity
Not Asking Questions
Lack of Knowledge and Skills
Not Getting it Right the First Time
Not Considering All Activities

Starting Without Clarity

Developers just break out their editors and start coding right away when they get a task. Almost always the task definitions will have gaps in them. It was probably written by a product manager or a technical architect. Only the developer will be aware of all the little questions that will arise while carrying out the task. So after taking a look at this task, ask questions that help you clear out any vagueness about the task.

Questions like ‘What errors must be returned?’, ‘Are there specific expectations on code quality?’, ‘Does the architect prefer using a certain design pattern?’, ‘Has the task considered its side effects?’ and so on, must be thought of by the developer when they start. Simply put, start with a clear end in sight.

Another thing that helps to achieve this, is to do test-driven development (TDD), where your tester first writes tests that are supposed to pass after the task is done. When you have actual tests, you have so much clarity on what needs to be done and can minimise the number of doubts that will rise later.

Not Asking Questions

There seems to be a lot of hesitation in today’s developers to ask questions. It mainly comes from either being an introvert, or by the doubt that your question might be stupid. Remember – Asking questions doesn’t make us stupid – making assumptions does.

And it also has a spiralling effect. The first time you might feel a little stupid to ask a question and carry on with your own assumptions, ignoring the fact that you are creating a gap in your own understanding. As time goes on and you avoid asking for clarity more and more, the gap keeps widening. At one point, you just can’t bring yourself to take a step back and ask your question for the fear of coming across as ignorant.

Stop worrying that your question might make you look less intelligent, or that you might be disturbing people if you raise questions. Software development is a lot about collaboration. People who don’t collaborate inevitably end up failing in their job.

Lack of Knowledge and Skills

As bitter as this factor may be, it is definitely a fact that lack of knowledge and skills in the developer is one of the top causes of slow delivery. There doesn’t even seem to be a motivation to learn for many developers. Even if the developer is dropped into a project where a new skills are required, often they don’t sit down and learn those skills.

Thanks to the wealth of snippets and quick-fixes available on Google and StackOverflow, it’s very easy to become a copy-paste developer. Just learn the basics of your programming language, and then use the internet for anything that’s required right? – Wrong. Even if every solution you find on the internet is suitable for you and you can finish most of your tasks by searching for answers on the internet, it’s going to make you a slow developer. A terribly slow developer.

You need to be able to do most of your work by yourself without searching for answers. The internet helps in situations when you are stuck. But if you need to search for almost everything you are doing, it means you are stuck all the time. It’s like repeatedly stopping to tie your laces while you are running a race. You will eventually finish, but your progress will be slow and rocky.

Invest time and effort to actually learn the tools you are using. Your language, the core libraries and frameworks you use, your IDE, your build system – these are the things that you should sit down and learn. There is a big difference in quality, speed and smoothness when a developer actually knows how to do their task rather than searching on the internet for every basic thing.

Not Getting It Right First Time

When you complete your development and merge it with the main codebase, you should be sure of what you have done. Have you thought about all error scenarios? All possible exceptions? What assumptions have you made? Have you communicated those assumptions and ensured they are correct? Have you thoroughly tested it? (Yes, a developer is supposed to test their development before sending it off to a tester)

Ensure that the tester won’t send it back to you. Or worse, it won’t end up as a user-reported bug. Or even worse, it won’t break the production system.

Making sure that your code doesn’t have mistakes and bugs are very easy when you are still writing it on your computer. Once your code passes on to the next phase, your mistakes become more expensive in terms of time to fix it. Think about finding a typo on your computer before you commit. Versus, the time to deploy it, have the user find the issue, reporting it back, it landing as a bug on your plate, then you fix and deploy again. Carefulness on your part would have saved that much time.

Time for doing bug fixes is usually not planned into project timelines. So whenever you introduce a bug and it has to be fixed later, it will take away time from something else. Imagine you are already assigned a task and suddenly a critical bug lands on your plate. You spend a couple days fixing the bug, and your original task now has two days lesser to complete.

The timeless proverb ‘a stitch in time, saves nine’ is one of the most important things you should keep in mind. When you work on completing a task, always remember, that you should get it right the first time. Don’t depend on the tester, or the user, to discover a bug. Consider it your responsibility and comb your work thoroughly to ensure you have covered everything – code bugs, user misuse, changes in integrated systems, unit tests, error messages, breaking compatibility – everything. Only then commit.

Not Considering All Activities

When you have a task and a target date, you need to consider what the target date means. Most of the time, when an execution team gets a target date, it means the task is supposed to be completed and available in production on that date. But in developer’s mind, it often gets registered as the date on which they should complete coding. Even when the developer is told specifically that the target date is when the task should be on production, it doesn’t create a concrete realisation in the developer’s mind.

You need to think about –

The time required for your code to be reviewed and merged
The time required for deployment on a QA stack
The time required for the tester to run tests
The time required for the tester to write new tests for your code
The time required for you to fix any errors discovered by the tester
The time required for your code to go to a pre-production environment
The time you need to wait (for safety) on the pre-production environment before going to production
The time required for documentation changes
The time you need to deploy on production

Those are just a few things that come to my mind. You might have more than that (or less), but the takeaway is that you must think about how many things need to be done to call your task ‘completed’. You might think there is a lot of time for you to finish your task, but it’s simply wrong to take all the time to write your code and your tester gets two hours to test before it gets deployed to production. For example, when I get 10 days to complete a task, I usually find that I have 2 – 3 days to finish development and hand over for testing.

Conclusion

Of course those are just a measly 5 factors among several others that contribute to delays in software development, but in my experience, when intentions are good and the project is still delayed, those are usually the main culprits. More importantly, as a developer, those are the 5 points that are actually in your control. Fix them and you’ll stand apart from mediocrity. Be awesome.

Benchmarking Java Methods using JMH with Gradle

JMH (Java Microbenchmarking Harness) is a library for benchmarking Java code. Running a benchmark using a library such as JMH is better than simply measuring execution time of methods using System.currentTimeMillis() or System.nanoTime().

The easiest way to run a benchmark using JMH is to use Maven/Gradle plugins. This article shows how to write and run a simple benchmark using the Gradle plugin.

I suppose you already have a Gradle project in which you want to use JMH, but if you wish to try out the walkthrough in this article, you can checkout the code from here as a starting point – https://github.com/heppydepe/measuring-execution-time-of-java-methods/tree/v1.0 which was used in the other article ‘Measuring Execution Time of Java Methods‘. It is a simple project with a class called ‘RandomNumbers’ that creates a list of 100 random numbers and stores it in memory when an object is instantiated.

Add The Plugin

The latest version of this plugin is 0.5.3 at the time of writing this post. You should probably use whichever latest version is available now.

plugins {
    id 'me.champeau.jmh' version '0.6.5'
}

Write Your Benchmarks

The plugin expects all our benchmarks are put in a src/jmh directory. I like this because it is similar to how we would keep our unit tests – in a separate src/tests folder. So create a jmh directory inside the src, create folder structure to represent your package inside this. That is, if your package name is com.example.mybenchmark, then create com/example/mybenchmark directory inside the jmh directory.

Create a java file for putting your benchmarks in. This is a benchmark –

@State(Scope.Benchmark)
public class RandomNumbersBenchmark {
    @Benchmark
    @OutputTimeUnit(TimeUnit.NANOSECONDS)
    @BenchmarkMode(Mode.AverageTime)
    public void initializeRandomNumbers(Blackhole bh) {
        bh.consume(new RandomNumbers());
    }
}

new RandomNumbers() is the piece of code that we are benchmarking. We are benchmarking the performance of creating new RandomNumbers objects. The Blackhole.consume() method ensures that JVM optimisations don’t come in the way of our benchmark. Without it, the JVM can see that we are not actually using the new RandomNumbers object, and optimise our code accordingly. JVMs do many such things and benchmarking frameworks like JMH give us ways to minimise JVM optimisations from hurting our benchmark numbers.

Run the Benchmarks

Running the benchmarks is quite simple. Just execute ./gradlew jmh from the project directory (where the build.gradle file is). You should be able to see the benchmark running and displaying it’s metrics as it runs. And the output would end with a summary like so –


Benchmark                                       Mode  Cnt     Score    Error  Units
RandomNumbersBenchmark.initializeRandomNumbers  avgt   25  2056.808 ± 84.686  ns/op

The full project for trying out can be downloaded from GitHub – https://github.com/heppydepe/measuring-execution-time-of-java-methods/tree/v2.0

Annotations

There are a ton of annotations in JMH which you can refer to in the documentation. But I’ll briefly write about the annotations used in the above sample.

@State is used to assign a “Scope” for the benchmark. Possible scopes are Benchmark, Group and Thread.

The @Benchmark annotation identifies this method as a benchmark. Similar to the @Test annotation that marks Unit tests.

@OutputTimeUnit annotation specifies what time unit your benchmark output should be in.

@BenchmarkMode annotation specifies in which modes the benchmark would run. Note that this annotation can supply multiple modes. Possible modes are AverageTime, SampleTime, SingleShotTime, Throughput and All.

For reference, all annotations and their meanings can be found in the JavaDoc for JMH.

Why JMH?

Microbenchmarking is a big step above simple time capturing. JMH does things like warmup, iterating and managing threads for us. It also helps us avoid common traps while benchmarking – for example, side stepping from JVM optimisations so that our benchmarks are accurate.

Picking My Game Development Tools

A process that used to excite me during my early days of development is to choose a tool or framework. But now as an experienced developer, I realise that’s probably the most exhausting yet not very useful to spend a ton of time and energy to make difficult choices at the beginning of a project (or in this case even before getting started with the project). So I will put my thoughts until lunch today. Make the quickest decisions that I can make. Stick with till I’m good.

Guiding Principles

I have to first learn a boatload of things. And it is yet to be seen whether I will stick to my commitment or I will get tired and give up. So
This is a last time I process this choice. Before I complete my first game, I will not go back on these decisions. If I don’t get some feature I need, I will sacrifice on it rather than think back and change these decisions.
Whenever possible, learn the art first. Invest money in tools later.

Time

The most important and difficult to get resource is going to be time. I have no intention of quitting my job and doing this full time. So my schedule would be to utilise the weekends and holidays to the fullest. It’s been very long and I have about a ton of gaps in knowledge to get where I want to be. So I don’t expect I’ll make anything significant at least for a year or even two. I’m also currently studying for masters, which will go on for one more year. So for one year this will be in the third priority. I can pickup a bit more speed after that. Strictly keep this to the weekends.

Hardware

I have a 13″ MacBook Pro 2015 and a 1080p monitor. I also have a Wacom Intuos drawing tablet too, in case I need something more than my Logitech Trackball mouse for drawing. I will purchase a small – maybe 25 keys – MIDI controller for making music. But only after 3 months. If I’m still going strong. Until then, I think I can simply use my computer keyboard to learn basic music composition. Decision here is that I already have everything I need. After 3 months, I will invest in a MIDI Controller – Akai MPK Mini MK3

Artwork

There wasn’t much questions here. Blender was the straightforward choice. But I need to learn to use Blender. Making 3D models can be considered my weakest skill in this whole endeavour. I practically do not know anything about it. I learnt some 3D modelling 20 years back (on a tool called Gmax – which has been discontinued now). I’m not sure whether I will remember anything or whether that knowledge will be useful now. I don’t think I will have to upgrade in future even after I become a good 3D modeller. Both Blender and Sketchbook are great tools even for professionals. So they should be good enough for me. So Blender and Sketchbook for artwork.

Music

There weren’t any good open-source music making software. I found LMMS, but it was far behind any decent commercial alternative. Almost nobody on the internet recommended it for a beginner. But fortunately, since I use a MacBook, I have GarageBand and it’s free. I will learn the basics of music composition and then upgrade my setup to a 25-key MIDI controller and FL Studio. I checked out another software called Reason – which appears to be great, but for a higher price. But in terms of popularity, FL Studio came out as a big winner, and so I believe as a beginner, it’s better I use FL Studio since I’ll have to be asking help from the community a lot. So decision is FL Studio.

Game Engine

A quick search on the internet showed me that I have two options Unity and Godot. I did some time-consuming research, before coming to the realisation that Unity is free-to-use. I don’t have to worry about its cost unless I am making significant revenue, which I don’t t think I will be making anytime soon. If you are making that much revenue, Unity’s licensing costs wouldn’t seem much. And Unity also didn’t appear to me as an evil company that would irrationally jack up license costs. So Unity it is.

So there. I have a big set of skills I’ll have to acquire in the next several months. Let me get cracking.

Rebooting the Game Development Dream

A very long time back – about 25 years ago, when I came across computer programming as a subject, the first thing that got me attracted to the field was that I could make my own computer games. More than all the fun of playing them, I was curious about how they were made. The more I found out, the more it made me curious.

But knowledge was very hard to come by. I mean really really hard. As a kid, for years I could not find anything about how they made games. I was able to convince my parents to send me to a teacher who taught BASIC programming for very little fees. Yet, all he was able to teach me was some ‘advanced’ basic. Like changing the screen mode, making the beep sound. But nothing about how to actually put a picture on the screen. This was 1995. All I could do was get the BASIC and C books from some grown ups and try to guess how those programs would work – just a pen and paper. I didn’t own a computer and I could get my hands on a computer for about a couple of hours in a year.

Fast forward to 2000, I got more frequent access to a computer thanks to one of my uncles who purchased a PC, and more importantly – an internet connection! The internet was brilliant – just so much information. And everything was free! I distinctly remember the first set of colourful graphics tutorials by David Brackeen (glad to see he is still keeping the tutorials online). Then I found the site called makegames.com (the site is closed now, but I found an archive here). And a big book called Action Arcade Adventure Set which it’s author Diana Gruber had released on the internet for free. But all this was still not enough. I was told to ‘stop playing with the computer’ and to study and get into a good college.

Fast forward to 2006, I finished college and it had been years since I tried my hand at programming anything related to video games. Surprisingly, the college teachers were completely disappointing when it came to this stuff. They knew the syntax of languages and were able to teach me enough to pass the subjects and attend job interviews at software companies. But absolutely no one from whom I could learn video game development. My dad bought me a computer, but still no internet. I could still have internet only when I went off to my uncles’ home for holidays. Until 2007. In 2007, I expressed to my dad that I was interested to become a video game developer.

He did not discourage me. He paid for an internet connection, he gave me money for a couple books, and surprisingly, he paid fees for a short video game programming course. It was the only video game related course that was available in the whole city at that time. Happy times! I made a lot of programs in this time – a fancy MP3 player, a simple C code editor, a couple DirectX-based games. But real life caught up.

It was almost 18 months after I finished college and I still couldn’t get a job. There were literally 3 video game companies I could interview at. Two of them quoted very low pay – where I wouldn’t even be able to feed myself. One promised a good salary, but couldn’t put together the team that they needed. So they shut shop and left the country. At this point, folks at my home lost patience. I was about to take up one of those game programming jobs, but they firmly forbade it at my home, because the pay was too low and in their view, I was wasting away my life.

So I took up job as an ERP developer at a reputed automotive company. It was quite easy compared to video game programming. I was able to grow quite fast and I didn’t end up wasting my life like my parents warned. It’s almost 13 years since. Right now I’m a lead Java developer for a very good company. The work is perfect and the pay is good. But still, the fact that I didn’t become a video game programmer keeps nagging me from inside. Whenever I play a video game, watch a trailer .. I can’t help but feel sad. If I had held on a bit longer .. If I had persevered .. I might have been on one of those teams, building those beautiful things.

Since it’s a feeling that keeps coming back, even after a decade, I’m thinking to do something about it. I’m not sure if I’m young enough to learn something completely new now and put effort in a large side project. And I’m quite sure that I don’t remember much from the old days. I have to start from scratch. But I’m not going to be able to put together a team at this point. I first have to learn a lot and become worthy of a team, make something small to prove myself. Then maybe … just maybe .. I would be able to become an actual video game developer. For now, all I have, is the decision that I’m going to give this a true, solid attempt.