Code Creek

Picking My Game Development Tools

A process that used to excite me during my early days of development is to choose a tool or framework. But now as an experienced developer, I realise that’s probably the most exhausting yet not very useful to spend a ton of time and energy to make difficult choices at the beginning of a project (or in this case even before getting started with the project). So I will put my thoughts until lunch today. Make the quickest decisions that I can make. Stick with till I’m good.

Guiding Principles

I have to first learn a boatload of things. And it is yet to be seen whether I will stick to my commitment or I will get tired and give up. So
This is a last time I process this choice. Before I complete my first game, I will not go back on these decisions. If I don’t get some feature I need, I will sacrifice on it rather than think back and change these decisions.
Whenever possible, learn the art first. Invest money in tools later.

Time

The most important and difficult to get resource is going to be time. I have no intention of quitting my job and doing this full time. So my schedule would be to utilise the weekends and holidays to the fullest. It’s been very long and I have about a ton of gaps in knowledge to get where I want to be. So I don’t expect I’ll make anything significant at least for a year or even two. I’m also currently studying for masters, which will go on for one more year. So for one year this will be in the third priority. I can pickup a bit more speed after that. Strictly keep this to the weekends.

Hardware

I have a 13″ MacBook Pro 2015 and a 1080p monitor. I also have a Wacom Intuos drawing tablet too, in case I need something more than my Logitech Trackball mouse for drawing. I will purchase a small – maybe 25 keys – MIDI controller for making music. But only after 3 months. If I’m still going strong. Until then, I think I can simply use my computer keyboard to learn basic music composition. Decision here is that I already have everything I need. After 3 months, I will invest in a MIDI Controller – Akai MPK Mini MK3

Artwork

There wasn’t much questions here. Blender was the straightforward choice. But I need to learn to use Blender. Making 3D models can be considered my weakest skill in this whole endeavour. I practically do not know anything about it. I learnt some 3D modelling 20 years back (on a tool called Gmax – which has been discontinued now). I’m not sure whether I will remember anything or whether that knowledge will be useful now. I don’t think I will have to upgrade in future even after I become a good 3D modeller. Both Blender and Sketchbook are great tools even for professionals. So they should be good enough for me. So Blender and Sketchbook for artwork.

Music

There weren’t any good open-source music making software. I found LMMS, but it was far behind any decent commercial alternative. Almost nobody on the internet recommended it for a beginner. But fortunately, since I use a MacBook, I have GarageBand and it’s free. I will learn the basics of music composition and then upgrade my setup to a 25-key MIDI controller and FL Studio. I checked out another software called Reason – which appears to be great, but for a higher price. But in terms of popularity, FL Studio came out as a big winner, and so I believe as a beginner, it’s better I use FL Studio since I’ll have to be asking help from the community a lot. So decision is FL Studio.

Game Engine

A quick search on the internet showed me that I have two options Unity and Godot. I did some time-consuming research, before coming to the realisation that Unity is free-to-use. I don’t have to worry about its cost unless I am making significant revenue, which I don’t t think I will be making anytime soon. If you are making that much revenue, Unity’s licensing costs wouldn’t seem much. And Unity also didn’t appear to me as an evil company that would irrationally jack up license costs. So Unity it is.

So there. I have a big set of skills I’ll have to acquire in the next several months. Let me get cracking.

Rebooting the Game Development Dream

A very long time back – about 25 years ago, when I came across computer programming as a subject, the first thing that got me attracted to the field was that I could make my own computer games. More than all the fun of playing them, I was curious about how they were made. The more I found out, the more it made me curious.

But knowledge was very hard to come by. I mean really really hard. As a kid, for years I could not find anything about how they made games. I was able to convince my parents to send me to a teacher who taught BASIC programming for very little fees. Yet, all he was able to teach me was some ‘advanced’ basic. Like changing the screen mode, making the beep sound. But nothing about how to actually put a picture on the screen. This was 1995. All I could do was get the BASIC and C books from some grown ups and try to guess how those programs would work – just a pen and paper. I didn’t own a computer and I could get my hands on a computer for about a couple of hours in a year.

Fast forward to 2000, I got more frequent access to a computer thanks to one of my uncles who purchased a PC, and more importantly – an internet connection! The internet was brilliant – just so much information. And everything was free! I distinctly remember the first set of colourful graphics tutorials by David Brackeen (glad to see he is still keeping the tutorials online). Then I found the site called makegames.com (the site is closed now, but I found an archive here). And a big book called Action Arcade Adventure Set which it’s author Diana Gruber had released on the internet for free. But all this was still not enough. I was told to ‘stop playing with the computer’ and to study and get into a good college.

Fast forward to 2006, I finished college and it had been years since I tried my hand at programming anything related to video games. Surprisingly, the college teachers were completely disappointing when it came to this stuff. They knew the syntax of languages and were able to teach me enough to pass the subjects and attend job interviews at software companies. But absolutely no one from whom I could learn video game development. My dad bought me a computer, but still no internet. I could still have internet only when I went off to my uncles’ home for holidays. Until 2007. In 2007, I expressed to my dad that I was interested to become a video game developer.

He did not discourage me. He paid for an internet connection, he gave me money for a couple books, and surprisingly, he paid fees for a short video game programming course. It was the only video game related course that was available in the whole city at that time. Happy times! I made a lot of programs in this time – a fancy MP3 player, a simple C code editor, a couple DirectX-based games. But real life caught up.

It was almost 18 months after I finished college and I still couldn’t get a job. There were literally 3 video game companies I could interview at. Two of them quoted very low pay – where I wouldn’t even be able to feed myself. One promised a good salary, but couldn’t put together the team that they needed. So they shut shop and left the country. At this point, folks at my home lost patience. I was about to take up one of those game programming jobs, but they firmly forbade it at my home, because the pay was too low and in their view, I was wasting away my life.

So I took up job as an ERP developer at a reputed automotive company. It was quite easy compared to video game programming. I was able to grow quite fast and I didn’t end up wasting my life like my parents warned. It’s almost 13 years since. Right now I’m a lead Java developer for a very good company. The work is perfect and the pay is good. But still, the fact that I didn’t become a video game programmer keeps nagging me from inside. Whenever I play a video game, watch a trailer .. I can’t help but feel sad. If I had held on a bit longer .. If I had persevered .. I might have been on one of those teams, building those beautiful things.

Since it’s a feeling that keeps coming back, even after a decade, I’m thinking to do something about it. I’m not sure if I’m young enough to learn something completely new now and put effort in a large side project. And I’m quite sure that I don’t remember much from the old days. I have to start from scratch. But I’m not going to be able to put together a team at this point. I first have to learn a lot and become worthy of a team, make something small to prove myself. Then maybe … just maybe .. I would be able to become an actual video game developer. For now, all I have, is the decision that I’m going to give this a true, solid attempt.

Measuring Execution Time of Java Methods

There are 2 ways to measure execution time of Java methods.

Wall Clock Time Vs. CPU Time

Wall Clock Time: Time that you actually wait for the method to complete. This metric is useful in measuring how fast a user perceives your method to be. For desktop apps and mobile apps, this measure is rarely accurate because we don’t know what other programs the user might be running in parallel when they are using our app. But this is very useful if you are measuring performance on server applications. Because they run on our server where we can control the capabilities and load from other programs. Wall clock time is what we measure in typical performance tests of web applications.
CPU Time: Time spent by the CPU executing the method. Since the CPU is shared by all processes running on a computer, this metric shows you how much of the resource your program is using – or in other words, for how much time, the CPU core was kept active by your program. In an ideal world, this would be exactly the same if you run the same code again and again, but our world is more ideal than ideal – both the JVM and the OS, execute code in quite complicated ways to optimise system usage. So there still might be differences between each execution but much less than wall clock time.

Source Code

You can get the source code for this article on GitHub – https://github.com/heppydepe/measuring-execution-time-of-java-methods/tree/v1.0

The Sample Class

We will use the following RandomNumbers class and measure it’s execution time. This is a neat little class for demonstrating time measurement – we can test the object creation because it creates 100 random numbers, we can test IO operation that happens when you execute writeToFile(), and you can test string conversion through the toString() method.

package com.codecreek.methodbenchmark;

import java.io.FileWriter;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;

public class RandomNumbers {
    private List<Integer> list;

    public RandomNumbers() {
        Random rand = new Random();
        list = new ArrayList<>();
        for (int i = 0; i < 100; i++) {
            list.add(rand.nextInt());
        }
    }

    public void writeToFile() {
        FileWriter fileWriter = new FileWriter("output.txt");
        fileWriter.write(this.toString());
        fileWriter.close();
    }

    public String toString() {
        return list.toString();
    }
}

Measuring Wall Clock Time

Wall Clock Time can be measured using the System.nanoTime() method. Capture the elapsed time when the execution starts, then capture it again when the execution ends. There are a dozen ways of capturing system time, but I like the simple nanoTime(). For some other ways you can refer this StackOverflow answer.

try {
    for (int i = 0; i < iterations; i++) {
        startTime = System.nanoTime();
        randomNumbers.writeToFile();
        endTime = System.nanoTime();
        if (i >= skipFirst) {
            measureList.add(endTime - startTime);
        }
    }
} catch (IOException ex) {
    ex.printStackTrace();
}

System.out.println("Wall Clock Time:\t" + measureList);

Measuring CPU Time

CPU Time can be measured using the ThreadMXBean. To get an instance, we use ManagementFactory.getThreadMXBean() and then it has a method getCurrentThreadCpuTime() to get elapsed time of the current thread. Again, there are other ways of doing this, but for this article I’m just keeping to the simplest ways. You don’t need external dependencies for ThreadMXBean.

ThreadMXBean tmb = ManagementFactory.getThreadMXBean();
try {
    for (int i = 0; i < iterations; i++) {
        startTime = tmb.getCurrentThreadCpuTime();
        randomNumbers.writeToFile();
        endTime = tmb.getCurrentThreadCpuTime();
        if (i >= skipFirst) {
            measureList.add(endTime - startTime);
        }
    }
} catch (IOException ex) {
    ex.printStackTrace();
}

System.out.println("CPU Time:\t" + measureList);

Output

I ran the code on my computer and this is what I got –

Wall Clock Time:
        Stats{count=10000, mean=278124.58609999955, populationStandardDeviation=368752.52698052896, min=111579.0, max=1.258139E7}

CPU Time:
        Stats{count=10000, mean=210671.20000000033, populationStandardDeviation=86977.8137835161, min=110000.0, max=1794000.0}

I used the Stats functions from Google Guava library. Even in this relatively naive test, I could notice that the wall clock time has much greater variance than CPU time.

But I do have to reiterate. This test is practically good for my use case. But might not be for others. Because I haven’t considered what optimisations my JVM and my OS may have done while executing the code. This is more of a ‘getting-started’ kind of article. For more serious benchmarking it’s better to use tools like JMH or Google Caliper .

Simple Command Line Java Project (using Gradle)

I like to create a simple command line application whenever I want to learn a new concept/product or when I’m trying to implement a complicated logic. I never actually work on command line projects – for several years now I’ve been exclusively working on web applications. But still my go to approach for implementing something new is to first try it out in a command line program and then when I am comfortable, write it into my web application. Isolating new attempts like this helps me keep a clear mind and learn the new concept in more depth. Also it’s much more efficient – otherwise, I have to load up the entire web application just to see whether my new approach worked.

Why Not Just `javac` and `java`

Yes, it’s quite simple to just write a *.java file and compile it with javac and run it with the java command. But specifying and managing dependencies is a headache. Using a tool like Gradle or Maven would enable us to avoid that whole activity and focus on our project. So even for a one-file command line application, I will use something like Gradle.

Setup Gradle

If you have never initialised a Gradle project, chances are the you never had to install Gradle on your computer. The gradlew command that is available in Gradle projects take care of downloading and installing the required version of Gradle for that project. Well now you are going to initialise a new project. You need a standard installation of Gradle to do this.

My preferred way to install Gradle is to use SDKMAN! Just sdk install gradle and I have a clean installation of Gradle. After that do gradle -v to check the version of Gradle installed. For this tutorial, it doesn’t matter much. If you’ve installed a relatively new version it’s fine.

Create A New Project

For this example, let’s create a project called test-project. Steps to create a new project are –

Create a directory – Open a terminal and type mkdir test-project and then switch into the new directory cd test-project
Initialize a Gradle project – execute gradle init and then answer the questions that the script asks. For this project the answers would be –
1. Type of project is ‘application’. ‘basic’ is basically an empty project – we don’t want that. Choose ‘application’
2. Implementation language is ‘Java’
3. Split into sub projects – ‘no’. We just need a simple project.
4. Build script DSL – ‘Groovy’. Selecting Kotlin doesn’t make much of a difference anyway. Being statically typed, Kotlin makes it easier for IDEs to do fancy things like auto-complete. But Groovy is quite fine for me for this project.
5. Test framework – ‘JUnit Jupiter’. I like JUnit 5 (also called JUnit Jupiter)
6. Project name – Accept the default – which is the directory name test-project. You can give a different name if you like.
7. Source package – If the project is never going to leave your computer, simply accept the default or type whatever you like. If you’re going to share it, it’s a good idea to put some unique string here. Real projects use their domain names to make package names unique – org.apache.commons etc.

Keep in mind about reserved keywords when you name things. Source package names like java.test or try.newtech etc. will fail. Because try is a reserved keyword in java and java itself is forbidden to be used in package names.

That’s it. You’ve made a neat little application for yourself to play with.

Run the Project

Simply execute ./gradlew run to run your new little application. As of today, the default project prints out the string Hello World!

You can open this folder in your favourite text editor or IDE and go about making changes. The source code would be inside the app/src/main directory.

Execute ./gradlew tasks to see what tasks are available for you to run. If like me, you are doing this to do a trial implementation of some new algorithm or library, ./gradlew run is all you need.

Get Cracking

Open the build.gradle file in your project folder to add dependencies. Use the app/src/main/java/{{source-package}}/App.java file as a starting point for your code.

Of course this is only one of the countless ways in which you could put together a stub project for trying out new things, but it’s a neat and simple way. And since it’s Gradle, pretty much all of your dependencies are easy to add into your project. And you don’t have to suffer with a bulky big project while you are trying to learn your new technology or implement your new idea.

Thoughts on Performance Analysis

The application that I’m working on – a REST API, has a full fledged performance test suite that’s run for every release. That test suite is owned by a separate set of people responsible for performance testing. As a developer I limit myself to take inputs from the performance tests, and tune the application wherever necessary. But I think there are some basics every performance test project should get down.

It is common knowledge that a basic performance test should measure both requests/second and response times. So the basic output would be a chart –

Simple, but probably useless performance report

The Problem

The commitment to your customer might be something like “less than 500 ms response times for load upto 500 requests/second“. The performance test will generate a chart like the one above, and you can use it to show the the performance is under 500 ms as committed for upto 500 req/s. Also, the chart shows that there is a disproportionate performance decrease after a certain req/s and then it gets exponentially worse. You can decide whether or not to address this based on your budget and whether your user might be expected to increase their work load to more than the problem point.

Now my problem is, how do I define ‘requests/second’. In almost all applications, there are vastly different types of requests. Just executing one type of request repeatedly and using it to derive a performance test would be absolutely useless. Measuring for all the different types of requests, and then aggregating by average or median is also quite useless. In fact, what this translates to, is that the “req/s” attribute is useless.

Define KPIs

You should define KPIs based on the functionality of the application. Instead of “req/s”, you have to create meaningful attributes like “user logins per second”, “orders created per second” or “get-product-by-id requests per second” etc. You need to define metrics also – almost always this is ‘response time’ because that’s the one metric that affects the users directly. But your application can have other metrics also, like volume of logs produced, number of network calls made, amount of RAM utilised. Many of these are not even measured nowadays because it has become common to drastically oversize the hardware required to run software. But still, maybe some are useful.

I believe the above point is one of the most ignored things in software development. You need to have clearly defined KPIs before you do your performance tests. Otherwise the performance testers will put whatever they feel like into the report, and you’ll end up with a vague and crowded document as a performance report.

Create an Environment

Second, you have to get down the environment right. The infrastructure you use for performance testing must ideally be the same size as the one you use for production. The data already present on the system must be the same size as production. For example, the number of products in the database, will most likely affect how fast one product can be retrieved. So you need to have a ‘seeding’ process where you populate your performance test stack with production-like data.

It helps to create a ‘seeding program’ that creates test data based on a configuration file. The config file might have settings like number of users to create, the number of products to create, the number of orders the users might already have etc. These settings can also be ‘ranges’ – eg., each user to have 0 to 30 orders distributed randomly. This might seem like overkill, but trust me, it will up your performance testing game to epic levels. It will give you the confidence to say you can performance test to any given scenario and consequently, future-proof your application.

Report to the Business

Make your report for the business to understand. Don’t make it a technical report like “response time vs. req/s”. Your report should tell stories – like “If number of logged in users increases by 50%, what is the response time for order registration’? If you have started your process by clearly defined KPIs then your report will come out great naturally. And it will communicate directly what needs to be done for the future, and what risks are there in the system.

“All requests must respond within a certain response time, if it doesn’t, just spend more money and throw more hardware at it” – seems to have become a common excuse for a proper performance tune up. Requests are not equal. One type of request might have to talk to the database. One type of request might have to wait for another service on the internet. It’s simply wrong, and very inconvenient to group all requests of an application together. Don’t do that. Go one step further and save some money by doing a proper performance analysis.