What Slows Down Software Development

Delays in delivery is so common in the software development world, that it has become a common practice to include a ‘buffer’ time when giving commitments. And the sad thing is, we face delays even after including such a buffer. There are two sides to this problem – either the task has not been sized correctly, or the developers were slow in delivering. I thought of things that causes slow development and could find these major factors slowing down development for me –

  1. Starting Without Clarity
  2. Not Asking Questions
  3. Lack of Knowledge and Skills
  4. Not Getting it Right the First Time
  5. Not Considering All Activities
Photo by Braden Collum on Unsplash

Starting Without Clarity

Developers just break out their editors and start coding right away when they get a task. Almost always the task definitions will have gaps in them. It was probably written by a product manager or a technical architect. Only the developer will be aware of all the little questions that will arise while carrying out the task. So after taking a look at this task, ask questions that help you clear out any vagueness about the task.

Questions like ‘What errors must be returned?’, ‘Are there specific expectations on code quality?’, ‘Does the architect prefer using a certain design pattern?’, ‘Has the task considered its side effects?’ and so on, must be thought of by the developer when they start. Simply put, start with a clear end in sight.

Another thing that helps to achieve this, is to do test-driven development (TDD), where your tester first writes tests that are supposed to pass after the task is done. When you have actual tests, you have so much clarity on what needs to be done and can minimise the number of doubts that will rise later.

Not Asking Questions

There seems to be a lot of hesitation in today’s developers to ask questions. It mainly comes from either being an introvert, or by the doubt that your question might be stupid. Remember – Asking questions doesn’t make us stupid – making assumptions does.

And it also has a spiralling effect. The first time you might feel a little stupid to ask a question and carry on with your own assumptions, ignoring the fact that you are creating a gap in your own understanding. As time goes on and you avoid asking for clarity more and more, the gap keeps widening. At one point, you just can’t bring yourself to take a step back and ask your question for the fear of coming across as ignorant.

Stop worrying that your question might make you look less intelligent, or that you might be disturbing people if you raise questions. Software development is a lot about collaboration. People who don’t collaborate inevitably end up failing in their job.

Lack of Knowledge and Skills

As bitter as this factor may be, it is definitely a fact that lack of knowledge and skills in the developer is one of the top causes of slow delivery. There doesn’t even seem to be a motivation to learn for many developers. Even if the developer is dropped into a project where a new skills are required, often they don’t sit down and learn those skills.

Thanks to the wealth of snippets and quick-fixes available on Google and StackOverflow, it’s very easy to become a copy-paste developer. Just learn the basics of your programming language, and then use the internet for anything that’s required right? – Wrong. Even if every solution you find on the internet is suitable for you and you can finish most of your tasks by searching for answers on the internet, it’s going to make you a slow developer. A terribly slow developer.

You need to be able to do most of your work by yourself without searching for answers. The internet helps in situations when you are stuck. But if you need to search for almost everything you are doing, it means you are stuck all the time. It’s like repeatedly stopping to tie your laces while you are running a race. You will eventually finish, but your progress will be slow and rocky.

Invest time and effort to actually learn the tools you are using. Your language, the core libraries and frameworks you use, your IDE, your build system – these are the things that you should sit down and learn. There is a big difference in quality, speed and smoothness when a developer actually knows how to do their task rather than searching on the internet for every basic thing.

Not Getting It Right First Time

When you complete your development and merge it with the main codebase, you should be sure of what you have done. Have you thought about all error scenarios? All possible exceptions? What assumptions have you made? Have you communicated those assumptions and ensured they are correct? Have you thoroughly tested it? (Yes, a developer is supposed to test their development before sending it off to a tester)

Ensure that the tester won’t send it back to you. Or worse, it won’t end up as a user-reported bug. Or even worse, it won’t break the production system.

Making sure that your code doesn’t have mistakes and bugs are very easy when you are still writing it on your computer. Once your code passes on to the next phase, your mistakes become more expensive in terms of time to fix it. Think about finding a typo on your computer before you commit. Versus, the time to deploy it, have the user find the issue, reporting it back, it landing as a bug on your plate, then you fix and deploy again. Carefulness on your part would have saved that much time.

Time for doing bug fixes is usually not planned into project timelines. So whenever you introduce a bug and it has to be fixed later, it will take away time from something else. Imagine you are already assigned a task and suddenly a critical bug lands on your plate. You spend a couple days fixing the bug, and your original task now has two days lesser to complete.

The timeless proverb ‘a stitch in time, saves nine’ is one of the most important things you should keep in mind. When you work on completing a task, always remember, that you should get it right the first time. Don’t depend on the tester, or the user, to discover a bug. Consider it your responsibility and comb your work thoroughly to ensure you have covered everything – code bugs, user misuse, changes in integrated systems, unit tests, error messages, breaking compatibility – everything. Only then commit.

Not Considering All Activities

When you have a task and a target date, you need to consider what the target date means. Most of the time, when an execution team gets a target date, it means the task is supposed to be completed and available in production on that date. But in developer’s mind, it often gets registered as the date on which they should complete coding. Even when the developer is told specifically that the target date is when the task should be on production, it doesn’t create a concrete realisation in the developer’s mind.

You need to think about –

  1. The time required for your code to be reviewed and merged
  2. The time required for deployment on a QA stack
  3. The time required for the tester to run tests
  4. The time required for the tester to write new tests for your code
  5. The time required for you to fix any errors discovered by the tester
  6. The time required for your code to go to a pre-production environment
  7. The time you need to wait (for safety) on the pre-production environment before going to production
  8. The time required for documentation changes
  9. The time you need to deploy on production

Those are just a few things that come to my mind. You might have more than that (or less), but the takeaway is that you must think about how many things need to be done to call your task ‘completed’. You might think there is a lot of time for you to finish your task, but it’s simply wrong to take all the time to write your code and your tester gets two hours to test before it gets deployed to production. For example, when I get 10 days to complete a task, I usually find that I have 2 – 3 days to finish development and hand over for testing.

Conclusion

Of course those are just a measly 5 factors among several others that contribute to delays in software development, but in my experience, when intentions are good and the project is still delayed, those are usually the main culprits. More importantly, as a developer, those are the 5 points that are actually in your control. Fix them and you’ll stand apart from mediocrity. Be awesome.

Picking My Game Development Tools

A process that used to excite me during my early days of development is to choose a tool or framework. But now as an experienced developer, I realise that’s probably the most exhausting yet not very useful to spend a ton of time and energy to make difficult choices at the beginning of a project (or in this case even before getting started with the project). So I will put my thoughts until lunch today. Make the quickest decisions that I can make. Stick with till I’m good.

Guiding Principles

  1. I have to first learn a boatload of things. And it is yet to be seen whether I will stick to my commitment or I will get tired and give up. So
  2. This is a last time I process this choice. Before I complete my first game, I will not go back on these decisions. If I don’t get some feature I need, I will sacrifice on it rather than think back and change these decisions.
  3. Whenever possible, learn the art first. Invest money in tools later.
Photo by Jo Szczepanska on Unsplash

Time

The most important and difficult to get resource is going to be time. I have no intention of quitting my job and doing this full time. So my schedule would be to utilise the weekends and holidays to the fullest. It’s been very long and I have about a ton of gaps in knowledge to get where I want to be. So I don’t expect I’ll make anything significant at least for a year or even two. I’m also currently studying for masters, which will go on for one more year. So for one year this will be in the third priority. I can pickup a bit more speed after that. Strictly keep this to the weekends.

Hardware

I have a 13″ MacBook Pro 2015 and a 1080p monitor. I also have a Wacom Intuos drawing tablet too, in case I need something more than my Logitech Trackball mouse for drawing. I will purchase a small – maybe 25 keys – MIDI controller for making music. But only after 3 months. If I’m still going strong. Until then, I think I can simply use my computer keyboard to learn basic music composition. Decision here is that I already have everything I need. After 3 months, I will invest in a MIDI Controller – Akai MPK Mini MK3

Artwork

There wasn’t much questions here. Blender was the straightforward choice. But I need to learn to use Blender. Making 3D models can be considered my weakest skill in this whole endeavour. I practically do not know anything about it. I learnt some 3D modelling 20 years back (on a tool called Gmax – which has been discontinued now). I’m not sure whether I will remember anything or whether that knowledge will be useful now. I don’t think I will have to upgrade in future even after I become a good 3D modeller. Both Blender and Sketchbook are great tools even for professionals. So they should be good enough for me. So Blender and Sketchbook for artwork.

Music

There weren’t any good open-source music making software. I found LMMS, but it was far behind any decent commercial alternative. Almost nobody on the internet recommended it for a beginner. But fortunately, since I use a MacBook, I have GarageBand and it’s free. I will learn the basics of music composition and then upgrade my setup to a 25-key MIDI controller and FL Studio. I checked out another software called Reason – which appears to be great, but for a higher price. But in terms of popularity, FL Studio came out as a big winner, and so I believe as a beginner, it’s better I use FL Studio since I’ll have to be asking help from the community a lot. So decision is FL Studio.

Game Engine

A quick search on the internet showed me that I have two options Unity and Godot. I did some time-consuming research, before coming to the realisation that Unity is free-to-use. I don’t have to worry about its cost unless I am making significant revenue, which I don’t t think I will be making anytime soon. If you are making that much revenue, Unity’s licensing costs wouldn’t seem much. And Unity also didn’t appear to me as an evil company that would irrationally jack up license costs. So Unity it is.

So there. I have a big set of skills I’ll have to acquire in the next several months. Let me get cracking.

Thoughts on Performance Analysis

The application that I’m working on – a REST API, has a full fledged performance test suite that’s run for every release. That test suite is owned by a separate set of people responsible for performance testing. As a developer I limit myself to take inputs from the performance tests, and tune the application wherever necessary. But I think there are some basics every performance test project should get down.

It is common knowledge that a basic performance test should measure both requests/second and response times. So the basic output would be a chart –

Simple, but probably useless performance report

The Problem

The commitment to your customer might be something like “less than 500 ms response times for load upto 500 requests/second“. The performance test will generate a chart like the one above, and you can use it to show the the performance is under 500 ms as committed for upto 500 req/s. Also, the chart shows that there is a disproportionate performance decrease after a certain req/s and then it gets exponentially worse. You can decide whether or not to address this based on your budget and whether your user might be expected to increase their work load to more than the problem point.

Now my problem is, how do I define ‘requests/second’. In almost all applications, there are vastly different types of requests. Just executing one type of request repeatedly and using it to derive a performance test would be absolutely useless. Measuring for all the different types of requests, and then aggregating by average or median is also quite useless. In fact, what this translates to, is that the “req/s” attribute is useless.

Define KPIs

You should define KPIs based on the functionality of the application. Instead of “req/s”, you have to create meaningful attributes like “user logins per second”, “orders created per second” or “get-product-by-id requests per second” etc. You need to define metrics also – almost always this is ‘response time’ because that’s the one metric that affects the users directly. But your application can have other metrics also, like volume of logs produced, number of network calls made, amount of RAM utilised. Many of these are not even measured nowadays because it has become common to drastically oversize the hardware required to run software. But still, maybe some are useful.

I believe the above point is one of the most ignored things in software development. You need to have clearly defined KPIs before you do your performance tests. Otherwise the performance testers will put whatever they feel like into the report, and you’ll end up with a vague and crowded document as a performance report.

Create an Environment

Second, you have to get down the environment right. The infrastructure you use for performance testing must ideally be the same size as the one you use for production. The data already present on the system must be the same size as production. For example, the number of products in the database, will most likely affect how fast one product can be retrieved. So you need to have a ‘seeding’ process where you populate your performance test stack with production-like data.

It helps to create a ‘seeding program’ that creates test data based on a configuration file. The config file might have settings like number of users to create, the number of products to create, the number of orders the users might already have etc. These settings can also be ‘ranges’ – eg., each user to have 0 to 30 orders distributed randomly. This might seem like overkill, but trust me, it will up your performance testing game to epic levels. It will give you the confidence to say you can performance test to any given scenario and consequently, future-proof your application.

Report to the Business

Make your report for the business to understand. Don’t make it a technical report like “response time vs. req/s”. Your report should tell stories – like “If number of logged in users increases by 50%, what is the response time for order registration’? If you have started your process by clearly defined KPIs then your report will come out great naturally. And it will communicate directly what needs to be done for the future, and what risks are there in the system.

“All requests must respond within a certain response time, if it doesn’t, just spend more money and throw more hardware at it” – seems to have become a common excuse for a proper performance tune up. Requests are not equal. One type of request might have to talk to the database. One type of request might have to wait for another service on the internet. It’s simply wrong, and very inconvenient to group all requests of an application together. Don’t do that. Go one step further and save some money by doing a proper performance analysis.