Why TDD is a must from a Return on Investment (ROI) point of view

1. Automated tests
2. TDD

Automated Tests

Automated tests are written to test out portions of code in an automated manner. That means no human intervention is required – such as a QA opening up the application and running the test cases by following several manual steps, entering data, performing actions. Rather, automated tests are code in themselves. They are written to invoke certain components, pass some test data and test the results. All these are done through a test case that is purposely written in code. Thus to run the test case, no human intervention is required. One only needs to run the automated test case and it will test out the functionality.

TDD

TDD is a concept where developers write automated test cases that test out the source code implementation. It is not any tool or technology; rather a practice to be followed. The practice dictates that before writing your implementation, you should write automated test cases. The TDD process primarily goes through 3 steps:

1. Red

This phase is when we do not have any implementation and we write test cases before them. Once we write the automated tests, we run them. They will fail as we have not implemented anything yet.

2. Green

Once we have our set of automated test cases ready, we start writing our implementation and re-run our test cases. Based on whether the test cases pass or fail, we adjust our implementation so that all test cases and correct expectations pass.

3. Refactor

Once the implementation is ready and the test cases are passing, it means we have achieved the functionality we wanted to. We can now refactor the code. And we can easily run the test cases again in a few seconds. That would save a lot of time and make refactoring easier as we do not have to test the entire application manually every time we make a change.

Concerns About TDD

The concept seems amazing and it is. It has some amazing benefits when followed. But there are still some concerns in the minds of people as to why they should follow this. Some people do not find any value in this. One of the main reasons for the people to reject TDD is the time taken to write test cases is an additional overhead. And as the application will be tested anyway in the QA phase, some feel the writing of automated test cases and that too before development leads to additional cost as it takes additional time to write them. In this article, we will be addressing this specific scenario and see how these claims are baseless with the help of an example.

The Example Setup and Premise

As a myth buster, I have considered a very simple example with the simplest of use case for demonstration. I have an API that does not do anything, it just returns a simple error response with status code and messages as below. It could be any simple method that can be called from the automated tests. In reality, the method would contain some other business logic and operations too; but for simplicity, the code has been omitted and only relevant portions have been retained.

Figure 1: API method returning a response with code and message

To test it, there is an automated test that calls this method and verifies the response data. The test ensures that the method behaves exactly as it is supposed to. As you can see below, the test just calls the method, receives the response and verifies that we have received the same data that we were supposed to. If not, the test case will fail.

Figure 2: Automated test case for testing the method and response data

There are several other test cases in the system to test the entire app and are divided into 2 types

• Unit tests that test only one component at a time
• Integration tests that test the entire end-to-end flow across component boundaries

Figure 3: The whole set of automated tests

The Test Premise

For the test execution, a simple scenario is considered. The requirement is to just change the message returned - from “Internal Server Error” to something else when the method is executed. During that execution, the time taken to update and test the scenario will be compared with respect to

• Implementing and testing the changes through manual debugging
• Implementing and testing the changes through automated tests

At the end, the results will be compared to arrive at a conclusion through statistics.

Implementation & Execution

The changes were implemented using both the approaches and the timings were noted.

Below is a breakup of the observations -

Figure 4: Implementation and test manually without TDD

Figure 5: Implementation and test with TDD

Analysis of the Findings

The timeline clearly shows that changing the same things manually took almost double the amount of time. And even in that, there are a lot of factors -

1. For the test, the changes were done in one go. But in reality, code has to be changed several times to get the implementation correct. In those cases, the difference has been much higher and manual approach will takes more time.
2. In the approach without TDD and automated tests, we haven’t even considered the scenarios for regression tests, i.e. testing all other functionalities. In real cases, that is an overhead and as you can see using TDD all the test cases took just 14 seconds to run whereas it could take hours to run the same without TDD and automation.
3. The test does not consider possible refactoring. In the case of automated tests, it is easier. In the other approach, every time refactoring is done, the whole cycle needs to be repeated resulting in more time.
4. The scenario considered in this test is really simple. In reality, it is way more complicated than that. Even in such simple scenarios, the difference is substantial. As the complexity and the size of the app grows, the manual approach would consume way more time as compared to TDD with automation.

To Sum Up

As per the statistics, it is quite clear that if the simplest scenarios like these can provide such huge benefits in terms of saving time and effort, real-world scenarios will lead to greater savings. Yes, there will always be difficulties with TDD like a learning curve but a little extra time at the start of development to establish TDD practices greatly outweighs the benefits in terms of Return on Investment (ROI).