Author: Andreas Grau

In large projects automated test reports are often not sufficient. Stake holders of the project are not familiar with the specific cases tested and the severity of the bugs found. Also, from a business perspective, it is often much better to release the software with known issues than waiting for a fix. Thus, the test report must enable the product owner to judge whether the new software version is better than the old one, what can and can’t be done with it and the implications for the User Experience.

Unit testing involves the individual testing of all the components of an application or website to ensure that the application meets its design requirements and behaves as intended. Since unit testing is the most intensive test that can be carried out on an application, it is often disregarded by software developers and considered time-consuming. But is Unit testing still worth the time and effort?

There are many latent benefits of performing Unit testing. It enables developers to make big changes to code quickly and also gives confidence that all units/components work exactly as they should. It also helps in finding problems early in the development cycle and helps with the documentation of coding progress, enabling developers to learn from previous mistakes.

Contrary to popular opinion, unit testing speeds up the development process as it makes it easy for developers to reuse/migrate code components more efficiently as the unit tests of these components can also be migrated as well.

In many situations, it was very helpful that someone dedicated to testing performed seemingly ineffective tests with somewhat random test steps. This way, the application is tested for both, for what it is supposed to do, and for what it is not supposed to do. You can even go a little further and use Fuzzing, that means completely random input. This way, you can find many security vulnerabilities.

This is no offence to the developers. Developers are as close to the code as it gets. Thus, they should write unit tests. But, developers should not write End-to-End tests. This is to reduce the chances of the developer running a biased tests or a lack of creativity in testing. Someone who knows what the software will be used for must write the tests. This is usually someone from the product team, e.g. the Product Owner.

Early testing ensures quick detection of bugs and reduces time and energy required to perform subsequent tests. It also builds confidence in developers to include new features in the product.

Client Needs are covered by Acceptance Testing.

Requirements are handled by End-to-End Testing.

Software Design is ensured by Integration Testing.

Code is checked by Unit Testing.

In practice, it is usually impossible to test all parameter combinations for real life applications. Say, you have 10 different parameters of a tax advice software where each parameter is true of false. Enumerating all parameter combinations leads to 3,628,800 test cases already. So, testing can always check a few combinations, only.

But, when the software finally goes public, I often see that rare usage patterns occur which have never been tested. There is a neat trick by a research group at NIST: Automated Combinatorial Testing for Software. Richard Kuhn, Raghu Kacker and Yu Lei observed that most errors occur by a change of one or two parameters, only. A two way test with 10 parameters can be done in 10 test cases. Even a 4-way test, where a specific set of 4 parameters is required to trigger a bug, is easy to do with 42 test cases.

A product roadmap is a strategic document that maps out the general stages of a product’s development. In startups or new businesses, a product roadmap usually serves as a filter to determine the most important tasks involved in a product’s development.

In Agile product development, activities such as split A/B testing are usually omitted from the product roadmap. This is because Agile product development is time-bound, hence no extra time is dedicated to testing which is often regarded as time-consuming.

To integrate testing into a product roadmap, the product management team first needs to understand and accept that testing is one of the most important tasks involved in product development, hence it deserves to be included in the roadmap. Also, the Engineering team should be duly included in the creation of the product roadmap from the beginning. This will ensure that Testing is performed at every step of product development. Finally, to save time testing, product management teams are advised to consider Automated testing.

User testing involves getting your users to test what you have developed. Although Manual or Automated software testing can help you find out whether or not your app functions properly, it cannot take the place of user testing. Through User testing, you can determine if your app really solves your users’ problems and it also provides an avenue for new product features to be suggested.

To perform User testing efficiently, here are 6 things you need to do:

1. Send out emails requesting your current users or waiting list to participate in the test.
2. Determine what questions you would like to ask your users during and after the test.
3. Ensure that the testing process is short and simple.
4. Take notes or audio/video recordings of all the feedback you receive from your users.
5. Extract the most insightful information from users’ feedback to be shared with the Engineering team.
6. Create a form of reward system for users who participated in the test.