Tests Coverage

As a tester you have a different way to think about the scenarios. You know that you need to think beyond the scenarios. So, how do you know when it will be enough ? When will you have 100% of tests coverage ?

You probably already found a bug out of the requirements, and in a specific sequence of steps. I normally find these kind of bugs with exploratory tests, when I have time to free my creative side and start to do different ways to test the same thing. Developers follow the requirements, they don’t do exploratory tests, usually they think even the user has the possibility to do the same step in a different way, they shouldn’t (Because it’s not the right way).

In my humble opinion, if your software allow to do the same function in 1000 different ways, you should be prepare to test every single “invalid” way, because this will increase the trust in your software. If I find a single stupid bug in an application, like an error when I send invalid characters, I start to think what type of software was delivered, like neither the basic simple stupid scenario of invalid characters was tested, imagine the more complex ones… This could be low priority, but if you ignore this, you need to face there are many people like me (critical detail vision), that see these kind of things and lose the confidence on the software and to be honest, the respect too.

Imagine you have all the requirements:

And you have the system software in this another circle:

But in the real world, we don’t have every requirement covered by the system, we have something like this:

Which means you will have some parts in your system not covered by your requirements and you have some parts in your requirements not covered by your system. It is exactly in this part that we, testers should start to think about. We need knowledge of both of the parts, and this takes time.

So, you don’t need to worry cover 100% of your tests in the beginning, you need to worry if you know everything about what you are testing, some scenarios you just figure out when you are testing, because you are pretending you are an user. You need a good background, someone to sit next to you, or some good documentation about what you will test, spend some time exploring the app before you start the scenarios. This will create your first impression of the software and you will be more into it and the user experience.

Finally, my advice to know if you have a good test coverage is:

• Exploratory tests, this will help you to find unknown scenarios between system and requirements. This is a type of test you can’t automate, it involves more about your creativity than objective steps. Sometimes is just a different sequence that you do and you can find a critical bug.
• Kick-off requirements, this is another thing that helps to reduce the unknown scenarios, like if you have an explanation about what a new function will do, you can raise and think in points which you already know and maybe nobody thought yet. As I said before, it’s better if you have a good background about what is coming.
• System flow, the last key is try to understand the gaps and the flow of the software, like what is the flow to a function update something in database. It seems very technical, but this will help you to think about scenarios that might crash when you do in a different sequence or if you do many times, or if you don’t wait a specific time, this is quality assurance 🙂

See you next week !