Risk-based testing

Within LeaseWeb there is a lot of testing done on the software the customers use. Because it is impossible to test everything, we use a risk based strategy to focus on the most important parts.

Professionals in the IT know it: You can’t test everything. Even a simple web page with one edit box can be tested and looked at from many perspectives. Capitals, normal characters, strange characters, numbers, negative numbers, leave it empty, big arrays of texts, using other fonts, …. Just think about it and you will have a big checklist when you’re through.

What should happen then, if you push the [Submit] button? Is the data from the edit box placed in a database or a text file? How does the database look? Should the data be saved secure or is a text file enough? If it is saved in a text file, how to deal with strange characters. Or is that of no importance to the customer? Is it maybe more important that a textbox pops up with the text in it? Or should it be send to another device via mail or another communication protocol. And if it is sent to another device, what about security of the communication channel. Encryption? Is that important? Should we get feedback if something is sent?

If you don’t know what’s important for the user, on what part of the software should you focus? Security of saving or sending data, or should you focus on functionality and usability of the software?

Mostly we would only have a limited time for testing. So we will have to have a approach to focus testing on our software.

Change of failure and damage

To get focus on only the parts that probably have a big risk in failing or causing a lot of damage when it gets in production. This means talking to the right persons to discuss the risks and the priority of those risks.

No damage or low damage when something goes wrong? Little or no testing. Things could go wrong, but only once a year? But has big consequences and thus big damage. Lots of testing there!

So doing a risk analysis is getting focus on the parts that matter for the people that will use the software or otherwise are involved in this software (the stakeholders).

Risks have a chance of failure and height of damage

The technical people (developers, designers, testers) often can estimate the chance of failure. The damage for the business is better estimated by the business people (sales, support, and management etc).

Classification of risks

  • Risk = Failure Probability X Damage / R = F X D
    • Risk: The chance that the product will fail in relation to the expected damage
    • Failure probability: The chance that a system or component will fail within specified circumstances or within a specific time period
    • Damage: Prejudice, loss, damage, injury, money lost

Format of risks

About the format: I would expect a risk to have a cause and something happening because of that.

Examples:

–       Because the user doesn’t know how to select the right product, he will order something he doesn’t want

–       If more than 1000 users will try to order something, the website will be too slow to use.

Examples of aspects of failure

  • Frequency of use
  • Complicated or unknown product
  • New tooling
  • Development of product is taken over from others
  • Time pressure
  • Functions in subsystems are complex
  • Functions that have been changed a lot

Examples of damage

  • Loss of image of company or organization
  • Costs of correction actions, also extra documentation
  • Employees cannot do their work
  • Clients can’t be supported
  • The business processes are less efficient (or not efficient)
  • There is arising a negative image of the ICT organization

Conclusion

With this approach the most risky parts of the software are identified and a developer, functional designer and the tester can use this as a checklist for doing the quality part of the job.

In a next blog some tips for doing product risks sessions yourself if you like this idea.

Share

Leave a Reply

Your email address will not be published. Required fields are marked *