Performance Risk Assessment – Risk Mitigation

Risk of a system or an application failing when released can be mitigated by properly testing it before launch and there are various strategies which this article looks at that can be employed to achieve this. It should be noted that there are risk factors that are present in testing that should be mitigated so that testing can be completed on time and within budget. The level of risk can be measured by multiplying the probability of the risk occurring by the impact of the risk if it occurred. By doing this you can come up with a classification for each risk identified as follows.

As can be seen, each risk can be assessed by the likelihood of it occurring and the impact it will have and can be treated differently in the testing phase in terms of time and effort spent on them

• Risk 1 – Low Impact Low Probability can potentially be ignored as there is not much value trying to mitigate these
• Risk 2 – Low Impact High Probability should be monitored to ensure risk remains low but not much effort and time should be spent to mitigate it
• Risk 3 – High Impact Low Probability should have a strategy to mitigate this risk
• Risk 4 – High Impact High Probability should have the most time and effort spent to minimize this risk

In performance testing, the following strategies can be applied so that risks can be mitigated.

1. Test requirements are agreed and signed off
2. Realistic business processes are identified and tested i.e. by interrogating the application log files and defining a performance model. This way transactions that are most frequent or load intensive are tested
3. SLAs are defined, and tests measured against them
4. Different types of load testing are performed i.e. normal expected load, peak hour etc. and comparisons made
5. Tests are repeated so that transaction response times and system behaviour are compared, and any differences investigated
6. Test results are compared to earlier releases and the differences are investigated
7. Test data setup is varied so that tests are realistic
8. The data volumes in the test environment are production like for realistic testing
9. Test environments are stable and production like for performance testing. If test environments are not comparable to production, then modelling needs be done to extrapolate the results
10. The application is thoroughly functionally tested before doing performance testing so that delays to test cycles can be minimized due to unstable and defective applications
11. Start the performance testing as soon as possible in order to identify any bottlenecks early, allowing time for remediation and retesting
12. Perform Stress testing to ensure that the system is capable of more than just the expected load
13. Perform Soak testing to ensure that the system is stable over a long period time and any memory leaks identified in the infrastructure
14. Failure scenarios are tested so that system stability can be assessed when servers or application instances fail
15. Check that the application has updated the system as expected e.g. database and log file updates
16. Identify the load at which the SLAs fail
17. Testing is performed during different times of the day such as overnight batch runs to gauge any difference in system behaviour
18. Monitor servers under test in terms of CPU, memory and correlate with response time statistics and load levels during tests to understand system behaviour and potential issues and bottlenecks
19. If test environments fail or have issues, are there alternatives where testing can be done
20. Ensure you have got the correct resources with the relevant skills to complete the testing
21. Ensure you have plans in place in case of system failures in production
22. Ensure you have high-level sponsorship of the project in order, to allow rapid escalation of any issues
23. Arrange support from the development, operational support, and database teams
24. Agree the performance test approach with the 3^rd party suppliers and include them in the results review process

In conclusion, by implementing the above strategies you can mitigate the risks that you have identified on a Performance testing project.

To see how SQA Consulting may assist your company in performance testing your applications, please contact us.