Risk of a system or an application failing when released can be mitigated by properly testing it before launch and there are various strategies which this article looks at that can be employed to achieve this. It should be noted that there are risk factors that are present in testing that should be mitigated so that testing can be completed on time and within budget. The level of risk can be measured by multiplying the probability of the risk occurring by the impact of the risk if it occurred. By doing this you can come up with a classification for each risk identified as follows.
As can be seen, each risk can be assessed by the likelihood of it occurring and the impact it will have and can be treated differently in the testing phase in terms of time and effort spent on them
Risk 1 – Low Impact Low Probability can potentially be ignored as there is not much value trying to mitigate these
Risk 2 – Low Impact High Probability should be monitored to ensure risk remains low but not much effort and time should be spent to mitigate it
Risk 3 – High Impact Low Probability should have a strategy to mitigate this risk
Risk 4 – High Impact High Probability should have the most time and effort spent to minimize this risk
In performance testing, the following strategies can be applied so that risks can be mitigated.
Test requirements are agreed and signed off
Realistic business processes are identified and tested i.e. by interrogating the application log files and defining a performance model. This way transactions that are most frequent or load intensive are tested
SLAs are defined, and tests measured against them
Different types of load testing are performed i.e. normal expected load, peak hour etc. and comparisons made
Tests are repeated so that transaction response times and system behaviour are compared, and any differences investigated
Test results are compared to earlier releases and the differences are investigated
Test data setup is varied so that tests are realistic
The data volumes in the test environment are production like for realistic testing
Test environments are stable and production like for performance testing. If test environments are not comparable to production, then modelling needs be done to extrapolate the results
The application is thoroughly functionally tested before doing performance testing so that delays to test cycles can be minimized due to unstable and defective applications
Start the performance testing as soon as possible in order to identify any bottlenecks early, allowing time for remediation and retesting
Perform Stress testing to ensure that the system is capable of more than just the expected load
Perform Soak testing to ensure that the system is stable over a long period time and any memory leaks identified in the infrastructure
Failure scenarios are tested so that system stability can be assessed when servers or application instances fail
Check that the application has updated the system as expected e.g. database and log file updates
Identify the load at which the SLAs fail
Testing is performed during different times of the day such as overnight batch runs to gauge any difference in system behaviour
Monitor servers under test in terms of CPU, memory and correlate with response time statistics and load levels during tests to understand system behaviour and potential issues and bottlenecks
If test environments fail or have issues, are there alternatives where testing can be done
Ensure you have got the correct resources with the relevant skills to complete the testing
Ensure you have plans in place in case of system failures in production
Ensure you have high-level sponsorship of the project in order, to allow rapid escalation of any issues
Arrange support from the development, operational support, and database teams
Agree the performance test approach with the 3rd party suppliers and include them in the results review process
In conclusion, by implementing the above strategies you can mitigate the risks that you have identified on a Performance testing project.
To see how SQA Consulting may assist your company in performance testing your applications, pleasecontact us.
