In this next instalment article from the RPA Implementation Lifecycle series, continuing our focus on the Testing phase, we will discuss about Non-functional testing needs that are necessary to be met in RPA implementation projects.
Non-Functional Testing Types applicable to RPA Robots
Non-functional testing of RPA robots is largely driven by the non-functional requirements that are important to define for a software robot as part of the requirements ‘gather’ activities. This is critical to ensure the resilience of a robot as it is operationalised, and to minimise the risk of any financial losses or reputation due to robot breaking down in production impacting the business continuity of an automated business process.
There are essentially three different kinds of non-functional testing to be performed on RPA robots and these are explained below along with the sub-levels involved in each of them.
The following performance testing types are to be carried out to ensure that the robot continues to perform within an acceptable level under varied load conditions – i.e. normal and peak loads.
Load testing is to confirm that a software robot continues to perform within acceptable execution timescales under normal load conditions. Normal load conditions it may imply that the number of transactions to be executed are within a specified threshold, health of the applications on which the robot operates is normal, robot execution configuration and hardware requirements are met. Load testing allows to ensure that the average transaction time and throughput that the robot can achieve. For instance, the number of transactions it can process in a day is within acceptable limits.
Stress testing allows us to establish the level of stressed or peak load conditions under which a software robot can continue to perform in an acceptable manner, and beyond which its performance starts to degrade resulting in execution failures. Stressed conditions can be achieved by increasing the number of concurrent execution threads and reducing the ramp-up time to reach the maximum number of threads hitting against target applications for transaction processing, and as well as the number of transactions per thread to execute. Given RPA robots predominantly uses UI of desktop applications, by ramping up robot threads and transactions per thread, robot execution limits can be established to see when target application becomes unresponsive or their response time slows down significantly to start causing robot failures.
Soak testing allows to confirm that a software robot who performs well under normal load conditions can continue to do so in case that the load conditions are elongated for an extended period of time. This can be achieved by making the robot execute a much larger number of back to back transactions in a day than normal, or before the resources for the robot are recycled, to ensure that the robot can continue to operate in an acceptable manner.
Security testing is one of the most important testing activities to minimise any cybersecurity risks while using software robots for production use. It is required to achieve at least the following as part of security testing.
Penetration testing and vulnerability assessment of individual cybersecurity issues
This is done by simulating real-world security attacks and influence a robot as a piece of software the way an attacker would do. The purpose of this testing is to identify any security vulnerabilities that can be exploited by a hacker. It is good practice to cover the most critical security risks such as OWASP top 10 to be applied and tested by an expert security testing professional internally or by an external certified pen tester.
Security Monitoring and Alerting
This is done on an ongoing basis by having the right level of logging and alerting capabilities set up in the first place within the software robot, and then by collecting, managing & analysing the logs to understand what has happened in case there has been a security incident.
Operational Acceptance Testing
The purpose of OAT is to ensure –
- Resilience of the robot to recover from any component failovers
- Ability to trigger alerts that are put in place in case of failures when specific thresholds have been met
- In case a specific system or application component is temporarily down or is unresponsive due to network issues, to ensure that the robot can continue to provide an acceptable level of operational capability while the remainder of the system or application/s still functions correctly by doing one or more of the following –
- Continue to execute a business process partially
- Restart the system or an application instance which is faulty
- Notify operations team in case the robot is unable to continue to operate so that necessary business continuity plans can be triggered
Read the other articles from this RPA Implementation Lifecycle series from here:
RPA Implementation Lifecycle
RPA Plan Phase- Benefit Assessment
RPA Plan Phase- Process Identification
RPA Design Phase- Attended or Unattended Robots
RPA Design Phase – UI or API
RPA Design Phase – Solution Architecture
RPA Developmental Phase- UI Object Selectors
RPA Development Phase- Development Framework Approach
RPA Testing Phase – Functional Testing of Low Code RPA Robots
Contact us at SQA Consulting, to see how we may assist you in developing the necessary skills needed for implementing RPA projects.