It is essential in a DevOps organisation that performance engineering becomes embedded within the processes. This article attempts to identify the core components of a performance engineering approach in a DevOps environment.
What is DevOps?
According to Atlassian “DevOps is a set of practices that works to automate and integrate the processes between software development and IT teams, so they can build, test, and release software, faster and more reliably. The term DevOps was formed by combining the words “development” and “operations” and signifies a cultural shift that bridges the gap between development and operation teams, which historically functioned in siloes.”
Why Move to DevOps?
Historically, a major issue in developing a software product/change has been the length of time it takes to bring it to market. Arguably? this issue has been caused by the processes being built around the software lifecycle.
In the past, for example, a typical project would be carried out in the following way:
- Design – Designers/Architects create and pass their designs on to the developers.
- Code – Developers write code based on the business requirements.
- Test – Testers (functional and non-functional) test the code passing any issues identified back to the developers to rectify. The testing cycle would then be repeated before the accepted code was ultimately released to the operations team.
- Deploy – Operation teams would then schedule and deploy the code into pre-production and production.
Approaching a project using the typical steps described above could inevitably present difficulties such as:
- Lengthy test cycles – generally accepted as a necessity to mitigate risk and improve quality.
- Accommodating changes to requirements – labour and time-intensive.
- Performance issues being discovered at later phases – more costly to fix.
Introducing a DevOps approach into an organisation can provide the following benefits:
- Continuous delivery of workable software.
- Flexibility in terms of the scope and requirements.
- Better End-user experience.
- Improved delivery time to market.
- Efficient and effective operations.
Performance Challenges with DevOps
Performance Testing in a DevOps environment could present the following challenges:
- Stringent Timelines.
- Scoping (what, when and where).
- Constant change.
- Ensuring Coverage.
DevOps Performance Engineering Approach
The most difficult part of the process is to build a performance engineering approach which will put and embed the processes, practices and tooling in place to ensure that change is delivered without introducing performance risk.
By adopting a combined “shift left” and “shift right” performance engineering approach, this can mitigate performance risks and cater for the above challenges in a DevOps environment.
The core components of implementing a Performance Engineering approach should consist of the following:
- Defining the tooling to be used.
- Defining performance requirements – Performance NFRs need to be reviewed and updated constantly.
- Undertaking a Performance risk assessment at the same time as planning and estimating. – In other words, as soon as the team start considering items in the backlog, they should concurrently start considering the risks to performance.
- Implementing continuous performance testing by integrating performance tests into the CI/CD process. Continuous performance testing means automating the whole process, including setting up the test data and the environment, running tests, and reporting and analysing results i.e. continuous delivery requires continuous performance.
- Defining the Reporting to be used.
- Defining data requirements – Refer to our article on data Data Considerations for Performance Testing.
- Performance Design – Review the design. The first step in a performance design is to review the implementation to make best use of the available resources, given goals, and expected load.
- Coding with performance in mind.
- Unit Testing – Assesses performance at a unit or profiling level.
- Service level Testing – focuses on the performance of an individual service.
- Product performance component testing – focuses on the performance of the entire product.
- Integration Performance testing – focuses on the performance from an integration point of view. This needs to be performed in a fully integrated environment refer to our article on Test Environment-Considerations for Performance Testing.
- Establishing a Performance Testing Review/Feedback Loop – critical for successful DevOps implementation.
- Perform a health check review – comparing current performance to past baselines and NFRs.
- Performance model validation – validating the performance model used for designing the performance tests. Refer to our article on Performance Modelling – Creating an accurate performance model.
- Testing in production – repeating performance testing in production.
- Defining the roles and responsibilities required e.g. resources with technical skills around scripting and automation, as well as business skills in creating and embedding processes within an organisation.
- Defining the key performance metrics to be measured/collected. Refer to our article on Key Performance Metrics.
- Automating performance monitoring by using performance monitoring tools and integrating them into the CI/CD process i.e. Performance monitoring, including real-user monitoring (RUM) – Refer to our article on A High-Level Insight into Performance Monitoring.
- Removing performance bottlenecks by continuously applying root cause analysis of performance issues found for each build of software.
In conclusion, by creating and implementing a Performance Engineering approach with the above components is key to the success of performance testing within a DevOps organisation.
To see how SQA Consulting can help and assist your company to validate performance in a DevOps environment, please contact us.
