Continuous Performance Testing

Continuous Performance Testing is the process of executing performance tests, scripts as part of the software delivery to obtain immediate feedback on the business risks and detecting performance issues quickly and locating them more easily.

Today’s fast development pace demands continuous performance testing is an integral part of continuous delivery.

For Continuous Performance Testing, the scope of testing extends from validating requirements to executing test scripts.

This is more helpful for product based companies as well as in agile development where you get builds and baselines more frequently.

Following is the process for Continuous Performance Testing:

1. Verify and understand the current business flow with stake holders. It will help you to Create / modify scripts accordingly.
2. Check the deployed build:
   • Verify deployed build, baseline and check if there is any change prior to earlier build, base-line.
3. Perform the sanity testing for each script. Email current working scripts status to stakeholder; it helps to take decision for performance schedule and which tests need to be run.
4. Create performance test data if required.
5. Database plays crucial role in performance testing, before starting any performance tests always take snapshot of DB. This will be helpful for repeating test scripts. One more benefit for snapshot is that you will get consistent results every time.
6. In the deployed application, for generating the log we may need to do some modification in configuration files, thus find out any simplest way to do this. Create checklist for how to take generate logs and revert that setting after completing the test.
7. Start all the performance monitors that are required to monitor/ captured data for further performance analysis.
8. Before starting performance testing start application manually.
9. Email performance test results to respective stake holder once the test is completed.

Testing types that will help you in continuous performance testing:

• Benchmark testing: This test is carried out with a nominal user load in order to benchmark the response time of all transactions against previous build benchmark tests.
  
  
• Load testing: This test is carried out with inflated user loads to check for performance bottlenecks in the application build.
  
  
• Continuous testing: In this test each script is run in isolation for 30 minutes with a user load of nominal users. Purpose of the test is to isolate the performance issues / bottlenecks based upon the scenario under the test which may further add value to the performance tuning.
  
  

  E.g. If we have total 15 scripts then schedule will look like:

  Run first script for 1 hour then take break for half an hour then start next script.
  
  

Today, Continuous Performance Testing has become essential and more advanced.  In Continuous performance testing we are time bound, so create checklist and setup standard process for each activity.