Analyze Results, Report, and Retest in Core Performance Testing

Final procedure in Performance Testing is to Analyze Results, Report, and Retest

Consider the following important points while analyzing the data returned by your performance test:

• Analyze the captured data and compare the results against the metric’s acceptable or expected level to determine whether the performance of the application being tested shows a trend toward or away from the performance objectives.
• If all of the metric values are within accepted limits and none of the set thresholds have been violated, the tests have passed and you have finished testing that particular scenario on that particular configuration.
• If the test fails, a diagnosis and tuning activity are generally warranted. {Explained: The Performance Tester’s Role in Diagnosis and Tuning}
• If you fix any bottlenecks, repeat the testing process from step 4 onwards until the test succeeds.
• Performance testing should produce meaningful performance tests mapping to the real world requirements
• Performance testing results should provide analysis to go deep into components and correlate back to the real world. Example performance testing contribution defined 2x orders increase with same infrastructure
• Performance testing should deliver informed architecture and business decisions

Note: If required, capture additional metrics in subsequent test cycles. For example, suppose that during the first iteration of load tests the process shows a marked increase in memory consumption, indicating a possible memory leak. In subsequent test iterations, additional memory counters related to generations can be captured, allowing you to study the memory allocation pattern for the application.

Considerations

• Immediately share test results and make raw data available to your entirel team.
• Talk to the consumers of the data to validate that the test achieved the desired results and that the data means what you think it means. Modify the test to get new, better, or different information now while the test objectives are fresh in your mind, especially if the results do not represent what the test was defined to determine.
• Filter out any unnecessary data. For example, if 87 of the 100 pages tested failed to meet their target response time, remove them from the graph to avoid redundancy.
• Make your report using strong but factual language, with supporting paragraphs and graphics if available. For example, “The home page fails to achieve target response time for 75% of the users.”
• Make your report using business language, not simply technical data. For example, do no say, “The CPU utilization is hovering at 85%”; rather, say, “The application server is not powerful enough to support the target load as the application is currently deployed/operating.”
• Use current results to set priorities for the next test.
• After each test, tell the team what you expect the next two tests to be so the team members can provide input concerning what they would like to have tested next while you are executing the current test.
• Always keep supporting data handy and deliver it in the Appendix.

Related Topics:

• Configuring Test Environment in Performance Testing
• Plan and Design Tests
• Identifying Performance Acceptance Criteria
• How to Identify Test Environment in web application
• Implement Test Design in Performance Testing
• How to Execute Core Performance Testing