Executing Core Performance Testing

Execute Test in Core Performance Testing

After the Implementation of Test Design have been completed to an appropriate degree, for the test you want to execute, it is worth the time to double check that the following items have been accomplished before starting a formal test execution:

• Validate that the test environment matches the configuration that you were expecting andte/or designed your test for.
• Ensure that both the test and the test environment are correctly configured for metrics collection.
• Before running the real test, execute a quick “smoke test” to make sure that the test script and remote performance counters are working correctly.
• Reset the system (unless your scenario is to do otherwise) and start a formal test execution.
• Make sure test scripts execution are representing the workload model, For example check for number of orders posted, searches and other business metrics
• Make sure the load test containing test scripts, is collecting key performance indicators and business indicators

Considerations

• If at all possible, execute every test twice. If the results produced are not very similar, execute the test again. Try to determine what factors account for the difference.
• Observe your test during execution and pay close attention any behavior you feel is unusual. Your instincts are usually right, or at least valuable.
• No matter how far in advance a test is scheduled, give the team 30-minute and 5-minute warnings before launching the test (or starting the day’s testing) if you are using a shared test environment. Inform the team whenever you are not going to be executing for more than 1 hour in succession.
• Do not process data, write reports, or draw diagrams on your load-generating machine while generating a load because this can skew the results of your test.
• Turn off any active virus-scanning on load-generating machines during testing to minimize the likelihood of unintentionally skew the results of your test.
• Use the system manually during test execution so that you can compare your observations with the results data at a later time.
• Remember to simulate ramp-up and cool-down periods appropriately.
• Do not throw away the first iteration because of application script compilation, web server cache building or other similar reasons. Instead, measure this iteration separately so you will know what the first user after a system-wide reboot can expect.
• Test execution is never really finished, but eventually you will reach a point of diminishing returns on a particular test. When you stop obtaining valuable information, change your test.
• If neither you feel like you are not making progress in understanding an observed issue, it may be more efficient to eliminate one or more variables/potential causes and run the test again.

Related Topics:

• Configuring Test Environment in Performance Testing
• Plan and Design Tests
• Identifying Performance Acceptance Criteria
• How to Identify Test Environment in web application
• Implement Test Design in Performance Testing