Performance & Load Testing

Mind Map Summary

• Topic: Performance & Load Testing
• Definition: A type of non-functional testing that evaluates how a system performs in terms of responsiveness, stability, scalability, and resource usage under a particular workload.
• Key Concepts:
  • Performance Testing: A broad term encompassing various tests to determine a system’s speed, responsiveness, and stability under a particular workload.
  • Load Testing: Assessing system behavior under anticipated peak load. It helps determine if the infrastructure can handle the expected number of users or transactions.
  • Stress Testing: Pushing the system beyond its normal operational limits to identify its breaking point and how it recovers from extreme conditions.
  • Scalability Testing: Determining the system’s ability to scale up or down to meet increasing or decreasing user demands.
  • Soak Testing (Endurance Testing): Testing the system under a typical load for a prolonged period to detect memory leaks or degradation over time.
  • Response Time: The time taken for the system to respond to a user request.
  • Throughput: The number of transactions or requests processed by the system per unit of time.
  • Error Rate: The percentage of requests that result in an error.
  • Concurrency: The number of simultaneous users or requests the system can handle.
• Benefits (Pros):
  • Identifies Bottlenecks: Pinpoints areas in the application or infrastructure that are causing performance degradation.
  • Ensures Scalability: Verifies that the application can handle increased user loads without compromising performance.
  • Improves User Experience: Ensures the application remains responsive and stable, leading to better user satisfaction.
  • Reduces Risk: Prevents system failures, crashes, or slowdowns in production under high traffic.
  • Optimizes Resource Usage: Helps in right-sizing infrastructure and optimizing code for better resource utilization.
• Challenges (Cons):
  • Complexity: Can be complex to design, set up, and execute, requiring specialized skills.
  • Tooling: Requires specific tools, which might have a learning curve or licensing costs.
  • Environment Setup: Needs a test environment that closely mimics production, which can be challenging to provision and maintain.
  • Result Interpretation: Analyzing and interpreting performance metrics to identify root causes can be difficult.
  • Cost: Can be resource-intensive in terms of infrastructure and personnel.
• Common Tools:
  • k6: Modern, open-source load testing tool written in Go, scriptable with JavaScript.
  • JMeter: Apache’s open-source, Java-based load testing tool, widely used for various protocols.
  • Azure Load Testing: A fully managed load testing service on Azure.
  • Locust: Open-source, Python-based load testing tool, allowing you to define user behavior in Python code.
  • Gatling: Open-source load testing tool based on Scala, Akka, and Netty.
• Practical Use:
  • Simulating realistic user traffic patterns.
  • Measuring system performance under various load conditions.
  • Identifying the maximum user capacity before degradation.
  • Validating service level agreements (SLAs).

Core Concepts

Performance and load testing are crucial for ensuring that applications can handle the expected user traffic and perform reliably under various conditions. Unlike functional tests that verify what the system does, performance tests verify how well the system performs.

Load testing specifically focuses on simulating anticipated user loads to ensure the system remains stable and responsive. Stress testing goes a step further, pushing the system beyond its limits to find its breaking point and observe how it behaves under extreme conditions.

Tools like k6 provide a modern, scriptable approach to defining load test scenarios, allowing developers to integrate performance testing into their CI/CD pipelines.

Practice Exercise

Using a tool like k6, write a simple load test script. The script should simulate 10 virtual users continuously hitting a specific API endpoint for 30 seconds. Run the test and analyze the results, such as average response time and request rate.

Answer

For this exercise, we’ll use k6.

1. Install k6

If you don’t have k6 installed, you can download it from the official website (k6.io) or use a package manager:

Windows (with Winget):

winget install k6

macOS (with Homebrew):

brew install k6

Linux (Debian/Ubuntu):

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 379CE192D401AB61
echo "deb https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
sudo apt update
sudo apt install k6

2. Create a k6 Test Script (test.js)

Let’s assume you have an API endpoint at http://localhost:5000/api/products.

// test.js
import http from 'k6/http';
import { sleep } from 'k6';

export const options = {
  // Simulate 10 virtual users (VUs)
  vus: 10,
  // Run the test for 30 seconds
  duration: '30s',
  // Define thresholds for pass/fail criteria (optional but recommended)
  thresholds: {
    // 95% of requests must complete within 200ms
    http_req_duration: ['p(95)<200'],
    // The error rate must be below 1%
    http_req_failed: ['rate<0.01'],
  },
};

export default function () {
  // Make a GET request to the API endpoint
  http.get('http://localhost:5000/api/products');

  // Pause for a short duration to simulate user think time
  sleep(1);
}

3. Run the Test

Open your terminal or command prompt, navigate to the directory where you saved test.js, and run the test:

k6 run test.js

4. Analyze the Results

After the test completes, k6 will output a summary to the console. Here’s an example of what you might see and how to interpret it:

          /\      |‾‾| /‾‾/   /\
     /\  /  \     |  |/  /   /  \
    /  \/    \    |   /  /  /    \
   /          \   |  /  /  /      \
  / __________ \  | /  /  /________\
 /_____________\]  |/  /_____________ 

     running (30.0s), 10 VUs, 300 complete and 0 interrupted iterations
     ✓ http_req_duration..............: avg=15.23ms  min=10.12ms med=14.56ms max=35.89ms p(90)=18.01ms p(95)=19.50ms
       http_req_failed..............: 0.00%   ✓ 0        ✗ 300
       http_req_waiting.............: avg=15.23ms  min=10.12ms med=14.56ms max=35.89ms p(90)=18.01ms p(95)=19.50ms
       http_reqs....................: 300    9.999999/s
       iteration_duration...........: avg=1015.23ms min=1010.12ms med=1014.56ms max=1035.89ms p(90)=1018.01ms p(95)=1019.50ms
       iterations...................: 300    9.999999/s
       vus..........................: 10     min=10     max=10
       vus_max......................: 10     min=10     max=10

Key Metrics to Analyze:

• http_req_duration: This is the most important metric. It shows the total time for the HTTP request, including DNS lookup, TCP connection, TLS handshake, and response receive time.
  • avg: Average response time (e.g., 15.23ms).
  • min, med, max: Minimum, median, and maximum response times.
  • p(90), p(95): 90th and 95th percentile response times. This means 90% (or 95%) of requests completed within this time. These are crucial for understanding user experience.
• http_req_failed: The percentage of failed HTTP requests. In this example, 0.00% indicates no failures. Our threshold rate<0.01 (less than 1% error rate) would pass.
• http_reqs: Total number of HTTP requests made during the test (e.g., 300).
• http_reqs / s: Requests per second, or throughput (e.g., 9.999999/s). This indicates how many requests the system can handle per second.
• iterations: Total number of times the default function (your test scenario) was executed.
• vus: Virtual Users. Shows the number of active virtual users during the test.

Analysis for this example:

• The average response time is very low (15.23ms), which is excellent.
• 95% of requests completed within 19.50ms, which is well within our 200ms threshold.
• There were no failed requests (0.00% error rate).
• The system handled approximately 10 requests per second.

This indicates that the API endpoint performed very well under the simulated load of 10 concurrent users for 30 seconds. If the http_req_duration percentiles were higher than expected, or the http_req_failed rate was significant, it would indicate a performance bottleneck or stability issue that needs investigation.