Performance Tests with Artillery

Hello guys, after a long break I am posting about a previous project where I created some performance tests to check the reliability of the server. Artillery is a npm library that helps you with load tests, is very simple to use and the scripts are written in .yml, so make sure the indentation is right.

So in the load-tests.yml file you will find this script:

  config:
    target: 'https://YOUR-HOST-HERE' //Here you need add your host url
    processor: "helpers/pre-request.js" //This is the pre-request function we are using to create the data
    timeout: 3 // What is the timeout for each request, it is going to stop the flow and tag the scenario as a failure
    ensure:
      p95: 1000 // Force artillery to exit with a non-zero code when a condition is not met, useful for CI/CD
    plugins:
      expect: {}
    environments:
      qa:
        target: "https://YOUR-HOST-HERE-QA-ENV" //Here you need add your QA env url
        phases:
          - duration: 600 //Duration of the test, in this case 10 minutes
            arrivalRate: 2 //Create 2 virtual users every second for 10 minutes
            name: "Sustained max load 2/second" //Run performance tests creating 2 users/second for 10 minutes
      dev:
        target: "https://YOUR-HOST-HERE-DEV-ENV" //Here you need add your Dev env url
        phases:
          - duration: 120
            arrivalRate: 0
            rampTo: 10 //Ramp up from 0 to 10 users with constant arrival rate over 2 minutes
            name: "Warm up the application"
          - duration: 3600
            arrivalCount: 10 //Fixed count of 10 arrivals (approximately 1 every 6 seconds):
            name: "Sustained max load 10 every 6 seconds for 1 hour"
    defaults:
      headers:
        content-type: "application/json" //Default headers needed to send the requests
  scenarios:
    - name: "Send User Data"
      flow:
      - function: "generateRandomData" //Function that we are using to create the random data
      - post:
          headers:
            uuid: "{{ uuid }}" //Variable with value set from generateRandomData function
          url: "/PATH-HERE"//Path of your request 
          json:
            name: "{{ name }}"
          expect:
            - statusCode: 200 //Assertions, in this case we are asserting only the status code
      - log: "Sent name: {{ name }} request to /PATH-HERE"
      - think: 30 //Wait 30 seconds before running next request
      - post:
          headers:
            uuid: "{{ uuid }}"
          url: "/PATH-HERE"
          json:
            name: "{{ mobile }}"
          expect:
            - statusCode: 200
- log: "Sent mobile: {{ mobile }} request to /PATH-HERE"

 

Now, for the function that creates the data you have a Faker library, that you need to install in your package with npm, then you need to export this function. You need to make the variables available using the userContext.vars and remember to always accept the parameters: userContext, events and done, so they can be used in the artillery scripts.

const Faker = require('faker')

module.exports = {
  generateRandomData
}

function generateRandomData (userContext, events, done) {
  userContext.vars.name = `${Faker.name.firstName()} ${Faker.name.lastName()} PerformanceTests`
  userContext.vars.mobile = `+44 0 ${Faker.random.number({min: 1000000, max: 9999999})}`
  userContext.vars.uuid = Faker.random.uuid()
  userContext.vars.email = Faker.internet.email()
  return done()
}

 

This is just an example, but you can see how powerful and simple artillery is on their website.

You can see the entire project with the endurance and load scripts here: https://github.com/rafaelaazevedo/artilleryExamples

See you guys !

Webinar Jmeter Pipeline – Improving the performance

Hi guys, just sharing a cool webinar that I watched this week about performance tests, but what I thought really interesting is the demo with Taurus and Jmeter.

The performance tests with Taurus look way more readable and simple than when creating on Jmeter.

Common questions about Performance Tests

 

When do I need to create a Performance Test ?

To validate the behavior of the system at various load conditions performance testing is done. So you can reproduce several user performs for desired operations Customer, Tester, Developer, DBA and N/W management team checking the behavior of the system. It requires close to production test environment and several H/W facilities to populate the load.

What all thing involves in Performance Testing Process?

    • Right testing environment: Figure out the physical test environment before carry performance testing, like hardware, software and network configuration
    • Identify the performance acceptance criteria: It contains constraints and goals for throughput, response times and resource allocation
    • Plan and design Performance tests: Define how usage is likely to vary among end users, and find key scenarios to test for all possible use cases
    • Test environment configuration: Before the execution, prepare the testing environment and arranges tools, other resources, etc.
    • Test design implementation: According to your test design, create a performance test
    • Run the tests: Execute and monitor the tests
    • Analyze, tune and retest: Analyze, consolidate and share test results. After that, fine tune and test again to see if there is any enhancement in performance. Stop the test, if CPU is causing bottlenecking.

What parameters should I consider for performance testing?

    • Memory usage
    • Processor usage
    • Bandwidth
    • Memory pages
    • Network output queue length
    • Response time
    • CPU interruption per second
    • Committed memory
    • Thread counts
    • Top waits, etc.

What are the different types of performance testing?

    • Load testing
    • Stress testing
    • Endurance testing
    • Spike testing
    • Volume testing
    • Scalability testing

Endurance vs Spike

    • Endurance Testing: It is one type of performance testing where the testing is conducted to evaluate the behavior of the system when a significant workload is given continuously
    • Spike Testing: It is also a type of performance testing that is performed to analyze the behavior of the system when the load is increased substantially.

How you can execute spike testing in JMeter?

In JMeter, spike testing can be done by using Synchronizing Timer.  The threads are jammed by synchronizing the timer until a specific number of threads have been blocked and then release at once, creating a large instantaneous load.

What is concurrent user hits in load testing?

In load testing, without any time difference when multiple users hit on the same event of an application under the load test is called a concurrent user hit.

What are the common mistakes done in Performance Testing?

    • Direct jump to multi-user tests
    • Test results not validated
    • Unknown workload details
    • Too small run durations
    • Lacking long duration sustainability test
    • Confusion on definition of concurrent users
    • Data not populated sufficiently
    • Significant difference between test and production environment
    • Network bandwidth not simulated
    • Underestimating performance testing schedules
    • Incorrect extrapolation of pilots
    • Inappropriate base-lining of configurations

What is the throughput in Performance Testing?

In performance testing, throughput is referred to the amount of data transported to the server in responds to the client request at a given period of time. It is calculated in terms of requests per second, calls per day, reports per year, hits per second, etc. Performance of application depends on throughput value, higher the value of throughput -higher the performance of the application.

What are the common performance bottlenecks?

    • CPU Utilization
    • Memory Utilization
    • Networking Utilization
    • S limitation
    • Disk Usage

What are the common performance problem does user face?

    • Longer loading time
    • Poor response time
    • Poor Scalability
    • Bottlenecking (coding errors or hardware issues)