Load tests: Jmeter vs K6

Hello all,

Today it’s the turn of Jmeter and K6 ! As always, remember to check your other options and see what better fits for your project.

Jmeter is a great and powerful tool, but depending on what you really need (something more lighter) then Jmeter might become an over complex, slow, hard to maintain tool.

Jmeter K6
In-built Protocols Support
  • HTTP
  • FTP
  • JDBC
  • SOAP
  • LDAP
  • TCP
  • JMS
  • SMTP
  • POP3
  • IMAP
  • HTTP 1.1
  • HTTP 2
  • WebSockets
Speed to write tests
  • Slow
  • Fast
Support of “Test as Code”
  • GUI oriented
  • Possibility to create scripts, but too complex and lack of documentation
  • Weak (Java)
  • Hard to maintain
  • Scripts oriented
  • JavaScript
  • Easier to maintain
Ramp-up Flexibility
  • Plugins available to be able to configure flexible load
  • Supports ramp-up phases and flexible load
Test Results Analyzing
  • Yes
  • Yes
Resources Consumption
  • Heavy to run tests with multiple users on a single machine, more memory consumption
  • Lightweight and doesn’t take up so much memory of your machine

Screenshot 2020-07-06 at 23.34.47

Easy to use with Version Control Systems
  • No
  • Yes
Number of Concurrent Users
  • Thousands, under restrictions
  • Thousands
Recording Functionality
  • Yes
  • No, but it allows to auto-generate a k6 script via an HAR file
Distributed Execution
  • Yes
  • Yes
Load Tests Monitoring
  • Add listeners, but consume more memory

Screenshot 2020-07-06 at 23.35.02

Jmeter is most used when:

  • You need to perform a complex load including different protocols
  • You can record scenarios
  • Robust support and training ecosystem
  • Require that a full scenario be written for every test
  • If you need to simulate specific load with some custom ramp-up patterns
  • If you just prefer UI desktop app for scripts creation, or you just do not know Javascript/YAML/JSON well enough

 

K6 solves some specific problems:

  • CLI tool with developer-friendly APIs.
  • You can use HAR files to generate record sessions
  • Checks and Thresholds – for goal-oriented, automation-friendly load testing
  • Open source, great support and documentation
  • Lightweight uses Javascript
  • Does not run in NodeJS and doesn’t run in a browser

 

Resources:

https://k6.io/

6 thoughts on “Load tests: Jmeter vs K6

  1. Why do you think K6 has distributed execution capabilities? Just because its Cloud Service provides distributed execution capabilities? I think it’s controversial that you simply wrote support

    1. Hello, you can run k6 in different instances if you need it, but unless you need more than 100,000-300,000 requests per second (6-12M requests per minute), a single instance of k6 will likely be sufficient for your needs. Cloud services support is another topic which jmeter and k6 were compared against 🙂

  2. https://stackoverflow.com/questions/58408705/are-k6-http-calls-asynchronous#:~:text=Fortunately%2C%20k6%20currently%20does%20not,or%20more%20generally%2C%20event%20loop.
    According to this post, k6 currently does not support asynchronous calls. This means our throughput is dependent on the response time of the API calls.
    Example: If I set VU = 50 and API response time is 1s, I will be able to generate a load of 50 * 60 = 3000 RPM.
    Jmeter will help us make async calls and we can generate load at higher RPMs for same number of VU

  3. https://stackoverflow.com/questions/58408705/are-k6-http-calls-asynchronous#:~:text=Fortunately%2C%20k6%20currently%20does%20not,or%20more%20generally%2C%20event%20loop.
    According to this post, k6 currently does not support asynchronous calls. This means our throughput is dependent on the response time of the API calls.
    Example: If I set VU = 50 and API response time is 1s, I will be able to generate a load of 50 * 60 = 3000 RPM.
    Jmeter will help us make async calls and we can generate load at higher RPMs for the same number of VU

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.