Today I am going to post a comparison of these two different load tests framework. I know most people use Jmeter and it has been longer in the market, but I have recently used Locust and also Artillery (which will post a comparison later) and the results are great, the team was able to improve the creation of the tests and also the maintainability.
In the end of the day you need to use the right tool for your needs, Jmeter is a great and powerful tool, but depending on what you really need (something more lighter) then Jmeter might become an overcomplex, slow, hard to maintain tool.
In-built Protocols Support
HTTP, but ability to add a customise protocol
Speed to write tests
Support of “Test as Code”
Possibility to create scripts, but too complex and lack of documentation
Hard to maintain
Easier to maintain
Plugins available to be able to configure flexible load
Ability to specify a linear load only (so no complex scenarios)
Test Results Analyzing
Heavy to run tests with multiple users on a single machine, more memory consumption
Light to run tests with multiple users on a single machine, less memory consumption
Doesn’t take up so many of your machines’ resources.
Easy to use with Version Control Systems
Number of Concurrent Users
Thousands, under restrictions
Load Tests Monitoring
Add listeners, but consume more memory
Ability to monitor a basic load
Jmeter is most used when:
You need to perform a complex load including different protocols
If you need the script recording functionality
If you need to simulate specific load with some custom ramp-up patterns
If you just prefer UI desktop app for scripts creation, or you just do not know Python well enough
Locust solves some specific problems:
You can write performance scripts pretty fast
Push to your VCS and easily maintain the scripts
Spend minimum time on maintenance without additional GUI applications
Simulate thousands of test users on local machine without the need to have multiple slaves
Today I am going to share this really interesting webinar of my friend Julio de Lima, he is one of the top QA influencers in Brazil, and in this video he is talking about how to reduce the scope of load tests using Machine Learning.
Load testing execution produces a huge amount of data. Investigation and analysis are time-consuming, and numbers tend to hide important information about issues and trends. Using machine learning is a good way to solve data issues by giving meaningful insights about what happened during test execution.
Julio de Lima will show you how to use K-means clustering, a machine learning algorithm, to reduce almost 300,000 records to fewer than 1,000 and still get good insights into load testing results. He will explain K-means clustering, detail what use cases and applications this method can be used in, and give the steps to help you reproduce a K-means clustering experiment in your own projects. You’ll learn how to use this machine learning algorithm to reduce the scope of your load testing and getting meaningful analysis from your data faster.
Hey guys, today I will post about a quick code to set properties in runtime from jmeter variables.
Where you could use this ?
If you have a list after extract with json path extractor and you want to set a property to use in another thread group, this could be useful.
For example, I will get the serials of a list extracted with json path extractor.
– Create a beanshell script with:
//Get the count of the serials
String countSerials = vars.get("serials_matchNr");
// create a for to create a property for each serial variable
//set each property + index with the variable of each serial in the list
So for today I will post the code I am using on Jmeter to get a random index from a list. So, if you have a json with several ids, for example, and you want to choose one of them to use on the next request, you need:
1 – Create a post processor to get the id node from your json response request. As you may know when you use this plugin, jmeter creates automatically the list of variables with the prefix you have chosen and the suffix “__index”. You can see how to use this plugin on my previous post.
2 – Create a Beanshell Sample
3 – Get a random index from this list of ids or you could use this method to get the random id directly. I found easiest get the index r at least it was the first thing which came into my mind.
String idSize = vars.get("ids_matchNr");
int max = Integer.parseInt(idSize);
int min = 1;
int idx = min + (int) (Math.random() * ((max - min) + 1));
String id = vars.get("ids_" + Integer.toString(idx));
So, we got the expression: $.data.token and now we can create the json path extractor with a default value (when jmeter doesn’t find the expression), jsonpath and the variable (which will contain the value of the expression). See:
Now, we can use the variable on a header or another request:
This example I am getting json path from an authentication request and use it in another group thread.
Remember install the json plugin for jmeter (If you don’t have), you can do that with homebrew command :
brew install jmeter --with-plugins
Create a test plan and set the web server and the port:
Create a Thread Group:
Create HTTP Request, use the variables for the server and the password. Put the path of the authentication page, username and the password.
Create Json Path Extractor and put the path Expression and the variable that you will use. You can test if your path is correct here.
Create a beanshell Assertion or Post processor and set the property:
Create a new Thread Group and a HTTP Header Manager and use the same variable you used before:
Create the Listener > View Result Tree and it’s done, you can run the jmeter and see if it’s getting the token and using in the next thread group