Monitoring and Alert Test Strategy Meetup

Hello all, in July I presented this meetup about Monitoring and Alerts Test Strategy for the Test Automation Talks Group.

Here are the slides:

The demos with different from the slides:

AWS Fault Injection Simulator

Chaos Engineering with Gremlin

ChaosMesh Doc

https://chaos-mesh.org/docs/basic-features/

KubeMonkey – Running Chaos Engineering

As always if you have any questions or feedback just ping me a message 😀

Dependency Injection with Selenium and Spring Boot

Hello guys,

Today I am going to share something that I learned recently. I am aware that so many people probably have been using this a long time ago, but I have moved away from java and Selenium quite some time ago and I recently got back into it.

So here is a boilerplate with an automated test project using Dependency Injection of Spring Boot with Selenium, Gradle, and Cucumber.

Feel free to contribute or give any feedback !!

Github project: https://github.com/rafaelaazevedo/Scrux

Benefits of AI in Test Automation

Photo by Kindel Media on Pexels.com

There are several interesting web app automation scenarios that we can improve using AI:

  • Reduce the execution time: Nowadays you have the feature target function already even without an AI test automation project, but with AI you can add this feature without having cucumber in place or even the need to tag the scenarios or features. The AI should be able to identify the features related to the change automatically.
  • Converted manual test cases to automation: you can use Natural Language Processing (NLP) to automatically translate manual test cases into automated test cases. I have seen this done with cucumber not AI yet, but totally possible as AI models work on datasets.
  • Creating different data combinations by training the AI to identify the possible combinations based on a dataset is possible. This would increase the data coverage and bring more confidence to the automation project.
  • Visual validations: Many tools perform this functionality already. I personally tried one tool ages ago called Percy, but you can also try some other popular tools like Applitools and Telerik
  • Test execution stability or self-healing automation: AI can automatically locate web elements when the primary locators fail. You can see this feature in some cutting-edge automation tools like Mabl and Xray and Functionize. Self-healing employs data analytics to identify objects in a script even after they have changed. When your script fails due to being unable to find the object it expected, the self-healing mechanism provides a fuller understanding and analysis of options. Rather than shutting down the process, it examines objects holistically, evaluates the attributes and properties of all available objects, and uses a weighted scoring system to select the one most similar to the one previously used.

Becoming a Domain Model Expert

Creating a model for your test automation requires a domain expert, therefore is critical to have a test automation specialist that also knows the business so the AI can bring the desired innovation. With such extensive use cases, AI systems will need different parameters from domain experts.

Machine Learning Algorithms In Layman's Terms, Part 1 | by Audrey  Lorberfeld | Towards Data Science
https://wordstream-files-prod.s3.amazonaws.com/s3fs-public/machine-learning.png

Be careful to not run more automated tests than you actually need it. A stage of supervision when the AI is learning the patterns is definitely needed it.

Resources:

Breaking The Bias – International Women’s Day

Hello guys,

As part of my STEM Ambassadors activity to bring more young women to the STEM area, I recorded a video telling my career history and how I ended up in tech.

As you know women are still a minority and one of the reasons why is that some people still believe there is a rule saying tech is a man’s job or a woman’s job. This is me when I hear these comments:

I wish the world was binary and simple like that.

Here is the presentation in case you are curious about how I ended up where I am now 😊

Javascript Executor with PageFactory Selenium

Photo by Antonio Batiniu0107 on Pexels.com

Hello guys,

Today I am going to post another problem that cost me a lot of research time and pulling out my own hair.

This was the initial code:

    public void javascriptExecutor(String script, WebElement element) {
       WebDriver driver = new ChromeDriver();  
       JavascriptExecutor jse = (JavascriptExecutor) driver;  
       jse.executeScript(script, element);  
    }

If you ever encounter the error complaining Selenium couldn’t find the Webdriver instance, then check the complete explanation of the problem here.

TL;DR; Webdriver when using PageFactory needs to extract the underlying WebElement with WrapsElement and then the WebDriver from it. Basically, you need to get the real element and then you will be able to use javascript executor:

    public void javascriptExecutor(String script, WebElement element) {  
       WebDriver driver = ((WrapsDriver)((WrapsElement)element).getWrappedElement()).getWrappedDriver();
       JavascriptExecutor jse = (JavascriptExecutor) driver;  
       jse.executeScript(script, element);  
    }

So, next time this problem comes up it will be smooth to fix and not days of trying to figure out what is happening 😂

Stale Element when using PageFactory – Selenium

Photo by luis gomes on Pexels.com

Hello guys,

The first post of the year will be a quick one. Took me a while to figure out since it has been ages I coded in Selenium/Java and used PageFactory (which I thought was deprecated, but it is just deprecated in C#)

If you ever come across an error like this when using PageFactory:

stale element reference: element is not attached to the page document

PageFactory initializes the elements the first time you run the automation and when the page changes (which happens on Angular and React pages) Selenium loses the reference to that element and needs to find it again with the new DOM.

You can do something like this:

protected void clickElement(WebElement element) {
   try {
      webDriverUtils.waitForElementToBePresent(element);
      element.click();
   } catch (StaleElementReferenceException e) {
       PageFactory.initElements(driver, this);
   }

Blockchain Tests Workshop at The National Software Testing Conference

Hello guys, here are the slides from the workshop on Blockchain Tests that I gave earlier this week, as well as some responses to the issues that were addressed during the session. I can say as my first presential workshop after pandemic, it was a great experience with a full room 😃


Unfortunately, there is no recording of it, but you may clone the repo and follow the coding instructions to build the test class and methods, then compare to version 2 of the project, which has the most recent version.

What is the Required() function in Solidity?

The require Solidity function guarantees validity of conditions that cannot be detected before execution. It checks inputs, contract state variables and return values from calls to external contracts.

Where the Address type comes from ?

The address type comes in two flavours, which are largely identical:

  • address: Holds a 20 byte value (size of an Ethereum address).
  • address payable: Same as address, but with the additional members transfer and send.

The idea behind this distinction is that address payable is an address you can send Ether to, while a plain address cannot be sent Ether.

Type conversions

Implicit conversions from address payable to address are allowed, whereas conversions from address to address payable must be explicit via payable(<address>).

Explicit conversions to and from address are allowed for uint160, integer literals, bytes20 and contract types.

Other tools that you can use to test Blockchain applications

  • Ethereum Tester,
  • Hyperledger Composer
  • Exonum test kit
  • Embark
  • Populus
  • BitcoinJ
  • Binance

Resources

Sending a JSON Parameter to Jmeter from pom-maven

Hello all,

Today I am going to post a quick snippet that I used recently and it quite made me spent a lot of time just because I didn’t read the Jmeter documentation in the first place 😅

In the pom.xml you will need to set some jmeter configurations:

 <?xml...    
 <project...
               <properties>                            
                  <jsonValue>[{"name": "Rafa", "age": 20},{"name": "Robin", "age": 5}]</jsonValue>                                     
               </properties>
...
<build>
   <plugins>
                 <plugin>
                   </configuration>
...
                        <propertiesUser>                            
                            <jsonValue>${jsonValue}</jsonValue>                                     
                        </propertiesUser>                                      
                   </configuration>
                </plugin>

How it should look like:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>1.0.0</modelVersion>

    <artifactId>jmeter-json-param-example</artifactId>
    <groupId>com.azevedorafaela.jmeter</groupId>
    <version>1.0</version>
    <packaging>pom</packaging>

    <properties>
        <host>www.google.co.uk</host>
        <jsonValue>[{"name": "Rafa", "age": 20},{"name": "Robin", "age": 5}]</jsonValue>
        <duration>30</duration>
        <threads>10</threads>
        <targetThroughput>1500</targetThroughput>
    </properties>

    <profiles>
        <profile>
            <id>performance</id>
            <build>
                <plugins>
                    <!-- execute JMeter test -->
                    <plugin>
                        <groupId>com.lazerycode.jmeter</groupId>
                        <artifactId>jmeter-maven-plugin</artifactId>
                        <version>2.9.0</version>
                        <executions>
                            <execution>
                                <id>test</id>
                                <goals>
                                    <goal>jmeter</goal>
                                </goals>
                            </execution>
                        </executions>
                        <configuration>
                           <generateReports>true</generateReports>
<ignoreResultsFailures>false</ignoreResultsFailures>
<testResultsTimestamp>false</testResultsTimestamp>
                            <propertiesUser>
                              <threads>${threads}</threads>
                              <targetThroughput>${targetThroughput}</targetThroughput>
                             <duration>${duration}</duration>
<!-- json parameter, host and port -->
                             <jsonValue>${jsonValue}</jsonValue>                                 
                             <server>${host}</server>
                            </propertiesUser>
                        </configuration>
                    </plugin>

                    <plugin>
                        <groupId>com.lazerycode.jmeter</groupId>
                        <artifactId>jmeter-analysis-maven-plugin</artifactId>
                        <version>1.0.6</version>
                        <executions>
                            <execution>
                                <goals>
                                    <goal>analyze</goal>
                                </goals>
                                <phase>post-integration-test</phase>
                            </execution>
                        </executions>
                        <configuration>
                            <!--
                            source file that contains jmeter result data. Needs to be XML format or a GZIPed XML format
                            -->
<source>${project.build.directory}/results/*</source>
                        </configuration>
                    </plugin>

                </plugins>
            </build>
        </profile>
    </profiles>
</project>

On the Test Plan configuration you will need to set the parameters getting it from the pom.xml:

  • The default value for this parameter is: [{"name": "Rafa", "age": 20},{"name": "Robin", "age": 5}]
  • It is optional to set a default value in case it is not sent.
  • Note that we will need to escape the commas when adding the value to the jmeter script:
${__P(jsonValue,[{"name": "Rafa"\, "age": 20}\,{"name": "Robin"\, "age": 5}])}

Now you can run your jmeter tests passing a json code to the maven command line and if you don’t pass this parameter, then Jmeter is going to use the default one.

Resources:

Blockchain Tests with Truffle (Solidity)

Hello guys !! After my talk about Testing Blockchain Applications on the #TechKnowDay,  I thought it was worth posting a guide on the same topic as I felt it was complex to also understand Blockchain. So, this is a quick tutorial on how to run tests with Truffle Framework.

If you are still wondering where you can use Blockchain, here it’s a table with the percentage of companies that are focusing on it and the use cases.

Requirements

We are going to use a bit of Javascript for the web part and Solidity for the blockchain project. For this project you will need to have installed NPM and Node already. We are going to install Truffle as part of the setup. You will also need MetaMask and Ganache, all of them you can find bellow:

Setup your Local Ethereum Blockchain

  • Open your MetaMask plugin and click on new Custom RPC Network. Type the following (Currency Symbol will be automatically populated after you type the Chain ID) and save.
  • Open Ganache, click on Quick Start (Ethereum). Double check if the server has this configuration clicking on the on top right and then Server tab.
  • Click on Show keys of the account you have created and copy the private key.
  • Back on MetaMask, click on Import Account and add the private key you have copied from the previous step.

Setting up the project

On your terminal run:

npm install -g truffle

If you want to follow the test steps only, then you need to download the first release that contains the installation files of the project already:

https://github.com/rafaelaazevedo/PetShopTestWorkshop/releases/tag/1.0

If you want to try the installation for yourself from the scratch, then just go to this link and follow the Getting Started guide. This guide won’t be focusing on the installation of the framework or the setup of the contracts and migrations.

Just a heads up that for this project I am using the pet-shop box instead of the MetaCoin from the Getting Started guide.

Installation

After downloading the project, you will need to run some commands to set it up. Open your terminal on the root of the project and run:

Run the development console

truffle develop

Compile and migrate the smart contracts. Inside the development console you don’t need to type the truffle command.

compile
migrate

To get out of the truffle console type

.exit

Create a new file inside of the test folder called TestAdoption.sol and import all the needed modules, such as assertions, new instances of deployed addresses and the contract that will be tested.

pragma solidity >=0.5.0;

import "truffle/Assert.sol";
import "truffle/DeployedAddresses.sol";
import "../contracts/Adoption.sol";

Then create the class and inside add the variables:

  • adoption that will get the Adoption contract that was deployed
  • expectedId this one will be the id of the pet under test (You can get the id of the pet you want to test from this file)
  • expectedAdopter which is the address of this current contract that we are creating in the next function
contract TestAdoption {
    Adoption adoption = Adoption(DeployedAddresses.Adoption());
    uint256 expectedPetId = 8;
    address expectedAdopter = address(this);
}

Testing

Asserting the adoption

  • Add the testUserCanAdoptPet() function below the variables block.
  • Note that here you are creating a function that will test you can adopt a pet and for this you will need to get the address related to the adoption transaction and compare the returnedId with the expectedPetId address
  • Try to explore and add other asserts like checking if it’s not returning null.
function testUserCanAdoptPet() public {
        uint256 returnedId = adoption.adopt(expectedPetId);

        Assert.equal(
            returnedId,
            expectedPetId,
            "Adoption of the expected pet should match what is returned."
        );
    }

Asserting the adopter of the pet

  • Add the testGetAdopterAddressByPetId() function below the previous function.
  • This function will check if the adopter address for that pet is the same from the adopters list
  • Try to explore and add other asserts like comparing the age of the pet is returning correctly, for that you would need to add the age on the Adoption.sol contract, then compile and migrate again.
function testGetAdopterAddressByPetId() public {
        address adopter = adoption.adopters(expectedPetId);

        Assert.equal(
            adopter,
            expectedAdopter,
            "Owner of the expected pet should be this contract"
        );
    }

Asserting the list of adopters

  • Now add the testGetAdopterAddressByPetIdInArray() function below the previous function.
  • This function will check if the memory address for this petId is the same as the expectedAdopter
  • Almost the same test as before, but this time we are explicitly storing adopters in memory rather than contract’s storage and then comparing them.
function testGetAdopterAddressByPetIdInArray() public {
        address[16] memory adopters = adoption.getAdopters();

        Assert.equal(
            adopters[expectedPetId],
            expectedAdopter,
            "Owner of the expected pet should be this contract"
        );
    }
}

You should have something like this:

pragma solidity >=0.5.0;

// The first two imports are referring to global Truffle files, not a `truffle` directory.
// Gives us various assertions to use in our tests.
import "truffle/Assert.sol";

// When running tests, Truffle will deploy a fresh instance of the contract being tested to the blockchain.
import "truffle/DeployedAddresses.sol";

// The smart contract we want to test.
import "../contracts/Adoption.sol";

contract TestAdoption {
    // The address of the adoption contract to be tested
    Adoption adoption = Adoption(DeployedAddresses.Adoption());

    // The id of the pet that will be used for testing
    uint256 expectedPetId = 8;

    //The expected owner of adopted pet is this contract
    address expectedAdopter = address(this);

    // Testing the adopt() function
    function testUserCanAdoptPet() public {
        uint256 returnedId = adoption.adopt(expectedPetId);

        Assert.equal(
            returnedId,
            expectedPetId,
            "Adoption of the expected pet should match what is returned."
        );
    }

    // Testing retrieval of a single pet's owner
    function testGetAdopterAddressByPetId() public {
        address adopter = adoption.adopters(expectedPetId);

        Assert.equal(
            adopter,
            expectedAdopter,
            "Owner of the expected pet should be this contract"
        );
    }

    // Testing retrieval of pet owner storing getAdopters in memory
    function testGetAdopterAddressByPetIdInArray() public {
        // Store adopters in memory rather than contract's storage
        address[16] memory adopters = adoption.getAdopters();

        Assert.equal(
            adopters[expectedPetId],
            expectedAdopter,
            "Owner of the expected pet should be this contract"
        );
    }
}

Running the tests

  • Open your terminal on the root of the project and run:
truffle test
  • If everything went okay you will see green checks on your terminal like this:

You can check the final code with the latest release (Spoiler alert: You will see pictures of my dog, my dog’s best friend, my friend’s cat and my previous dog)

git clone git@github.com:rafaelaazevedo/PetShopTestWorkshop.git

Open a tab on your terminal and run the development local server

npm run dev

You can also check truffle config file to check the host and port where we are running the project and edit if you need to.

Releases of the project:

  • Release 1.0 Contains the Solidity contracts and Truffle Migrations only (No UI)
  • Release 1.1 Contains the Solidity contracts, Truffle Migrations and Truffle Initial Tests only (No UI)
  • Release 1.2 Contains the Solidity contracts, Truffle Migrations, Truffle Initial Tests and Initial UI
  • Release 1.3 Contains the Solidity contracts, Truffle Migrations, Truffle Initial Tests, Final version of UI

Testing Trends 2021

Hello all, I have been wondering what will be the trends for this year. Last year, I noticed an increase in accessibility and performance tests concerns because of the increased number of users going online due to the pandemic.

I believe this is going to continue this year and then we will definitely have an increase on the other trends that were already around for the past years. I have condensed some of these trends here:

Blockchain

Think about how the world changed after the internet, everybody is connected. Billions and billions of transactions are made per second, because of that, the concern with the security is even more important. Blockchain arrived with the idea of making it difficult or impossible to change, hack, or cheat the system. Blockchain is essentially a digital ledger of transactions and smart contract (chaincode) services to applications that is distributed across an entire network of computer systems.

This has been a hot topic in the past years, so nothing really new, but you will definitely see an increase as people are still learning and understanding what are the benefits of it and what are the problems that Blockchain solves. Once you understand you never go back.

Augmented Reality

Again nothing really new here, you can see videogames have been using this a lot and it was definitely a boom when you could catch Pokemons in the middle of the street !

With Augmented reality (AR) you can have a real-world interactive experience. The objects from the real world are enhanced by the computer-generated perceptual information, or even combined with digital created objects. You can see an increase on this trend as again people are getting more and more online and avoiding going to the stores because of the pandemic. Now you have the ability to try clothes online for example.

I have never really joined a project with Augmented Reality, but this would be another challenge since you need to keep in mind things like speed you move, lights, objects and other variants.

Chatbots and Artificial Intelligence

I know many people hate chatbots but, for someone that hates calls or just wants to make a quick question, or even just want things to be solved as fast as possible like me, this is the right option. Of course there is still a lot to improve on the machine learning side of the technology, but depending on the how you were able to train the bot, you can even think you are talking to a real person.

I am not going to deny that there is a lot of training of words and phrases until it feels like talking to a human but, some chatbots are really useful, for example if you need to return a purchase on Wish app. Today, chatbots are used most commonly in the customer service space, assuming roles traditionally performed by living, breathing human beings such as Tier-1 support operatives and customer satisfaction reps.

Artificial Intelligence (AI) is here to stay since the beginning of its creation. Big companies like Tesla, Amazon (Alexa), Apple (Siri) are more and more sophisticated. You can expect Machine Learning to be used in the automated tests more and more and also to think about the tests you will need to perform on AI projects.

Mobile Apps (Fitness, Mental Health, E-commerce, Services in general)

It is rare to find a person that doesn’t have a smartphone nowadays, even kids need to have so the parents can contact them to see if everything is okay. The pandemic brought more people to look for mental health and fitness apps than ever, is this going to stay even after our current situation ?

Mobile tests are way more full of details, things you need to consider, like the speed of the internet connection, orientation of the screen, sending the app to the background and opening again… The market is flooded with millions of apps and we have too many options now, so how will new apps find their place? Quality of the service, Usability and Price.

Usability and Accessibility Testing

This is another hot topic as the pandemic pushed more people to go for online services. Keeping accessibility tests in mind you will help people with disabilities to be able to navigate and interact with websites and tools. It also means that they can contribute equally without barriers.

Usability Tests is more about the design of the products. People don’t want to tap 300 times to be able to login or pay for a product, your app needs to be efficient, effective and satisfying. A lot of people don’t realise how this is important, but once you lose a client because of a small mistake it might be hard to get it back.

Internet Of Things (IoT)

Share data and automate services are a must have ! Nowadays you need to have the option to share your fitbit steps with your scale, for example. There is a huge demand to access, create, use and share data from any device.

The thrust is to provide greater insight and control, over various interconnected IOT devices. Which means you need to have the option to tell Alexa to turn on or off the lights when you are too tired to get out of the bed to do it.

I particularly didn’t see too much movement in this area last year, maybe because of all the other more urgent things we had to deal with, but this is another thing that is going to continue increasing, even though I can see a lot of security concerns around having everything connected. Imagine if somebody is able to hack your printer and then getting access to your front door lock ?

Data Protection and Security Tests

This is something you need to keep in mind every time you have a new feature coming up, or even updating a current feature. We need to identify the threats in the system and measure its potential vulnerabilities, so the threats can be encountered and the system does not stop functioning or can not be exploited.

Security tests were always important, but I always felt it was a bit neglected until something bad happens, then was treated as first priority. Here in Europe we need to be super careful with the GDPR rules and make sure we are not exposing personal data anywhere. So, another thing to keep in mind as if we have more people accessing the internet, it means we have more people trying to steal data as well.

Coding Skills and QAOps

Another name for the QA role, like we don’t have enough 😂 (QA, Tester, SDET, Software Developer in Test, Test Automation Engineer, Full Stack QA and the list goes on…). What is this new QAOps name ?

It is a combination of Quality Assurance (QA) and software operations. QAOps combines QA practices with software development and IT operations to develop a long-term, integrated operational delivery model.

Resources