Celebrating Ada Lovelace Day !

Ada Lovelace : London Remembers, Aiming to capture all memorials in London
Ada Lovelace house’s plaque in London

Today we are celebrating Ada Lovelace Day !!

Ada Lovelace was an english mathematician and is recognised as “the first computer programmer” due to the creation of an algorithm for a computing machine in the mid-1800s. She was one of my first inspirations after my parents and my oldest brother to come to the Technology area.

Her childhood background also resembles mine, she had ability with math since little, focused on studying as much as possible and not having a father figure when little ! She is a truly inspiration to me and hopefully to many other women out there.

The Ada Lovelace Day was created in 2009 by Suw Charman-Anderson and it is now held every year on the second Tuesday of October. It is an international celebration of the achievements of women in science, technology, engineering and maths (STEM). The goal of this celebration is to motivate and bring more women in STEM, creating new role models and support women already working in this area.

You can find more about this day and how to support here: https://findingada.com/

Test Data Management Strategies

Hello all,

Today I am going to talk about some different approaches to handle your test data when running automated tests and the trade-offs.

 

Database

Injecting the data before running the tests with SQL, mysql or postgresql scripts are one of the most common approaches. So, you can inject the data you will need for the tests and skip all the setup, which is not the goal of all your scenarios, right ?

For the scenarios that you actually need to test the creation of the data then you won’t use this kind of script. For example in javascript, you would add a setup/data management class, a @BeforeAll and then something like this:

var mysql = require('mysql');
var con = mysql.createConnection({
     host: "localhost",
     user: "root",
     password: "12345",
     database: "javatpoint"
});  

con.connect(function(err) {
     if (err) throw err;
       console.log("Connected!");
       var sql = "INSERT INTO employees (id, name, age, city) VALUES ('1', 'Ajeet Kumar', '27', 'Allahabad')"; 
       con.query(sql, function (err, result) {
     if (err) throw err;
       console.log("1 record inserted");  
     });
});

Then you can have a @TearDown, @AfterAll function to delete the data that was created to be used during the tests.

Files

If, for example, you are running some API tests you might want to have static data ready to be injected for each scenario. You can create a json file and add all the fields and values that are going to be used during your automation:

 { 
   name: "John", 
   age: 31, 
   city: "New York" 
},
{
   name: "Rafa", 
   age: 29, 
   city: "London" 
}

Then you can load this file to be used during your tests. You can create this data upfront, but then you need to make sure that this data is always going to be there otherwise you need to create it again (during your tests or manually).

 

Objects

You can create Objects with the data that you are going to need for the automated tests, so for example you can create a dictionary in Javascript:

var dict = {
  FirstName: "Rafa",
  Age: 30,
  Country: "UK"
};

Then again you need to make sure you are going to create this data during runtime, maybe in a @BeforeAll function or a Setup class, or maybe this is something you have created in the environment already and you need to make sure this is going to be there when running the tests, otherwise you need to create it again.

 

Docker

If you can control the database or the deployment of your QA environment, then it means you can also manipulate the database when running the tests.

If you use docker to create the environment you can add a Volume or even seed the database with docker-compose.

Volume

Volumes are often a better choice than persisting data in a container’s writable layer because a volume does not increase the size of the containers using it, and the volume’s contents exist outside the lifecycle of a given container.

You can push the database (json file, .db) entirely to the docker container:

 docker run -it --name my-directory-test -v /hostvolume:/containervolume centos /bin/bash

Seed

Write a small script that generates randomized and varying data and writes it to the database. Then you can wrap this script into your own Docker image in order to execute them automatically via docker-compose.

 

In this example I am using a mongoDB database:

docker-compose.yml

version: '1.0'

services:

  mongodb:
    image: mongo
    container_name: mongo
    ports:
      - 27017:27017


  mongo-seed:
    build: .
    environment:
      - MONGODB_HOST=mongo
      - MONGODB_PORT=27017
    volumes:
      - ./config/db-seed:/data
    depends_on:
      - mongo
    command: [
      "mongoimport --host mongo --port 27017 --db testautomation --mode upsert --type json --file data.json --jsonArray"
      ]

data.json

[
  {
    "name": "Peter Parker",
    "email": "spiderman@gmail.com",
    "age": 28
  },
  {
    "name": "Bruce Wayne",
    "email": "batman@gmail.com",
    "age": 48
   }
]

 

Scenarios

If you are working with Gherkin syntax, it means you can also add the data in the middle of the scenario and then use it during the automation. So, something like:

Scenario: Correct number of movies found by superhero
Given I have the following movies
| Batman Begins | Batman |
| Wonder Woman | Wonder Woman |
| Wonder Woman 1984 | Wonder Woman |
When I search for movies by superhero Wonder Woman
Then I find 2 movies

Then you can get this data from the step definitions and use during yours tests.

You might have other ways to create and manage the test data, but whatever the approach you decide, make sure the scenarios are independent and if you can clean up the environment data after (unless you have decided to have static data in the environment for now) then clean it.

 

Resources:

https://forums.docker.com/t/seeding-data-volume-containers-mongodb/2214

https://stackoverflow.com/questions/31210973/how-do-i-seed-a-mongo-database-using-docker-compose

https://www.baeldung.com/cucumber-data-tables

https://docs.docker.com/storage/volumes/

https://phauer.com/2018/local-development-docker-compose-seeding-stubs/

TestProject New Python SDK

I have adventured myself to test the new TestProject Python SDK this week and I can say it has been quite straight forward. Also, the documentation is extensive and cover a lot of different scenarios and setups.

For those who don’t know, TestProject is a Free Automation Platform that wrappers open source test frameworks (Selenium and Appium) integrating your automation scripts. It consolidates all the needed drivers to run your test automation without additional setup.

1- To start you need to get SDK token from the TestProject Portal (you can register for free here)

2- Download and install TestProject Agent

3- Run the agent locally and verify the status

4- Install the latest version of python (the min. supported Python version is 3.4)

pip install testproject-python-sdk

5- Generate and copy your developer token

6- Create your first test, for example

7- Then you can see the reports published on your TestProject account, for example

 

You can find a lot more examples on their README file.

 

Resources:

https://testproject.io/

https://github.com/testproject-io/python-sdk

Contract Testing with Pact.js + GraphQL

Contract Tests vs Integration Tests

  • Trustworthy like the API tests, even though the contract test is mocking the provider/consumer, you know it is mocking based on the contract that was generated.
  • Reliable because you don’t depend on your internet connection to get the same consistency on the results (When your API does’t have third parties integration or you are testing locally).
  • Fast because you don’t need internet connection, everything is mocked using the contract that was generated.
  • Cheap because you don’t spend huge amount of time to create a pact test or to run it, even less to maintain.
Contract Tests API Tests
Trustworthy Trustworthy
Reliable Not realiable
Fast Slow
Cheap Expensive

Remember contract tests are NOT about testing the performance of your microservice. So, if you have API Test that are taking ages to (execute/perform), failing due server no replying fast enough or timeouts, this means you have a performance problem, or it is just your internet connection. In either case you need to separate the problem and create targeted tests that are going to verify the performance of your server and not the expected response body/code.

How it works

You can use a framework like Pact which will generate the contract details and fields from the consumer. You need to  specify the data you are going to send and in the verification part you will use the same function the app would use to do the requests to the API.

Contract test is part of an integration test stage where you don’t really need to hit the API, so it is faster and reliable, completely independent of your internet connection. It is trustworthy since you are generating the contract based on the same function and the same way you would do when using the consumer to hit the provider. Pact is responsible to generate this contract for you, so you just need to worry about passing the data and add the assertions, like response code, headers, etc. If It seems pretty straight forward to know who is the consumer, who is the provider and the contract that you are going to generate, but imagine a more complex real life scenario where you have a structure like:

In this case you have multiple microservices communicating with each other and sometimes this service is the provider and sometimes the same service is the consumer. So, to keep the house organised when maintaining these services you need to create a pact between each one of them.

 

The fun part

So let’s get hands-on now and see how we can actually create these contracts.

Create a helper for the consumer to setup and finalise the provider (this will be the pact mock where the consumer is going to point when creating the pact.

import { Pact } from '@pact-foundation/pact'
import path from 'path'

jasmine.DEFAULT_TIMEOUT_INTERVAL = 10000

export const provider = new Pact({
   port: 20002,
   log: path.resolve(process.cwd(), 'logs', 'mockserver-integration.log'),
   dir: path.resolve(process.cwd(), 'pacts'),
   pactfileWriteMode: 'overwrite',
   consumer: 'GraphQLConsumer',
   provider: 'GraphQLProvider'
})

beforeAll(() => provider.setup())
afterAll(() => provider.finalize())

// verify with Pact, and reset expectations
afterEach(() => provider.verify())

 

Then create a consumer file where you add what is the data you want to check and the response you are expecting from the graphQL API.

import { Matchers, GraphQLInteraction } from '@pact-foundation/pact'
import { addTypenameToDocument } from 'apollo-utilities'
import gql from 'graphql-tag'
import graphql from 'graphQLAPI'

const { like } = Matchers

const product = {
  id: like('456789').contents,
  disabled: false,
  type: like('shampoo').contents,
}

describe('GraphQL', () => {
  describe('query product list', () => {
    beforeEach(() => {
      const graphqlQuery = new GraphQLInteraction()
        .uponReceiving('a list of products')
        .withRequest({
          path: '/graphql',
          method: 'POST'
        })
        .withOperation('ProductsList')
        .withQuery(print(addTypenameToDocument(gql(productsList))))
        .withVariables({})
        .willRespondWith({
          status: 200,
          headers: {
            'Content-Type': 'application/json; charset=utf-8'
          },
          body: {
            data: {
              productsList: {
                items: [
                  product
                ],
                nextToken: null
              }
            }
          }
        })
      return provider.addInteraction(graphqlQuery)
    })

    it('returns the correct response', async () => {
      expect(await graphql.productsList()).toEqual([product])
    })
  })
})

When you run the script above, pact is going to create a .json file in your pacts folder and this will be used to test the provider side. So, this is going to be the source of truth for your tests.

This is the basic template if you are using jest, just set the timeout and then you need to use the same functions that you are going to use for the consumer to communicate with the provider. You just need to decide how you are going to inject the data in your local database, you can pre-generate all the data on the beforeAll or a pre-test and then add a post-test or a function in your afterAll to clean the database once the tests are done.

The provider.js file should be something similar to this one:

import { Verifier } from '@pact-foundation/pact'
import path from 'path
import server from 'server'

jest.setTimeout(30000)

beforeAll(async () => {
         server.start('local')
})

afterAll(async () => {
         server.tearDown()
})

describe('Contract Tests', () => {
       it('validates the pact is correct', () => {
         const config = {
                  pactUrls: [path.resolve(process.cwd(), 'pacts/graphqlconsumer-graphqlprovider.json')],
                  pactBrokerPassword: "Password",
                  pactBrokerUrl: "https://test.pact.com/",
                  pactBrokerUsername: "Username",
                  provider:'GraphQLProvider',
                  providerBaseUrl:server.getGraphQLUrl(),
                  publishVerificationResult:true
         }
         return new Verifier(config).verifyProvider()
       }
})

In the end you just need to verify that the contract is still valid after your changes on provider or consumer, for this reason you don’t need to add edge scenarios, just exactly what the provider is expecting as data.

 

Resources:

https://docs.pact.io/

https://docs.pact.io/pact_broker/advanced_topics/how_pact_works

https://medium.com/humanitec-developers/testing-in-microservice-architectures-b302f584f98c

Swift and XCUI tests for beginners

I went to a workshop last year #TechKnowDay and I saved this one about Swift for beginners in my draft. I didn’t have a chance to participate, but I followed the instructions on the slide of the project: https://github.com/ananogal/Workshop-Swift-for-beginners

I took the chance to do some automation on this project and created the scenarios (without BDD) here:

https://github.com/rafaelaazevedo/Templet

It is really basic and simple, but it is a start point for everybody who wants to learn how to create the tests.

You can also record the actions, just open your project on Xcode:

  • Create a New Target

 

  • Type a name for your UITest target

 

  • Select the UI Testing template

 

  • Then you just need to click inside of the test function and the record button (red circle) will show up on the bottom of the screen

 

Thank you Ana for this great workshop !

Deep learning with python

Hello guys, last year I joined to a day of workshops from #TechkNowDay here in London. This workshop is about deep learning with python and I am able to share the power point explaining the exercise with you.

You can download the Power Point presentation here and the list of files for this exercise on this link.

Nice coding everyone !

Thank you so much for the workshop and all the explanation Bianca Furtuna !

Injecting cookies in your testcafe automation

Hello guys,

Today I am going to post an alternative to authenticate without going throught the login page. I have done this before generating a token for Keycloak to authenticate, but in my last project I generated the cookies and added them as a header with testcafe intercepting the HTTP requests to the website.

This is useful for when you don’t need to test the login process or you have a separated feature to test the login page. Then you will be able to save time when running the automation and avoiding to have to sign in every time you launch the scenario.

I had to take this approach for another reason as well, which was this bug here that happened because TestCafe uses a URL-rewritten proxy internally and this proxy is forced to handle cookies manually because the URL of the tested website is changed during test execution.

You will need to add this in a Before hook and generate the cookies before running the scenarios.

So first, install keygrip and cookies in your package:

npm run keygrip base-64

Second you will need to create the cookie based on your authentication process. For example if your cookie is generated with a json like this. You will also need to set the cookie like this and then you can add it in your utils/helper/support class:

import { RequestHook } from 'testcafe';
import Keygrip from 'keygrip';
import Base64 from 'base-64';

class addCookie extends RequestHook {

    constructor (requestFilterRules) {
        super(requestFilterRules);
    }

    async onRequest (event) {
       const cookieName = "name-of-your-cookie-here"; //Change the value with the name of your authentication cookie 
       const cookieSigName = "name-of-your-cookie-here.sig"; //Same as above, but this is the signature cookie 

       let cookieValue = { "name":"username", "email":"username@email.com" }; //Here you have the value that is inside the cookie 
       cookieValue = Base64.encode(JSON.stringify(cookieValue)); //Encode to Base64 the string if you need it 

       const keys = new Keygrip(["SECRET_KEY"]); //Here you will add your secret
       const hash = keys.sign(`${cookieName}=${cookieValue}`); //This is where you are going to sign the cookie with your secret key

       const myDate = new Date();
       myDate.setFullYear(2020);
       event.requestOptions.headers[cookieName]= `${cookieValue};expires=${myDate};domain=${domain};path=/`;
       event.requestOptions.headers[cookieSigName]= `${hash};expires=${myDate};domain=${domain};path=/`; 
    } 

    async onResponse (event) {
       // Anything you need to add when you have the response 
    }

 }

 

You will need to import the page class and then in the feature level:

import addCookie from 'add-cookie.js';

const setCookies = new addCookie(/https?:\/\/your-url.com/);

fixture`Your Feature`
 .page(baseUrl)
.requestHooks(setCookies);

 

This is just an idea of how to skip the login screen page when doing automation and saving you sometime, but there are some other suggestions as well, like generating the token and add them to the header.

 

Resources:

https://github.com/crypto-utils/keygrip

https://github.com/pillarjs/cookies

https://devexpress.github.io/testcafe/documentation/test-api/intercepting-http-requests/creating-a-custom-http-request-hook.html

Chaos Engineering: Why Breaking Things Should be Practiced

Hello guys,

Last week I went to the WebSummit 2018 Conference in Lisbon and I managed to join some of the AWS talks. The talk that I am posting today is about chaos engineering, which specifically address the uncertainty of distributed systems at scale. The aim of this practice is to uncover the system weakness and build confidence in the system’s capability. 

The harder it is to disrupt the steady state, the more confidence we have in the behavior of the system.  If a weakness is uncovered, we now have a target for improvement before that behavior manifests in the system at large.

Today I am going to post the video on the exact moment that this talk starts.

https://player.twitch.tv/?autoplay=false&t=02h05m17s&video=v333130731

This talk is presented by AWS Technical Evangelist Adrian Hornsby.

You can find tools to help you with the tests in this repo:

https://github.com/dastergon/awesome-chaos-engineering#notable-tools

 

References:

https://principlesofchaos.org/

https://www.twitch.tv/videos/333130731

Amazing repo with content/links about the topic: https://github.com/dastergon/awesome-chaos-engineering

How do you help developers to test ?

Hello guys,

I have joined to some webinars from Test Masters Online, not sure if you heard about them, but this one really called my attention. The title looks a bit extreme, but you will see that is more about how testers and developers can work together to improve the quality of the team. The ideal is to have specialised QAs that can teach developers about automation, performance, security tests and so on. It is more about giving awareness about tests when developing.

The challenge nowadays is changing the developers mindset to contribute along with QAs with the automation tests and also getting support from the managers (which is something that I’ve always highlighted). If you want a team without a bottleneck and where everybody is max contributing for the quality and speed of the deliveries, then yes you need to think about a team sharing all the responsabilities and having a specilised person to guide and teach.

Basically you will see how Joel Montvelisky discusses the issues which prevents software developers to test, and what testers can do to change that.

Thank you !! See you in the next post 🙂

Using Artificial Intelligence to Speed up your Test Automation

Hey guys, today I am going to share with you this awesome webinar that I watched last weekend ! I suggest to follow IIST as they are always sharing good webinars about test.