PACT CI/CD Workshop

Hello all, after a small burnout and a month break from my extra work activities (2nd lockdown here in London and winter vibes definitely didn’t help), I am finally back on track and I thought it was worth sharing this CI/CD Pact workshop that was published in the middle of this year.

They use Node, but you don’t need to be an expert to try this workshop. Once you have finished, you will be able to fit Pact and Pactflow into your CI/CD pipelines and understand the workflow when changing the consumer and provider.

I followed all the steps and after around 1h20m I finished it. It is pretty straight forward and it includes the new feature pending pact that is equivalent to the pending tests tag, just make sure you are copying the right tokens and using the right URLs.

The link to the workshop with the instructions and requirements are here:

https://docs.pactflow.io/docs/workshops/ci-cd/

In the end I got the results:

Travis Passed

✅ Provider Passed Locally

✅ Consumer Passed Locally

✅ Pactflow verified

Thanks to Matt Fellows and Beth Skurrie for that !

Special thanks to my friend Marie who suggested me to use Excalidraw for my awesome diagrams !

Preparing yourself for the CTFL Exam

A friend of mine has recently passed the CTFL Exam after one week of a lot of study and effort. CTFL is a popular Certified Tester Foundation Level exam in software testing. It examines your professional knowledge around software testing discipline. The exam has 40 questions and takes 60 minutes.

Should I take the exam ?

This is a very debatable subject and people can discuss for hours about it’s value.

If you are thinking about if this is for you or not, these are some things that you can keep in mind and help you to decide:

  • Experience always has more value than any certification. Certifications don’t provide the exposure and training you get while working on real life projects !
  • THIS CERTIFICATION CAN HELP YOU TO GET YOUR FIRST QA JOB. If you are changing careers, never worked with Software Testing before, then this certification might help you to be selected for an interview. This would be part of your portfolio and as you might not have any experience in this field, this can be used as a parameter when filtering the candidates.
  • THE CERTIFICATION DOESN’T MEAN YOU ARE AN EXPERT OR THAT YOU ARE GETTING THE JOB. Most of the professionals applied to this certification in the beginning of their career and then never study the Syllabus again. This is because you start to use your experience more than the base knowledge you got when studying for the exam. Again the certification doesn’t mean you will get the job, experience and exposure to different projects will.

Online Course

Mock Exams/Material and Syllabus links

When do I know I am ready ?

One technique that I use for actually most of the exams that I apply is practicing the mock exams and once I pass 3 times in a row then it means I am mostly ready to take the real one. You might have other ways to see when you are actually ready, but whatever you follow make sure to prepare and dedicate yourself !

Celebrating Ada Lovelace Day !

Ada Lovelace : London Remembers, Aiming to capture all memorials in London
Ada Lovelace house’s plaque in London

Today we are celebrating Ada Lovelace Day !!

Ada Lovelace was an english mathematician and is recognised as “the first computer programmer” due to the creation of an algorithm for a computing machine in the mid-1800s. She was one of my first inspirations after my parents and my oldest brother to come to the Technology area.

Her childhood background also resembles mine, she had ability with math since little, focused on studying as much as possible and not having a father figure when little ! She is a truly inspiration to me and hopefully to many other women out there.

The Ada Lovelace Day was created in 2009 by Suw Charman-Anderson and it is now held every year on the second Tuesday of October. It is an international celebration of the achievements of women in science, technology, engineering and maths (STEM). The goal of this celebration is to motivate and bring more women in STEM, creating new role models and support women already working in this area.

You can find more about this day and how to support here: https://findingada.com/

Test Strategy Templates

Hello people,
 
Following up these meetups: Developing a Test Strategy and Desenvolvendo uma Estratégia de Testes, I realised it would be useful to share some templates and examples that I have seen in my previous projects.
 
Every company/project adapted this document and had their own template. There is no right or wrong as long as you have the needed information there to the best of your knowledge at the time is okay.
 
 

 

Template 1

 

Template 2

 

Template 3

 

Template 4

 

Template 5

 

You can mix them, pick one session from one and another session from the other, feel free to create your own Test Strategy according to what you need !

AI for Testing: Beyond Functional Automation webinar

Hello guys, I joined a webinar some months ago (15/07/2020) about AI for Testing: Beyond Functional Automation by Tariq King which was really interesting ! I know how it’s hard to keep up with all the online events now, so I always try to keep the recording of the ones that I couldn’t join and are interesting to listen to when I have time.

So thought about sharing with you as well in case you missed. You will learn about reinforcing learning by giving scores to the right actions and about training bots to recognize good and bad designs with examples. This allows the framework to be more robust when searching for a particular query or asserting the scenarios:

 

Here it is the link to the recording:

Thanks Tariq King !

Test Data Management Strategies

Hello all,

Today I am going to talk about some different approaches to handle your test data when running automated tests and the trade-offs.

 

Database

Injecting the data before running the tests with SQL, mysql or postgresql scripts are one of the most common approaches. So, you can inject the data you will need for the tests and skip all the setup, which is not the goal of all your scenarios, right ?

For the scenarios that you actually need to test the creation of the data then you won’t use this kind of script. For example in javascript, you would add a setup/data management class, a @BeforeAll and then something like this:

var mysql = require('mysql');
var con = mysql.createConnection({
     host: "localhost",
     user: "root",
     password: "12345",
     database: "javatpoint"
});  

con.connect(function(err) {
     if (err) throw err;
       console.log("Connected!");
       var sql = "INSERT INTO employees (id, name, age, city) VALUES ('1', 'Ajeet Kumar', '27', 'Allahabad')"; 
       con.query(sql, function (err, result) {
     if (err) throw err;
       console.log("1 record inserted");  
     });
});

Then you can have a @TearDown, @AfterAll function to delete the data that was created to be used during the tests.

Files

If, for example, you are running some API tests you might want to have static data ready to be injected for each scenario. You can create a json file and add all the fields and values that are going to be used during your automation:

 { 
   name: "John", 
   age: 31, 
   city: "New York" 
},
{
   name: "Rafa", 
   age: 29, 
   city: "London" 
}

Then you can load this file to be used during your tests. You can create this data upfront, but then you need to make sure that this data is always going to be there otherwise you need to create it again (during your tests or manually).

 

Objects

You can create Objects with the data that you are going to need for the automated tests, so for example you can create a dictionary in Javascript:

var dict = {
  FirstName: "Rafa",
  Age: 30,
  Country: "UK"
};

Then again you need to make sure you are going to create this data during runtime, maybe in a @BeforeAll function or a Setup class, or maybe this is something you have created in the environment already and you need to make sure this is going to be there when running the tests, otherwise you need to create it again.

 

Docker

If you can control the database or the deployment of your QA environment, then it means you can also manipulate the database when running the tests.

If you use docker to create the environment you can add a Volume or even seed the database with docker-compose.

Volume

Volumes are often a better choice than persisting data in a container’s writable layer because a volume does not increase the size of the containers using it, and the volume’s contents exist outside the lifecycle of a given container.

You can push the database (json file, .db) entirely to the docker container:

 docker run -it --name my-directory-test -v /hostvolume:/containervolume centos /bin/bash

Seed

Write a small script that generates randomized and varying data and writes it to the database. Then you can wrap this script into your own Docker image in order to execute them automatically via docker-compose.

 

In this example I am using a mongoDB database:

docker-compose.yml

version: '1.0'

services:

  mongodb:
    image: mongo
    container_name: mongo
    ports:
      - 27017:27017


  mongo-seed:
    build: .
    environment:
      - MONGODB_HOST=mongo
      - MONGODB_PORT=27017
    volumes:
      - ./config/db-seed:/data
    depends_on:
      - mongo
    command: [
      "mongoimport --host mongo --port 27017 --db testautomation --mode upsert --type json --file data.json --jsonArray"
      ]

data.json

[
  {
    "name": "Peter Parker",
    "email": "spiderman@gmail.com",
    "age": 28
  },
  {
    "name": "Bruce Wayne",
    "email": "batman@gmail.com",
    "age": 48
   }
]

 

Scenarios

If you are working with Gherkin syntax, it means you can also add the data in the middle of the scenario and then use it during the automation. So, something like:

Scenario: Correct number of movies found by superhero
Given I have the following movies
| Batman Begins | Batman |
| Wonder Woman | Wonder Woman |
| Wonder Woman 1984 | Wonder Woman |
When I search for movies by superhero Wonder Woman
Then I find 2 movies

Then you can get this data from the step definitions and use during yours tests.

You might have other ways to create and manage the test data, but whatever the approach you decide, make sure the scenarios are independent and if you can clean up the environment data after (unless you have decided to have static data in the environment for now) then clean it.

 

Resources:

https://forums.docker.com/t/seeding-data-volume-containers-mongodb/2214

https://stackoverflow.com/questions/31210973/how-do-i-seed-a-mongo-database-using-docker-compose

https://www.baeldung.com/cucumber-data-tables

https://docs.docker.com/storage/volumes/

https://phauer.com/2018/local-development-docker-compose-seeding-stubs/

Quality? Who Cares?

Steve Watson's avatarSteve Watson - Musings of a Test Manager

A few weeks ago I was really fortunate enough to be involved in delivering a Unicom talk on Quality, followed a week later by a round table talk which I hosted, and was set up by Billy Senior.

The topic of Quality is something that intrigues me in the context of software engineering. Most of my career has been dedicated to checking whether the work of someone else does what it is meant to do, and doesn’t do something that it shouldn’t. It sounds bizarre when you think of testing as just that – we are validating that a software engineer has written code that meets the requirements and expectations of an individual or group of individuals. 

But quality is not just about testing to see if something works as it should – plenty of things ‘work’, but the experience is awful, or it takes a long time to…

View original post 628 more words

TestProject New Python SDK

I have adventured myself to test the new TestProject Python SDK this week and I can say it has been quite straight forward. Also, the documentation is extensive and cover a lot of different scenarios and setups.

For those who don’t know, TestProject is a Free Automation Platform that wrappers open source test frameworks (Selenium and Appium) integrating your automation scripts. It consolidates all the needed drivers to run your test automation without additional setup.

1- To start you need to get SDK token from the TestProject Portal (you can register for free here)

2- Download and install TestProject Agent

3- Run the agent locally and verify the status

4- Install the latest version of python (the min. supported Python version is 3.4)

pip install testproject-python-sdk

5- Generate and copy your developer token

6- Create your first test, for example

7- Then you can see the reports published on your TestProject account, for example

 

You can find a lot more examples on their README file.

 

Resources:

https://testproject.io/

https://github.com/testproject-io/python-sdk

Open Banking Functional Conformance Suite Test Cases

What is Open Banking ?

Open banking allows the use of open APIs enabling third-party developers to build applications and services around financial institutions. It comes to bring more financial transparency options for account holders ranging from open data to private data.

Open Banking Use Cases (for Users) | by Ştefan Alexandru Băluţ ...

Open Banking Functional Conformance Suite

To be able to get the Functional Conformance Certificate, Open Banking provides a Functional Conformance Tool to allow implementers to check if your API has successfully developed all required functional elements of the OBIE Read/Write API Specifications.

This Open Banking tool allows an ASPSP (Account Servicing Payment Service Provider) and a TPP (Third Party Provider) to test the response of any API endpoint and validate that the JSON and data formats meet the schema, permissions and interfaces against the Functional API standard.

How to identify Test Cases covered in the OB Functional Conformance Suite ?

How do you know what else needs to be covered and if there is indeed something more to cover ? After digging into the project on bitbucket, I found some useful json files where you can check the assertions for each test case, the test cases itself and another file to translate the list of the assertions.

So, you can find the asserts that are being done for each test case inside the manifests folder.

For example, this one contains the assertions for this test case: The x-fapi-interaction-id is replayed for an Account. You can find the file with the accounts transactions test cases here.

Screenshot 2020-07-13 at 17.48.08

Then you would need to check what this assertion actually means, and you can find the dictionary of the assertions on this file.

Screenshot 2020-07-13 at 17.52.47

Remember that all the tests currently assume that consent is granted at the ASPSP portal for each requested PSU Consent (Payment Service User Consent).

Also, you will find that some test cases are missing for instance what should happen when you send an invalid token to the payments endpoint, but you can see there is a test case for the accounts endpoints for when you send a token without the required permissions to get a 401 response.

In this example, you can see that for payments the consent model is a bit different because each access token doesn’t have a range of permissions, but is associated with a single payment consent id. So, in order to get a 401 response, the request can present the wrong token along with a payment call or present no token at all. The conformance tool is not sending any token in this instance.

So make sure you are aware and cover the missing test cases with another approach.

I found quite hard to have a straight answer about what are all the test cases they are covering and also the details, so hope this helps to have a bit more clarity in case you are having the same issues.

Resources:

https://openbankinguk.github.io/knowledge-base-pub/conformance-tools/

https://openbanking.atlassian.net/wiki/spaces/DZ/pages/1061716467/Functional+Conformance

https://medium.com/zoidcoin-network/open-banking-use-cases-for-users-8678d11d770b

https://en.wikipedia.org/wiki/Open_banking