Testing Blockchain Applications

Hello all, today I am going to post about a topic that I am a huge enthusiast about: Blockchain !

Nowadays data is one of the most valuable product that you can own, and as we are surrounded by fake data coming from thousands of different places, it is hard to know what is trustworthy or not.

What is Blockchain ?

It is basically the second phase of the internet. The Internet democratized the exchange of information, blockchain promises to democratize the exchange of real value. It was created in 2008 by Satashi Nakamoto, which the identity is still unknown.

An example of how blockchain can be used is when you want to make a transaction of £10 from a X to Y. Nowadays what happens is this transaction goes through a third-party app or payment processing system. Then, X’s bank will need to check the details of Y’s bank and once this is done the transfer of the amount will initiate with a certain amount of deductions, of course. In the end both banks will record the transaction deducting the transaction fee, meaning Y will receive something like £9.95. Even though this process is quite secure and has many redundancies in place to make sure that it stays secure and accurate, there are some basic fundamental issues with this process:

  • Delay in processing (using this middle agent and all the bureaucracy)
  • Dependency on a single intermediary whose effectiveness is never 100 percent (dependency on a national state)
  • In case of any gap in the transaction, no one takes the responsibility and people keep on blaming each other (not concrete proof to check the transaction)

Blockchain is a system where you no longer have to worry about these problems, you just need to perform a transaction and there is a chain of people who are able to validate your transactions every second. This mechanism is called as Proof of block and this is done based on the public key provided for the encrypted data, and it is done by all listeners in the peer-to-peer network. They are real people/identities, not bots. Since there is not a single transaction validating authority (banks, national state) who are centralising the transaction service, the process is effectively decentralized, public and transparent for everybody to see.

Once a specified number of people validates the transaction, the transaction details are stored in the form of a block and that block is added to the existing blockchain. Once the blocks are validated, nobody can change them, just add another one on top, they are immutable. These blocks have a specific hash associated with every block. These hashes are like fingerprints and are unique to every block. Every person that is validating the transaction process is called a miner. The more miners, the better efficiency of the transaction.

A block contains data, a hash, and the hash of the previous block. Since it contains a hash of a previous block, all the blocks contain data for the previous blocks and it becomes almost impossible for a blockchain to be corrupt.

What types of tests can I perform in a Blockchain app ?

  • API Testing: In API testing, we ensure that the interaction between applications in the blockchain ecosystem is as expected
  • Block/Peer/Node Testing: All the blocks on the Network should be tested individually to ensure proper cooperation. All diverse nodes on the Network must be tested independently to ensure smooth cooperation. Do the nodes in the network sync with other validating peers? Is the integrity of the network and shared ledger maintained throughout the testing?
  • Functional Testing: In Functional Testing, we evaluate the work of various functional parts of the Blockchain (e.g., smart contracts).
  • Performance Testing: Details like network latency based on block size, network size, expected transaction size, and how long a query takes to return the output with the specialized authentication protocol
  • Security Testing: Here, we ensure that the application is not vulnerable to attacks and the systems can protect the data and are capable of handling malicious attacks, etc.
  • Integration Testing: In Integration testing, we ensure that all the components of the application are integrated properly and performing the actions appropriately
  • Smart Contract Testing: Smart Contract testing is about performing detailed functional testing of business logic and process. It refers to the set of software constructs that automatically execute transactions when predefined conditions and business logic are met. Testing smart contracts involve simulation of all possible expected and unexpected variables for every contract and the triggers that execute transactions.

Benefits of Blockchain Apps

  • Decentralized System: No need to have a middle agent for the transactions. Beneficial in various industries like finance, real estate etc.
  • Better Security: As it uses multiple nodes to complete and authenticate transactions, you don’t rely only on a transaction system that is controlled by a centralized group of identities.
  • Authenticity: Allows the unique algorithm to process data, like a fingerprint.
  • Increased Capacity: Increases the capacity of the entire Network as it is completely decentralized and real people can verify the blockchain.

Challenges in Blockchain Testing

  • Understanding the Technology: Blockchain is a new technology and understanding it is very important in order to test it. Where does regulatory compliance overlap with quality? How do you fix a defect that has been deployed as part of an immutable smart contract? How do you predict transaction fees and determine the correct behaviour of your application if transaction fees and network volume increase unexpectedly?
  • Lack of Blockchain Testing Tools: Blockchain-based applications testing is all about tools. Selecting the right tool as per application is one of the important decisions.
  • Defining Test Strategy: Designing Test Strategy for Blockchain application is quite complex exactly because you need to have a good understanding and in-depth knowledge of the technology.
  • Block and Chain Size: Testing for block size and chain size is quite important as the application may fail without proper validation of block size and chain size and creating a false-positive verification.
  • Integration Testing: Should be done properly and frequently to test that all the components are properly integrated.
  • Performance and Load: You need to perform load testing to give better insight into how the Blockchain application performs in the real world, check if it is scalable and what are the limits. Check the example that was done to Ethereum and the Cryptokitties.
  • Security: Securing the data should be the most important thing in a Blockchain Application, to be honest, in any application. Even bitcoin is not bug free !

What do you need to consider when testing a Blockchain Application?

Here it goes an example with Bitcoin transaction tests:

  1. Block Size: The maximum fixed limit of a block is 1 megabyte. After the introduction of Bitcoin, the average size of a block for the first 18 months came out to be under 30 KB. But in December 2017, it hovered around 1 MB. What if the size of a block exceeds beyond 1 MB?
  2. Chain Size: There is no limit on the size of the chain. So, it is fun to test it for its function and performance. For example, the Bitcoin chain’s size keeps on increasing day by day.
  3. Load: With so many people on the blockchain, the load becomes a major parameter to test in a blockchain. Bitcoin currently has a maximum throughput of 3.3 – 7 transactions per second, but what if the transaction/second increases as in the case of Visa(2000), Paypal(193), etc? The load remains a major problem with blockchain because performance drops when the load increases.
  4. Security: Since there are many miners involved with a transaction, ensuring security is a little complex. Well, there is a multi-layered security system in a blockchain. If one of the layers has been hacked, the instantaneous transactions cannot be stopped. It is, therefore, to be tested that one security layer doesn’t affect the other.
  5. Transmission of data: Encrypted and decrypted data is transferred from computer to computer, so it is necessary to test if the transmission process is working flawlessly. Is the data being sent received on the other end, or is there a loss in between?
  6. Addition of block: Every new block is added to the chain once the transaction’s validity is authenticated. So, it must be tested that there should not be any leak in the block addition system and the block must be added after authentication.
  7. Cryptographical data: Cryptography is the backbone of blockchain technology. Therefore, it is necessary to make sure that the data is properly encrypted and decrypted.

Blockchain Testing Tools

  1. Ethereum Tester: It is an open-source testing library available as a Github repo. Its setup is pretty easy with a manageable API support for various Testing requirements.
  2. BitcoinJ: It is a Java-based framework built for Bitcoin-based apps that enables you to interact with the real BTC network and various testing activities. In order to use it, you don’t have to download the standard BTC Core files from Bitcoin.com. You can even approach a user forum in case you need clarification or are facing hiccups in the testing process. It is an open network available for assistance.
  3. Populus: This framework has the testing functionality of Ethereum embedded in the form of a set of features for test contract deployment. It’s developed around the py.test framework. Hence, it is relatively easy to implement.
  4. Truffle: It’s a commonly referred name for Ethereum developers, which brings in good testing features, such as automated contract testing. The framework holds capabilities beyond just testing functionality within the Blockchain application.
  5. Embark: It is a testing framework that focuses on developing decentralized applications (dApps) that run on various systems or nodes. It has integrations with Ethereum blockchain, IPFS, and a decentralized communication platforms such as Whisper and Orbit.

Future of Blockchain

Even though Blockchain is often discussed in financial services and cryptocurrencies, this technology offers a broader range of potential applications. The use case of Blockchain technology is picking up pace in various industries and segments. A World Economic Forum report predicts that by 2025, 10% of Global Gross Domestic Product (GDP) is stored on Blockchains or Blockchain-based technology.

Free courses

Resources:

https://www.etestware.com/why-companies-need-blockchain-testing/
https://www.guru99.com/blockchain-testing.html
https://www.cigniti.com/blog/5-popular-tools-for-testing-blockchain-applications/
https://www.testingxperts.com/services/blockchain-application-testing/gb-en
https://blog.b9lab.com/testing-blockchain-applications-not-just-for-testers-bb5932981df4
https://dzone.com/articles/all-you-need-to-know-about-blockchain-testing
https://blogs.iadb.org/caribbean-dev-trends/en/blockchain-technology-explained-and-what-it-could-mean-for-the-caribbean/
https://www.youtube.com/c/exodusmovement/videos

PACT CI/CD Workshop

Hello all, after a small burnout and a month break from my extra work activities (2nd lockdown here in London and winter vibes definitely didn’t help), I am finally back on track and I thought it was worth sharing this CI/CD Pact workshop that was published in the middle of this year.

They use Node, but you don’t need to be an expert to try this workshop. Once you have finished, you will be able to fit Pact and Pactflow into your CI/CD pipelines and understand the workflow when changing the consumer and provider.

I followed all the steps and after around 1h20m I finished it. It is pretty straight forward and it includes the new feature pending pact that is equivalent to the pending tests tag, just make sure you are copying the right tokens and using the right URLs.

The link to the workshop with the instructions and requirements are here:

https://docs.pactflow.io/docs/workshops/ci-cd/

In the end I got the results:

Travis Passed

✅ Provider Passed Locally

✅ Consumer Passed Locally

✅ Pactflow verified

Thanks to Matt Fellows and Beth Skurrie for that !

Special thanks to my friend Marie who suggested me to use Excalidraw for my awesome diagrams !

AI for Testing: Beyond Functional Automation webinar

Hello guys, I joined a webinar some months ago (15/07/2020) about AI for Testing: Beyond Functional Automation by Tariq King which was really interesting ! I know how it’s hard to keep up with all the online events now, so I always try to keep the recording of the ones that I couldn’t join and are interesting to listen to when I have time.

So thought about sharing with you as well in case you missed. You will learn about reinforcing learning by giving scores to the right actions and about training bots to recognize good and bad designs with examples. This allows the framework to be more robust when searching for a particular query or asserting the scenarios:

 

Here it is the link to the recording:

Thanks Tariq King !

Test Data Management Strategies

Hello all,

Today I am going to talk about some different approaches to handle your test data when running automated tests and the trade-offs.

 

Database

Injecting the data before running the tests with SQL, mysql or postgresql scripts are one of the most common approaches. So, you can inject the data you will need for the tests and skip all the setup, which is not the goal of all your scenarios, right ?

For the scenarios that you actually need to test the creation of the data then you won’t use this kind of script. For example in javascript, you would add a setup/data management class, a @BeforeAll and then something like this:

var mysql = require('mysql');
var con = mysql.createConnection({
     host: "localhost",
     user: "root",
     password: "12345",
     database: "javatpoint"
});  

con.connect(function(err) {
     if (err) throw err;
       console.log("Connected!");
       var sql = "INSERT INTO employees (id, name, age, city) VALUES ('1', 'Ajeet Kumar', '27', 'Allahabad')"; 
       con.query(sql, function (err, result) {
     if (err) throw err;
       console.log("1 record inserted");  
     });
});

Then you can have a @TearDown, @AfterAll function to delete the data that was created to be used during the tests.

Files

If, for example, you are running some API tests you might want to have static data ready to be injected for each scenario. You can create a json file and add all the fields and values that are going to be used during your automation:

 { 
   name: "John", 
   age: 31, 
   city: "New York" 
},
{
   name: "Rafa", 
   age: 29, 
   city: "London" 
}

Then you can load this file to be used during your tests. You can create this data upfront, but then you need to make sure that this data is always going to be there otherwise you need to create it again (during your tests or manually).

 

Objects

You can create Objects with the data that you are going to need for the automated tests, so for example you can create a dictionary in Javascript:

var dict = {
  FirstName: "Rafa",
  Age: 30,
  Country: "UK"
};

Then again you need to make sure you are going to create this data during runtime, maybe in a @BeforeAll function or a Setup class, or maybe this is something you have created in the environment already and you need to make sure this is going to be there when running the tests, otherwise you need to create it again.

 

Docker

If you can control the database or the deployment of your QA environment, then it means you can also manipulate the database when running the tests.

If you use docker to create the environment you can add a Volume or even seed the database with docker-compose.

Volume

Volumes are often a better choice than persisting data in a container’s writable layer because a volume does not increase the size of the containers using it, and the volume’s contents exist outside the lifecycle of a given container.

You can push the database (json file, .db) entirely to the docker container:

 docker run -it --name my-directory-test -v /hostvolume:/containervolume centos /bin/bash

Seed

Write a small script that generates randomized and varying data and writes it to the database. Then you can wrap this script into your own Docker image in order to execute them automatically via docker-compose.

 

In this example I am using a mongoDB database:

docker-compose.yml

version: '1.0'

services:

  mongodb:
    image: mongo
    container_name: mongo
    ports:
      - 27017:27017


  mongo-seed:
    build: .
    environment:
      - MONGODB_HOST=mongo
      - MONGODB_PORT=27017
    volumes:
      - ./config/db-seed:/data
    depends_on:
      - mongo
    command: [
      "mongoimport --host mongo --port 27017 --db testautomation --mode upsert --type json --file data.json --jsonArray"
      ]

data.json

[
  {
    "name": "Peter Parker",
    "email": "spiderman@gmail.com",
    "age": 28
  },
  {
    "name": "Bruce Wayne",
    "email": "batman@gmail.com",
    "age": 48
   }
]

 

Scenarios

If you are working with Gherkin syntax, it means you can also add the data in the middle of the scenario and then use it during the automation. So, something like:

Scenario: Correct number of movies found by superhero
Given I have the following movies
| Batman Begins | Batman |
| Wonder Woman | Wonder Woman |
| Wonder Woman 1984 | Wonder Woman |
When I search for movies by superhero Wonder Woman
Then I find 2 movies

Then you can get this data from the step definitions and use during yours tests.

You might have other ways to create and manage the test data, but whatever the approach you decide, make sure the scenarios are independent and if you can clean up the environment data after (unless you have decided to have static data in the environment for now) then clean it.

 

Resources:

https://forums.docker.com/t/seeding-data-volume-containers-mongodb/2214

https://stackoverflow.com/questions/31210973/how-do-i-seed-a-mongo-database-using-docker-compose

https://www.baeldung.com/cucumber-data-tables

https://docs.docker.com/storage/volumes/

https://phauer.com/2018/local-development-docker-compose-seeding-stubs/

Developing a Test Strategy

Hello everybody,

In case you have missed here it is the link for the meetup about Developing a Test Strategy 30/07/2020.

 

If you can speak portuguese and don’t feel comfortable with english yet you can also watch the portuguese version here:

Quality? Who Cares?

Steve Watson - Musings of a Test Manager

A few weeks ago I was really fortunate enough to be involved in delivering a Unicom talk on Quality, followed a week later by a round table talk which I hosted, and was set up by Billy Senior.

The topic of Quality is something that intrigues me in the context of software engineering. Most of my career has been dedicated to checking whether the work of someone else does what it is meant to do, and doesn’t do something that it shouldn’t. It sounds bizarre when you think of testing as just that – we are validating that a software engineer has written code that meets the requirements and expectations of an individual or group of individuals. 

But quality is not just about testing to see if something works as it should – plenty of things ‘work’, but the experience is awful, or it takes a long time to…

View original post 628 more words

TestProject New Python SDK

I have adventured myself to test the new TestProject Python SDK this week and I can say it has been quite straight forward. Also, the documentation is extensive and cover a lot of different scenarios and setups.

For those who don’t know, TestProject is a Free Automation Platform that wrappers open source test frameworks (Selenium and Appium) integrating your automation scripts. It consolidates all the needed drivers to run your test automation without additional setup.

1- To start you need to get SDK token from the TestProject Portal (you can register for free here)

2- Download and install TestProject Agent

3- Run the agent locally and verify the status

4- Install the latest version of python (the min. supported Python version is 3.4)

pip install testproject-python-sdk

5- Generate and copy your developer token

6- Create your first test, for example

7- Then you can see the reports published on your TestProject account, for example

 

You can find a lot more examples on their README file.

 

Resources:

https://testproject.io/

https://github.com/testproject-io/python-sdk

Open Banking Functional Conformance Suite Test Cases

What is Open Banking ?

Open banking allows the use of open APIs enabling third-party developers to build applications and services around financial institutions. It comes to bring more financial transparency options for account holders ranging from open data to private data.

Open Banking Use Cases (for Users) | by Ştefan Alexandru Băluţ ...

Open Banking Functional Conformance Suite

To be able to get the Functional Conformance Certificate, Open Banking provides a Functional Conformance Tool to allow implementers to check if your API has successfully developed all required functional elements of the OBIE Read/Write API Specifications.

This Open Banking tool allows an ASPSP (Account Servicing Payment Service Provider) and a TPP (Third Party Provider) to test the response of any API endpoint and validate that the JSON and data formats meet the schema, permissions and interfaces against the Functional API standard.

How to identify Test Cases covered in the OB Functional Conformance Suite ?

How do you know what else needs to be covered and if there is indeed something more to cover ? After digging into the project on bitbucket, I found some useful json files where you can check the assertions for each test case, the test cases itself and another file to translate the list of the assertions.

So, you can find the asserts that are being done for each test case inside the manifests folder.

For example, this one contains the assertions for this test case: The x-fapi-interaction-id is replayed for an Account. You can find the file with the accounts transactions test cases here.

Screenshot 2020-07-13 at 17.48.08

Then you would need to check what this assertion actually means, and you can find the dictionary of the assertions on this file.

Screenshot 2020-07-13 at 17.52.47

Remember that all the tests currently assume that consent is granted at the ASPSP portal for each requested PSU Consent (Payment Service User Consent).

Also, you will find that some test cases are missing for instance what should happen when you send an invalid token to the payments endpoint, but you can see there is a test case for the accounts endpoints for when you send a token without the required permissions to get a 401 response.

In this example, you can see that for payments the consent model is a bit different because each access token doesn’t have a range of permissions, but is associated with a single payment consent id. So, in order to get a 401 response, the request can present the wrong token along with a payment call or present no token at all. The conformance tool is not sending any token in this instance.

So make sure you are aware and cover the missing test cases with another approach.

I found quite hard to have a straight answer about what are all the test cases they are covering and also the details, so hope this helps to have a bit more clarity in case you are having the same issues.

Resources:

https://openbankinguk.github.io/knowledge-base-pub/conformance-tools/

https://openbanking.atlassian.net/wiki/spaces/DZ/pages/1061716467/Functional+Conformance

https://medium.com/zoidcoin-network/open-banking-use-cases-for-users-8678d11d770b

https://en.wikipedia.org/wiki/Open_banking

Load tests: Jmeter vs K6

Hello all,

Today it’s the turn of Jmeter and K6 ! As always, remember to check your other options and see what better fits for your project.

Jmeter is a great and powerful tool, but depending on what you really need (something more lighter) then Jmeter might become an over complex, slow, hard to maintain tool.

Jmeter K6
In-built Protocols Support
  • HTTP
  • FTP
  • JDBC
  • SOAP
  • LDAP
  • TCP
  • JMS
  • SMTP
  • POP3
  • IMAP
  • HTTP 1.1
  • HTTP 2
  • WebSockets
Speed to write tests
  • Slow
  • Fast
Support of “Test as Code”
  • GUI oriented
  • Possibility to create scripts, but too complex and lack of documentation
  • Weak (Java)
  • Hard to maintain
  • Scripts oriented
  • JavaScript
  • Easier to maintain
Ramp-up Flexibility
  • Plugins available to be able to configure flexible load
  • Supports ramp-up phases and flexible load
Test Results Analyzing
  • Yes
  • Yes
Resources Consumption
  • Heavy to run tests with multiple users on a single machine, more memory consumption
  • Lightweight and doesn’t take up so much memory of your machine

Screenshot 2020-07-06 at 23.34.47

Easy to use with Version Control Systems
  • No
  • Yes
Number of Concurrent Users
  • Thousands, under restrictions
  • Thousands
Recording Functionality
  • Yes
  • No, but it allows to auto-generate a k6 script via an HAR file
Distributed Execution
  • Yes
  • Yes
Load Tests Monitoring
  • Add listeners, but consume more memory

Screenshot 2020-07-06 at 23.35.02

Jmeter is most used when:

  • You need to perform a complex load including different protocols
  • You can record scenarios
  • Robust support and training ecosystem
  • Require that a full scenario be written for every test
  • If you need to simulate specific load with some custom ramp-up patterns
  • If you just prefer UI desktop app for scripts creation, or you just do not know Javascript/YAML/JSON well enough

 

K6 solves some specific problems:

  • CLI tool with developer-friendly APIs.
  • You can use HAR files to generate record sessions
  • Checks and Thresholds – for goal-oriented, automation-friendly load testing
  • Open source, great support and documentation
  • Lightweight uses Javascript
  • Does not run in NodeJS and doesn’t run in a browser

 

Resources:

https://k6.io/

Load Tests: Jmeter vs Gatling

Hello guys,

Continuing on reviewing some performance test tools, today is the turn of Jmeter and Gatling, which looks like more and more people are using nowadays. Remember always check your other options and see what better fits for your project.

 

Jmeter is a great and powerful tool, but depending on what you really need (something more lighter) then Jmeter might become an overcomplex, slow, hard to maintain tool.

Jmeter Gatling
In-built Protocols Support
  • HTTP
  • FTP
  • JDBC
  • SOAP
  • LDAP
  • TCP
  • JMS
  • SMTP
  • POP3
  • IMAP
  • HTTP
  • JMS
  • MQTT
Speed to write tests
  • Slow
  • Fast
Support of “Test as Code”
  • GUI oriented
  • Possibility to create scripts, but too complex and lack of documentation
  • Weak (Java)
  • Hard to maintain
Ramp-up Flexibility
  • Plugins available to be able to configure flexible load
  • Supports ramp-up phases and flexible load
Test Results Analyzing
  • Yes
  • Yes
Resources Consumption
  • Heavy to run tests with multiple users on a single machine, more memory consumption
  • Lighweight and doesn’t take up so much memory of your machine

Screenshot 2020-06-20 at 14.38.36

Easy to use with Version Control Systems
  • No
  • Yes
Number of Concurrent Users
  • Thousands, under restrictions
  • Thousands
Recording Functionality
  • Yes
  • Yes
Distributed Execution
  • Yes
  • Yes
Load Tests Monitoring
  • Add listeners, but consume more memory
  • Yes, logs through the console and reports are created at the end
Screenshot_2020-06-20 Gatling Stats - Global Information

 

Jmeter is most used when:

  • You need to perform a complex load including different protocols
  • You can record scenarios
  • Robust support and training ecosystem
  • Require that a full scenario be written for every test
  • If you need to simulate specific load with some custom ramp-up patterns
  • If you just prefer UI desktop app for scripts creation, or you just do not know Javascript/YAML/JSON well enough

 

Gatling solves some specific problems:

 

Resources:

gatling.io/