Blockchain Tests Workshop at The National Software Testing Conference

Hello guys, here are the slides from the workshop on Blockchain Tests that I gave earlier this week, as well as some responses to the issues that were addressed during the session. I can say as my first presential workshop after pandemic, it was a great experience with a full room 😃


Unfortunately, there is no recording of it, but you may clone the repo and follow the coding instructions to build the test class and methods, then compare to version 2 of the project, which has the most recent version.

What is the Required() function in Solidity?

The require Solidity function guarantees validity of conditions that cannot be detected before execution. It checks inputs, contract state variables and return values from calls to external contracts.

Where the Address type comes from ?

The address type comes in two flavours, which are largely identical:

  • address: Holds a 20 byte value (size of an Ethereum address).
  • address payable: Same as address, but with the additional members transfer and send.

The idea behind this distinction is that address payable is an address you can send Ether to, while a plain address cannot be sent Ether.

Type conversions

Implicit conversions from address payable to address are allowed, whereas conversions from address to address payable must be explicit via payable(<address>).

Explicit conversions to and from address are allowed for uint160, integer literals, bytes20 and contract types.

Other tools that you can use to test Blockchain applications

  • Ethereum Tester,
  • Hyperledger Composer
  • Exonum test kit
  • Embark
  • Populus
  • BitcoinJ
  • Binance

Resources

Blockchain Tests with Truffle (Solidity)

Hello guys !! After my talk about Testing Blockchain Applications on the #TechKnowDay,  I thought it was worth posting a guide on the same topic as I felt it was complex to also understand Blockchain. So, this is a quick tutorial on how to run tests with Truffle Framework.

If you are still wondering where you can use Blockchain, here it’s a table with the percentage of companies that are focusing on it and the use cases.

Requirements

We are going to use a bit of Javascript for the web part and Solidity for the blockchain project. For this project you will need to have installed NPM and Node already. We are going to install Truffle as part of the setup. You will also need MetaMask and Ganache, all of them you can find bellow:

Setup your Local Ethereum Blockchain

  • Open your MetaMask plugin and click on new Custom RPC Network. Type the following (Currency Symbol will be automatically populated after you type the Chain ID) and save.
  • Open Ganache, click on Quick Start (Ethereum). Double check if the server has this configuration clicking on the on top right and then Server tab.
  • Click on Show keys of the account you have created and copy the private key.
  • Back on MetaMask, click on Import Account and add the private key you have copied from the previous step.

Setting up the project

On your terminal run:

npm install -g truffle

If you want to follow the test steps only, then you need to download the first release that contains the installation files of the project already:

https://github.com/rafaelaazevedo/PetShopTestWorkshop/releases/tag/1.0

If you want to try the installation for yourself from the scratch, then just go to this link and follow the Getting Started guide. This guide won’t be focusing on the installation of the framework or the setup of the contracts and migrations.

Just a heads up that for this project I am using the pet-shop box instead of the MetaCoin from the Getting Started guide.

Installation

After downloading the project, you will need to run some commands to set it up. Open your terminal on the root of the project and run:

Run the development console

truffle develop

Compile and migrate the smart contracts. Inside the development console you don’t need to type the truffle command.

compile
migrate

To get out of the truffle console type

.exit

Create a new file inside of the test folder called TestAdoption.sol and import all the needed modules, such as assertions, new instances of deployed addresses and the contract that will be tested.

pragma solidity >=0.5.0;

import "truffle/Assert.sol";
import "truffle/DeployedAddresses.sol";
import "../contracts/Adoption.sol";

Then create the class and inside add the variables:

  • adoption that will get the Adoption contract that was deployed
  • expectedId this one will be the id of the pet under test (You can get the id of the pet you want to test from this file)
  • expectedAdopter which is the address of this current contract that we are creating in the next function
contract TestAdoption {
    Adoption adoption = Adoption(DeployedAddresses.Adoption());
    uint256 expectedPetId = 8;
    address expectedAdopter = address(this);
}

Testing

Asserting the adoption

  • Add the testUserCanAdoptPet() function below the variables block.
  • Note that here you are creating a function that will test you can adopt a pet and for this you will need to get the address related to the adoption transaction and compare the returnedId with the expectedPetId address
  • Try to explore and add other asserts like checking if it’s not returning null.
function testUserCanAdoptPet() public {
        uint256 returnedId = adoption.adopt(expectedPetId);

        Assert.equal(
            returnedId,
            expectedPetId,
            "Adoption of the expected pet should match what is returned."
        );
    }

Asserting the adopter of the pet

  • Add the testGetAdopterAddressByPetId() function below the previous function.
  • This function will check if the adopter address for that pet is the same from the adopters list
  • Try to explore and add other asserts like comparing the age of the pet is returning correctly, for that you would need to add the age on the Adoption.sol contract, then compile and migrate again.
function testGetAdopterAddressByPetId() public {
        address adopter = adoption.adopters(expectedPetId);

        Assert.equal(
            adopter,
            expectedAdopter,
            "Owner of the expected pet should be this contract"
        );
    }

Asserting the list of adopters

  • Now add the testGetAdopterAddressByPetIdInArray() function below the previous function.
  • This function will check if the memory address for this petId is the same as the expectedAdopter
  • Almost the same test as before, but this time we are explicitly storing adopters in memory rather than contract’s storage and then comparing them.
function testGetAdopterAddressByPetIdInArray() public {
        address[16] memory adopters = adoption.getAdopters();

        Assert.equal(
            adopters[expectedPetId],
            expectedAdopter,
            "Owner of the expected pet should be this contract"
        );
    }
}

You should have something like this:

pragma solidity >=0.5.0;

// The first two imports are referring to global Truffle files, not a `truffle` directory.
// Gives us various assertions to use in our tests.
import "truffle/Assert.sol";

// When running tests, Truffle will deploy a fresh instance of the contract being tested to the blockchain.
import "truffle/DeployedAddresses.sol";

// The smart contract we want to test.
import "../contracts/Adoption.sol";

contract TestAdoption {
    // The address of the adoption contract to be tested
    Adoption adoption = Adoption(DeployedAddresses.Adoption());

    // The id of the pet that will be used for testing
    uint256 expectedPetId = 8;

    //The expected owner of adopted pet is this contract
    address expectedAdopter = address(this);

    // Testing the adopt() function
    function testUserCanAdoptPet() public {
        uint256 returnedId = adoption.adopt(expectedPetId);

        Assert.equal(
            returnedId,
            expectedPetId,
            "Adoption of the expected pet should match what is returned."
        );
    }

    // Testing retrieval of a single pet's owner
    function testGetAdopterAddressByPetId() public {
        address adopter = adoption.adopters(expectedPetId);

        Assert.equal(
            adopter,
            expectedAdopter,
            "Owner of the expected pet should be this contract"
        );
    }

    // Testing retrieval of pet owner storing getAdopters in memory
    function testGetAdopterAddressByPetIdInArray() public {
        // Store adopters in memory rather than contract's storage
        address[16] memory adopters = adoption.getAdopters();

        Assert.equal(
            adopters[expectedPetId],
            expectedAdopter,
            "Owner of the expected pet should be this contract"
        );
    }
}

Running the tests

  • Open your terminal on the root of the project and run:
truffle test
  • If everything went okay you will see green checks on your terminal like this:

You can check the final code with the latest release (Spoiler alert: You will see pictures of my dog, my dog’s best friend, my friend’s cat and my previous dog)

git clone git@github.com:rafaelaazevedo/PetShopTestWorkshop.git

Open a tab on your terminal and run the development local server

npm run dev

You can also check truffle config file to check the host and port where we are running the project and edit if you need to.

Releases of the project:

  • Release 1.0 Contains the Solidity contracts and Truffle Migrations only (No UI)
  • Release 1.1 Contains the Solidity contracts, Truffle Migrations and Truffle Initial Tests only (No UI)
  • Release 1.2 Contains the Solidity contracts, Truffle Migrations, Truffle Initial Tests and Initial UI
  • Release 1.3 Contains the Solidity contracts, Truffle Migrations, Truffle Initial Tests, Final version of UI

Testing Blockchain Applications

Hello all, today I am going to post about a topic that I am a huge enthusiast about: Blockchain !

Nowadays data is one of the most valuable product that you can own, and as we are surrounded by fake data coming from thousands of different places, it is hard to know what is trustworthy or not.

What is Blockchain ?

It is basically the second phase of the internet. The Internet democratized the exchange of information, blockchain promises to democratize the exchange of real value. It was created in 2008 by Satashi Nakamoto, which the identity is still unknown.

An example of how blockchain can be used is when you want to make a transaction of £10 from a X to Y. Nowadays what happens is this transaction goes through a third-party app or payment processing system. Then, X’s bank will need to check the details of Y’s bank and once this is done the transfer of the amount will initiate with a certain amount of deductions, of course. In the end both banks will record the transaction deducting the transaction fee, meaning Y will receive something like £9.95. Even though this process is quite secure and has many redundancies in place to make sure that it stays secure and accurate, there are some basic fundamental issues with this process:

  • Delay in processing (using this middle agent and all the bureaucracy)
  • Dependency on a single intermediary whose effectiveness is never 100 percent (dependency on a national state)
  • In case of any gap in the transaction, no one takes the responsibility and people keep on blaming each other (not concrete proof to check the transaction)

Blockchain is a system where you no longer have to worry about these problems, you just need to perform a transaction and there is a chain of people who are able to validate your transactions every second. This mechanism is called as Proof of block and this is done based on the public key provided for the encrypted data, and it is done by all listeners in the peer-to-peer network. They are real people/identities, not bots. Since there is not a single transaction validating authority (banks, national state) who are centralising the transaction service, the process is effectively decentralized, public and transparent for everybody to see.

Once a specified number of people validates the transaction, the transaction details are stored in the form of a block and that block is added to the existing blockchain. Once the blocks are validated, nobody can change them, just add another one on top, they are immutable. These blocks have a specific hash associated with every block. These hashes are like fingerprints and are unique to every block. Every person that is validating the transaction process is called a miner. The more miners, the better efficiency of the transaction.

A block contains data, a hash, and the hash of the previous block. Since it contains a hash of a previous block, all the blocks contain data for the previous blocks and it becomes almost impossible for a blockchain to be corrupt.

What types of tests can I perform in a Blockchain app ?

  • API Testing: In API testing, we ensure that the interaction between applications in the blockchain ecosystem is as expected
  • Block/Peer/Node Testing: All the blocks on the Network should be tested individually to ensure proper cooperation. All diverse nodes on the Network must be tested independently to ensure smooth cooperation. Do the nodes in the network sync with other validating peers? Is the integrity of the network and shared ledger maintained throughout the testing?
  • Functional Testing: In Functional Testing, we evaluate the work of various functional parts of the Blockchain (e.g., smart contracts).
  • Performance Testing: Details like network latency based on block size, network size, expected transaction size, and how long a query takes to return the output with the specialized authentication protocol
  • Security Testing: Here, we ensure that the application is not vulnerable to attacks and the systems can protect the data and are capable of handling malicious attacks, etc.
  • Integration Testing: In Integration testing, we ensure that all the components of the application are integrated properly and performing the actions appropriately
  • Smart Contract Testing: Smart Contract testing is about performing detailed functional testing of business logic and process. It refers to the set of software constructs that automatically execute transactions when predefined conditions and business logic are met. Testing smart contracts involve simulation of all possible expected and unexpected variables for every contract and the triggers that execute transactions.

Benefits of Blockchain Apps

  • Decentralized System: No need to have a middle agent for the transactions. Beneficial in various industries like finance, real estate etc.
  • Better Security: As it uses multiple nodes to complete and authenticate transactions, you don’t rely only on a transaction system that is controlled by a centralized group of identities.
  • Authenticity: Allows the unique algorithm to process data, like a fingerprint.
  • Increased Capacity: Increases the capacity of the entire Network as it is completely decentralized and real people can verify the blockchain.

Challenges in Blockchain Testing

  • Understanding the Technology: Blockchain is a new technology and understanding it is very important in order to test it. Where does regulatory compliance overlap with quality? How do you fix a defect that has been deployed as part of an immutable smart contract? How do you predict transaction fees and determine the correct behaviour of your application if transaction fees and network volume increase unexpectedly?
  • Lack of Blockchain Testing Tools: Blockchain-based applications testing is all about tools. Selecting the right tool as per application is one of the important decisions.
  • Defining Test Strategy: Designing Test Strategy for Blockchain application is quite complex exactly because you need to have a good understanding and in-depth knowledge of the technology.
  • Block and Chain Size: Testing for block size and chain size is quite important as the application may fail without proper validation of block size and chain size and creating a false-positive verification.
  • Integration Testing: Should be done properly and frequently to test that all the components are properly integrated.
  • Performance and Load: You need to perform load testing to give better insight into how the Blockchain application performs in the real world, check if it is scalable and what are the limits. Check the example that was done to Ethereum and the Cryptokitties.
  • Security: Securing the data should be the most important thing in a Blockchain Application, to be honest, in any application. Even bitcoin is not bug free !

What do you need to consider when testing a Blockchain Application?

Here it goes an example with Bitcoin transaction tests:

  1. Block Size: The maximum fixed limit of a block is 1 megabyte. After the introduction of Bitcoin, the average size of a block for the first 18 months came out to be under 30 KB. But in December 2017, it hovered around 1 MB. What if the size of a block exceeds beyond 1 MB?
  2. Chain Size: There is no limit on the size of the chain. So, it is fun to test it for its function and performance. For example, the Bitcoin chain’s size keeps on increasing day by day.
  3. Load: With so many people on the blockchain, the load becomes a major parameter to test in a blockchain. Bitcoin currently has a maximum throughput of 3.3 – 7 transactions per second, but what if the transaction/second increases as in the case of Visa(2000), Paypal(193), etc? The load remains a major problem with blockchain because performance drops when the load increases.
  4. Security: Since there are many miners involved with a transaction, ensuring security is a little complex. Well, there is a multi-layered security system in a blockchain. If one of the layers has been hacked, the instantaneous transactions cannot be stopped. It is, therefore, to be tested that one security layer doesn’t affect the other.
  5. Transmission of data: Encrypted and decrypted data is transferred from computer to computer, so it is necessary to test if the transmission process is working flawlessly. Is the data being sent received on the other end, or is there a loss in between?
  6. Addition of block: Every new block is added to the chain once the transaction’s validity is authenticated. So, it must be tested that there should not be any leak in the block addition system and the block must be added after authentication.
  7. Cryptographical data: Cryptography is the backbone of blockchain technology. Therefore, it is necessary to make sure that the data is properly encrypted and decrypted.

Blockchain Testing Tools

  1. Ethereum Tester: It is an open-source testing library available as a Github repo. Its setup is pretty easy with a manageable API support for various Testing requirements.
  2. BitcoinJ: It is a Java-based framework built for Bitcoin-based apps that enables you to interact with the real BTC network and various testing activities. In order to use it, you don’t have to download the standard BTC Core files from Bitcoin.com. You can even approach a user forum in case you need clarification or are facing hiccups in the testing process. It is an open network available for assistance.
  3. Populus: This framework has the testing functionality of Ethereum embedded in the form of a set of features for test contract deployment. It’s developed around the py.test framework. Hence, it is relatively easy to implement.
  4. Truffle: It’s a commonly referred name for Ethereum developers, which brings in good testing features, such as automated contract testing. The framework holds capabilities beyond just testing functionality within the Blockchain application.
  5. Embark: It is a testing framework that focuses on developing decentralized applications (dApps) that run on various systems or nodes. It has integrations with Ethereum blockchain, IPFS, and a decentralized communication platforms such as Whisper and Orbit.

Future of Blockchain

Even though Blockchain is often discussed in financial services and cryptocurrencies, this technology offers a broader range of potential applications. The use case of Blockchain technology is picking up pace in various industries and segments. A World Economic Forum report predicts that by 2025, 10% of Global Gross Domestic Product (GDP) is stored on Blockchains or Blockchain-based technology.

Free courses

Resources:

https://www.etestware.com/why-companies-need-blockchain-testing/
https://www.guru99.com/blockchain-testing.html
https://www.cigniti.com/blog/5-popular-tools-for-testing-blockchain-applications/
https://www.testingxperts.com/services/blockchain-application-testing/gb-en
https://blog.b9lab.com/testing-blockchain-applications-not-just-for-testers-bb5932981df4
https://dzone.com/articles/all-you-need-to-know-about-blockchain-testing
https://blogs.iadb.org/caribbean-dev-trends/en/blockchain-technology-explained-and-what-it-could-mean-for-the-caribbean/
https://www.youtube.com/c/exodusmovement/videos

Blockchain Testing Tools

If you are wondering what Blockchain is, I will give you a quick introduction. Blockchain is a data structure that is distributed at once in many different places and as you can’t ever delete from it, it is extremely difficult to make amendments. This makes the record more secure and more trustworthy.

So, what are the kinds of test that you can perform ? You can use the traditional testing, since it is just normal development work with normal testing criteria. So, boundary value analysis, decision tables, test driven development and behavior driven development techniques.

There is also a set of questions that can help you to build your test scenarios, for example:

  • How does it handles valid and invalid inputs?
  • How does it cope with a wide range of input data?
  • How does it handle missing state, or existing state?
  • How does it handle error cases?
  • How does it handle security and access control?

You don’t need to test the Blockchain because the algorithms are well-established, because it is a distributed system, but the transactions still require some kind of validation. For example, you may need to check if your transaction is valid before it can be approved. There are approval authorities for different blockchains, and they must test the integrity of the transactions.

 

What is Smart Contract ?

Smart Contract is an API and defines the rules for transactions in a Blockchain network. A Smart Contract is a set of rules in the form of programmable constructs that are capable of automatically enforcing themselves when pre-defined conditions are met.

It has public functions which might be called by anyone registered on the Blockchain network. However, unlike a traditional API, a Smart Contract cannot call external web APIs.

 

What do you need to know to test Ethereum Smart Contracts ?

Test automation requires that the platform being tested must have hooks so that external automated scripts can instruct the platform, observe the outcome, and verify that the outcome is what is expected. Legacy platforms in banking often do not have these hooks, and that makes automation much more difficult. When you compare smart contracts to older software used in banks, you can automate testing much earlier and much faster.

 

I will show some of the tools that you can use to perform tests on Blockchain applications:

 

Ethereum Tester

This github has a project for you to test Ethereum Blockchain Applications. You will need to clone Eth-Tester. The Eth-tester library strictly enforces some input formats and types.

 

Truffle

Truffle is one of the most popular Ethereum development frameworks and has testing functionality, it is a scaffolding framework for smart contracts used by UI builders. You have the ability to write automated tests for your contracts in both JavaScript and Solidity and get your contracts developed quickly.

Ganache

Ganache is the most-used library for testing Ethereum contracts locally. It mocks a blockchain that gives you access to accounts you can run tests, execute commands, etc.

 

Populus

By default tests run against an in-memory ethereum blockchain and as you can see here Populus supports writing contracts that are specifically for testing.

 

Manticore

Manticore is a symbolic execution tool for analysis of binaries and smart contracts. It is supported on Linux and requires Python 2.7. Ubuntu 16.04 is strongly recommended. It has a Python programming interface which can be used to implement custom analyses. You can see more about here on the wiki.

 

Hyperledger Composer

Hyperledger Composer supports three types of testing: interactive testing, automated unit testing and automated system testing. It’s a framework for rapidly building Blockchain business networks on top of a Blockchain platform, such as Hyperledger Fabric.

This framework allows you to automatically deploy the business network definition, and then interact with it by submitting queries and transactions in order to test how that business network really behaves.

 

Corda Testing Tools

Corda is a blockchain-inspired, open-source distributed ledger platform. There are several distinct test suites each with a different purpose: Unit tests, Integration tests, Smoke tests, Load tests and other. These tests are mostly written with JUnit and can run via Gradle.

 

BitcoinJ

This tool BitcoinJ allows you to interact with Bitcoin connecting directly to the bitcoin network. So, you can simulate send and receive transactions in real time, also you don’t need to have a local copy of the Bitcoin Core.

You can follow this guide to get start with this tool.

 

Resources:

https://www.joecolantonio.com/2018/02/01/blockchain-testing-tools/

https://joecolantonio.com/testtalks/175-blockchain-application-testing-rhian-lewis/

http://searchsoftwarequality.techtarget.com/answer/Heres-everything-you-need-to-know-about-testing-blockchain

https://www.capgemini.com/2017/01/testing-of-smart-contracts-in-the-blockchain-world/

http://www.bcs.org/content/conWebDoc/56020

https://medium.com/@mrsimonstone/test-your-blockchain-business-network-using-hyperledger-composer-c8e8f112da08