Dappeteer vs Synpress

You asked you got it !! JK 😂 , this was in my backlog for a long time and finally had time to write something about it. This literally summarises my life:

Not many people out there talking about Blockchain Tests, maybe Oleksandr Romanov and Rhian Lewis, so I usually resource to communities (Web3Tests, Synpress…) to see what others are doing.

It is a bit shocking as there are many well-known attacks in this technology and on top of that after deploying something to the Blockchain you can’t go back (immutability feature), so how come this phase is so neglected as making sure everything is alright is so crucial ?

I am doing a quick benchmark about the End-to-End Tests Web3 Frameworks that are out there, and yes you can always mock the Web3 and then you don’t rely on third party integrations, which you would do any Web2 application already. But hey, this is one option to use when you need it !

Here it is what I got from the two major and most popular tools to test Web3 Apps (DApps):

CriteriaSynpressDappeteer
Platform & Application SupportWeb3 applications (Dapps)Web3 applications (Dapps)
Supported TechnologiesTypeScript, JavascriptTypeScript, Javascript
Wallets supportedMetaMaskMetaMask snaps, but probably is outdated.
Testing ScopeE2E TestsE2E Tests
Testing Framework IntegrationPlaywright (as a plugin)
Support for Cypress is coming soon.
Playwright (as a plugin)
Puppeteer (as a plugin)
Ease of Use & Learning CurveEasy learning curve, requires JavaScript/Typecript knowledgeRequires JavaScript/Typescript knowledge
Reporting & AnalyticsDetailed reports with dashboards and integrationsBasic reporting, may require additional tools
Ease of setupEasy to set up with Node.js, yarn and npmEasy to set up with Node.js, yarn and npm
Browsers– Chrome/Chromium (+ Edge, Opera, Brave, Chromium-based browsers)
– Firefox
– Webkit (Safari)
– Chrome/Chromium (+ Edge, Opera, Brave, Chromium-based browsers)
– Webkit (Safari)
Community & SupportLarge and active community, extensive documentation and supportSmaller community, growing resources and documentation
Cost & LicensingOpen source and free to useOpen source and free to use
Scalability & IntegrationHighly scalable, integrates with CI/CD pipelinesLess information available, may require custom integrations
AccessibilitySupports accessibility testing with pluginsInformation not available.
CustomizationHighly customizable with plugins and extensionsHighly customizable with plugins and extensions

Let me know in the comments which option you went for and how was your experience with it 😃

Web3 Test Series: Lewis Prescott

Lewis Prescott

QA Lead at Cera Care (one of Europe’s fastest-growing companies). He talks about Contract API Testing: 99 minute workshops with Ministry of Testing, on Testers Island Discs and many meetups.

He provides services on how to get started with API Contract Testing, Contract Testing for Microservices, End to End Testing, Acceptance Test Driven Development!

What are contract tests, and how do they differ from smart contract tests?

 Contract testing is a framework for testing integrations between two services, for example an API and a mobile app. The contract is defined by the mobile application who requests data, shares the request and expected response with the API service which verifies the implementation matches the expectations. Imagine when you start developing a feature that has frontend and backend components. They must communicate with each other perfectly, in happy path and negative scenarios. These details are usually detailed within some form of documentation, but who is verifying that both sides have implemented according to the documentation? That’s where contracts come in.

Smart contract tests are also focused on testing agreements with the terms between contracts written into code, but the difference is that the contract that is compared against to is a contract deployed to a Blockchain. Same concept as normal contract testing but this time focused on the decentralised applications (DApps) and blockchain ecosystems.

Why are contract tests important in software development, and how do they differ from smart contract tests in blockchain development? 

Contract tests offer the ability to scale integration tests by using mocks and act as unit level tests. Large applications with many integration points can be verified quickly and without significant maintenance of test environments. There is plenty of logic such as http codes, required fields and error contents which can be verified via contracts without the need for heavy duty integration environments.

Smart contracts are the backbone of decentralized applications (DApps) and business logic. They help ensure that the code deployed on a blockchain accurately represents the intended business logic, functions correctly in different scenarios, and is resistant to security vulnerabilities, specially given the irreversible and immutable nature of blockchain transactions.

What are some of the challenges developers face when writing contract tests versus smart contract tests?

Contract tests has a bit overhead on setup, with provisioning the broker to store the contracts. Deciding whether you want to self host or use a SaaS solution. As well as the process shift of teams changing how they go about testing integrations. Moving from the authors of the API service writing their own tests to the consumer such as the mobile app writing tests. Often when implementing contract tests you already have integration tests, so persuading people to duplicate the effort can be a hard sell as well.

Smart Contract tests same as the Contract tests are not straight forward when you are starting specially as you need to have special knowledge to setup test networks, wallets, and tools. Another important point is the costs associated when deploying and testing smart contracts, especially when running large-scale or repeated tests. On top of those there is the blockchain specific challenges and security.

How do contract tests and smart contract tests differ in terms of the tools and frameworks used to implement them?

Contract testing most commonly uses Pact & Pactflow which supports multiple languages and frameworks such as graphql. Offering matchers and the ability to compare results makes implementing contract testing much easier. Pactflow offers a SaaS solution to store contracts securely and additional features to visualise dependents.

Smart contract tests are often implemented using dedicated blockchain development tools and language such as Truffle, Hardhat, Solidity. These frameworks provide additional features like contract deployment management, fixture data handling, and integration with test networks. Tools like Ganache provide local blockchain test networks that developers can use to deploy and test smart contracts.

How do contract tests and smart contract tests differ in terms of their level of complexity and the skills required to write them?

Contract testing gets complicated when you need to setup different states before the tests can run, for example data needs to be available within the API in order to test the specific scenario. The key skill though required is good communication and collaboration. Due to the teams now relying on each other to deploy and make changes. It’s important to communicate effectively and support each other’s changes. Contract tests also live next to the code, requiring a decent level of programming skills and knowledge of the application under test.

Smart contracts tests need to consider factors such as consensus mechanisms, gas costs, and security vulnerabilities, blockchain network, deploying contracts, and handling transactions. Additionally, skills in testing blockchain-specific functionality, such as testing for gas usage optimization and handling different network environments, are essential for writing effective smart contract tests.

Thanks for the participation Lewis ! It is always a pleasure to learn from one of the biggest leaders in the area ! 🙌

Web3 Test Series: Rhian Lewis

Rhian Lewis

Rhian Lewis is a consultant software engineer and former digital journalist at The Times who is a regular international conference speaker and panellist on all things blockchain and cryptocurrency. She launched the altcoin portfolio tracker countmycrypto.com, co-founded the London Women in Bitcoin meetup group in 2014 and has acted as an advisor and strategist on various blockchain projects for the last seven years. She blogs on cryptocurrency and is the author of The Cryptocurrency Revolution (Kogan Page) and Understanding Decentralized Finance (Kogan Page). She is based near Plymouth, UK.

How can we test the performance and scalability of our DApp on Web3, especially when it comes to handling high volumes of transactions?

Testing dApps challenges many of our assumptions about testing, primarily the idea that we can provide a controlled test environment that closely mirrors Production. Of course. It is still important to adhere to all our good practices for the non-Web3 parts of the application: if we are making requests to APIs for the non-blockchain parts of the dApp or if the front end is complex, we would apply all the normal strategies and tools that we would apply for a standard Web 2.0 application.

However, as the back end for the dApp is effectively supplied by a public blockchain, our application can only be as scalable and performant as the underlying network itself. Choice of blockchain will influence our approach: we may opt to use a Layer 2 solution, for example, rather than directly using a blockchain like Ethereum.

But sometimes all the testing in the world can’t help you. There have been several instances where Ethereum has more or less ground to a halt where the system is under a lot of pressure, such as the doomed Bored Ape Yacht Club Otherside NFT drop. In cases like this, even the best designed dApp will fail to deliver: the role of the tester in this case is to have ensured that the behaviour of the dApp when the system is under such pressure is such that it communicates to users exactly what is happening and – as far as possible – attempts to conserve the current state so that transactions can happen later. So you should always ensure your tests cover the “very unhappy path” scenario.

What tools and frameworks are available for testing smart contracts and other components of Web3 applications?

The good news is that we have a lot of choice when it comes to tools and frameworks, especially for automated tests. Most of these involve wrappers around existing test tools, or custom matchers for assertion libraries: for example, Waffle and Hardhat both provide their own versions of Mocha and Chai packaged with their one-stop development-test-and-deployment toolboxes.

Truffle, the original all-in-one Ethereum development solution, offers various choices for testing including the opportunity to run tests natively in Solidity with the `truffle test` command. Any of the frameworks I’ve mentioned could be used by someone with experience of JavaScript testing or development.

But there are also solutions based around Rust (Foundry) or Python (Brownie). Your choice of tools and language will, of course, be heavily influenced by the blockchain that your team are developing on top of and the smart contract languages that are used.

All of these I’ve mentioned are focused around testing smart contracts, but it’s also important to test the other elements that make up a dApp, including the front end. For that you could consider using Dappeteer, which is a fork of Puppeteer originally developed by the developers at Decentraland.

How can we ensure that our DApp is secure and resistant to attacks, such as denial-of-service (DoS) attacks, double-spending attacks, and others?

This is a very relevant question, when we consider the billions of dollars worth of value that has been stolen or lost from the ecosystem through hacks and bugs.There are some very simple things that developers can do to close some of the more common loopholes, such as using pre-existing secure libraries such as OpenZeppelin during development.

And there are various good security practices such as protocols around how private keys are stored and how any client funds are stored, which are not specifically the tester’s responsibility to ensure, but which are a whole-team responsibility that everyone should be involved in.

One of the most important things a tester can do is to think outside the box: if I was a hacker, how would I behave? Manual exploration and curiosity is your primary tool here.

I mention the importance of this because many of the so-called bugs that have happened in Web3 are not actually software bugs at all – they are bugs in the requirements caused by product owners failing to anticipate that people may use the application in a certain way. Many so-called exploits on DeFi exchanges have been traders simply using the smart contract as it was designed: it was just that no one anticipated they would be able to do this.

Most reputable teams will hire an audit team and also offer bug bounties to white-hat hackers because the level of complexity of smart contracts is such that the expertise needed to spot potential attack vectors goes far beyond what an average developer or tester can spot. However, we still need to rule out all the bugs we can reasonably find at an early stage of development so we are not paying specialists to find issues that could have been caught earlier.

What are some common bugs and vulnerabilities that can arise in Web3 applications, and how can we detect and address them?


If we look at a list of the biggest Web3 hacks and attacks, one thing that jumps out is the vulnerability of bridges, which are a special type of smart contract that enables value and information to be exchanged between blockchains. These are notoriously fragile – and should be approached with caution.

Another area to be aware of is issues caused by the existence of the mempool in Ethereum and Ethereum-like chains. When your transactions are public, and when there is a delay between your contracts becoming public and actually being written to the blockchain, you need to think very carefully about whether there is information there that could be exploited. Some of this may be legitimate, like MEV, but in some cases it can allow attackers to benefit from this knowledge.

In the last question, you mentioned the double spend problem, which is something that always needs to be guarded against. Using static analysis tools such as Slither is also really important for defining potential weaknesses in smart contracts.


How can we collaborate with other developers and testing experts to improve the quality and reliability of our Web3 applications since it is a bit of a wild west at the moment ?

I feel this is something that is really lacking in Web3 at the moment, especially when it comes to testers. Your Web3 testing community is definitely the sort of thing that will make a difference. The main issue is that the whole ecosystem is in its infancy, and many projects are so early-stage that they don’t even have test teams. I believe this will change organically over time, but in the meantime, communities like the one you are creating has a chance to shape the way we test dApps in the future.

What resources and communities are available for learning and collaborating on Web3 development?

Chainacademy, of course! And Web3 testing community. And I would also encourage people to check out the B9Lab Academy, with whom I have worked in the past. They have been around since 2014 and Elias, the co-founder, spoke at the very first Ethereum DevCon. They have a great Cosmos program running, among other courses. The Ethereum Foundation has a good list of links, and you should also look out for great developer relations people like Camila Ramos (Fuel Labs) and Nader Dabit (Lens), who have written a lot of excellent tutorials.

And finally, nothing beats getting your hands dirty building your own dApp, or even playing around with tools like Remix. The more people who get involved in this community, the better.

Thanks for the participation Rhian ! You rock 🤘