Tag: test automation with ai

Using AI to Accelerate Test Automation

Hello hello peeps 👋

I have been a bit of a workaholic lately, but all for a good cause 😊

Not sure if you know already, but I started to work on a project The Chaincademy helping Developers (SDETs, Engineers, Coders, Programmers, Test Automation Engineers…), especially the junior ones that are coming to Tech to find their first job 💻

We have launched our MVP before Xmas, and we are testing it with our audience (Junior Developers). So, in case you want to accelerate your career (for now, only web3) and get your first experience as a developer, sign up for our Newsletter to get access 🎉

First time I actually adventured myself with AI and Machine Learning was back in 2018 in a Machine Learning Workshop. I had to create this iOS app where AI was replacing my face with an emoji based on my expressions 😆 Really simple, but back in that time, AI was not so good as it is right now (As we say in Brazil: Na minha época isso aqui era tudo mato – Back in my day, this place was all woods)

And since the launch of chatGPT to speed up all my work, I have been using AI on a daily basis, more Bard actually (Think it is much better than ChatGpt nowadays), so here I am going to give some tips on how I have been using it in test automation:

Test Automation

1. Test Case Generation:

  • Scenarios: You need to pass user stories and acceptance criteria, to generate corresponding test cases with detailed steps and expected outcomes.

    Prompt Example: Given a user tries to register with an invalid email address, describe the steps they would take and verify that the system displays an appropriate error message.
  • Edge Cases: Ask to suggest potential edge cases or corner scenarios to test, ensuring comprehensive coverage of your application’s functionality.

    Prompt Example: For the checkout process, what happens if the user's internet connection drops while entering their payment information? List potential scenarios and expected outcomes.
  • Data-Driven Testing: Generate test data sets based on specific criteria.

    Prompt Example: Generate 10 test cases for the login feature, covering cases with valid and invalid username/password combinations and different user types (admin, regular user).

2. Coding:

  • Test Script Automation: Describe the test actions:

    Prompt Example: I want to test clicking the 'Submit Order' button and verifying the order confirmation page appears. Write a Cypress with javascript script for this scenario.
  • Code Completion: Get test assertions, locator identification, and handling complex interactions.

    Prompt Example: In my Cypress test, I'm trying to assert that the element contains the text 'Welcome back'. Please suggest the next line of code with assertion syntax.
  • Refactoring: Analyze your existing test scripts and suggest improvements like removing redundancy, increasing reusability, or optimizing execution time.

    Prompt Example: Analyze my Pull request for the search functionality. Can you add comments and suggest ways to improve readability, reduce redundancy, and speed up execution?

3. Test Planning and Management:

  • Prioritization and Risk Assessment: Provide the test case details and application knowledge, so it can help prioritize tests based on risk or impact.

    Prompt Example: Given these 20 test cases for the new feature, rank them based on potential impact, speed of delivery and risk of failure. Explain your reasoning for each.
  • Maintenance: Identify outdated or irrelevant test cases and suggest updates or new tests to maintain coverage.

    Prompt Example: The application updated its login page layout. Identify test cases needing modification and suggest relevant updates based on the new UI.

4. Environment Management:

  • Mocks: Describe data needs for specific tests, and generate mock data or API responses, reducing reliance on real environments and dependencies. Remember you can also use contract tests (with Pact for example) and this can be done automatically from the code.

    Prompt Example: Generate mock API responses for the payment gateway integration test, simulating successful and failed scenarios based on test case requirements
  • Environment Configuration: Configurations for different test environments based on your application and testing requirements.

    Prompt Example: Suggest configurations for a staging environment replicating the production database but with limited user access. Include details for network settings and resource allocation.

Thanks to Abel from Graph Protocol 👏 to send over these great resources that I have been using to learn about how to better prompt for Software Development are:

Benefits of AI in Test Automation

Photo by Kindel Media on Pexels.com

There are several interesting web app automation scenarios that we can improve using AI:

  • Reduce the execution time: Nowadays you have the feature target function already even without an AI test automation project, but with AI you can add this feature without having cucumber in place or even the need to tag the scenarios or features. The AI should be able to identify the features related to the change automatically.
  • Converted manual test cases to automation: you can use Natural Language Processing (NLP) to automatically translate manual test cases into automated test cases. I have seen this done with cucumber not AI yet, but totally possible as AI models work on datasets.
  • Creating different data combinations by training the AI to identify the possible combinations based on a dataset is possible. This would increase the data coverage and bring more confidence to the automation project.
  • Visual validations: Many tools perform this functionality already. I personally tried one tool ages ago called Percy, but you can also try some other popular tools like Applitools and Telerik
  • Test execution stability or self-healing automation: AI can automatically locate web elements when the primary locators fail. You can see this feature in some cutting-edge automation tools like Mabl and Xray and Functionize. Self-healing employs data analytics to identify objects in a script even after they have changed. When your script fails due to being unable to find the object it expected, the self-healing mechanism provides a fuller understanding and analysis of options. Rather than shutting down the process, it examines objects holistically, evaluates the attributes and properties of all available objects, and uses a weighted scoring system to select the one most similar to the one previously used.

Becoming a Domain Model Expert

Creating a model for your test automation requires a domain expert, therefore is critical to have a test automation specialist that also knows the business so the AI can bring the desired innovation. With such extensive use cases, AI systems will need different parameters from domain experts.

Machine Learning Algorithms In Layman's Terms, Part 1 | by Audrey Lorberfeld | Towards Data Science
https://wordstream-files-prod.s3.amazonaws.com/s3fs-public/machine-learning.png

Be careful to not run more automated tests than you actually need it. A stage of supervision when the AI is learning the patterns is definitely needed it.

Resources: