Category: Test Automation

How to test angular and non angular pages with protractor

As you know Protractor is known as the best compatible automation framework for angular sites, since it awaits for angular to do his work and you don’t need to use waits methods. But what if you have an angular site that has some non angular pages ? How can you proceed ?

 

Protractor provides the means to test angularjs and non angularjs out of the box. The DSL is not the same for angular and non angular sites.

AngularJS:

element.find(By.model('details'))

The element keyword is exposed via the global, so you can use in any js file without need to require it. You can check on your runner.js that you are exporting all the necessary keywords.

// Export protractor to the global namespace to be used in tests.
    global.protractor = protractor;
    global.browser = browser;
    global.$ = browser.$;
    global.$$ = browser.$$;
    global.element = browser.element;

 

NonAngularJS: You may access the wrapped webDriver instance directly by using browser.driver.

browser.driver.find(By.model('details'))

You can also create an alias for this browser.driver. This will allow you to use elem.find(by.css(‘.details’)) instead of using browser.driver. for example:

onPrepare: function(){
      global.elem = browser.driver;
      }

So, how can you use the same DSL for non-angular and angular pages ? You will need to ignore the sync, keep in mind that once you set this value, it will run for the entire suite. This will allow you to start using the same DSL.

onPrepare:function(){
   global.isAngularSite = function(flag) {
     browser.ignoreSynchronization = !flag;
   };
}

You can add a Before for each angular/non angular scenario, you just need to tag each scenario indicating which one will run on an angular page, example:

 this.Before({tags: ['~@angular'] },
function(features, callback) {
   isAngularSite(false);
   callback();
 });


 this.Before({tags: ['@angular'] },
function(features, callback) {
   isAngularSite(true);
   callback();
 });

 

Hope this helps you guys !

Mobile Automation Strategy

Critical scenarios

First of all, you need to build a set of the most critical/important scenarios. So, create a smoke tests with the critical basic features and divide them into phases. Also, remember to add the most frequent scenarios, those that are used in a daily basis.

 

Continuous integration

Implement your continuous integration since the beginning so you can follow when a scenario has broken and if you have false positives. The reason why is you need to trust on your automation, for this reason in the beginning you will need to pair the manual tests with the automation until you have confidence on your tests.

 

Devices

It is impossible to make your tests run on all the existent devices in the world. So, what you need to do is getting the information about what are the most used devices for your app. Exactly, this needs to follow your app, your users. If you don’t have and there is no possibility to get this data, then you can follow the most used devices in general. Focus on your app and your client in the first place.

In this category we can include the different OS’s, screen resolutions, etc.

 

Network

Mobiles are trick because you need to test the network, so you will need to have specific scenarios to simulate the 3G, 4G, WiFi. Remember to have the expected behaviour with poor connection or if the connection drops down and back again.

 

Language (Localisation Testings)

If you have a multiple language app, you also need to worry with the translation.

  1. You can add the language after all the smoke tests are done, since this is easier and faster to test manually.
  2. You can add a specific scenario to go through all the pages and check the translation against the database.
  3. You can specify on your automation that you will run each time with a different language and add the checks along the scenarios.

My suggestion is go for a specific scenario going through all the main pages and checking the translations (2). If you go with option 3 remember, your automation will take longer since it is performing all the scenarios again but with different languages, when a simple assertion on the page without any functionality check would be enough.

 

Screen Orientation

As for mobile, you can have portrait or landscape, so you need to remember to add scenarios related to the orientation. You can start the tests including both of the orientations. You will need to set this in the beginning of the automation or you can have specific scenario to test the orientation for the main screens.

 

Emulators vs Real Devices

Another aspect for which “balance” is a good mantra is testing on real devices vs. emulators. Emulators can’t give you the pixel-perfect resolution you might need for some testing or allow you to see how your app functions in conjunction with the quirks of real-life phone hardware. But they do allow you to do cost-efficient testing at scale and are a powerful tool to have in your mobile testing arsenal. A healthy mix of real device and simulator/emulator testing can give you the test coverage you need at a reasonable price.

 

Be sure you are leaving room for growth, both of the marketplace and your own needs. You need to always choose the best tools and practices that fit your needs, but at the same time you need to think about what is coming in the future. So, expand your automation thinking about what could come next, and minimize the threat of having to spend more time and resources redoing or revising systems that are out of date. Choose always flexibility: Cross-platform testing tools and scalable third party infrastructure are a good example of how to keep it.

Docker – Courses

Hey guys, today I will post a link for you that are interested to learn docker. You can register yourself and follow the steps. It is really easy to understand all the usage and how to create/run containers.

 

But why should I use Docker ?

Remember when you need to create all your data before you run your automated tests ? So, with Docker you don’t need to code this part anymore, you just need to build an image automatically with all the basic data you need for the tests.

If your tests are creating the data upfront via API, you won’t be able to test this part anymore. But now your tests will be focused on the main goal of your project. Also, you will avoid all the data creation instability issues.

For instance, let’s say you have an UI automation and you need to create some users as a prerequisite to test the sort of those users. You will save time not coding this part, not waiting for the server response, so the test will be independent from the API calls and more focused, you will save time while executing, your repository is kept up-to-date with code changes automatically.

Give it a try: training.docker.com/category/self-paced-online

Thank you guys !

Layout responsive tests with Galen

If you ever thought about check layout specifications on different browsers and sizes, there is a framework called Galen, also you can code this framework on java or javascript. I don’t usually create layout tests that specific, but this is very useful since functional tests could pass even with the layout messed up.

Galen, validates the responsive design by checking the location of objects relatively to each other on page using a special syntax and rules.

Keep these tests together with your unit tests.

Java:

http://galenframework.com/docs/reference-java-tests/

There is an example of Galen with TestNg here:

https://github.com/galenframework/galen-sample-java-tests

 

Javascript:

http://galenframework.com/docs/reference-javascript-tests-guide/

A javascript example here:

https://github.com/galenframework/galen-sample-tests

and another here:

http://axatrikx.com/test-responsive-design-galen-framework/

 

Resources:

http://galenframework.com/

https://github.com/galenframework/

http://axatrikx.com/test-responsive-design-galen-framework/

How to deal with data tables in Cucumber and Protractor

 

So, today I will give a code snippet about how to deal with Cucumber data tables and protractor.

Following the example:

 

  • Scenario with data table:
 Scenario: Register multiple users from different countries
   Given I want to register multiple users
    | user  | country  |
    | nicko | uk       |
    | pitty | brazil   |
    | slash | us       |
   When I send the form
   Then all the users should be registered

 

Remember to create the data table with the headers. Don’t forget that data tables are different from examples, here the first step will create a hash table with the data from the table and send it as parameter. Examples are used to run the same scenario with different variables in each run.

 

  • Related Step Definitions:
'use strict';

var protractor = require('protractor');
var browser = protractor.browser;
var _ = require('lodash');
var Q = require('q');

var RegisterPageSteps = function() {

 this.Given(/^I want to register multiple users$/, function(data) {
    var promises = [];
    var rows = data.hashes();
    //For each row you will get the user and the country
    _.each(rows, function(row) {
       var user = row.user;
       var country = row.country;

       //Here you can add the promises to perform sequentially
       //you can call the promises passing the user and the 
       //country as parameters
       promises.push((addUserCountry(user, country));
    });

    //You can also have promises to be performed that don't need
    //the parameters from the data table
    promises.push(navigateToSubmit());

    //Here you return all the promises
    return Q.all(promises);
  });

 this.When(/^I send the form$/, function(callback) {
    callback();
 });

 this.Then(/^all the users should be registered$/, function(callback) {
    callback();
 });

};

module.exports = RegisterPageSteps;

 

I have not implemented other functions and steps as the aim here is to show how to deal with the data table in your scenario.

Thank you !

Automation Plan and Strategy

Hey guys, in this article I will not talk about automation tools or automation frameworks, but I will talk about how to set the strategy and choose the key scenarios and priorities to build your automation strategy.

 

Who is most interested to have automation tests running ? Quality Assurance team.

 

Why ? Because the regression tests that are always performed by the QA engineers will now be performed by automated scripts. They use the automation to run detailed, repetitive, and data-intensive tests automatically.

It helps to improve software quality and make the most of their always-limited testing resources. Automated testing will shorten your development cycles, avoid repetitive tasks and help improve software quality.

 

Why should we implement an automation test project ?

  • It helps to test faster (Failures will be reported ASAP to be fixed)
  • Allows them to test substantially more code (Able to build deeper tests)
  • Improves test accuracy (Avoid human mistakes)
  • Frees up QA engineers so they can focus on tests that require manual attention and their unique human skills (Exploratory tests)

 

For a manual testing project the cost consuming factors are:

  1. People
  2. Tools – Test/defect management
  3. Infrastructure – environment
  4. Time
  5. Training

 

For an automation project, in addition to the above items it needs also:

  1. Automation tools
  2. Add-in for test management tool integration
  3. Add-in to support AUT (like SAP, oracle etc)
  4. Framework set up
  5. Tool specific training

 

What determines the success of your automation ?

“Are you able to generate a better ROI (Return on Investment) in comparison to the manual route”? – If not immediately, eventually.

 

Test Plan

 

  1. Decide what Test Cases to Automate
  2. Setup a Continuous Integration Box
  3. Select the Right Automated Testing Tool
  4. Divide your Automated Testing Efforts
  5. Create Good, Quality Test Data
  6. Create Automated Tests that are Resistant to Changes in the UI

 

Scope

 

  • Add repetitive tests that run for multiple builds.
  • Add tests that tend to cause human error.
  • Add tests that require multiple data sets.
  • Add frequently used functionality that introduces high risk conditions.
  • Add tests that are impossible to perform manually.
  • Add tests that run on several different hardware or software platforms and configurations.
  • Add tests that take a lot of effort and time when manual testing.
  • Divide your projects in phases and prioritise them.

 

Strategy

 

  • Name convention
  • Reports
  • Select the right automation tool
  • Browsers and its versions, desktop or mobile size (Layout responsive)
  • Platforms (Windows, Mac, Linux)
  • Multiple mobile devices (Portrait and landscape)
  • Continuous integration (Frequency which you will run the tests)
  • Locations for test assets to be stored

 

Resources:

https://smartbear.com/learn/automated-testing/best-practices-for-automation/

http://www.softwaretestinghelp.com/automation-test-palnning/

Headless browsers vs Real browsers

Hi guys, I have noticed that many developers doesn’t know the purpose of the headless browsers and when you can consider to use them. Everything depends of your project and what is the purpose of your tests.

Headless browser is like any other regular browser, but without the GUI (Graphical User Interface). The networking component, javascript interpreter, rendering and layout engines are still present. They are particularly useful for testing web pages as they are able to render and understand HTML the same way a browser would, including styling elements such as page layout, colour, font selection and execution of JavaScript and AJAX which are usually not available when using other testing methods.

 

You can use headless browser for:

  • Test automation in web applications (Best performance as you don’t have a GUI).
  • Taking screenshots of web pages.
  • Running automated tests for JavaScript libraries.
  • Scraping web sites for data.
  • Automating interaction of web pages.
  • Perform DDOS attacks on web sites.
  • Increase advertisement impressions.
  • Automate web sites in unintended ways e.g. for Credential stuffing.

 

Headless browsers became a known technology back in 2009 when Google revealed it was opened to using one to parse and index AJAX-based websites.

Headless browsers have been known to be used in DDOS attacks, brute-force attacks, and also for falsely increasing ad revenue by faking page loads and user interactions.

 

Why not to use them:

  • The gain in the speed is not that big compared with real browsers, depending of how many tests you have.
  • Hard to debug.
  • It is not a real browser, so in the end, it is not the same experience your users have.
  • Headless browser can have issues of its own that do not show up in other browsers (a bug found using a headless browser like PhantomJS may actually not be a bug).

 

Tradeoffs:

  • Speed vs Consistency.
  • More specific code vs more general code.

 

Hope I have helped you guys to understand when it is possible to use them and when it is not, since it depends of your project and your requirements. You can judge what is the best option for your project in the moment.

Thank you guys, see you later !

 

Resources:

http://news.softpedia.com/news/what-is-a-headless-browser-and-what-s-it-good-for-485162.shtml#ixzz4Ntp2UsDK

http://blog.arhg.net/2009/10/what-is-headless-browser.html

http://phantomjs.org/

http://www.ecommercetimes.com/story/80194.html

http://www.business2community.com/tech-gadgets/headless-browser-botnet-used-150-hour-ddos-attack-0688767#XhLlg0ZU610scl9l.97

http://www.itproportal.com/2014/04/01/headless-browsers-legitimate-software-enables-attack/

https://webmasters.googleblog.com/2009/10/proposal-for-making-ajax-crawlable.html

Real vs Headless Browsers for Automated Acceptance Tests

 

Hooks vs Backgrounds (Cucumber)

Sometimes you need some pre conditions to run your scenario or a group of scenarios sharing the same steps repeatedly. You can use background or hooks to setup these conditions. How to know what is the best to use ? Well, depends of the case. So today, I will give some examples with best practices when you should use background and when you should use hooks.

Step definition files have a corresponding method available in the before(condition) do . . .method, which has however a matching after(condition) do . . . method as well.

 

Use Background only to setup a pre-condition that the user needs to know

 

If it is not a trivial information to the user, set it up in the implementation (hooks), not in the test steps. Remember feature files should focus on What, and not How. These should be high level steps. We want to keep this simple. So, for this reason you avoid give too many details like this type of steps: “When I press the button”.

 

When using hooks

 

You can use hooks to run before/after each scenario, a group of scenarios according to the tags, all the scenarios in a feature or all the scenarios of your project. They will run before the first step of your scenario, like the background, but it won’t need any step in your feature file.

 

@Before and @After each scenario

Similar to JUnit @Before and @After tagging a method with either of these will cause the method to run before or after each scenario runs. Common functionality like starting or stop browsers are nice to place in these hooks. They reduce the number of common test steps in each scenario. Before hooks will be run before the first step of each scenario. They will run in the same order of which they are registered. After hooks will be run after the last step of each scenario, even when there are failing, undefined, pending or skipped steps.

You may want to finish the tests after the first failure (could be useful in some cases like Continuous Integration when fast feedback is important), for those cases add the command (ruby) in your hook:

  Cucumber.wants_to_quit = true if s.failed?

 

@AfterStep

You have also the possibility to create an after step hook and add for example a take screenshot action. This hook will run after each step of you scenario and you can also filter for certain scenarios using tags. This is only available for Ruby language at the moment, not for Java.

 

@Around

Around hooks will run “around” a scenario. This can be used to wrap the execution of a scenario in a block. The Around hook receives a scenario object and a block (Proc) object. The scenario will be executed when you invoke block.call.

The following example (ruby) will cause scenarios tagged with @fast to fail if the execution takes longer than 0.5 seconds:

Around('@fast') do |scenario, block|
  Timeout.timeout(0.5) do
    block.call
  end
end

 

Tagged hooks

You can filter what are the scenarios that will run this hook every time before start the scenario or after the scenario ends. The condition which enables the before/after block is the tag (false or nil). Tagged hooks can have multiple tags, and follow similar tagging AND/OR rules that the runner does. You can OR and AND tags in much the same way as you can when running Cucumber from the command line.
e.x. @Before(‘@mobile’, ‘Ëś@login’) for tests needing a mobile browser launched and are not tagged as login

e.x. @Before(‘@mobile, Ëś@login’) for tests needing a mobile browser launched or are not tagged as login

 

@Before all scenarios

If you have a hook you only want to run once before all the scenarios, use a global variable. Example (ruby):

Before do 
  $dunit ||= false # define a variable before we can reference its value
  return $dunit if $dunit                  # bail if $dunit TRUE
  step "run the really slow log in method" # otherwise do it.
  $dunit = true                            # don't do it again.
end

 

@AfterConfiguration

You may also provide an AfterConfiguration hook that will be run after Cucumber has been configured. This hook will run only once; after support has been loaded but before features are loaded. You can use this hook to extend Cucumber, for example you could affect how features are loaded or register custom formatters programatically.

 

 

When using background

 

Short Backgrounds

When using background keep it as short as possible. These steps won’t be written out each time the user reads the scenario, so it’s best to have something simple that the user can remember while reading through your feature file.

 

Vigorous Backgrounds

Similar to the above, since these steps won’t be listed with each scenario, the more vivid, the test step is, the easier time the user will have remembering it.

 

Short Feature files

It’s best to keep these feature files smaller, so that the Background information is more readily available. The general rule of thumb is to keep the file small enough to still see the Background test steps at the top of page when reading any scenario.

 

These two methods are powerful tools, but be aware to not use them excessively.

 

Resources:

https://github.com/cucumber/cucumber/wiki/Hooks

https://github.com/cucumber/cucumber/wiki/Cucumber-Backgrounder

https://seleniumbycharan.wordpress.com/2015/08/25/use-of-background-hooks-tags-in-cucumber-jvm/