BDD Best practices

Here I am again talking about BDD  haha 😀

I am trying to implement the best practices of BDD in my company, it’s quite hard because most of them are developers and they have their own way to think which is different for Business and QA professionals. Other day I was discussing with a guy who had read a post about BDD and suddenly he entitled himself as an expert ! Really, this is the most common attitude you will find in IT atmosphere.

57221765

 

So, the structure is exactly like this and some people like to write the Given step in the past, but this is the thing, you don’t really need to write the first step in the past, some cases you can write in the present tense.

 Like:

Given I have xxx…
When I do xxx…
Then I should see xxxx…

The first step is a situation you have now like I am logged on the system, you don’t need write always in the past, this is not a rule. You need to write something is evident, this is the aim. So, if you want write in the past the Given, you will have something like:

Given I have logged xxx…
When I do xxx…
Then I should see xxx…

Except the Given that you can choose what is the best option for your scenarios, the When and Then you need to follow the right tenses, When in present Tense and Then in the future tense.

BDD structure:

bdd-best-practices-e1433515152640

The features must have .feature as the extension of the file and you need put the features in the correct path, if you don’t do that Cucumber won’t work. If you want to run with Jbehave you don’t need follow the right structure, you can change this on the pom file if you are using Maven.

 

Where should feature files be kept

Some people don’t like to keep them on git with the project, they would like to see the scenarios on JIRA, but you will need to update and maintain in both of the places, for this reason I always put on github.

 

How should we write feature files

 

There are generally two ways of writing feature files – Imperative and Declarative

Imperative style of writing a feature file, is very verbose, contains low level details and too much information.

Pros: person reading the feature file can follow the step-by-step

Cons: Because of too much detail, the reader can lose the point of the story and the tests. The feature file becomes too big, difficult to maintain and likely to fail due to UI updates.

Declarative style of writing a feature file is concise and to the point, contains only relevant information about the story.

Pros: The declarative style is more readable as it contains less steps in the scenario. The reader can easily understand the scope of the test and quickly identify if any key elements are missing.

 

Independence of Scenarios

Scenarios should be independent of one another. This means that each and every scenario is stand-alone and should not refer to or depend on functionalities/conditions in other scenarios.

 

Redundancy and Refactoring

  • Avoid repetitions! Repetitions or identical steps indicate the need forrefactoring. Refactoring denotes that a scenario should be split into multiple ones while the original meaning is preserved.
  • How many Givens, Whens and Thens (including Ands andButs) should you use? As a rule of thumb, there should be amaximum of 3 to 4 consecutive Givens and Thens. Otherwise this indicates a need for refactoring. Whens should be used very sparingly! Usually only include one When.
  • Generally, favour many simple scenarios over few complex ones.


Resources
:

http://www.testingexcellence.com/bdd-guidelines-best-practices/

https://blog.grandcentrix.net/gherkin-guidelines-and-best-practices/

Pros and Cons of Robotium

Hi guys, I am writing just a summary about Robotium, because maybe I will start to work with it. So, if you have any tips or suggestion, please feel free to comment here.

Robotium is a popular Android automation framework for testing native and hybrid Android apps using the black box method. Licensed under an Apache 2.0 license and first released in 2010.

To use Robotium, you need either the source code or apk file for the app, Eclipse for building a test project, ADT (Android Development Tools), SDK (Software Development Kit), JDK (Java Development Kit), and the Robotium.jar file.

Benefits:
• You can develop powerful test cases, with minimal knowledge of the application under test.
• Allows user to test more flexible and convenient for analyzing results.
• Robotium allows us to take screenshots anywhere in the test (both for Emulator and Device) and save it to device internal memory or SD Card or Emulator
• The framework handles multiple Android activities automatically.
• Minimal time needed to write solid test cases.
• Readability of test cases is greatly improved, compared to standard instrumentation tests.
• Test cases are more robust due to the run-time binding to GUI components.
• Blazing fast test case execution.
• Automatic timing and delays.

Limitations of Robotium:
• Tied to JUnit 3 Instrumentation on device.
• Tied to one app process.
• It can’t work with different Applications in on test – if your application call another one (like Camera) – Robotium can’t “see” it and press any buttons there.

Parallel tests:

I’ve found this API: https://github.com/square/spoon, but I believe that we can configure parallel tests with Jenkins too.

 

Thank you guys ! See you next week 🙂

 

Sources:

https://saucelabs.com/resources/articles/open-source-tools-robotium-android-appium

https://code.google.com/p/robotium/

https://www.linkedin.com/grp/post/3769150-5852687643892002817

http://blog.mobinius.com/robotium-best-testing-framework-for-android/

Configure Thucydides (Report for JBehave) – BDD

What is ?

Basically, this report of your scenarios/features:

Screen Shot 2015-06-17 at 12.34.50

 

Project Structure

+ src
   + main
      + java
         + com.mycompany.pages
            - HomePage.java   

   + test
      + java
         + com.mycompany.pages
            + requirements
               - Application.java
            + steps
               - EndUserSteps.java
            - SearchByKeywordStoryTest.java 

      + stories
         + com.wakaleo.webtests.wikipedia
            - SearchingForCats.story

 

Starting

If you have maven installed, go to the terminal or IDE and:

– Create a new project:

mvn archetype:generate -Dfilter=thucydides

– Open your settings.xml and write:

<?xml version="1.0" encoding="UTF-8"?>
<settings>
   <pluginGroups>
       <pluginGroup>net.thucydides.maven.plugins</pluginGroup>
  </pluginGroups>
</settings>

– In the pom.xml file, update the default JUnit dependency to at least 4.11

– Add a dependency to hamcrest-all and thucydides-junit. Thucydides uses SLF4J for its logging, so add an SLF4J implementation (e.g. slf4j-simple) as well.

– If you are not using Maven 3, make sure you configure the Maven compiler plugin to use Java 5.

– Finally, add the thucydides-maven-plugin, which provides the Thucydides reporting services. The resulting pom.xml file should look something like this:


  <groupId>com.wakaleo.webtests.wikipedia</groupId>
  <artifactId>wikipediawebtests</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>wikipediawebtests</name>
  <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <thucydides.version>0.9.228</thucydides.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
        </dependency>
        <dependency>
            <groupId>org.hamcrest</groupId>
            <artifactId>hamcrest-all</artifactId>
            <version>1.1</version>
        </dependency>
        <dependency>
            <groupId>net.thucydides</groupId>
            <artifactId>thucydides-junit</artifactId>
            <version>${thucydides.version}</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.6.1</version>
            <type>pom</type>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.5</source>
                    <target>1.5</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>net.thucydides.maven.plugins</groupId>
                <artifactId>maven-thucydides-plugin</artifactId>
                <version>${thucydides.version}</version>
            </plugin>
        </plugins>
    </build>
</project>                      

 

– Run this command to be sure that everything it is running correctly: mvn package

– Create a new class contains the list of “features” that make up the application, and stories related to each feature. These stories are not the tests themselves – rather they are used to model the application requirements.

– Writing your first test:

public class Application {
	    @Feature
	    public class Search {
	        public class SearchByKeyword {}
	        public class SearchByAnimalRelatedKeyword {}
	        public class SearchByFoodRelatedKeyword {}
	        public class SearchByMultipleKeywords {}
	        public class SearchForQuote{}
	    }

	    @Feature
	    public class Backend {
	        public class ProcessSales {}
	        public class ProcessSubscriptions {}
	    }

	    @Feature
	    public class Contribute {
	        public class AddNewArticle {}
	        public class EditExistingArticle {}
	    }
	}

 

– Create a new test class in a package of your choice calledSearchByKeywordStoryTest.java

@Story(SearchBySingleKeyword.class)
@RunWith(ThucydidesRunner.class)
public class SearchByKeywordStoryTest {

    @Managed(uniqueSession = true)
    public WebDriver webdriver;

    @ManagedPages(defaultUrl = "http://www.wikipedia.com")
    public Pages pages;

    @Steps
    public EndUserSteps endUser;

    @Test
    public void should_display_article_about_cats() {
        endUser.is_on_the_wikipedia_home_page();
        endUser.searches_by_keyword("cats");
        endUser.should_see_article_with_title("Cat - Wikipedia, the free
 encyclopedia");

 

PS: @Managed and  @ManagedPages are required to take care of our page objects.

 

– Create a step library called EndUserSteps:

public class EndUserSteps extends ScenarioSteps {

    public EndUserSteps(Pages pages) {
	super(pages);
    }

    @Step
    public void searches_by_keyword(String keyword) {
        enters(keyword);
        performs_search();
    }

    @Step
    public void enters(String keyword) {
    }

    @Step
    public void performs_search() {
    }

    @Step
    public void should_see_article_with_title(String title) {
    }

    @Step
    public void is_on_the_wikipedia_home_page() {
    }
}

 

– You will need create your own classes inside of each Step.

– Run the command to generate your report: mvn verify thucydides:aggregate

– Your report will be in target/thucydides directory, open the home.html

 

If you want to see a complete model, go to this link.

If you want to learn more about maven commands, go to this link.

If you want to see the post of John Smart with his complete example, go to this link.

 

Thank you guys  🙂

 

Sources:

https://github.com/thucydides-webtests/thucydides/wiki/Getting-Started

https://weblogs.java.net/blog/johnsmart/archive/2011/10/31/getting-started-thucydides-%E2%80%93-using-thucydides-maven-archetypes

http://stackoverflow.com/questions/23297718/thucydides-install-from-scratch-archetype-all-tests-are-skipped

http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html

https://github.com/thucydides-webtests/thucydides-demos

Gherkin BDD comparison

Hi guys, I will post a research about the BDD engines, it is a bit old (2012), but despite this I found very interesting. I have summarised the relevant informations. The actual language to write tests with BDD is called Gherkin. And it has different implementations adopted to different programming language, the most famous:

Cucumber for Ruby

JBehave for Java

NBehave and SpecFlow for C#

Freshen for Python Behat for PHP

All of them have some common set of supported features but there’re some restrictions and abilities specific to the actual engine. So, we will collect useful features for each listed above engine and present it in some comparable form. Key features to be mentioned are:

  • Documentation availability
  • Flexibility in passing parameters
  • Auto-complete
  • Steps, scenario and feature scoping
  • Complex steps
  • Hooks and pre-conditions
  • Binding to code
  • Formatting flexibility
  • Built-in reports
  • Input data sources support
Grade Criteria
0 No support at all
1 Functionality exists but with serious restrictions
2 Major functionality exists
3 Full-featured support

Documentation Availability

  • Cucumber:
    • Cucumber group on LinkedIn – quite populated place with big number of active discussions
    • Cukes Tutorial Site
  • Freshen – honestly speaking I didn’t find any specialized resourse dedicated to freshen only. Most likely it can be discussed in the more general forums dedicated to BDD in general.
  • JBehave:
  • SpecFlow:
  • Behat:


With this sources to look the documentation of each engine, we can evaluate the criteria and the grades:

GRADE CRITERIA
1 Documentation is available in general (it makes grade 1 at once)
2 Every feature is described and has examples (if it fits it makes grade 2)
3 There’re additional well-grown resources (forums, blogs, user groups) where we can find additional information about the engine

At the end we have:

Engine Documentation availability
Cucumber 3
Freshen 2
JBehave 3
NBehave 1
SpecFlow 3
Behat 3

Other grades:

Flexibility in passing parameters:

Engine Regular expressions support Tables support Multi-line input support Extra features
Cucumber 3 3 3 0
Freshen 3 3 3 0
JBehave 2 3 0 3
NBehave 2 3 0 2
SpecFlow 2 3 3 2
Behat 3 3 3 0

Auto-Complete:

Engine Auto-complete support
Cucumber 1
Freshen 1
JBehave 1
NBehave 1
SpecFlow 3
Behat 1

Step Scenario and feature scoping:

Engine Tagging support Scoped steps support
Cucumber 3 0
Freshen 3 0
JBehave 3 1
NBehave 0 0
SpecFlow 3 3
Behat 3 0

Complex Steps:

Engine Composite steps
Cucumber 3
Freshen 3
JBehave 3
NBehave 2
SpecFlow 2
Behat 3

Hooks and pre-conditions:

Engine Backgrounds Hooks
Cucumber 3 3
Freshen 3 3
JBehave 1 1
NBehave 0 1
SpecFlow 3 3
Behat 3 3

Binding Code:

Engine Binding to code
Cucumber 2
Freshen 3
JBehave 3
NBehave 3
SpecFlow 3
Behat 2

Formatting Flexibility:

Engine Formatting
Cucumber 3
Freshen 3
JBehave 1
NBehave 3
SpecFlow 3
Behat 3

Reports:

Engine Built-in reports
Cucumber 3
Freshen 2
JBehave 2
NBehave 2
SpecFlow 2
Behat 3

Input data sources support:

Engine External Data Inclusions
Cucumber 0 0
Freshen 2 3
JBehave 3 2
NBehave 0 0
SpecFlow 0 0
Behat 0 0

Conclusion:

Table


Source
http://mkolisnyk.blogspot.co.uk/2012/06/gherkin-bdd-engines-comparison.html

Sublime and BDD

A simple guide about how to write formatted BDD features in Sublime. – Download what you want in Package Control. If you don’t have the package control in your sublime, follow these steps: – Open the console:

ctrl ` shortcut

– If your sublime is 2:

import urllib2,os,hashlib; h = 'eb2297e1a458f27d836c04bb0cbaf282' + 'd0e7a3098092775ccb37ca9d6b2e4b7d'; pf = 'Package Control.sublime-package'; ipp = sublime.installed_packages_path(); os.makedirs( ipp ) if not os.path.exists(ipp) else None; urllib2.install_opener( urllib2.build_opener( urllib2.ProxyHandler()) ); by = urllib2.urlopen( 'http://packagecontrol.io/' + pf.replace(' ', '%20')).read(); dh = hashlib.sha256(by).hexdigest(); open( os.path.join( ipp, pf), 'wb' ).write(by) if dh == h else None; print('Error validating download (got %s instead of %s), please try manual install' % (dh, h) if dh != h else 'Please restart Sublime Text to finish installation')

– If your sublime is 3:

import urllib.request,os,hashlib; h = 'eb2297e1a458f27d836c04bb0cbaf282' + 'd0e7a3098092775ccb37ca9d6b2e4b7d'; pf = 'Package Control.sublime-package'; ipp = sublime.installed_packages_path(); urllib.request.install_opener( urllib.request.build_opener( urllib.request.ProxyHandler()) ); by = urllib.request.urlopen( 'http://packagecontrol.io/' + pf.replace(' ', '%20')).read(); dh = hashlib.sha256(by).hexdigest(); print('Error validating download (got %s instead of %s), please try manual install' % (dh, h)) if dh != h else open(os.path.join( ipp, pf), 'wb' ).write(by)

– Hit Enter and restart the sublime after downloaded the package.

  How to install Packages from Package Control ?

– ‘Command+Shift+P’ – Type ‘Install Package’ and enter

– Choose the package as :

Cucumber

Gherkin[Cucumber] Formatter

Behat

Behat Snippets

How to use ?

You can try any of these commands to generate respective file.

– ‘Command+Shift+P’ – Type: ‘Gherkin’

– ‘Command+Shift+P’ – Type: ‘Feature’

– ‘Command+Shift+P’ – Type: ‘Scenario’ or ‘Scenario Outline’

– ‘Command+Shift+P’ – Type: ‘Example’

  If you want some snippet of Given/When/Then/etc, you can use the command  ‘Command+Shift+P’ – Type: ‘Given’

Screen Shot 2015-04-01 at 21.46.24

Easy and very useful, for this reason I like Sublime ! lol Thank you guys 🙂

Sourceshttp://shashikantjagtap.net/speed-up-your-bdd-with-sublime-text-2/ https://packagecontrol.io/installation

BDD – Summarized in 13 questions/tips and 1 example

Hi guys, today I will post a summary of a meet up that I went about BDD. It was very interesting and more clear than the others texts/posts that I found about.

1 –  Communication with all teams (business, developers, tests since beginning) is the key – Everyone should participate.

2 – What is the value of the scenario ?

3 – Could you write in another way/different steps ?

4 – Could you use the same steps ?

5 – Could you join some steps in other scenario to spend less time when you run the feature ?

6 – Your flow is decreasing the possible points of misunderstood gaps ?

7 – Are you doing the scenarios with participation of everyone in the beginning of the project ? When everyone is understanding the same thing and you starts to worry about tests in the beginning of the project, you spend less time/money than write test cases after development. Pay attention in your flow.

8 – Be curious ! Ask !

9 – Everything should be automated ? Remember that the effort spent when you automate something can be less valuable than manual testing.

10 – How many steps are you using in the scenarios, try don’t use more than 4 or 5. Make it simple.

11 – Use page object model – Better structure

12 – Be clear and objective when write the steps

13 – The feature is your documentation for end users, testers, developers. Following one of the Scrum rules, you don’t need more than this in your documentation process. Less time spent, more clear than huge documents (Because it uses examples), keeps the documentation together with the test.

14 – Use 4 layers – Example :

bdd

Thanks John F. Smart !