State of Testing Survey 2018

Hey guys, the #StateofTesting survey is closing today. It doesn’t take too long to complete and will help the QA community around the globe to see what are the trends and challenges in our area.

The link to vote is http://qablog.practitest.com/state-of-testing/

Starting with security testing

Security test is a group of measures to secure an application against unforeseen actions that can cause the application to stop or to expose data. These actions can be intentional (caused by hackers) or unintentional. So apart from the obvious reasons why you should be sure your application is not vulnerable, you have at organization level:

  • Responsibility
  • Corporate responsibility
  • Regulatory bodies
  • Compliance
  • Legal
  • Financial

And at the technical point of view:

  • Integrity
  • Authorisation
  • Confidentiality
  • No repudiation
  • Availability
  • Authentication

 

So, what should you do when creating some security tests ?

You need to seek permission before you start, then try to learn on sandbox applications or virtual machine, not real environments. Keep focused when doing the tests and prepare in advance threat modelling/survey sessions.

The web security vulnerabilities are prioritized depending on exploitability, detectability and impact on software.

  • What is needed to exploit the security vulnerability? 

Highest exploitability when the attack needs only web browser and lowest being advanced programming and tools.

  • How easy is it to detect the threat? 

Highest being the information displayed on URL, Cookies, Form or Error message and lowest being source code.

  • How much damage will be done if the security vulnerability is exposed or attacked?

Highest being complete system crash and lowest being nothing at all

 

Threat modelling

A range of techniques for analyzing the security of an application, examples:

  • Data flow diagrams
  • Threat categorization
  • Trust levels

 

You need to understand how the data is manipulated in all levels:

  • Entered
  • Stored
  • Transported
  • Presented

 

What are the assets you want to protect, which can be physical or abstract:

  • Reputation
  • Customer data/client
  • Corporate data

 

Threat Modelling – STRIDE

  • Spoofing
  • Tampering
  • Repudiation
  • Information Disclosure
  • Denial of service
  • Elevation of privilege

 

There are so many ways to check if your application is vulnerable, like:

  • User Inputs
  • Error messages
  • URLs
  • Files and attachments
  • Request/Response headers
  • Insecure User Credentials
  • APIs
  • Cookies

 

If you want to test injection flaws for example, you can exploit by submitting untrusted data, also it is very common and easy to exploit causing extremely damaging problems if not fixed. This type of issue can affect UI elements, URLs, parameters in requests and cookies. Examples of Injection are SQL Injection and Cross Site Scripting,

 

Broken Authentication and Session Management

The causes of a insecure session management may lead to session hijacking, spoofing or fixation and to escalation of privilege. Fairly common, simple to exploit, but can have wide ranging and damaging impact. It can affect session tokens, user credentials, request parameters.

 

Tools

 

Resources:

https://google-gruyere.appspot.com/

https://www.pluralsight.com/courses/hack-yourself-first

https://www.owasp.org/index.php/Main_Page

https://www.ministryoftesting.com/

https://www.guru99.com/web-security-vulnerabilities.html

How to integrate Guice + Cucumber + WebDriver

Hello guys,

Today I am going to post about something that I have been studying. So, in this project you can have an idea of how to implement Dependency Injection using Guice, a Google framework which helps you to reduce the need of new in your Java code. I found only projects that have only the Guice + Cucumber integration or only the Guice + Page Objects, so I think this one might be a good example of how to integrate everything.

Also, the project contains Page Objects and a Driver Factory, so you will be able to include other browsers and implement each one’s singularity when creating the browser, and also you still have the good practices of the POM.

I’ve added comments along the code, but feel free to ask anything or even suggest improvements.

https://github.com/rafaelaazevedo/dog-lightning

Filling a ticket – Best practices

Search for duplicates

Before reporting a bug it is necessary to make sure that the bug does not already exist.

Pick a good summary

When we’re looking through lists of lots of issues, the summaries are essential in helping us to understand what an issue is about. This means that you should err in favour of putting too much information in the summary, rather than too little.

Some examples:

Bad: Space export doesn’t work
Good: Space export fails with NullPointerException when performed by the anonymous user

Bad: Administrator attachment control
Good: Allow configuration of maximum attachment sizes per space or per page

Set the Priority, Severity and Impact

You need to select the priority, severity and impact of this bug.

Set the “Fix version”, release version

Set the fix version once you know the test has passed and this is going in the next release, unless the release is based on what needs to go and not in what is ready.

Choose appropriate components

You need to select all the components that are related to the ticket. So, if you need to make changes on different components, this is the field that you need to specify them. It will be useful for the QA engineer to know what are the components affected and all the components that need to be tested.

Attach images/logs/videos/files

Often, especially when describing bugs, an image/log/video/file will help tremendously to explain the issue. If you are testing on mobile platform, try to attach tablet logs, videos and images. For web an image or a video should be fine and for the server platform you can attach the response and of course the steps to reproduce.

If you need to include a large piece of information in an issue, such as a stack trace, please do not put it in the Description field. Instead, add it as an attachment or a comment. (This makes it a lot easier to view and parse issues in most browsers.) I suggest you to use Jing to capture the images and video.

Link issues

If you know that a feature request or a bug is similar (but not identical) to an existing one, please link it using a link type of “relates to”. This will help to get a broader view of the problem, and may be able to solve not only one but two issues.

Write a detailed description

For all issues, please provide a description that can be understood by all in the community, not just yourself or other developers. Also, the description should allow the QA engineer to understand the implementation details of the story, down to the code level. This informs their decisions on risks, what testing needs to be added, and what testing is unnecessary. If you need to include some techno-babble, in addition to the plain language, for other developer types to understand the full details that’s fine.

First and foremost, put yourself in the place of someone else trying to solve the issue based on information in the Jira ticket alone. That means put as much information as you can into the ticket so that the next person can work the issue without having to follow up. Even if the ticket is for an issue you plan to work yourself, the more information provided, the better.

Every ticket must include:

  • Detailed steps to test
  • Given/When/Then following BDD
  • Impacts, any integration with other components/features/functions. So, the tester will be able to think about edge cases
  • Write which environment this needs to be tested (QA is the default if not specified). For mobile platform, we test directly from each component’s master branch, unless written in the ticket that the app was deployed on qa/dev environment.

Every bug must include:

  • Detailed steps to reproduce
  • Impacts, any integration with other components/features/functions. So, the tester will be able to think about edge cases
  • What you expected to happen
  • What happened instead
  • Don’t forget to mention the environment you are testing. Make sure to include any information which might be relevant to the bug (eg. your screen resolution, browser, etc)

 

Don’t reopen issues that have been resolved and shipped

Don’t reopen resolved/closed issues once they have been shipped. If an issue similar to yours is marked as resolved, but you’re still experiencing a problem with it, create a new issue and provide a link to the original one. In the case of resolved bug-fix-issues, this will help to evaluate whether it is in fact the same problem, or actually a new issue with some of the same symptoms.

Testing a new story/feature

If you find a bug while testing a new feature:

  • Create a Story bug and link it to the main ticket, remember to prioritize this bug. Also, put back the story ticket to in progress again
  • Once the related Story bugs are fixed, the developer should move the story containing the bugs to Ready for QA, so any QA will be able to check if all bugs have been fixed and close the main story if there was not introduced new bugs.

Resources:

https://confluence.sakaiproject.org/display/MGT/Sakai+Jira+Guidelines#SakaiJiraGuidelines-Community
https://wiki.collectionspace.org/display/collectionspace/Guidelines+for+Filing+a+JIRA+Bug+Report
https://confluence.atlassian.com/jirakb/using-jira-for-test-case-management-136872198.html
http://www.3pillarglobal.com/insights/manage-test-cases-bugs-using-jira
http://blogs.atlassian.com/2013/12/jira-qa-process/
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=30746287
https://confluence.atlassian.com/display/DEV/JIRA+usage+guidelines

Usability Testing

Hello guys,

Today I am going to write about some usability tests techniques that you can use in your day-to-day tests. Usability tests is where you can figure out if your design is working or not, what can be improved and what is not straight forward.

Jakob Nielsen created the 10 usability heuristics, which includes:

  • Visibility of system status
  • Match between system and the real world
  • User control and freedom
  • Consistency and standards
  • Error prevention
  • Recognition rather than recall
  • Flexibility and efficiency of use
  • Aesthetic and minimalist design
  • Help users recognize, diagnose, and recover from errors
  • Help and documentation

 

Know your user

When you know your users, you can focus on their goals, the characteristics that they have and the attitudes that they display. You should also examine what the user expects from the product.

User personas are created from other forms of user research and thus offer an real life portrait that is easy for the team to keep in mind when designing the products. User personas have a name and a story.

 

Simplify

We need to start with the idea of a user in your worst case scenario when doing UX testing, someone who knows nothing about your product, is distracted when they use the system, has bad cell reception, etc. By watching that person use and fumble through your product, you can quickly identify areas where the app is not simple, clear or fast enough.

 

Trust your intuition

It’s important to remember what specific problem you’re setting out to solve. Trust your guts: early on, it’s especially important when larger decisions are so fragile. Remember that you can’t build something that pleases everyone: trying to do so normally results in a weak release. Stay focused on the use-case you want to nail and avoid trying to solve all the use cases at once. The smallest details can be the difference between a product that has a good experience for the user and one that has not.

 

Efficiency

It lies in the time taken by the user to do a task. For instance, if you are an E-commerce site, the efficiency of your site depends on the time and the number of steps that the users take to complete the basic tasks like buying a product.

 

Recall

This is one of the most important aspects to examine the person’s memory concerning the browsing process and the interface they used a while ago. If your design is simple and straight forward it will be easier to remember how to complete a task.

 

Emotional response

This helps to analyze the user’s feelings after they have used the product. Some might feel happy while others might feel down and there are others who are so deeply impressed that they recommend the product to their friend.

 

Resources:

https://uxmastery.com/beginners-guide-to-usability-testing/

https://www.nngroup.com/articles/how-many-test-users/

https://thenextweb.com/dd/2013/08/10/13-ways-to-master-ux-testing-for-your-startup/#.tnw_3ngxAqEM

https://www.invisionapp.com/blog/ux-usability-research-testing/

https://www.interaction-design.org/literature/article/7-great-tried-and-tested-ux-research-techniques

https://uxdesign.cc/ux-tools-for-user-research-and-user-testing-a720131552e1

https://www.cso.com.au/article/626752/data-breach-notification-just-it-business/

Hiring QAs, Headless vs Real Browsers, Automated tests, Consumer contracting tests

Hi guys, I went to the #18 Agile Roundabout meetup here in London and I found really interesting to share. The first video is a talk about some of the challenges that we have when hiring QAs and about automation on headless browsers vs real browsers. The second is about some of the challenges that we have when implementing automated tests in a company and the third one is about how to do consumer contracting tests with Pact.

I do recommend everyone to watch the videos since it was really interesting to hear these experiences and you might be facing one of these situations.

Common mistakes when you hire QAs

 

Automated Tests

 

Consumer Contract Testing

 

 

Thank you guys for this great meetup !

How to deal with common situations in QA area

I know you are probably thinking, why do you need to talk about it ? Unfortunately, I can spend hours here talking about the situations that a big part of QAs go through everyday. Maybe you are not a QA, but you have to deal with one of these problems as well.

So, I will talk about the most common problems and how you can deal with them:

 

When you have been telling management about a problem for weeks and now you’re just like

 


This is a classical problem, QAs should raise their concerns about some issue that they see it is coming. As a QA you know how the end user uses the product and you can see some upfront failures, scalability issues, etc.

Your job is to point the issue and raise the concern. Now, it is up to your manager to act on the problem or not.

 

 

 

 

When you hear about “small last-minute changes”

 

This is completely common in agile (personally, I don’t think this is a major issue), we just need to be aware of the risks. If we keep this in mind, it is okay. In the end we are not machines, and most of people don’t perform well when working under pressure compared to a normal day.

My advice is to be cold blooded when you have these last minute demands, just go to a place where you can focus and ignore any outside distractions, put some music on if you think it is better to concentrate.

 

When a new feature comes to QA in the last day of the sprint

 

Be realistic, as a QA you probably already know the percentage of the bugs you will find for each developer’s ticket in the first round of tests. This experience improves according to the time you stay in the company and know the quality of the work of each developer.

When this feature arrives to QA, be realistic and raise the point that it has good chances to be back in development and not finished until the end of the sprint.

 

When a bug slips through to the production environment

 

Don’t cry ! haha This is completely common, and this is not always QA’s fault. QA is just the tip of the iceberg and for this reason you need to know the feature not only on the technical, but also on the business point of view. For this reason the kickoff meetings and all the details are important.
When a bug slips to production it is a series of mistakes, this means that maybe the development team was not aware of some scenario, or the PO was not aware of what the users really wanted, and consequently QA didn’t know about some edge cases since they were not involved in the technical and business discussions. You know when you missed something and you know when you didn’t test something because you were not aware of the implications/impact.

 

 

When no one recognizes QA

 

This is really sad, but after some years you get used to it. I mean, it is not ideal, most of the recognition comes from the developers themselves or  the QAs that work with you.
Learn to motivate yourself without waiting for any recognition from your managers. I mean, if you love what you do, you don’t need anybody to say how good you are at it. If you have a feedback about your work, take the positive and constructive criticism to improve yourself, ditch the negative ones. Do what you like, as long as you are learning and you are happy, it is okay.