Headless browsers vs Real browsers

Hi guys, I have noticed that many developers doesn’t know the purpose of the headless browsers and when you can consider to use them. Everything depends of your project and what is the purpose of your tests.

Headless browser is like any other regular browser, but without the GUI (Graphical User Interface). The networking component, javascript interpreter, rendering and layout engines are still present. They are particularly useful for testing web pages as they are able to render and understand HTML the same way a browser would, including styling elements such as page layout, colour, font selection and execution of JavaScript and AJAX which are usually not available when using other testing methods.

 

You can use headless browser for:

  • Test automation in web applications (Best performance as you don’t have a GUI).
  • Taking screenshots of web pages.
  • Running automated tests for JavaScript libraries.
  • Scraping web sites for data.
  • Automating interaction of web pages.
  • Perform DDOS attacks on web sites.
  • Increase advertisement impressions.
  • Automate web sites in unintended ways e.g. for Credential stuffing.

 

Headless browsers became a known technology back in 2009 when Google revealed it was opened to using one to parse and index AJAX-based websites.

Headless browsers have been known to be used in DDOS attacks, brute-force attacks, and also for falsely increasing ad revenue by faking page loads and user interactions.

 

Why not to use them:

  • The gain in the speed is not that big compared with real browsers, depending of how many tests you have.
  • Hard to debug.
  • It is not a real browser, so in the end, it is not the same experience your users have.
  • Headless browser can have issues of its own that do not show up in other browsers (a bug found using a headless browser like PhantomJS may actually not be a bug).

 

Tradeoffs:

  • Speed vs Consistency.
  • More specific code vs more general code.

 

Hope I have helped you guys to understand when it is possible to use them and when it is not, since it depends of your project and your requirements. You can judge what is the best option for your project in the moment.

Thank you guys, see you later !

 

Resources:

http://news.softpedia.com/news/what-is-a-headless-browser-and-what-s-it-good-for-485162.shtml#ixzz4Ntp2UsDK

http://blog.arhg.net/2009/10/what-is-headless-browser.html

http://phantomjs.org/

http://www.ecommercetimes.com/story/80194.html

http://www.business2community.com/tech-gadgets/headless-browser-botnet-used-150-hour-ddos-attack-0688767#XhLlg0ZU610scl9l.97

http://www.itproportal.com/2014/04/01/headless-browsers-legitimate-software-enables-attack/

https://webmasters.googleblog.com/2009/10/proposal-for-making-ajax-crawlable.html

Real vs Headless Browsers for Automated Acceptance Tests

 

Running Selenium Headless Firefox in UBUNTU

If you want run your tests with Selenium without open the browser Firefox, I will write how you can do this:

Verify if you have the last version of Firefox.

1 – Install Firefox headless in Ubuntu

  • Open the file: /etc/apt/sources.list and add the line:
    ppa:mozillateam/firefox-stable
  • Open the terminal and run the commands:
     sudo apt-get update
     sudo apt-get install firefox

 

2 – Install Xvfb – the X Virtual FrameBuffer. This piece of software emulates the framebuffer using virtual memory which lets you run X-Server in machines with no display devices. This service is required to make browsers run normally by making them believe there is a display available.

  • Open the terminal and run the command:
    sudo apt-get install xvfb
  • Lets run the xvfb service in a display number which is less likely to clash even if you add a display at later stage. For this example, we will assume a display, 5. The parameter -ac makes xvfb run with access control off. The server should be running now.sudo Xvfb :5 -ac

 

3 – Start browser headlessly in Ubuntu

  • Run the command:
    export DISPLAY=:5
    firefox

If there was no error on the terminal, then you have successfully running firefox headlessly in Ubuntu. The command should keep running until you kill the process by pressing ctrl+ C or similar. There wont be any output. Now, you can run selenium server as you will run in your local machine.

 

Fonthttp://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/

 

 

 

How to run Selenium Headless Browser in Ubuntu ?

Hello again !!!

Today I will post a tutorial about how configure and run selenium headless in Ubuntu using Mozilla Firefox as your primary browser.

Install Firefox headless in Ubuntu

Download the latest version of Firefox on your PC. In case you don’t have firefox or using older version of firefox, follow the steps to upgrade firefox.

  • In /etc/apt/sources.list add the following line.
	ppa:mozillateam/firefox-stable
  • Run the following commands to upgrade or install latest version of Firefox to work with Selenium.
	sudo apt-get update
	sudo apt-get install firefox

Install Xvfb – the X Virtual FrameBuffer

This piece of software emulates the framebuffer using virtual memory which lets you run X-Server in machines with no display devices. This service is required to make browsers run normally by making them believe there is a display available. To install xvfb in ubuntu or Debian, run the following command:

sudo apt-get install xvfb

Now run the xvfb service in a display number which is less likely to clash even if you add a display at later stage. For this example, you can assume a display, 10.

sudo Xvfb :10 -ac

The parameter -ac makes xvfb run with access control off. The server should be running now.

Start browser headlessly in Ubuntu

Before you can run a browser, you need to set the environment variable DISPLAY with the display number at which xvfb is running. Before we add the environmental variable as a persistent one, let us check whether everything works as expected.

export DISPLAY=:10
firefox

If there was no error on the terminal, then you have successfully running firefox headlessly in Ubuntu. The command should keep running until you kill the process by pressing ctrl+ C or similar. There wont be any output.

Now that you have a browser running headless in Ubuntu server, you can run selenium server as you will run in your local machine.

Easy, don’t you think ? If you have any questions, just write below !

Thank you ! Have a good weekend !

 

Fonts: http://www.installationpage.com/selenium/how-to-run-selenium-headless-firefox-in-ubuntu/

What is headless browser ?

Every browser must to do 3 things:

  1. With an address, to access any site
  2. Render the content in DOM, run scripts inside of tag script and turns it able on the page
  3. Render the DOM in a visual content

A browser that do only the first item is called of  “text-based”. A browser that surround the 3 itens is the browsers that we know nowadays. A headless browser surround both firsts itens.

So,  headless browser is a browser that don’t have graphical interface, means: We don’t see the content of the page in our screen, but the browser runs any action on the URL.

What is its function?

Two things: As crawler and as a browser that do faster tests. The rising use of headless browsers is the capacity of run tests a faster way than a browser with graphical interface.

It turns more faster to don’t need load a lot of visual contents, as css styles on elements and load of images.

Think in a Continuous integration, each behaviour(commit, time, version…) a system of CI has to process (compile, unit tests with/or not integration, deploy, acceptance tests). In this process run the tests daily in a environment of CI will spend a lot of time to give us a feedback that we need.

What the people used to do is separate in few parts the most important scripts (smoke, sanity) with known tools (Selenium, Watir, Robot Framework), but these tests still spend a precious time on the execution because of the visual contents that they need.

Other people create scripts or just update them to execute on headless browser. This give to us a huge gain on execution and give to us a more faster feedback.

What are the headless browsers that we can use?

There are a lot of headless browsers on the market and the most part of them (and the best) are open source. They are developed on WebKit and Javascript. The most popular headless browsers:

  • PhantomJS
  • CasperJS
  • SlimerJS (Gecko)
  • HtmlUnit
  • ZombieJS

And Selenium/WebDriver can run tests in the most popular headless browsers:

  • HtmlUnit through of HtmlUnitDriver
  • PhantomJS through of GhostDriver

If you have any doubt or suggestion, just write below !

Thank you again guys 🙂