About End-to-End tests:
“Focus on the user and all else will follow”
- Developers like it because it offloads most, if not all, of the testing to others.
- Managers and decision-makers like it because tests that simulate real user scenarios can help them easily determine how a failing test would impact the user.
- Testers like it because they often worry about missing a bug or writing a test that does not verify real-world behaviour; writing tests from the user’s perspective often avoids both problems and gives the tester a greater sense of accomplishment.
Building the Right Feedback Loop
- It’s fast. No developer wants to wait hours or days to find out if their change works.
- It’s reliable. No developer wants to spend hours debugging a test, only to find out it was a flaky test.
- It isolates failures. To fix a bug, developers need to find the specific lines of code causing the bug.
Unit Tests
Unit tests take a small piece of the product and test that piece in isolation. They tend to create that ideal feedback loop:
- Unit tests are fast. We only need to build a small unit to test it, and the tests also tend to be rather small.
- Unit tests are reliable. Simple systems and small units in general tend to suffer much less from flakiness. Furthermore, best practices for unit testing – in particular practices related to hermetic tests – will remove flakiness entirely.
- Unit tests isolate failures. Even if a product contains millions of lines of code, if a unit test fails, you only need to search that small unit under test to find the bug.
Unit Tests vs. End-to-End Tests
With end-to-end tests, you have to wait: first for the entire product to be built, then for it to be deployed, and finally for all end-to-end tests to run.
Although end-to-end tests do a better job of simulating real user scenarios, this advantage quickly becomes outweighed by all the disadvantages of the end-to-end feedback loop.
Integration Tests
Unit tests do have one major disadvantage: even if the units work well in isolation, you do not know if they work well together. For that, you can use an integration test (more simple than end-to-end tests). An integration test takes a small group of units, often two units, and tests their behaviour as a whole, verifying that they coherently work together.
Even with both unit tests and integration tests, you still need do a small number of end-to-end tests to verify the system as a whole. End to end tests are very important even that you let this for the last phase.
What is the right balance ?
To find the right balance between all three test types, the best visual aid to use is the testing pyramid. Here is a simplified version of the testing pyramid from the opening keynote of the 2014 Google Test Automation Conference.
The bulk of your tests are unit tests at the bottom of the pyramid. As you move up the pyramid, your tests gets larger, but at the same time the number of tests (the width of your pyramid) gets smaller.
Google often suggests a 70/20/10 split: 70% unit tests, 20% integration tests, and 10% end-to-end tests. The exact mix will be different for each team, but in general, it should retain that pyramid shape.
How can I choose what will be the tests for each phase ?
- End to end tests: Choose the flows/functions most used for the client.
- Integration tests: Focus in the integration between the functions. E.g. If you register a new client, you should be able to see the new client in the search of the admin site, etc.
- Unit tests: This part you will see if the function is working (single function).You should test the validations, negative scenarios, positive scenarios… E.g. If you register a new client, you should be able to check if the message in the end is ok, if the client will appear in the database, if you can register an invalid client…
Hope you like it guys, I read the original article and it seemed to me that the guy doesn’t think it is too important end-to-end tests. I don’t agree with it, maybe for this reason I changed the way he was speaking about (If you still want to do end-to-end tests). End-to-End tests is important as any other test phase, the difference is that you don’t need to do all the small tests in this phase. It is just my thought… And you can write your comment, suggestion too in this post.
Thank you ! See you next week 🙂
Rafaela Azevedo 