Today I will post more about my experiences in developing the automation project from scratch. I developed 2 test mobile automation from scratch until now: one was with Calabash and Cucumber and the other one was with Robotium and Cucumber.
- Support – Be sure that you will have a lot of support from the framework team. If it’s an open source you could have fast support or not, depends of how the developers are busy. So, make sure that you will have a good support when you start to use the framework, go to the github of the project and look for the open issues and what is the frequency they reply and when it was the last bug fixed. Pay attention the frequency of the answers in forums on google groups/linkedin groups/stackoverflow, etc.
- Stability – Is stable enough ? How long this framework is in the market ? Really, you don’t want to start your automation with a lot of problems because the framework that you are using is still in beta phase or is still improving a lot of things. You need to be sure that your choice will depend more in you and your code than the framework you’ve chosen.
- Your app – Yes, you need to know first if your app is stable, with good performance and what are the objectives of your automation. Some frameworks work very well with stable apps, but when you need to test an app with memory leaks or performance problems you won’t be able to even start a scenario. I worked with Calabash most of my mobile automation experience and to be honest I didn’t have any problems to test some unstable apps, but when I did a POC with Espresso, the first simple scenario couldn’t even go further the first step, just because the app was not stable enough (and this it wasn’t the first priority of the automation – of course if you know that you have performance issues and they are not relevant enough you should be able to carry on the automation).
- Developers – Again, I will use the experience that I had with Espresso. The developers are from Google, which is a famous company. You could think, of course I will choose this one, because google is taking care of it. No, to be honest, I don’t really care about the company, I may consider the fact of the framework being developed from a good company, but in first place I see all the priorities above. In this example: Espresso is relative new if you compare with robotium or calabash, it takes serious about the performance of the app, so it won’t go further the automation if you have performance problems, the support is really fast and you can find a lot of people who already started use it.
- Pressure – You need to consider this, probably you will have a developer who will need to push you to use the framework he thinks it is really good (OMG Espresso is being developed by Google, we need to use it, because reasons and stuffs). I think all of us already worked with people like this, I am not saying that you need to ignore them, but just pay attention about what is more important and WHAT IT WILL WORK FOR YOUR COMPANY/APP. Please, not all the apps or companies work in the same way, you don’t need to follow the crowd, just follow a single tip to figure out which framework is better: POC.
- POC (Proof of concept) – The last tip and probably the most important one. If you want to know which framework will work better with your app it’s easy, take one scenario, a basic one, and automate it for the frameworks that you are in doubt 🙂
I hope this helps someone as well. If you have any suggestions/questions, please fell free to comment below. See you next week 🙂
Rafaela Azevedo 