How to Evaluate a Mobile Test Automation Tools for Your Application?

How to Evaluate a Mobile Test Automation Tools for your Application?

This article provides a recommended checklist for selecting a mobile test automation tool.

There are several must-have requirements in a mobile test automation tool. Support for all object identification methods: more specifically, the tool should be able to identify an object by its Native ID, Image Recognition, Text Recognition and Web HTML5 (DOM). This is crucial otherwise the maintenance of the tests will be very poor. For example - tools that only support image recognition will be extremely sensitive to any change in the graphical appearance of the object and will therefore require huge amount of maintenance.

Checklist

Following is a detailed, recommended criteria when evaluating a mobile test automation tool.

Criteria Description Tools Identification methods Can the tool identify objects Native ID See Test, UIAutomation, Selenium, Robotium by their Native ID? Web HTML5(DOM) Can the tool identify objects in Web/HTML5 using DOM See Test, Selenium commands commands? Can the tool identify objects Image recognition See Test, MonkeyTalk using Image Recognition? Can the tools identify objects OCR (text recognition) Appium,Selenium,MonkeyTalk,See Test using Text Recognition? Devices Exactly like the actual user has: No Jailbreak/Rooting Make sure the tool you are using does not require any change in the devices under test. Specifically make sure there is no need Ability to test devices taken to Jailbreak or root the MonkeyTalk, See Test, Appium, UIAutomation, out of a retail store and test devices as this changes the Robotium(Jail breaking not required), Selenium them as-is. device compared to what an actual user would be experiencing. Otherwise you might be “chasing ghosts” trying to debug non-existing bugs. Supported Phones Phones UIAutomation, Appium, MonkeyTalk, See Test iOS (iPhone, iPad) Appium, Robotium, MonkeyTalk, See Test, Android (inc. tablets) Selenium

BlackBerry See Test, Selenium Windows Mobile See Test The ability to launch applications on latest devices means also the ability to test on the latest devices even before they are commercially Support for phones that are not The existing tools have the ability to stand with available in the market. Make yet commercially released upcoming phones. sure your solution supports these devices. You do not want to wait to test your applications – as time to market is critical.

Emulators UIAutomation, Appium, MonkeyTalk, iOS (iPhone, iPad) Selenium, See Test Android (inc. tablets) Appium, Robotium, MonkeyTalk, See Test, BlackBerry Selenium Windows Mobile See Test, Selenium See Test, Selenium Plugs-in to existing test environments Verify that the test tool you select seamlessly integrates into the existing test The mobile automation tool environments including the should integrate into your latest versions of the existing test environment. This environment e.g. : way you can leverage your QTP (inc. V 11) investment in existing MSTest infrastructure (such as HP’s Appium, Selenium, See Test and Robotium. NUnit QTP, QC, LoadRunner or any Rational other test environment your TestComplete organization already uses). Java Python Perl Recorder Enables easy and quick test Appium, UIAutomation, MonkeyTalk, See Test, Ability to record tests creation. Selenium, Robotium(Using TestDroid app) Same test can run on different devices For example, a script Different devices of the same recorded on one Android Appium, UIAutomation, MonkeyTalk, OS device can run on another Robotium, Selenium, See Test Android device For example, a script Different devices of a different recorded on an Android See Test, MonkeyTalk OS device can run on an iPhone device Test Execution

Ability to automatically run Serial test execution on several different scripts one Appium, Robotium, UIAutomation, Selenium, after the other on several different devices MonkeyTalk, See Test different devices connected to the station Very fast script execution High speed test execution Selenium, UIAutomation, See Test for effective overnight runs Full Functionality Support On-screen multi-touch Appium, UIAutomation, Robotium, Multi-touch gestures such as rotation of MonkeyTalk, See Test, Selenium an image, text box Appium, MonkeyTalk, Robotium, MonkeyTalk, Swipe Swipe in all directions See Test, Selenium Appium, UIAutomation, Robotium, Drag and drop Drag and drop on screen MonkeyTalk, See Test, Selenium -Zoom in Appium, UIAutomation, Robotium, Zoom -Zoom out MonkeyTalk, See Test, Selenium - Scroll in all screen configurations -Scroll until event Appium, MonkeyTalk, Robotium, MonkeyTalk, Scrolling -Scroll to find elements not See Test, Selenium currently presented on the screen Appium, MonkeyTalk, Robotium, MonkeyTalk, Click on Security alerts See Test, Selenium Appium, MonkeyTalk, Robotium, MonkeyTalk, Click on System Alerts See Test, Selenium Appium, Robotium, See Test, MonkeyTalk, Reboot device Selenium, UIAutomation Awake device from sleep Appium, UIAutomation, See Test, Robotium, mode MonkeyTalk Operate device Back, Appium, UIAutomation, Robotium, Call/Hang Call, Home, Menu MonkeyTalk, Selenium buttons Appium, UIAuotmation, Robotium, Landscape mode MonkeyTalk, See Test, Selenium Ability to receive mobile UIAutomation and System Trace(Instrument in device CPU usage Xcode to find the CPU usage) Ability to receive mobile Leaks(Instrument in Xcode to find the Memory device memory usage Leaks) - All configurations (alphabet, punctuation and Device keyboard Input numbers, and punctuation Appium, UIAuotmation, MonkeyTalk, See Test

and symbols) - All languages - Support for Special Appium, UIAutomation, Robotium, PC keyboard Input symbols and characters MonkeyTalk, See Test, Selenium

- Support for Capitalization Ability to trigger incoming Simulating incoming call Research and Developement call Ability to trigger incoming Simulating incoming SMS Research and Development SMS Device Connectivity Connectivity Options: Appium, UIAutomation, Robotium, USB MonkeyTalk, See Test, Selenium WiFi MonkeyTalk, See Test Cloud (On-Site Private Cloud) See Test

By,

Suvesh T.K and Sanoj S

QA Team, RapidValue Solutions