T23 Class 10/6/2011 3:00 PM "Selecting Mobile Application Automation Tools" Presented by: Pradeep Kumar Govindasamy Cognizant Technology Solutions Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com Pradeep Govindasamy Cognizant Technology Solutions With more than twelve years of experience in the information technology industry, Pradeep Govindasamy currently holds the lead role for the Automation and Mobile Testing Center of Excellence at Cognizant. Pradeep started—and is now the research and development unit lead for—the automation and mobile practices which has more than 1500 experts servicing global customers. Pradeep has spoken on topics of automation, test data management, and tools at conferences worldwide including STARWEST, STAREAST 2011, Swiss Testing Day, HP Universe, and EuroStar. Mobile Testing Selecting Mobile Application Automation Tools Thursday, October 06, 2011 3:00 PM | ©2011, Cognizant 1 | ©2011, Cognizant Agenda Demystifying Mobile Platforms Understanding & Challenges l Introduction Testing Tool selection Criteria o to Understanding Cognizant tools framework Right Automatin the frameworks Mobile class termining n e i D Best 2 | ©2011, Cognizant Mobile Operating Systems Part – 1 Demystifying the Mobile World 3 | ©2011, Cognizant Mobile Platform iPhone OS Android Windows Phone Symbian Blackberry OS Current version 4.3 Current Version 3 Current Version 7 Symbian^4 Current version 6.0 Closed Source Open Source Closed Source Open Source Closed Source Apple Google Microsoft Visual Multitasking RIM Restricted Multi tasking Multi‐tasking Restricted Multi‐tasking Multi‐touch interface Multi‐tasking Safari web browser Chrome/ Webkit Browser Multi touch interface Mozilla Browser Webkit Browser Sync Patch updates Over the air updates I.E / Trident Sync/Patch OTA Sync/Patch OTA 4 | ©2011, Cognizant World Wide OS Market Share Trend Worldwide Smartphone operating system (OS) market share in 2009‐2015, according to Gartner Operating Systems 2009 2010 2011 2015 market share market share market share market Share Android 3.9% 22.7% 38.5% 48.8% BlackBerry 19.9% 16.0% 13.4% 11.1% iOS 14.4% 15.7% 19.4% 17.2% Symbian 46.9% 37.6% 19.2% 0.1% Windows Phone/Mobile 8.7% 4.2% 5.6% 19.5% Others 6.1% 3.8% 3.9% 3.3% Total smartphones sold 172 million 297 million 468 million 631 million 5 | ©2011, Cognizant What is Mobile App Mobile Applications are a rapidly developing segment of the global Mobile Market. They consists of software that runs on a mobile device and performs certain tasks before the user of the Mobile Phone. They can be downloaded physically through USB / WIFI from a desktop or can be downloaded by a web server over internet. IQ Video Players City Guide Audio Players Currency Converter Streaming Players Translators Graphical Viewers Multimedia Travel Weather Presentation Viewers Itinerary Calendars MOBILE Email Clients Calculators Productivity Application Diary Communication IM Clients Banking / Finance Browser apps Directory Services Social Network Apps Utilities Address Book manager File Manager Call Manager 6 | ©2011, Cognizant Types of Mobile Applications Native Applications Hybrid Applications Web Based Applications • Applications that reside on the device •Third party applications that are • Accessible through mobile browsers and are accessible at any point of time installed in the device •Interfaces with external •May require network connectivity • Requires network connectivity. systems/servers • Messaging, Telephony, Multimedia, • Google Chat / Messenger, shopping • Wireless Application Protocol (WAP) or PIM, etc… application on iPhone Hyper Text Transfer Protocol (HTTP) • Mobile Banking, Reservation system Deciding the type Depending on the requirements. Some apps are a better fit with web technologies than others. Knowing the pros and cons of each approach will help you make the right decision about which path is appropriate for your situation. 7 | ©2011, Cognizant Stop! – Quality check for Mobile Application Being the first Increase of Mobile Hurry‐to‐market increases greater Applications pressures usage of Applications •Failure and Security losses destroys the brand reputation •Test coverage •Unhappy Customer Remedy End to End testing is required for thorough validation of Mobile Applications. Mobile Automation testing is vital to ensure a high application quality and faster time to market in a highly competitive market Mobile Automation Testing becomes Paramount 8 | ©2011, Cognizant Image Area Part 2‐ Understanding the Components of Mobile Automation Testing ©2011, Cognizant 9 | ©2011, Cognizant Mobile Testing ? Mobile Testing Hardware / Software Device Testing Testing Certificate Application Content Factory testing Testing testing testing 10 | ©2011, Cognizant Mobile Testing ? Mobile Testing intends to enable greater mobile experience Mobile Application testing tests application on various parameters to enable defect free mobile applications Automating Mobile Application Testing can deliver greater benefits in-terms of cost and quality Factory Testing Hardware or Device Testing Certificate Testing Mobile Testing Application Testing Software Testing Content Testing 11 | ©2011, Cognizant Mobile App Testing Challenges DEVICE DIVERSITY HARDWARE • Multiple Platforms CHALLENGES • MMltilultiple Browsers • Limitations in • Rendering differences processing speed • Mobile devices have • Limitations of Memory different application size of mobile runtimes. • Differences in Communication Protocols of devices WAP/ HTTP. NETWORK CHALLENGES • Multiple type of Networks(GSM / GPRS / Wi‐Fi / Wi‐Max etc ) • Unpredictable time taken for data transfer • Different speed of connectivity across geographies • Multiple Network Operators with customized Network features 12 | ©2011, Cognizant Mobile Application Testing Eco System Test Tool Automation Tools Simulators Emulators Mobile Application Testing Real Devices Virtual Devices 13 | ©2011, Cognizant Mobile Automation Testing? Usual Rare Net ‐ Net Highly Quicker Though there are challenges the benefits reliable Why? of Automation supersedes the demerits Cheaper Robust In numerable OS/ Platforms variations Hardware devices/ OEM PiProcessing speed Limited Resources Small memory CHALLENGES Numerous Wi‐Fi, Wi‐Max, Connectivity modes Cellular networks Accelerometer Varying features Radio trasmitter 14 | ©2011, Cognizant A Typical Test Automation setup for mobile Emulator Automation tool Real Device Connectivity •Bluetooth •Wi‐Fi •USB •Remote 15 | ©2011, Cognizant Typical Mobile Automation Mobile applications under Test (AUT) App 1 App 2 App N Test Data Real Devices/ Emulator/ Remote Test Automation Engine Schedule and Management track test Tool Starts and automates test execution Interface Scheduler cases on the application s s s working on device and / or Acce emulator Defects Report Defects 16 | ©2011, Cognizant Mobile Automation Approach Understand Requirements and Budgets Non Traditional i.e Compatibility, Type of test Location based etc automation Traditional i.e Regression etc Determine feasibility of Automation Classify Application as Native, Hybrid , Web Based Scan markets to identify tools for Automation Determine the test tool for maximum automation coverage Evaluate cost benefit analysis of patch/ tool creation for AtAutoma tion Perform manual testing and create Execute manual testing and create test scripts Scripts for Automation Execute Automation test cycles Execute Automation test cycles 17 | ©2011, Cognizant Mobile Automation Approach Non Traditional i.e Traditional i.e Regression etc Compatibility, Location based etc Classify Application as Native, Understand Determine feasibility of Requirements and Hybrid , Web Based Automation Budgets Determine the test tool for Scan markets to identify tools for maximum automation coverage Automation Types of Test Perform manual testing and Automation Evaluate CBA of patch/tool create Scripts for Automation creation for Automation Manual execution, creation and Execute Automation test cycles execution of Automation test cycles 18 | ©2011, Cognizant Candidates for Mobile Automation Typical Candidates for Mobile Automation Business Testing Types Testing Tools • Complex business logic • Smoke and build acceptance • Less Dynamic UI components • Lengthy business flow Testing. • Across platform testing • Data Driven Test scenarios • Regression Testing • Testing with different carriers • Frequently changing • Synchronization Testing across various location Application under test • Compatibility Testing • User’s interaction with the • GUI TiTesting applicat ion • Test steps spanning across multiple test cases 19 | ©2011, Cognizant Image Area Part 3: Tools and Best Practices ©2011, Cognizant 20 | ©2011, Cognizant Tool Types Mobile Testing Tools Remote Based Desktop Based Industry Based Open Source • Squish • Robotium Device Anywhere • FoneMonkey • Selenium • JamoSolutions • Robot Perfecto Mobile • Keynote Selenium • Eggplant 21 | ©2011, Cognizant Image Area Part 3.1: Mobile Web Automation ©2011, Cognizant 22 | ©2011, Cognizant Mobile Web Automation Web Pages render differently on various mobile phone making Automation of Mobile Testing of Web Page Key to its success Automation Testing Approach User Agent Based Emulator / Device based Remote based Automation Testing Automation Testing Automation Testing Classify the testing types Compliance Testing •Compliance •Automated Testing tool to Gather Requirement for •End to End testing validate the content at