Introduction to Mobile App Testing
Total Page:16
File Type:pdf, Size:1020Kb
Introduction to Mobile App Testing Presented by Elena Houser May 30, 2012 | Agenda 1. What is Mobile? 2. Software Testing Overview 3. Mobile App Testing 4. Mobile Platforms 5. iOS vs. Android 6. Wrap Up & Questions | 2 The Challenge What is Mobile? | 3 What is Mobile? • Mobile Device – portable device powered by a battery that is carried around by the user and uses wireless communications. • Mobile activity primarily consists of two categories – Web: Customized websites intended for viewing on smartphones and tablets over a cellular or wireless internet connection. – Mobile/Native apps: Software programs developed specifically for smartphones and tablets. They access data over a cellular network or wireless internet connection. | 4 Mobile Web vs. Native App Some Companies use a Hybrid of Native App and Web for Mobile – ESPN ScoreCenter - A mobile app with updated scores within the app - Directs users to its mobile website for further articles – Pandora - A native app for listening to music, with small ads - HTML5-powered website to stream music and eventually deliver more content – For use on mobile and other devices | 5 Smart Phone vs. Computer Smartphones have full operating systems, that can be updated, making them almost multi-purpose computers whereas features phones are more limited. Today’s smartphones boast a multitude of operating systems. The most popular ones in the US are Google’s Android operating system, Apple’s iOS, and RIM's Blackberry | 6 Mobile Market Trends Mobile Market • Web and native apps were developed due to the growth of the Smartphone and tablet market and their ability to keep users connected more consistently • Smartphone applications are ubiquitous and mainstream (banking/finance, games, productivity, fitness) • Abundance of free apps Opportunities for testing “…because your app is free and has a built-in market doesn’t mean you can lower your standards or ignore what end users like. .. You need to fully understand how the program can be useful on-the-go and focus solely on the features that would be handy and increase productivity in a mobile, untethered setting.” http://blog.utest.com/why-testing-your-business-app-is- important/2012/05/ | 7 Mobile App Success Mobile Success To succeed Mobile should provide value to the user based on the following types of services: Games: Most downloaded app category -65% (Nielsen Wire) – Ux/entertainment and performance Social: latest news and share content with friends Retail: location intelligent, m-commerce Media: news stream, offline access to content Travel: mobile bookings, check-ins, maps, deals http://www.razorianfly.com/2012/03/07/heres-the-itunes-app-stores-100-all-time-top-apps-direct-links/ | 8 Why Test Mobile Apps? Mobile Apps and Usage are a Big Deal 49.7% of US mobile subscribers owned Smartphones in February 2012 (http://www.newmediatrendwatch.com/markets-by-country/17-usa/855- mobile-devices) • By 2015, International Data Corporation (IDC) predicts that 182.7 billion mobile apps will be downloaded, close to a 1600% increase from the 10.7 billion apps downloaded in 2010 • Diverse user base, multiple devices, manufactures, platforms and fragmentation Our focus and goals • First Impression for the brand/product is important in the app stores • Identify technical, design and functional challenges early to help reduce the cost of failure and repair • Evaluate readiness for release | 9 The Challenge Software Testing Overview | 10 Definitions & Terminology • What is an App? – Container for a product and/or service that is of value it its users. • What is an App’s Purpose? – It should provide value to the user based on a type of a service (e.g., email, social media, game or productivity app) • What is Quality? – Value to some person (Jerry Weinberg) . Different users have different opinions about the app quality . Ratings . User feedback • What is a Bug? – Any threat to the value of the product to any stakeholder who matters (James Bach) | 11 What is Testing? Testing - An empirical, technical investigation conducted to provide stakeholders with information about the quality of the product or service under test. - Cem Kaner Testers design and run tests in order to gain useful information about the products quality. http://www.testingeducation.org/BBST/exploratory/ Exploratory Testing - An interactive process of simultaneous learning, test design, and test execution. - James Bach | 12 What is Testing? Different objectives require different testing tools and strategies and will yield different tests, test documentation and test results. Black Box Software Testing – Foundations Cem Kaner & James Bach http://www.satisfice.com/tools/satisfice-tsm-4p.pdf | 13 The Challenge Mobile App Testing | 14 Quality & Mobile Context Driven App Testing •Quality - is Value to Some Person (Jerry Weinberg) – The value being tested may not be of value to you – As a Test professional, you test for that value(s) •Testing – Whether it’s mobile, desktop or other types of testing the core approaches are the same •Context Driven Testing (CDT) – There are no "best practices" in testing, but rather testing is a set of skills that allow the tester to select or invent testing practices to suit each unique situation. •Mobile Context Driven Testing (MCDT) - applies the mobile world - Project Scope (Functional, Showstoppers/Crashes, Ux, Battery, etc.) - Devices (phones, tablets) - Tools (DDMS & Jing, LogCat) - Mindset, difficulty, challenges | 15 Quality & Mobile Context Driven App Testing http://www.kohl.ca/articles/ISLICEDUPFUN.pdf http://moolya.com/blog/wp- content/uploads/2012/04/COP-FLUNG-GUN1.png | 16 Quality & Mobile Context Driven Testing http://unifiedtestinginitiative.org/files/utc_android_v1.pdf | 17 The Challenge Mobile Platforms | 18 Overview of the Main Mobile Platforms | 19 The Challenge iOS vs. Android | 20 Android Fragmentation •15-month-old Gingerbread operating system (63.7%) •2nd most popular version - 23-month-old Froyo OS (23.1%). •1st unveiled in October 2009 and currently at 6%, Android 2.1 Eclair is still found on nearly as many devices as Honeycomb and Ice Cream Sandwich combined. http://www.engadget.com/2010/12/07/visualized-the-real-android-fragmentation/ | 21 Android Fragmentation Android fragmentation highlights include: • 5 OS versions on devices (Donut, Eclair, Froyo, Gingerbread, and Ice Cream Sandwich) • Devices in various sizes, from small phones to TVs • Device manufacturers branching their own versions of Android | 22 Android Fragmentation | 23 iOS Fragmentation http://www.redmondpie.com/study-reveals-that-ios-apps-crash-more-frequently-than-android/ http://www.marco.org/2011/11/30/more-ios-device-and-os-version-stats-from-instapaper | 24 Which one is easier to test – Android or iOS? http://411droid.com/humor-surrender-iphone-have-surrounded/ | 25 Which one is easier to test – Android or iOS? | 26 Which one is easier to test – Android or iOS? iOS Android I 1. iTunes or iPhone Configuration Utility 1. Device: "Settings -> Applications -> Unknown sources" n 2. With UDID provisional and executable or just 2. Download -email attachments, Dropbox, a barcode scanner s t executable - download and drag them to your iTunes program, etc. a library (1st provisional, then the executable file) or 3. Accept the permissions and install the application. l iPhone Config Utility 4. App can be launched via shortcut or widget (needs to be added l 3. iDevice test apps can also be downloaded Over-The-- Settings>Applications>Manage Applications available). After a t Air (OTA) via a secure link to authorized devices. initially launching a test app, it can be called by pressing and i holding the native device Home button and then tapping on the o respective icon under Recent Apps list. n S Hold down the home button and press the power Digital cameras can be used to take a photo or video of the s c h button at the top right. Either email or sync your Android device's screen. Also Dalvik Debug Monitor r o device with either iTunes or a photo management Server (DDMS), Droid Explorer Android and AtScreen programs e t e program of the user's choice. Digital cameras can be and TechSmith's Jing to capture the steps to reproduce a problem. s n used for videocaptureing. L Console logs - iPhone Configuration Utility; Crash Log Collector or CatLog, o Reports - sync up with iTunes DDMS/Eclipse(http://developer.android.com/sdk/index.html) g (https://developer.apple.com/library/ios/#technotes/t s n2151/_index.html) O Speedtest - for easy connection testing (Download, Upload, and Ping) t h e r | 27 Wrap Up Few Tips and Recommendations •“Software testers who are assigned to test applications on mobile devices need to go beyond thinking “if I tap on the icon, does the application start” or “if I press the cancel button, does the operation/function get cancelled.” •Functional application testing is important but, the application is a part of an entire system and not just a singular entity. The application is completely dependent on the entire system, to function within the system, is a part of the system. •Temperature, memory, CPU Speed, Database searches are all affected when a device is charging and these tests are continually not considered. •A mobile app tester’s responsibility is not simply to find the bugs, their goal must be to increase the quality of the app, finding bugs, is just a small part of this. The testing performed must also take into consideration, the goal of the app, the target audience etc. •By collaborating with the app developer, a mobile app tester can not only significantly increase the app’s quality but also their involvement can drive forward the project with alternative design suggestions and usability feedback. http://www.testing4success.com/t4sblog/mobile-device-testing-vs-mobile-applications-testing/ | 28 The Challenge Thank You Questions? | 29 .