Survey Research Using Emerging Technologies

Survey Research using Emerging Technologies:

Considerations for Design, Data Collection, Sampling and Recruitment for Smartphone and Tablet Based Surveys

JOS Conference June 10 and 11, 2015 Trent D. Buskirk, Ph.D.

JOS AAPOR30th Anniversary Short Course, Conference 2014 Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Instructor: Trent D. Buskirk, Ph.D.

Current Position: Vice President of Statistics and Methodology @ Marketing Systems Group Lives in: Saint Louis, MO Focus on: Use of new technologies for data collection Mobile Phone Sampling Dual Frame Designs and Weighting Mobile Analytics and Paradata Collection and Analysis Addicted to: Playing Prince to my little Princesses APPS and iPhones SAS, SUDAAN and R http://www.m-s-g.com/Web/Index.aspx Tennis Boost 101.9 (www.boost1019.com)

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Acknowledgements and Thanks!

Colleagues at MSG for their assistance and support and for providing thumb drives for participants

My Family for their patience as I persisted through course prep

JOS Conference Organizing Committee for the Invitation to Present

3 3 JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Day Agenda & Goals 1

Background: Nuts and Bolts of our new Mobile Universe Understanding the landscape that is before us to set the stage New types of data available – burden, privacy and availability Framework for Data Collection on Mobile Devices Active versus Passive data collection methods Developing Emerging Best Practices: A blend of old and new Types of Mobile Surveys (and what we know now from the Literature) Free Resources for Mobile Web Surveys Recommendations for Internet/App based survey solutions (Top 14 EBPs)

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Day Agenda & Goals 2

Developing Emerging Best Practices: A blend of old and new Recommendations for Internet/App based survey solutions (Top 14 EBPs) Computing – just how do you make these surveys? From Websites to Mobile Web Surveys (Web design methods for Mobile) Overview of “Mobile Versions” for Popular Online Survey Packages Goal: Leave with a solid appreciation of: Potential role of new technologies in data collection; Different ways you can use smartphones to collect survey data Some practical “best practices” for deploying mobile surveys

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Reference Note

I have included web links as well as QR codes for nearly every data source we will discuss today.

Some popular QR code reader APPS for your smartphone include: I-nigma Kaywa Reader BeeTagg

The following website allows you to enter your exact mobile provider and provides a recommendation as to the QR code that is optimized for your phone’s camera http://www.mobile-barcodes.com/qr-code-software/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 6:59 Section 2: Emerging Technology and the Rise of Smarter Cell Phones

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

What are Feature Phones?

A feature phone is a mobile phone that combines the functions of a PDA and a mobile phone. These models serve as portable media players, web browsers and camera phones.

Feature phones generally refer to devices that are more low-end compared to higher- end Smartphones.

The most important difference between a feature phone and a Smartphone is the kind of operating system installed in it.

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 What is a Smartphone?

A Smartphone is a high-end mobile phone that combines the functions of a personal digital assistant (PDA) and a mobile phone.

In general, a Smartphone will be based on an operating system that allows it to run applications, use e-mail, and surf the internet. Apple's iPhone runs the iOS BlackBerry Smartphones run the Black -Berry OS Other devices run Google's Android OS, HP's webOS, and Microsoft's Windows Phone.

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

For the Visual Learners in the Room – a Summary

The Picture Summary!

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Smart vs. Feature Phones Worldwide

11 Source:http://www.nielsen.com/us/en/newswire/2013/how-the-mobile-consumer-connects-around-the-globe.html

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Smartphone penetration for US Mobile Subscribers over the past two years “To every thing there is a season, and a time to every purpose under the heaven,” Ecclesiastes 3:1

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 What is a Tablet Computer?

A tablet personal computer (tablet PC) is a portable personal computer equipped with a touchscreen as a primary input device, and running a modified desktop OS designed to be operated and owned by an individual.

Tablet PCs are larger than smartphones. Common form factor dimensions are (height, width, thickness and weight):

5-inch Tablets: 6" x 3.2" x .4" @ .75 lbs. 7-inch Tablets: 7.5" x 5" x .5" @ 1 lbs. 9-inch Tablets: 9.3" x 6" x .5" @ 1.2 lbs. 10-inch Tablets: 9.8" x 7" x .5" @ 1.4 lbs. 13-inch Tablets: 12.5" x 8" x .5" @ 2 lbs.

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Devices come in all sizes (and shapes)

Screen dimensions are measured in pixels (px) both for height and width. Another measure that combines these to give an overall measure is dots per inch (dpi) 1 megapixel = 1,000,000 pixels. Common widths (in pixels) for various devices:

768 1024 1680 px px px 320 px

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Mobile Devices introduced in 2013

15 Source: Millennial Media, 2013: http://www.millennialmedia.com/mobile-intelligence/mobile-mix/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Mobile Devices Covering the USA

240 Million Mobile Users (age 13+) in the U.S. 91% of All U.S. Adults have a Cell Phone 156 Million Smartphone Owners in the U.S. (age 13+) 64% of U.S. Adults have a Smartphone 10% of American Adults are Mobile-Only Internet Users; 15-21% among 18-29 YOs 82 Million Tablet Owners in the U.S. 42% of U.S. Adults have a Tablet

Sources: comScore (2015) and Pew Research (2015) 16 http://www.comscore.com/Insights/Presentations-and-Whitepapers/2015/2015-US-Digital-Future-in-Focus

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Mobile Devices Covering the World

4.77 Billion People Worldwide Using Mobile Phones 2.03 Billion People Worldwide Using SmartPhones 400 Million People Worldwide Own Tablet Devices 788 million mobile-only internet users by 2015 worldwide (Cisco, 2014)

Sources: http://www.emarketer.com/Article/Smartphone-Users-Worldwide-Will-Total-175-Billion-2014/1010536 17 http://www.techvibes.com/blog/tablet-adoption-rate-exceeding-that-of-smartphones-2013-10-15

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Smartphone Users Worldwide: 2012 – 2018

18 Source: emarketer (2014): http://bit.ly/1uWDLtg

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Mobile device penetration continues to rise in the U.S.

19 Source: Comscore, 2014 http://bit.ly/1hh9j8L

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Tablet Penetration in the EU-5 (2012-2018)

UK Spain Germany France Italy 70.0%

60.0%

50.0%

40.0%

30.0%

20.0% Share of populationShare 10.0%

0.0% 2012 2013 2014 2015 2016 2017 2018

20 Source: http://www.statista.com/statistics/271001/penetration-rate-of-tablets/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Smartphones in the U.S. - They’re almost everywhere!

Just over three in four (77%) mobile subscribers in the U.S. owned a smartphone during the most recent three-month period (Oct-Dec 2014), Nielsen, 2015

Android Operating System most popular smartphone OS in US followed by Apple IOS. Source: Nielsen (2015): http://bit.ly/1MDPB4j

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Most Common Smartphone Operating System Per Country

Note: These estimates are based on Mobile Web Activity rather than Device Ownership

22 Source: StatCounter: GlobalStats (2015) http://gs.statcounter.com/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 A different picture for Tablets

SOURCE: MILLENNIAL MEDIA’S Q2 2013 MOBILE MIX™ REPORT http://www.millennialmedia.com/mobile-intelligence/mobile-mix/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Most Common Tablet Operating System Per Country

Note: These estimates are based on Mobile Web Activity rather than Device Ownership

24 Source: StatCounter: GlobalStats (2015) http://gs.statcounter.com/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Smartphone Penetration in the US by Key Demographic Features

Source: Nielsen (2014): Source: Nielsen (2015): http://bit.ly/1pyhyfG http://bit.ly/1MDPB4j

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Age and Income Level Strongly Associated with Smartphone Ownership

Source: Nielsen Wire (2012) : http://www.nielsen.com/us/en/newswire/ 2012/survey-new-u-s-smartphone- growth-by-age-and-income.html 2012

2013

Source: Pew (2013) http://www.pewinternet.org/2013/06/05 /smartphone-ownership-2013/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Slightly Different Picture for Tablets Bigger Screen, Smaller Income The age distribution of Tablet users seems to skew older compared to smartphone users (comScore, 2013)

The annual HH income distribution for Smartphone users appears to skew richer than for tablet users (comScore, 2013)

27 Source: comScore (2013) http://bit.ly/1msbiaV

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Internet traffic by device over the last 3 years (11-13)

28 Source: http://www.mediabehavior.com/articles/the-whole-story-internet-use-by-platform/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Mobile Devices and Internet Use Today

According to comScore’s US Digital Future in Focus report (2014): Smartphone and Mobile devices account for 57% of U.S. Internet usage Smartphones alone have exceeded Desktops http://bit.ly/1lFEDlj

According to mobiThinking (2012) approximately 25% of U.S. Mobile Web users are mobile-only. many mobile-only are older people and many come from lower income households http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats/b#mobile-only

According to Pew Research’s Home Broadband Report (2013) approximately 10% of Americans indicate that they do not have a broadband connection at home but that they do own a smartphone http://www.pewinternet.org/2013/08/26/home-broadband-2013/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Mobile Only versus Desktop Only Internet Users (U.S.)

30 Source: comScore (2015): http://bit.ly/1EPWBJl

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Average daily minutes spent online by internet users worldwide as of 2nd quarter 2014, by device

PC, desktop or tablet Mobile 400

350 93 108.6 300 74.4

250 258.6 267.6 256.8 200

150

100

Daily Daily onlinetime in minutes 50

0 2012 2013 2014

31 Source: GlobalWebIndex; ID 319732

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Multiple Tasks for Multiple Platforms

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 With the surge in Mobile Internet Use comes the rise of unintentional mobile respondents

33 Source: Maritz Research http://www.maritzresearch.com/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Mobile, Mobile Everywhere! We have to care!

With Mobile Device penetration rising along with Increases in internet activity on mobile Increases on the email opening activity on mobile Rises in unintentional mobile survey respondents The case is clear – we need mobile survey research strategies, best practices and resources to meet our respondents where they seem to be going! Mobile cannot be ignored!

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 6:59 Section 3: Developing Best Practices in the New Landscape…

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Thinking Differently About Technology & Survey Data Collection Technologies often viewed as “survey-enabling” tools, but not as “data collection vehicles” in their own right The advancement in apps and hardware for these mobile devices and the social networks that are accessed using them has opened the avenue for a broader spectrum of data that can be collected as part of a “survey” Location data via captured GPS coordinates for both respondents and field workers Scagnelli, et al. 2012 Olson and Wagner, 2015 Picture data captured via an app or via phone Scagnelli et al. 2012, Michaud, Buskirk and Saunders, 2014 Data Collection Gigs via Crowdsourcing Duan, Lai and Link, 2013

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 A new era of survey research ushers in new data collection opportunities

In-person, in-the moment, round the clock data are now being collected, compared and explored in the context of surveys Health related outcomes via apps and peripheral devices (Bluetooth) and hardware Gregoski, et al. 2013 Abroms, et al., 2011 Kumar, 2012 Au-Yeung et al., 2012 (http://www.proteus.com/assets/AuYeung_NetworkedSystemForSelfManagement1.pdf) Real-time diaries for wellness, activities and time use and trip/location data Koskinen and Salminen, 2007 Fernee and Sonck, 2013 Graham and Cobb, 2013 Lai et al., 2010 Nitsche et al. , 2012

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

GPS analytics provided additional trip details and metrics not easily captured via survey (Source: Nielsen, 2012)

38 Source: Scagnelli, Bailey et al. 2012

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 “Grab & Go” Purchase Locations

Each green dot is a Grab & Go store where a Nielsen panelist made a purchase

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Mobile Devices aren’t just for Respondents!

We are also using these devices ourselves to collect data In-person data collection via tablets is also on the rise Seal, 2012 – http://www.burke.com/Library/Articles/John%20Seal_Article_CASRO%202012-2013%20Journal.pdf Paudela, et al. 2013 – http://www.ghspjournal.org/content/1/2/277.full.pdf+html https://www.dooblo.net/ http://www.surveypocket.com/

Using Mobile phone–assisted personal interviewing (MCAPI) van Heereen et al., 2014 van Heereen et al., 2013 – http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3713928/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Interface & Flow

User interface: usually an important consideration in data collection, with new technologies it is essential New & extended human-computer interaction concepts: Visual appeal Intuitive flow (more than simply “logical”) Ease of use /functions are readily apparent Few, if any, text instructions Optimize mode capabilities and reduce user anxiety and frustration Study of why people participate in mobile surveys (Bosnjak et al 2010): Perceived enjoyment; attitudes toward participation; self-expressiveness, & trust factors most important Blend current understanding of CAI systems with expanding literature on human-computer interactions (use & interaction)

41 Source: Bosnjak et al, 2010 http://ssc.sagepub.com/content/28/3/350.short

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Thinking about Mode Choice on Multiple Devices

The emergence of devices with internet access has created unprecedented choices for online survey completion: computer (lap/desk top), Tablet, Smartphone, FeaturePhone (for the brave) Other device (Sony PSP, your Smart TV, your Fridge, etc.)

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Thinking about Mode Choice on Multiple Devices

Buskirk, Walton and Wells (2013) developed a study to investigate factors related to survey mode choice Study employed a subset of smartphone, tablet and computer users from a nationally representative probability online survey panel and replicated this study for a large opt-in panel of similar users Study preference data analyzed using Conjoint Analysis. Factors included: Survey Mode (Smartphone, Tablet and Computer as well as Paper and Pencil), Time/Incentive (5 to 60 minutes; 2 to 25 dollars), survey sponsor, Incentive Type (Cash, Card, eCode, Charity) and Survey Topic). Among the combination of survey time, incentive amt and type and mode, Tablet and Computer mode had the top 5 combinations while smartphone only had one combination preferred in the top 25 (ranked 21). The degree of preference for smartphones increased as prior exposure with survey modes (other than computer) increased. Similarly, preference for computer mode decreased as prior survey exposure to on alternate modes increased.

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Thinking Differently About Technology & Survey Data Collection and Respondent Burden Technologies often viewed as “survey-enabling” tools, but not as “data collection vehicles” in their own right

What we normally ask ourselves: How can we use technology to conduct a traditional survey?

We should be asking: What are the questions / data elements we need to answer/collect to provide insights into phenomena of interest? How can technology be utilized to acquire that understanding / those data? What does respondent burden look like in the new landscape? Are there new data available that we can collect passively using new technologies that haven’t been easily collected before?

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Respondent Burden

The degree to which a respondent perceives participation in a task as “The concept of difficult, time consuming, or respondent burden is emotionally stressful. like that of the weather: everyone talks about it, Typically measured in terms of length but no one does anything about it.” or administration time of a survey. Norman Bradburn, Most common approach to countering 1978 respondent burden: incentives

Focus on “traditionally defined” Respondent Burden is no longer enough …

Source: Norman Bradburn 45 http://www.amstat.org/sections/srms/proceedings/papers/1978_007.pdf

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Respondent Burden in the Mobile World

Respondent burden in mobile world is similar to respondent burden in the regular world with a few differences Two Groups: Personal and Technoogical Personal: Safety Respondents aren’t just responding at home anymore! Steeh, Buskirk and Callegaro (2007) report respondents completing cell phone surveys while boarding a helicopter. Personal: Privacy New data types like photo/voice may mean information is collected from more than the consenting respondent Age of assent on internet is younger than consent via phone (13 vs 18/19) Personal: Time To complete survey tasks To load survey pages

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Respondent Burden in the New Mobile World, Cont.

Technological Data consumption Both receiving and sending information (survey questions/content and submitting responses) Nielsen (2012, Q3/Q4) Estimates that 96% of mobile phone users have a data plan and that the average monthly bill is $66 (all mobile) and $93 for smartphone users. Bandwidth Slower bandwidth increases time required to send and receive data Higher data consumption may lead to throttling by the provider – meaning reduced bandwidth in future Battery Drain Persistent GPS capture drains mobile device batteries Extra HTTP requsts can drain mobile device power

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Internet Access Method varies by Mobile Device Type

Internet access for US smartphone users seems to be predominantly through Mobile Connections rather than wi-fi connections; However this pattern is different across operating system

48 Source: comScore (2013) http://bit.ly/1msbiaV

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Smartphone Use Statistics by Country

https://think.withgoogle.com/mobileplanet/en/

49 Source: Our Mobile Planet, Google, 2013: http://bit.ly/1HSHFwO

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Data Plan Limits and Smartphone Users

50 Source: Pew Research Center (2015) http://pewrsr.ch/1FlmRbv

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Respondent Expectations

Being involved in a scientific survey is a relatively rare occurrence Little, if any, expectations by respondents Respondent experience with new technologies is much different Respondents are more savvy Technology changes rapidly – some respondents will find some tasks difficult or not easy to use. [Example: Michaud, Buskirk and Saunders, 2014 Voice Data Entry] Developing expectations from these experiences Ease of use/intuitive Speed Usefulness/utility or fun/entertainment or both Ability to share experiences w/ others Location awareness Auto detection/passive collection

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

The new paradigm: MOBILE Surveys

Smartphone Survey Design Research

Social Science Human Computer Computer Science Research Interaction and Web Buskirk and Andrus (2012) Development Link and Buskirk (2012) Parush and Yuviler-Gavish Kinesis White Paper (2010) (2004) Koch, P-P. (2010) Buskirk et al. (2011) Jones et al. (1999) Campaign Monitor (2012) Callegaro (2010) Callegaro and Macer (2011) Sweeney and Crestani Kriss (2013) (2006) Peytchev and Hill (2010) Medallia (2013) Zahariev et al. (2009) Fitts (1954) Smith (2011) EmailVision (2013) Goldberg, Fariden and Bosnjak, Metzger, Graf (2010) Html5rocks.com (2013) Alterovitz (2007) Lai et al. (2010) Pelletier (2013) Couper (2010) Zichermann and Millar and Dillman (2012) Cunningham (2011) Return Path (2013)

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 On Emerging Best Practices in the New Landscape…

Traditional Components of Best Practice: Length/administration time of a survey Respondent burden (time, number of phone minutes, number of surveys) Follow-up approaches/frequency, survey field periods Incentives – cash or other tokens (gift cards, minutes, etc.)

New Components of Best Practices: Balancing user engagement with user expectations/experience Ease of use / intuitive Speed of tool / interface / app Respondent privacy expectations Respondents sharing their experiences… Visual Appeal, user interaction with tool and gamification

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

16:59 Section 4: Types of Mobile Browser Surveys and What we Know so Far about Mode Effects

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Various Approaches for Survey Data Collection via a Smartphone Use the regular phone mode

Use Short Text Messages (SMS) Flash polls, Invitations, Synchronous Survey Interviews

App-based/administered surveys

Online surveys accessed via Smartphone Browser Passive and Active versions

Hybrid Approach: App-Like Smartphone Browser Surveys

Source: Buskirk and Andrus (2012) www.surveypractice.org.

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

SMS Surveys via Mobile Devices (Phones)

SMS Surveys involve a series of short communications between surveyor and respondent (i.e. text messages)

Surveyor sends a question with answer choices to the respondent (short snipets of usually 140 characters or less)

Respondent sends back the appropriate answer

Process continues as a series of asynchronous communications

Some 79% of cell phone owners say they use text messaging on their cells (Pew, 2012)

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 SMS Surveys via Mobile Devices (Phones), Cont.

Nielsen estimates that on average, 764.2 text messages were sent/received per month among all U.S. mobile post-paid/contract users. http://bit.ly/1QkUUFY

Studies using SMS for survey data collection are emerging, for example: Conrad et al. (2013) http://bit.ly/1gNVoms Yan et al. (2013) http://bit.ly/QAL5LV Mavletova and Couper (2014) http://bit.ly/1RGbEdk Schober et al. (2014) http://bit.ly/1M5sqiO

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Online Surveys via Mobile Web Browser

Passive: Surveys intended to be completed online are sometimes completed via mobile phone web browser (passive mobile web browser approach).

Active: The online web-based survey is optimized for the mobile environment. (think mobile versions of survey websites)

Many survey software programs (e.g. SNAP, Qualtrics, etc.) have mobile publishing options that optimize an otherwise online survey for the mobile browser.

If you know the type of devices on which those sampled will complete the survey, you can optimize the survey design for the particular browser/screen resolution (e.g. Safari / iPhone).

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Passive versus Active Smartphone Surveys

Passive Mobile Survey Appearance Active Mobile Survey Survey from Hotwire after a “double tap”

59 Both viewed on an iPhone 5

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

The Literature on Passive Mobile Browser Survey Mode Effects is now ACTIVE! Passive mobile browser surveys are surveys developed and intended to be completed via computer. Buskirk & Andrus (2012) hypothesized that passive mobile browser surveys could have negative survey consequences. (http://bit.ly/1f0Qvez) Fast forward 2 years: rise of mobile unintentionals + mass use of passive mobile browser surveys = empirical findings… Lower overall completion rates/higher break-off rates for Passive Mobile Surveys accessed on smartphones (Decipher, 2013; Wells et al., 2013; Poggio et al. 2013) Overall break-off rates among tablet users appear consistent with computer users (Decipher, 2013; Wells et al., 2013; Poggio et al., 2013) Overall median/mean completion time longer for smartphone users compared to computer users (Wells et al., 2013 [~2 minutes longer; Decipher, 2013 [~3 minutes longer], Cunningham et al., 2013 [about ¼ minute longer]). Respondents on Smartphones tend to be younger, female and non-Caucasian and primarily internet through phone users (Peterson et al., 2013; Wells et al. 2013) Faster Time to Hit Survey Start Page (Cunningham et al. 2013 [~170 min sooner]; Peterson, 2012 [~1-5 hours sooner (median)] 60

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 The Perfect Storm Goes Mobile: The intersection of non-mobile surveys and the rise of the unintentional mobile respondent

A recent internal study by Maritz (2013) has corroborated earlier findingsSecond by Medallia Quarter 2013(2012): Device Analysis Completion rates for Mobile users are lower than for computer users Completion for surveys designed% of Total for completion% of Total online% of Mobile via computerCompletion browsers. Time Relative Starts Completes Completes Percentage to Desktop All mobile phones Second14.0% Quarter 12.4%2013 Device 100.0% Analysis 75.0% 132.4% iPhone 9.3% 8.4% 67.6% 72.1% 129.7% Android 4.3% 3.7% 29.8% 72.1%Completion 137.3% Blackberry 5.0, 6.0 & 7.0% 0.2%of Total % 0.2%of Total % of 1.2% Mobile Completion 67.4% Time 184.4% Relative Other mobile phonesStarts 0.2%Completes 0.2%Completes 1.3%Percentage 68.2%to 133.1% Desktop iPadAll mobile phones 14.0%12.4%100.0%75.0%132.4%7.2% 7.6% NA 88.4% 99.7% iPhone 9.3%8.4%67.6%72.1%129.7% Desktop 78.8% 80.0% N/A 85.7% 7.7 minutes Android 4.3%3.7%29.8%72.1%137.3% Blackberry 5.0, 6.0 & 7.00.2%0.2%1.2%67.4%184.4% Other mobile phones0.2%0.2%1.3%68.2%133.1%Source: Maritz Research iPad 7.2%7.6%http://www.maritzresearch.comNA / 88.4%99.7% Desktop 78.8%80.0% N/A 85.7%7.7 minutes 61

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Passive surveys being reported in the literature

Articles dealing with online surveys being made (passively) available to respondents are making their way into the published literature and conferences. 1. Cunningham et al. (2013) http://bit.ly/RIyKGI 2. Bosnjak et al. (2013) http://bit.ly/1hckNtB 3. Poggio et al. (2013) http://bit.ly/1icvRCV 4. Poduska & Johnson (2010) http://bit.ly/PquJVf 5. Peterson (2012) http://bit.ly/1gP5Fim 6. Peterson et al. (2013) http://bit.ly/1i5WYo0

1 2 3 4 5 6

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Active and Passive Surveys being Reported in the Literature

Baker-Prewitt and Miller (2013) http://bit.ly/PqzePP Experiment Compared Active and Passive Mobile Browser Surveys to Desktops and Laptops Survey completion time, drop-out rates, straight-lining rates were all worse for passive version compared to active mobile version Peterson et al. (2013) http://bit.ly/1i5WYo0 Experiment to compare several active mobile versions of the survey to a passive version as well as an app-based version and two versions for PCs. Survey length (about 1 to 3 minutes longer) and break-offs (2 to 3 times) were longer/higher among passive mode compared to any of the active mobile versions and both types of mobile had longer completion times than PCs Survey satisfaction (4 Qs) ratings significantly lower for passive compared to any active mobile or pc version. (Won’t do again rating 2 to 5 times as high for passive compared to any active mobile approach)

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

APP-LIKE Mobile Browser Surveys

APP-LIKE Mobile Browser Surveys (A-LMBS) represent a hybrid of the APP and the Active Mobile Browser Approaches. A-LMBS are completed via the mobile web, but they rely on active browser refreshment using a combination of PHP Programming and Javascript to create “Active” Mobile web pages. A-LMBS can be developed to run without a persistent internet connection on multiple mobile browsers (HTML5) A-LMBS offer native, APP like functionality under the mobile browser context. In today’s browser vernacular, app-like surveys might fall under the class of Web apps which are defined as web pages that function and appear almost app like but are accessed via browsers

64 Source: Buskirk and Andrus (2012) www.surveypractice.org.

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 A-LMBS Example- The Got Healthy Apps Study

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

A-LMBS Compared to Active Mobile Web Browser

Active Active A-LMBS A-LMBS Mobile Mobile Approach Approach Browser Browser (iPhone) (Android) Approach Approach (iPhone) (Android)

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 APP-LIKE Mobile Browser Surveys in the Literature

Buskirk and Andrus (2014) report on an experiment comparing mode effects for app-like and online computer surveys http://bit.ly/1n8JLxA Completion rates for computer mode was significantly higher than for iPhone Item missing rates were very low across both computer and iPhone survey modes Survey completion times for iPhone mode was significantly less than Computer Significantly higher app ownership reported from iPhone mode vs Computer Mavletova and Couper (2013) present the results of an experiment that randomly assigned panel members (Russia) to complete a survey about sensitive topics using an app-like mobile version or PC version. One month later, respondents were invited for a follow-up survey on the other mode. http://bit.ly/1rl1aD4 Higher nonresponse, break-off ad longer completion times for Mobile Web compared to PC Measurement differences reported across the two modes for Alcohol consumption and income with less daily consumption and lower incomes being reported on mobile vs. PC More surveys completed outside the home in the presence of strangers on mobile vs PC No differential affect of satisficing by type of device uses (mobile or PC).

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

App-based/administered surveys

App-based Smartphone surveys administer and collect data via an (native) app that is installed on the sampled user’s Smartphone.

Surveys can be “pushed” to the app and then executed by the end user without the need for perpetual internet connection and can be designed to take full advantage of the phone’s capabilities like camera, voice, video, image capture, etc. Data transfer for completed surveys can occur once internet connection is established.

One popular example of this approach comes from Confirmit/Techneos’ Survey On-Demand Application (SODA), see: http://www.confirmit.com/Home/Community/Data-Sheets/January-2014/Confirmit-SODA.aspx

68 Video File Courtesy of Frits de Geest of Confirmit

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 App Attack – The rise of Mobile Device Apps

According to comScore (2013) Engagement with apps was 7 times that of engagement with web browsing, on average for iPhone owners compared to only 2 times for iPad owners. According to Flurry Analytics (2014) Time spent on a mobile device for US consumers, on average, has risen to 2 hrs and 42 minutes per day (4 minute increase over 2013) 2 hours and 19 minutes on average per day on Apps and 22 minutes per day, on average for the mobile web.

69 Source: Flurry (2014): http://blog.flurry.com/?month=4&year=2014 Source: comScore (2013): http://bit.ly/1c9vneo JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

App usage among Smartphone Owners by Country

70 Source: Our Mobile Planet, Google, 2013: http://bit.ly/1HSHFwO

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Top 15 Apps within the US Measured by % Reach

71 Source: comScore Mobile Metrix (2015): http://bit.ly/1ycKQM4

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Considerations for an APP-Based Approach

The APP based approach offers the most options in terms of formatting, gamification, and other options for data input And it offers more control per operating system One disadvantage is that it might be too much technology for the task at hand: Does a one-time survey need an app for normal survey tasks? Apps must be downloaded before data collection occurs Run the risk of “App Install Breakoff” Apps themselves count against respondent data plans/onboard memory required (download and memory) Potentially increases fielding time (development and deployment);

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 App-Based Survey Approaches in the Literature

Link and colleagues (2013) explored recruitment for an app-based diary study that was not part of a panel, but for a cross-sectional collection of television viewing. http://bit.ly/QCXkb4 Recruitment was conducted via telephone based on dual frame RDD samples for two local markets. Once screened for eligibility, respondents were sent app-registration details

Bailey and Wells (2012) compared mode effects between an app-based survey and an online (computer-based) version of the survey http://bit.ly/1k4dBjs Completion rates and completion times similar across the two modes Mobile app respondents typed more characters, on average, compared to computer respondents on a consumer behavior question iPhone Rs consistently entered longer responses in less time compared to android Rs

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Resources for Mobile Surveys

An overview of the survey research to date on mobile devices, capabilities, privacy and legal issues and some discussion of what’s on the horizon can be found in Chapter 1 of the: AAPOR Emerging Technologies Task Force Report

Mobile Web Best Practice Guidelines produced by the World Wide Web Consortium (W3C) Gives very detailed suggestions on how to create web pages for mobile devices, in general. Some of the information on touch interfaces, font sizes and mobile versioning of online websites may directly pertain to survey development and practice. http://www.w3.org/2005/MWI/BPWG/Group/Drafts/BestPractices-2.0/ED-mobile-bp2-20101202/#bp-devcap

Survey Practice eJournal Generally has a short publishing time from submission to online posting and contains many shorter articles highlighting experiments conducted with mobile devices. 74

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Resources for Mobile Surveys, Take 2

Mobile Marketing Research Association Has a Knowledge Center where content on mobile market research is searchable by keywords. http://www.mmra-global.org/search/all.asp?c=&bst=mode+effects Mobile Glossaries uSamp Mobile Glossary Contains detailed definitions for many things mobile including API, GeoFencing and many other terms relevant to mobile devices and use. http://www.usamp.com/learnmore/mobileglossary/?utm_source=RFL+Communcations& The European Society for Opinion and Market Research (ESOMAR) Guidance for Mobile Market Research Report http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and- guidelines/ESOMAR_Guideline-for-conducting-Mobile-Market-Research.pdf Guidance for Mobile Survey Research Report http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Codes- and-Guidelines_Conducting-survey-research-via-mobile-phone.pdf

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Resources for Mobile Surveys, Take 3

Our Mobile Planet Google Site Contains data from several countries on mobile behaviors by basic demographics of the user by year (from 2011 onward) for a host of countries around the world. http://think.withgoogle.com/mobileplanet/en/ Mobi-Thinking – Website Offered by dotMobi to educate practitioners about the mobile arena Special Section dedicated to Global Mobile Statistics http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats Flurry Analytics (Service and Blog) Track user activity in the app with events that are defined by the app owner. May be useful to collecting paradata for app related surveys. Also have a blog that provides regular estimates of app usage across a variety of platforms. http://blog.flurry.com/ and http://www.flurry.com

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 16:59

Section 5: Recruiting Methods and Coverage

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

From new devices to new sampling units!

The proliferation of devices has also provided new types of sampling units and new avenues for sampling. Take for example universes defined by owners of a certain app (like the Yahoo App). The sampling unit might be the App itself or alternatively the id of the downloader, stratified by Market Store (Google Play, App Store, etc.) The type or version of the app (phone or tablet) may also be a stratification variable RDD, ABS sampling frames may not provide access to the universe at adequate levels (low incidence of a given app). Perhaps selection generated from the app itself, much like push notifications, is one possible way to implement selection.

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Mobile device Reach and Coverage Biases

If you deploy a solution (such as an app) using only iOS app store, then you effectively will not be covering a fairly moderate percentage of smartphone users. iOS captures 42% of Smartphone Market in US compared to Android which captures 52% (Nielsen, 2014)

Similarly, if you design an online version of the survey with no mobile optimization, smartphone users will be present (unintentional mobile respondents) but those who receive internet only through mobile devices may be missing approximately 25% of Mobile Web users or 10% of the U.S. adult population

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Android Users versus iPhone Users

Annual HH income Share of Users Engaged by Activity Share by Platform

Share of Users between 18 and 24

80 Source: comScore, 2013: http://bit.ly/QiM43x

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Android Owners vs iPhone Owners, Part 2

Distribution among Distribution among Demographic Variable iPhone Users Android Users (U.S. Adults) (U.S. Adults) Men 45% 51% 18-34 36% 43% 55+ 27% 20% High School or Less 20% 32% College Degree (or More) 54% 39% Non-Hispanic Black 8% 19% Non-Hispanic White 80% 70% <30K annual HH income 15% 28% >75K annual HH income 51% 35%

Source: Estimates Recomputed from Tables Included in Pew’s 81 Smartphone Ownership – 2013 Update (Smith, 2013): http://bit.ly/Qnt4Rr

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Android and iDevices and Geography!

Geographic differences may also exist among the iPhone and Android User base within the U.S. Study conducted by Mobify in 2013 Using 20,000 partner websites with over 200 million visits through 2012 93% of mobile traffic came from either an iOS or Android mobile devices (tablets and smartphones)

The more red the state, the higher the percentage of Android mobile page views in 2012. The more blue the state, the higher the percentage of iOS views. States close to 50 / 50 will be a shade of purple. More blue=higher percentage of views from iOS

More Red = Higher Percentage of views from Android

82 Source Mobify (2013): http://www.mobify.com/blog/ios-vs-android-in-2012/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Survey Recruiting and Mobile Devices

There are many tools/strategies that are available for facilitating survey data collection using mobile devices. To point selected respondents to your survey you could use a QR code printed on a postcard or other type written material. The QR code (short for Quick Response Code) is a matrix representation of bytes of information associated with your web address (or location of a survey app in one of the app markets). GOQR: http://goqr.me/ because “QR Codes created on goQR.me are completely free of charge (commercial and print usage allowed).” Requires a QR scanner to be installed however; Use a simplified/shortened web address (URL) from a service like bitly (https://bitly.com/)

http://surveypractice.org/index.php/SurveyPractice/article/view/63/pdf bit.ly/14etvB3

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Example of a missed mobile opportunity

I recently received an invitation to complete the JD Power and Associates Vehicle Reliability and Service Survey. The main invitation came via postal mail A survey reminder came via postcard here:

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Keeping Recruiting close to the vest (or at least the mobile device)

An alternative to QR codes and shortened URLs could be the use of NFC (near field communication, within 4 inches) tags to transfer/receive data In areas where internet connectivity is spotty or unreliable, field workers could collect survey data loaded to their mobile devices via NFC from central computers and then when interviewing is complete, transfer of data back to laptop or central computer could again be made with NFC – all without requiring any internet connectivity. Passive NFC tags could be embedded in survey invitation papers/postcards or on posters or via kiosks and activated using respondents’ mobile devices. Similar to QR codes, but requires no additional apps or steps to “go online” from within the QR code reader app.

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Recruitment for Mobile Surveys via Email

Email Invitations and Web Addresses for Smartphone Surveys should be as simple and straightforward as possible.

S3DR Tip Wherever possible, email invitations should be designed responsively for easy reading and S3DS #1 navigation on smartphone and tablets.

Place the key information (e.g. survey sponsor, incentive…) at the beginning of the email subject. Place the survey link as soon as you can in the email invitation, to avoid scrolling. Minimize the number of “special characters” in the survey web address. If at all possible try to avoid “aliased” web addresses (especially for mobile web version).

86 S3DR = Smarter Smartphone (and Tablet) Survey Design Recommendation

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 The Got Health Apps Study Survey Invitation iPhone Invitation Process (Buskirk et al., 2011)

All panelists initially invited on their lap/desktops via email

iPhone panelists were asked to point their iPhone web browser to:

http://mobilehealth.slu.edu/1234567890 B

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Email Opens by Platform…Mobile on the rise giving way to unintentional mobile respondents

Source: Return Path (2013) http://www.returnpath.com/wp-content/uploads/resource/email- mostly-mobile/Return-Path-Email-Mostly-Mobile1.jpg

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Email on Smartphones and Tablets

Email and the Mobile Device Reality: 52% of all cell phone owners use email on their phone (Pew, 2013) 78% of smartphone owners check email on their phone. IDC and Facebook – “Always Connected” (2013) 10% Of iPad users say Mobile is their preferred device to read email and 9% for sending it. –Perion “iPad Owners Survey” (2013) Among mobile email users, If an email does not display correctly, BlueHornet “Consumer Views of Email Marketing” (2012) 69.7% will delete it immediately; 17.7% will view it on a computer 7.6% read it anyway on their device . Men open 20% more emails on mobile, but WOMEN click 10% more often on mobile e-mail TailoredMail – “It’s time to wake up and mobilize!” (2012)

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

If you send a respondent and email invitation, where is it likely read?

According to Pew Research (2013, 2011) 52% of Cell Owners use email on their phone and 54% of Tablet owners use email daily The 2013 US Consumer Device Preference Report by Movable Ink reports that:

At least 35% of Apple/Android Smartphone users spent over 15 seconds or more viewing an email.

90 Source: https://movableink.com/downloads/us_consumer_device_preference_report_Q42013

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Email Opening by Device Varies by State

Percentage of Emails State Opened on Smartphone (1) Texas 57.60% (2) Mississippi 57.27% (3) Indiana 56.22%

Percentage of Emails State Opened via Desk/Laptops (1) Maine 48.85% (2) Vermont 47.23% (3) Washington 47.19%

91 Source: https://movableink.com/downloads/us_consumer_device_preference_report_Q42013

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Email Formatting Recommendations

EmailVision provides additional recommendations pertaining to optimizing email deliverability For Subject lines and sender options, EmailVision recommends: Rather than generic emails such as [email protected] use relevant email addresses such as: [email protected], [email protected] [email protected] enable the recipient to understand the intent of the email and potentially reduces auto spam classifications.

http://www.emailvision.com/sites/default/files/email vision-deliverability-bestpractices-2011-03.pdf

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Email subject: Smartphone View

Android Smartphone resolution 480 by 800 3.7. inches screen size

93 JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 9 3

Responsive Email Designs for Survey Invitations

Campaign Monitor’s E-Publication “Responsive Email Designs” offers several considerations for developing mobile responsive emails including: Single column layouts (500 – 600 pixels) Links/Buttons minimum target area of 44 X 44 pixels Minimum font displayed on iPhones is 13 pixels – smaller font than this will be upscaled and formatting may change Call to action (i.e. “start survey button” or survey link) should appear toward the upper portion of the email to avoid scrolling For CSS programming use display:none option to limit social sharing buttons and other extraneous details in the mobile environment.

SOURCE: Campaign Monitor Responsive Email E-Book (2013): 94 http://www.campaignmonitor.com/guides/mobile/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Why focus on Responsive Mobile Emails?

Email Invitation on Survey that appears iPhone after tapping “Yes”

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Mobile Optimized and Non-Optimized Emails

Mobile Optimized Not Mobile Optimized

SOURCE: Campaign Monitor Responsive Email E-Book (2013): http://www.campaignmonitor.com/guides/mobile/design/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Example of Responsive Email from Amazon.Com

Column Dropping

Same email viewed on Smartphone

Email sent to Amazon Customers Viewed on Desktop/Laptop (responsive)

Source: http://marketingland.com/four-responsive-email-layouts-15858

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

From Responsive Email to App-Like Mobile Survey…

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 From Email to Mobile Web…

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Responsive Email  Quick Survey (Square, 2014)

100

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Example of an Email Invitation – Fresh off the Servers!

S C R O L L

101

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Internet Use, Time of Day and Device, US

102 Source: comScore, 2013 http://bit.ly/1c9vneo :

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Internet Use, Time of Day and Device, UK

103 Source: comScore, 2013 http://bit.ly/1QjphfK

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Resources for Mobile Optimized Emails

Smith (2013) “Responsive Email: Avoid Defections By Improving Mobile Experiences” Provides an overview of three common approaches to optimize emails for reading on mobile devices Mobile Friendly, Mobile Optimized and Responsive http://marketingland.com/responsive-email-creating-better-mobile-inbox-experiences-63014 Studabaker (2012) “Four Responsive Email Layouts” Examples of responsive email invitations from major vendors. http://marketingland.com/four-responsive-email-layouts-15858 Source: comScore, Studabaker (2012) “Choose The Best Responsive Email Layout For 2013 Your Message” http://bit.ly/1QjphfK : Offers examples of typical layout patterns that are commonly used in responsive email templates Shrink Wrap, Column Drop and Layout Shifting http://marketingland.com/responsive-email-layout-patterns-29378

104

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Mobile Recruiting Survey Protocols

Once you have optimized the email content and message, when should the email be delivered? 32% of mobile users often read email on their smartphone during the weekend; 31% sometimes reads their mobile email on weekends

54% reads email on their smartphone just before they go to bed sometimes or even more often

49% reads email on their smartphone immediately when they wake up sometimes or even often.

Source: Apsis “The email Barometer, Email on mobile devices” (2013) 105 http://www.emailmonday.com/mobile-email-usage-statistics

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

16:59 Section 6: Smartphone Survey Data Collection and Design Recommendations

106

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Smarter Smartphone Surveys Development/Design Recommendations (S3DR)

These recommendations come from our experience in developing surveys for smartphones from the java/html frameworks directly and are based on extensive review of the existing literature and practice. Some of them are based on the current technical and screen characteristic (i.e. size) of mobile devices – aspects likely to change over time. The design suggestions presented here are our recommendations meant as guidance not guideline.

107 JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 1 0 7

S3DR #1: Paradata

Successful implementation of multimode surveys requires that the survey software include comprehensive device detection functionality. Device detection is needed to automatically identify the type and brand of each respondent’s device, and then display the survey according to that device’s specifications.

-Cazes, et al. Kinesis Mobile Landscape Report, 2010 http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf

Plan to collect paradata from mobile sample including device type, size and browser size. Browsers matter and can be key to an overall S3DR #1 survey’s functionality on a respondent’s device.

108

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 User agent strings

“A text field in an HTTP request header that contains the name and version of the Web browser” (PC magazine definition) Web resource for locating your User Agent Strings: http://user-agent-string.info User agent strings look like this: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en- US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.41 Safari/534.7

Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_2 like Mac OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko) Version/4.0 Mobile/7D11 Safari/528.16 BlackBerry9700/5.0.0.351 Profile/MIDP-2.1 Configuration/CLDC-1.1 VendorID/123

109 JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 1 0 9

Device Feature Detection – Moving Beyond User Agent Strings

Sorting out the contents of User Agent Strings Resource: http://blog.trasatti.it/2010/10/sorting-user-agent-strings-out.html

In addition to phone types and operating systems, features of devices/browsers can be obtained through various device description repositories (DDR) maintained by third parties and could be used in advance to determine which information (i.e. javascript, HTML markup, etc) to send to the phone to render the mobile survey page. Device Atlas https://deviceatlas.com/device-data/devices (Browse library of information available on each phone model, for example). www.deviceatlas.com (User Agent String tool) WURFL (Wireless Universal Resource FiLe) http://wurfl.sourceforge.net/

110

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Using RDRs in action – WURFL Example

http://wurfl.sourceforge.net/

111

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Beyond User Agent Strings  Feature Detection

Feature detection is often deployed using a JavaScript library like modernizr (www.modernizr.com) and is done through a series of queries made to the phone.

If you are interested or concerned about tracking mode effects (especially primacy or recency effects), knowing the type of device/operating system may not be enough.

Keep in mind that while most smartphones (and tablets) use the native browser (safari for iDevices and Android Browser for Android devices) users can and do change these defaults.

Interpretation of the survey scripts can be interpreted differently across browsers and some functionality will gracefully degrade or not be offered as intended number entry presented as text; photo upload not offered

112

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Browser Types on Desktops (April 2014- April 2015)

A closer look at the Variability in Browsers across device types using desktop internet traffic data collected over the past year (from Stats Counter)

113 Source: StatsCounter: GlobalStats (2015) http://gs.statcounter.com/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Browser Types on Smartphones (May, 2014 – April, 2015)

114 Source:StatCounter GlobalStats (2015) http://gs.statcounter.com/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Here’s a look at why browsers matter

Ultimately, device detection via user agent strings helps to deliver overall content to the correct type of device (phone versus tablet versus computer).

However, the way the device translates the survey programming (HTML markup etc) is determined by: The browser in use The features of the browser The features of the device (e.g. hard keyboard versus virtual, front facing camera or not)

115

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Same view – different view!

When a media query is issued for the size of the viewport for each of these smartphone the result is 320 pixels. However, each of these phones have extremely different browsers with different capabilities.

116 Source: http://www.slideshare.net/yiibu/pragmatic-responsive-design

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Self-Demo 1: Explore Your Device’s UA String and Size

Using the following web-resources, determine your Smartphone’s user agent string and screen size.

User Agent String: http://user-agent-string.info

Screen Size: http://whatsmy.browsersize.com/

117

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Specific Example: Same Phone…Different Browser

118 http://user-agent-string.info/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Why browsers matter for Mobile Surveys!

Galaxy Note 3 Android (KitKat Android Browser Chrome Mobile 4.4.2) Smartphone (default/native) Browser (activated)

119

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

HTML5 and the rise of app-like functionality

The advent of HTML5 has given programmers and researchers a lot of flexibility and versatility in creating mobile versions of websites with many supported input types for mobile forms and capabilities: Geolocation collection Media capture (voice, photo and video) Improved and expanded input types Web storage For survey researchers, this new development means that our active mobile browser surveys can be made more and more app like with standard, predefined enhancements that have been include in HTML5. But this is not yet a silver bullet!! Care must be taken!

120 http://html5test.com/index.html

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 HTML 5 functionality across mobile browsers

Mobile HTML 5 Resources are available to determine if certain functionality of interest is available on a particular mobile browser. http://mobilehtml5.org/ http://caniuse.com http://detectmobilebrowsers.com/

121

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Alternate HTML 5 Browser Functionality Tool

Another comprehensive tool for tracking availability of HTML5 functionality across browser types is http://caniuse.com

122

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Case Study Example of HTML 5 in Action

Walmart in partnership with Mattel Hot Wheels launched a loyalty program for shoppers. Shoppers who purchased Hot Wheels products could receive points redeemable for various rewards by scanning their receipts showing the Hot Wheels item purchased. The loyalty program required NO app, yet enabled mobile photo capture! Scanning/photo functionality and uploads to the Walmart program were managed via an online website optimized for mobile visitors with (modern) browsers

123

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Hot Wheels Redemption Program on iPhone

SOURCE: Hot Wheels (2013) https://hotwheelsrewards.com/

124

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Practical Example from New Study!

Michaud, Buskirk and Saunders (2014) designed a study in partnership with Decipher and Research Now to compare various touch and data collection possibilities across three modes – computers, tablets and smartphones. A photo capture question was deployed to all devices detected as either “Tablet” or “Smartphone” using user agent string classification. Kindle Fire Tablets Respondents Those with devices without this running the native Silk browser had functionality had to press continue twice to continue past this screen. NO photo uploads as expected from browser limitations 125

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

S3DR #2: Mobile Survey Approach

Select an implementation approach that: (a) optimizes presentation of survey content S3DR #2 (b) is consistent with the type of survey data desired (c) is appropriate for the context.

126

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Share of Demographic Audiences by Platform

http://bit.ly/1D2zXhQ

Source:127 comScore Media Metrix Multi-Platform, U.S., Age 18+, Dec 2014 / Dec 2013

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Focusing on Digital Double and Triple Vision

Percentage of Specific Tablet Brand Owners who own Smartphone of a given brand

Source: Pew Research, 2012 http://bit.ly/1ttquIX

128 Source2: comScore (2013) http://bit.ly/1msbiaV

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Not all Tablets are Created (and used) Equally

129 Source: http://www.statista.com/chart/2075/how-different-tablet-users-spend-their-time/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Daily Social Networking Visits and Sharing by Country

70 61 60 56 56 56 52 53 53 50 41 40

30 26 23 22 18 19 20 16 14 12 10

0 Germany Sweden UK USA Visiting Social Networking Sites (via smartphone) Visiting Social Networking Sites (via Computer) Social Sharing via Smartphone Social Sharing via Computer

130 Source: Our Mobile Planet, Google, 2013: http://bit.ly/1HSHFwO

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Tablet versus Smartphone Usage by Task (European Technographics Consumer Technology Online Survey)

131 Source: Forrester Research, 2012: See: http://tcrn.ch/1SHLGaO

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Sweden Internet Users, TV + 1 Device

Source: Inizio “Multiscreen 2013” See: http://bit.ly/1FJyMSW

132

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 S3DR #2b: Survey Apps Should be Developed using versions of the Toolkit that are at least one-version back from current

While IOS devices update frequently, Android device updates are initiated via Cell Phone Providers and can often lag behind currently available versions.

The functionality of smartphone components such as S3DR#2b camera, scanners etc. will in part be governed by both device and operating system.

By using the older version of the OS to develop the app, the functionality will adapt in newer phones and not excluded from older devices.

Moreover, your survey app will be “found” in Market searches across operating system versions.

133

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Global Android version distribution since Dec. 2009

Dec. 2010 Feb. 2011 Oct. 2011 June 2012

134 Source: http://en.wikipedia.org/wiki/Android_version_history

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Thank you! [email protected]

314-695-1378

www.m-s-g.com

@trentbuskirk

135

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Survey Research using Emerging Technologies:

Considerations for Design, Data Collection, Sampling and Recruitment for Smartphone and Tablet Based Surveys

JOS Conference June 10 and 11, 2015 Trent D. Buskirk, Ph.D.

136

JOS AAPOR30th Anniversary Short Course, Conference 2014 Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Day Agenda & Goals 2

137

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

S3DR #4: Survey Length

Smartphone surveys should be designed to be as short as possible, but not necessarily abbreviated versions of longer online S3DR #4 surveys. Dead spots still exist for Smartphones In both voice and data (i.e. edge, roaming) Internet speeds vary also by area Sprint has both 4G and 3G speeds in St Louis, for example. Smartphone users multitask Three good reasons to allow for users to return to the survey site Start over? Begin where they left off (approximately)

138

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 S3DR #5: Survey Layout

The overall survey layout should minimize the need for scrolling (either horizontally or vertically) to the extent possible. Survey layout should also minimize the need for pinching/zooming. S3DS #1

S3DR #5 The number of questions per screen should generally be determined in order to minimize scrolling (implied scrolling) Be mindful for the device and place key actions in “good” zones Some devices don’t support zooming Answer choice layout may create need for scrolling on some devices. 139

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Putting Responses within Reach, for mobile devices!

Source: http://www.kickerstudio.com/2011/01/activity-zones-for-touchscreen-tablets-and-phones/

140

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Illustrating an ACTIVE Mobile Web Browser Survey: Peytchev & Hill Study (2010)

Experiment with 92 adults provided with a Samsung Blackjack phone 2.2. inches and resolution of 320 by 240 pixels

Several embedded experiments to test question wording effects, image effects, scrolling effects and open ended effects

141 JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 1 4 1

Horizontal Scrolling Experiment (Peytchev & Hill, 2010)

Q u e M s o t r i e o n s

More Responses No Significant Differences Noted

142

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Example of Survey from Frontier Airlines (2014) with Horizontal Scrolling Prompts

143

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

The Rotate Screen Prompt has Debuted!

144

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 S3DR #6: Question Formats

Consider reducing the number of response options for grid questions or reorganizing grid questions as single questions per screen. Consider organizing response options vertically or horizontally with scale labels provided per S3DR #6 occurrence (see Peterson, 2013).

If Multiple choice/select all that apply require a long list of alternatives, consider converting question into free response with as many single-text fields as answers expected (i.e. what are three of your current research interests – list up to three).

145

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Some Grid Examples on Mobile Devices

Example of Grids – reworked for mobile surveys

146 Source: Peterson et al. (2013) http://bit.ly/1i5WYo0

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Got Health Apps Study Screenshots Illustrating Question Layout and Response Choice Feedback

Source: Buskirk and Andrus (2014) http://fmx.sagepub.com/content/early/2014/04/08/1525822X14526146.full.pdf?ijkey=jZzKaocZyiG6YNn&keytype=ref

147

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Paging Versus Scrolling Question Presentation

Scrolling question presentation in mobile surveys presents a series of survey items on a single page Requires respondents to scroll (vertically) in order to see (and answer) all survey questions on the page Paging question presentation in mobile surveys limits the number of questions per page Generally one question per page and the respondent navigates multiple pages in order to complete the survey Buskirk and Andrus (2014) present a hybrid with two questions per page to limit (a) the vertical scrolling required on mobile; (b) the overall number of pages needed for survey completion and (c) increase comparability for desktop version

148

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 S3DR #7: Scrolling Versus Paging

(a) Consider an optimal combination of these methods for presenting questions. (b) If you have a series of questions with similar response options and type then scrolling may S3DR #7 help move a respondent along more quickly and simply. (c) If you have multiple skip pattern logic in your survey, paging may be preferred (at least for that section). (d) Be careful that scrolling is suggested or implied so respondents won’t skip (especially if Next/Back are at the top)

149

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Closer look at Paging versus Scrolling

Survey Outcomes for Mobile Respondents Paging Scrolling

Ease of Completion (Mavletova and Couper, 2014)

Lower Break-off rates (McGeeney and Marlar, 2013; Mavletova and Couper, 2014)

Shorter Completion Times (Mavletova and Couper, 2014)

Lower Overall Item Nonresponse (Mavletova and Couper, 2014)

Fewer Technical Difficulties when Completing Survey

Greater variety of question types/inputs Maximizes stored data if internet connection is lost and respondent returns to the webpage for completion Easy Facilitation of skip pattern logic

150 Source: http://bit.ly/1nqe6Ib

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 An example of Scrolling in practice

Requires tapping here; then entering value; then tapping out then scrolling and repeating.

151

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Example of a Survey w/ a Heavy Scrolling Section

152

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 S3DR #8: Question Types

Sliderbars, sum tallies and drag and drop sorters may not work on all devices. These question types should be tested extensively using emulators prior to their adoption in your mobile survey. Text entry fields should be wide enough so that

S3DR #8 users can see their entire entries without scrolling horizontally. Use this type of entry judiciously. Decisions about types of input/forms or input should include consideration of number of taps or clicks required (NTR or NCR) to enter (and register) an answer.

153

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

A quick note about sliders

Be wary of the default starting positions – could grossly impact final results Michaud, Buskirk and Saunders, 2014 Peterson et al., 2013 Can’t distinguish a “3” from missing if default start is midpoint

154 Source: Peterson et al. (2013) http://bit.ly/1i5WYo0

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Using the NTR metrics to help weigh design choices, in practice.

Michaud, Buskirk and Saunders (2014) included an experiment to compare radio button and list inputs for various substantive questions.

Initially respondents that were randomized to lists received this format for all survey questions including demographic questions about gender, education, race, etc..

Pilot testing and the NTR metrics changed our plans for the demographic questions…

155

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Using the NTR metrics to help weigh design choices, in practice.

Radio Button List/Dropdown iOS Selector Android Version (all devices) and Stock Android (some browsers)

156

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Beware of the Pagination/Scrolling Mashup!

Just because it’s optimized for smaller screens doesn’t mean it’s optimized for respondents! Presentation of questions only without their response options will save space on a screen, but makes data collection/completion a mine field! Example from Major League Baseball! https://www.fansatbat.mlb.com/PORTAL/default.aspx

The survey question (depicted on an iPhone 5): “Are any of these networks included in your current cable lineup?”

157

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Taking Advantage of New Input Types from HTML5

With the release of HTML 5 there are several new input types available that might be of interest to survey researchers and questionnaire designers including: Number Tel Email Range (a.k.a. sliders) Time These new input types when used wisely will likely reduce: The overall amount of additional scripting that is required to format the survey questions and response options Reduce survey page load times The NTR metric for a given question

158 Source: MobiForge, 2013 http://bit.ly/1aH3vnM

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Numeric, open ended questions using new HTML input options Number and Tel

How many miles per year do you drive the car that you personally drive most often. (Enter a number between 0 and 50000)

Question Page Default Input Using HTML5 “number” 159 input type

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Making Numeric Input Similar across Devices

By using the HTML5 “tel” input type, a numeric keypad similar to what is displayed when making calls is displayed on all devices.

Formats consistent across devices in terms of number entry. Note however, that if decimals are required, then iOS keypad does not offer this capability directly.

160

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Labels for Open Ended Fields

Labels Top Aligned Labels Within Fields

161

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Labels for Open Ended Questions Top or Within?

Pros for Labels at Top Cons for Pros for Labels Cons for Labels Labels at within within top Minimize completion time Requires Saves vertical space Hint text can be Easier coding (no indents/ more space misconstrued as a tables) completed item Accessibility: label then Results in Reduces page length Depending on tap field in order longer pages for pages with location the hint text Supports multiple screen (requiring multiple open- may not disappear resolutions scrolling) ended questions from the respondents answer More space for the item Can save on Once respondent (horizontally) horizontal scrolling begins typing, the full Reduces horizontal scrolling required compared question or item is to left aligned labels no longer fully visible

162 Source1: Wroblewski, 2012 http://bit.ly/1eFrXaG Source 2: Penzo, 2006: http://bit.ly/1iuOFC2

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Example of Label Alignment in practice

From Wroblewski, 2012 http://bit.ly/1eFrXaG

163

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

S3DR #9: Navigation/Progress

Limited screen landscape requires compromise and judgment when using logos, progress bars, disclaimers and help links in mobile surveys. Consider using abbreviated progress bars S3DS #1 Placement of next and back buttons should be oriented toward the top of the screen or at the S3DR #9 bottom of the screen as long as they are persistent and visible without scrolling

Disclaimers should be placed on the welcome page.

164

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Illustrating Back Button and Progress Bar Placement Next/Back Buttons

Progress Bars

165

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Another Take on Next/Back and Progress Bars

166 Strategic Reflections Quick Serve Restaurant Survey, 2014

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Another Example of Persistent Progress Bars (Expedia Survey, 2015)

167

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

S3DR #10: Buttons, Tabs and More Buttons

Response selections, action buttons and navigation tabs in surveys should be made as large as possible on mobile surveys.

Touch input differs from Mouse input. S3DS #1 Fattest Fingers – Average width of index finger is 11 3 S DR #10 mm with a range of under 7 for babies to over 19 for tall athletes.

Make sure to include padding around important navigation tabs (next/continue/enter) to avoid accidental taps .

168

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Importance of Button Sizes

Make butons/tabs large enough to not be missed/avoid mistaps 40 pixels (7 mm by 7mm) GOOD 50 pixels (9mm by 9 mm ) BETTER 30 pixels (5 mm by 5 mm) OK Give enough buffer around/between tabs/buttons Padding of at least 2mm (10 pixels) between targets

169 Source: Microsoft Touch Guidance

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Touch Reference: iPhone IOS Requirements

iOS requires app icons to be 57 square pixels for iPhones

iOS requires app icons to be 72 square pixels for iPads

Knowing the resolution of the devices is also important for rendering. Retina display phones and tablets for example usually have double the size allowance for icons (and images). Check out this icon reference chart by graphic illustrator Vicki Wenderlich: http://www.vickiwenderlich.com/2012/09/app-icon-size-reference-chart/

170

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 One accidental tap could be missing data!

From Michaud, Buskirk and Saunders (2014) We included slider bars as input options for scale questions. The slider bar button was made large enough to drag across phone and tablet devices. On smartphones such as iPhone, the “continue” tab was too close to the upper answer choices, so people who dragged to either 7 or 8 could have tapped the continue button by accident on the way to a rating of 9 or 10. In the radio button version, people attempting to tap an 8 9 or 10 could have tapped continue on the way there and recorded no data.

171

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

S3DR #11: Flash, Video, Audio, Images

Resolution of graphic images should be reduced in proportion to expected screen resolutions of mobile devices prior to transmission to devices (avoid using strictly responsive only web designs for mobile surveys if these surveys will contain images. Prefer RESS approach or adaptive RWD instead). Flash content should be limited if iPhone Users are partS3DS of #1 sampling frame. Alternate video playing capabilities are emerging through HTML5 but not ubiquitous across devices S3DR #11 yet. Video files should be embedded via link to YouTube or like “player” Can’t rely on QuickTime, windows media player, etc. Consider the size of the audio, image and video file sizes as they have implications for both bandwidth and data consumption.

172

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 A quick overview of IMAGE Types with focus on Mobile Device Surveys

Image Common Use Notes Type JPEG Typically used for photographic images A small reduction in quality (80% vs 95%) can reduce overall image weight by more than ½. 50% quality images can render quickly and without visible differences on many mobile devices/websites. PNG PNG is superior to GIF. It produces PNG files of photographs can be bloated in size smaller files and allows more colors. PNG compared to corresponding jpeg images (5 to 7 : 1) also supports partial transparency. PNG files however can be compressed further and However not all browsers support this still deliver the same image without loss of detail. function (IE). PNG files are also the only Try PNGCrush and OptiPNG or Smush.it image type with lossless compression http://www.smushit.com/ysmush.it/ supported across the web. GIF If the image has fewer than 256 different For images with greater numbers of color or detail, colors or contains areas of similar color GIF compression may miss some of the colors. But then GIF will be optimal. feature phones may not have a palette that goes beyond the 256 standard gif palette anyway. WebP Newer format developed by Google Can provide more compression of images of https://developers.google.com/speed/webp/docs/we comparable quality saved as jpeg. 173 bp_study

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Example of PNG Image Compression

BEFORE: AFTER: 13.66KB 11.01 KB

http://www.smushit.com/ysmush.it/

174

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Additional Issues with Collected Images

Images captured by mobile devices contain meta data that includes: Camera information (aperture, shutter speed, focal length) Date and Time information Geo Location Orientation of the image (portrait or landscape) This information along with the contents of the photo (respondent plus others who may not have consented to be included in the study) have potential privacy concerns. Consider stripping the collected images of this meta data and save it in it’s intended orientation prior to image upload. Not only reduces privacy concerns about location but also reduces the overall size of the transferred file if done client side.

175 http://cloudinary.com/blog/top_10_mistakes_in_handling_website_images_and_how_to_solve_them

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

S3DR #12: Recommendation for Mobile Surveys that have a Multilingual Audience/Various Language Versions

Remember the higher incidence of mobile users among Hispanics and Asians in the U.S.

When deciding on spacing of answer choices, resulting font size implications and layout for a given set of questions, make choices in light of the languages that will be presented. S3DR #12 For example, if the Spanish version of the answer choices are actually longer, then optimize for these choices first rather than always optimize for English versions.

176

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Optimizing for English Versions may not be sufficient!

ENGLISH VERSION SPANISH VERSION Questions or their answer choices (in English) that appear to be well optimized and presented using mobile browsers may not be optimally presented in another language

177

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

S3DR #13: If GPS location data are required, user opt-in/out notifications should be pushed Apps, unlike mobile web browsers, can enable GPS location services via the Smartphone’s GPS Antenna directly. Geolocation data can also be collected via browsers that support the HTML5 functionality (accuracy varies by cellular and wi-fi connectivity. In IOS GPS permissions are granted/controlled on aS 3“perDS #1 app” basis; S3DR #13 Android devices generally use global controls; Should your survey require GPS data collection (passive), the respondent should be given the option to opt out/in via a push notification at the time of the app launch. Don’t assume that once an opt in is given once that it persists across survey data collection points.

178

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Geolocation Data via Mobile Devices

Respondents – where art thou? GEOLOCATION is the capability to detect and record where a mobile phone is located using various methods: IP address, Mac address, RFID, Wi-Fi connection location or GPS coordinates. GPS or Global Positioning System is a satellite-based navigation system. GPS uses triangulation to calculate a user’s exact location. Smartphones that are GPS-enabled allow an app to determine a phone’s exact location. Using HTML5, geolocation can be estimated form mobile deices via the mobile web browser (wi-fi versus cellular networks) GEOFENCING Technology that provides the ability to create a virtual fence around a geographic location in the real world. Smartphones that are location-enabled detect when someone enters or exits these fences, (Big: city block or Small: retail store).

179

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Geolocation detection through BROWSERS!

A number of different sources are used to attempt to obtain the user’s location, and each has their own varying degree of accuracy. A desktop browser is likely to use WiFi (accurate to 20m) or IP Geolocation which is only accurate to the city level Mobile devices tend to use triangulation techniques such as GPS (accurate to 10m and only works outside), WiFi and GSM/CDMA cell IDs (accurate to 1000m). Accuracy on mobile phones depends on several factors including: Whether wifi is activated or not (for android phones this has a huge impact) Is phone in an urban or rural area? Is phone moving or stationary? Is a VPN turned on? http://html5doctor.com/finding-your-position-with-geolocation/ http://www.andygup.net/html5-geolocation-api-%e2%80%93-how-accurate-is-it-really/

180

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Geolocation through Browsers

There are some pros and cons for geolocation through browsers that we need to be aware of in order to better use these types of data Not all browsers support the Geolocation functionality of HTML5 (IE 8 does not, for example). Different browsers use different Geolocation services (mapping) Chrome browsers=Google Location Services Safari browsers on iOS devices=Apple Location Services Requires and internet connection and explicit user permission (as per HTML 5 Geolocation Guidelines for use). May not be useful for persistent Geo-location tracking or for geo-aware notifications such as used in Geo-fencing Does not generally have the same drain on batteries as the actual GPS hardware. Does not require an app to be installed in order to identify location of respondent. 181

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Accuracy of Geolocation via Browsers

Accuracy can be impacted by whether or not WiFi is turned on, for example. Here is a test link and the results I obtained with and without WiFi using an iphone 5 (ios6) http://andygup.net/samples/html5geo/

182

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Smartphone GPS Data Vendors

Locately: (http://www.locately.com/) Point of interest matching Trip Analytics Data processing (No integrated API) Panel built from anonymous user data

Loc-Aid (http://www.loc-aid.com/) Integrated API Allows for custom geo-fencing

183

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

7-Eleven and am/pm stores that one respondent visits

Gym Home 3 Visits: on way to 1 Visit Gym

4 Visits: on way to School

School

184

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 S3DR#14: Pre-Testing Mobile Surveys

There is a great deal of functionality that is consistent across mobile smartphones- but there is enough inconsistency to require extensive pre-testing of your mobile survey across multiple platforms.

Consider using smartphone emulators to pretest the mobile version of your survey or alternatively a virtual

S3DR #14 device laboratory to test mobile versions of your survey. Test availability of flash content using a resource such as: http://www.adobe.com/flashplatform/supported_devices/smartphon es.html (or on your mobile phone/emulator) http://www.adobe.com/software/flash/about/ 185

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Smartphone Emulators

Suggested Emulators for Popular Smartphones

iPhone: http://iphonetester.com/ Android Devices: http://www.addictivetips.com/windows- tips/download-google-android-emulator/ Blackberry Devices: http://www.blackberry.com/developers/downloads/simulat ors/index.shtml Microsoft Devices: http://www.microsoft.com/download/en/details.aspx?displa ylang=en&id=25191

Source: http://sixrevisions.com/web- 186 development/mobile-web-design-best-practices/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Mobile Emulator Software Example: Keynote’s DeviceAnywhere

Many types of devices centrally located in one source (host of tablets and smartphones)

http://www.keynotedeviceanywhere.com/da-free-product-overview.php

187

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

16:59 Section 7: Web Design Approaches to Mobile Surveys (DIY)

188

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Active Mobile Browser Survey Development

A multi-screen strategy is a must to succeed with today’s constantly-connected consumers. But one size doesn’t fit all when it comes to creating the perfect configuration. Many factors come into play, and you know your own business needs best. – Google 2013 http://bit.ly/1i9e3P1 and SURVEY Three main approaches to developing an active mobile browser survey: Responsive Web Design (RWD) for Survey Pages Mobile-First RWD Adaptive Designs Responsive Design with Server Side Components (RESS) Separate Mobile Web site developed for Surveys

189

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Responsive and Mobile Responsive Web Design

Responsive web design (RWD) is a method for web design that uses fluid grids, images and media queries to change the displayed layout based on the size of the device (view port size). Media queries govern when and how the fluid layouts appear on given devices and is managed by the respondent’s device Single HTML source code delivered to all devices (one size fits all) Mobile-first Responsive Web design is a method that begins with the mobile version first and pushes the web page to respond to larger devices via progressive enhancement. Additional layout attributes and survey functionality can be added progressively, in real time for respondents accessing website using larger devices. References Frost (2011) http://bit.ly/1gOcOPK Passani (2013) http://bit.ly/1i9G8Wg Wroblewski (2011) http://bit.ly/1gOcS2d

190

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 RWD and Mobile First RWDs for Surveys

RWD for Surveys would be a strategy that takes an online (computer survey) and thinks about how to render it down to tablet and then down to smartphones. Mobile First RWD would take a mobile survey and progressively push it to larger devices with potentially different functionality.

191 Source: http://web3canvas.com/what-is-mobile-first-responsive-design/

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Responsive Web Design Example: www.SweetHatClub.org

192

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Responsive Web Design Example: www.SweetHatClub.org

193

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Adaptive Designs

There are various versions of Adaptive web designs but the approach that has a lot of traction recently is Responsive Web Design with Server Side Components (RESS) RESS is sometimes referred to as a hybrid approach between Server- side implementation (rendering unto Caesar what is Caesar’s) and RWD. RESS uses device detection on the server side to send device specific survey web page content/scripting The device specific code can include components that allow it to progressively enhance the survey page depending on browser features it detects on the respondent’s device. References: S. Rieger (2011) http://slidesha.re/1k1HLUH Pratap (2013) http://bit.ly/IK6Q8A Ronan (2011) http://bit.ly/1tuHNcJ B. Rieger (2011) http://slidesha.re/1pnzBeA

194

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Images RESPONSIVE WEB used in DEISGN – 10000 Survey Feet Overview Computer Scripting and HTML Markup Survey (all devices) Questions

www.takesurvey.com

195

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

What Kind of Images Device is it? User for Agent String PHONES Says: Computer iPhone Scripting and HTML Markup Survey ( ) PHONES Questions

www.takesurvey.com RESPONSIVE WEB DEISGN with Server Side Components 10000 Feet Overview

196

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Giving your Online Survey Site a Mobile Sibling

Separate mobile version of the site is the ultimate adaptive strategy Special scripts are created to program survey functionality specifically for the mobile devices The survey website will generally be a stand-alone website and could potentially require a different web address. If you know a priori that you are recruiting mobile respondents, then a unique mobile link and QR code could be provided. If you are interested in field work or in-person data collection using mobile devices and you don’t want to use an app, then a separate mobile web survey optimized for mobile devices could be a cost effective and viable solution. Frost (2012) offers a great comparison or responsive and separate versions of presidential websites from the 2012 election. 197

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

198

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Data COLLECTION! 199

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

How to choose a survey design approach?

Responsive Web Design Pros All HTML scripting required for survey deployment across any device is contained in a single packet and is delivered to every device (OSFA) Survey content will be rendered seamlessly across a wide array of devices (even ones that aren’t invented yet or that you didn’t plan for) Single Survey URL serves all devices/sizes/types Survey content will automatically adjust to Horizontal or Vertical orientations Responsive Web Design Cons All programming code required to render surveys on any device is downloaded to the respondent’s device for processing (unnecessarily large data transmission size and potentially longer rendering times) Images are generally much larger in size than they need to be for mobile devices (again, increased data transferred to phone/tablet than required) Typically does not take full advantage of mobile device functionality Survey layout cannot be controlled as easily and in many cases pages will be longer and require more scrolling. 200 Source: http://bit.ly/QBI4Lv

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 How to choose a survey design approach?

Mobile First Responsive Web Design Pros Because programming and layout focus on mobile, images and therefore entire HTML mark-up will be smaller compared to RWD designs Mobile functionality can be built in to the survey coding from the ground up Total number of HTTP requests between the mobile devices and the survey host will be minimized in order to render the same content – thus reducing latency and loading times. Has been used successfully by Survey Researchers to collect data from both smartphone and computer respondents Bruijne and Wijnant (2013) http://bit.ly/1lxs0UX Mobile First Responsive Web Design Cons Requires re-engineering online survey website (computer version) from the beginning with mobile surveys as the basis ** Survey layout cannot be controlled as easily and in many cases pages will be longer and require more scrolling.

201 Source: http://bit.ly/1nl50MT

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

How to choose a survey design approach?

RESS Approach Pros Because content can be packaged for devices, http requests and overall sizes of markup tends to be smaller (decrease load time and data transferred) Can include components of progressive enhancement to build out features that are detected on respondents device Single Survey URL serves all devices/sizes/types Survey content will automatically adjust to Horizontal or Vertical orientations

RESS Approach Cons In order to send appropriate versions of survey scripts, a device detection data base/repository is required and would require updating To maximize information sent from the server and to minimize subsequent http requests for rendering survey cookies can be used May require informed consent to place cookies (EU Cookie Directive) Server side computing resources and specific programming is required

202

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 How to decide the approach used here?

203

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Resources for Assessing Size and Programming Features of your Survey Webpages

YSLOW – a downloadable/executable program from Yahoo to provide a grade for your website in terms of Web Performance Best practices Includes Image Assessment, Overall size assessment and more https://developer.yahoo.com/yslow/

PageSpeed – executable online program from Google to assess page performance and user experience criteria based on Web performance best practices Online version includes Image Assessment, overall size, speed and user experience criteria for both MOBILE and ONLINE with suggestions for improvements http://developers.google.com/speed/pagespeed/insights/

204

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 A Worldwide Wonderland for the World Wide Web (Mobile included!)

Responsive Web Design Example: www.disney.com

mini

205

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Let’s Look Under the “Hood” of the Disney Site.

An example of a bloated design includes background images (skin) along with many foreground images. www.disney.com This took about 10 seconds to load using my iphone 5 over the LTE network of Verizon

206 http://developer.yahoo.com/yslow/ JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 From Google PageSpeed: Disney.com

207

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

The other side of RWD…LAPTOP Pinch and Zoom!

Responsive web design can be great at adapting content to smaller viewports (or devices) but one of the drawbacks for surveys is the bringing back the pinch and zoom issue to lap/desktops as illustrated here – a laptop and iphone view of a question from a sample survey from CheckMarket. https://www.checkmarket.com/mobile-surveys/ iPhone Laptop View View

208

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Focus on a survey design that is technically sound

On mobile websites there is a smaller margin of error for: Image rendering, browser requests, survey page size, waiting times. 77% of cell internet users say they experience slow download speeds that prevent things from loading as quickly as they would like. Of those cell internet users, 46% face slow download speeds weekly or more frequently. (Pew, 2012) 97% of time on a single web page is spent on the front end – meaning the loading time of the page (Everts, 2011) Why do we need to focus on techniques that can optimize survey websites? Overall Respondent burden Lag/Latency Waiting time, Time on Tasks, data consumption/exchange, effort Break-off, partial completion, full completion Keep in mind that a web survey is a series of consecutive and linked web pages (each serving up survey tasks to complete).

209

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Practical Tips for Reducing Respondent Device Burden

There are several practical guidelines to think about for reducing overall respondent burden Image Resizing Deliver smaller images to devices with smaller resolutions Avoids having to transmit entire image to device and then resize at the device level Device Specific Scripting Only If displaying survey contents to a smartphone, don’t send deskop/tablet specifications, additions or modifications, for example Connectivity Detection Not only for cellular networks, but also for weak wi-fi networks (i.e. laptops accessing weak signal @ airports or conference hotels)

210

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 16:59 Section 8: Making it Work! Software Considerations

211

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Development Frameworks and Resources

JQuery Mobile: (http://jquerymobile.com/) mobile framework that allows you to design a single web site or app that will work on all popular smartphone, tablet, and desktop platforms. Pelletier’s E-book: Mobile App Manual: The Blueprint? Provides an introduction to developing apps using J-Query Mobile and the PhoneGap build environment. User friendly overview of the approach rather than emphasis on the computer code required. http://mobileappmanual.com/

Sencha Touch: (www.sencha.com) A comprehensive development environment for mobile apps that uses the HTML5 platform

Pure from Yahoo: http://purecss.io/ This collection of CSS modules enables the users to create responsively designed websites (for a myriad of devices) using an entire collection of CSS pages that have been designed to be as small as possible to avoid bloating and increased loading/unpacking times on mobile devices.

212

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Software Services Solutions for Mobile Survey Design/Development/Deployment WebFirst’s PhiCollect (Paid) www.webfirst.com/phicolect Open source data collection platform – flexible, scalable, cloud-based Web, Android, iOS and SMS options (Tablet and Phone) Location, multimedia, sms data collection Has been deployed for mHealth and education applications worldwide. Qualtrics (Paid) http://www.qualtrics.com/research-suite/mobile-surveys/ Provides feedback on survey questions and answers and formats with recommendations for better mobile optimization. Uses responsive web design with server side components allows customization of content delivery based on device type detection: ipad, android phone, palm, windows mobile, etc.

213

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

Example from a Basic Qualtrics Survey

214

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Software Services for Mobile Surveys, Cont.

Columbia University Modi Research Group’s formhub (Free) http://www.formhub.org/ Python/Django Open source, free development platform for mobile data collection Relies on ODK Collect app or mobile browser (modern browser that supports HTML caching is required) Surveys are designed and managed using Microsoft Excel XLSForms spreadsheets One tab is questions; the other is answer options Surveys are presented more like data completion forms (scrolling) and multiple completes can be managed from a single device (Tablet and Mobile Phone Assisted Interviewing) Formhub Demo Survey Available at http://bit.ly/1nvbXbj

215

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

The Actual Survey in FormXLS

216

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Software Services for Mobile Surveys, Cont.

CheckMarket Surveys (Paid) https://www.checkmarket.com/ Company headquartered in Belgium and offers responsive- like design for mobile surveys that has some app-like functionality. Their mobile solution has implemented several of the emerging best practices and provides some flexibility to the survey designer (controlling questions per page, size of windows for open ended questions) New data collection techniques for media are available as well as a ranking question type that is implemented smoothly in the mobile environment Example Demo Survey is available here: http://bit.ly/1K3hzqu

217

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

App-Based Survey Solutions

Confirmit’s SODA (Survey on Demand Application) App. (Paid) Probably the most widely known and used App-based survey solution among the survey research community. Version 3.0 was released in January, 2014 Continuous GPS tracking, Push Notifications, Multiple field editing enhancements http://bit.ly/1lllwfQ Conduit Mobile (Paid) Offers app development in a hosted environment. Can build your own app and can include data collection forms and formats using their web-based app-building platform http://mobile.conduit.com/

218

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 The Future of Survey Research is Coming: Fast!

The percentage of unintentional mobile respondents has reached arguably non-negligible levels and will likely increase as email and internet activity on mobile devices increases. There is still a lot to learn about usability and design considerations for surveys optimized for Smartphones. Even if you are using a survey platform that can handle mobile devices you will still have to make many decisions in terms of questionnaire design. Be mindful of variability across platforms- iPhone release may not be sufficient for all types of Smartphones associated members of your sampling frame. Here’s Hoping that All our Surveys Continue to Get Smarter…

219 JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 2 1 9

Final Comment!

I leave you with this final word from Sir Richard Branson to encourage all of you to continue to: invest in experimentation in the mobile survey arena and share your results

so that we can continue to develop and use best practices for these very cool (and smart) devices!

220

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 Thank you! [email protected]

314-695-1378

www.m-s-g.com

@trentbuskirk

221

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

References

AAPOR Cell Phone Task Force Report, 2010 available at: http://www.aapor.org/Cell_Phone_Task_Force_Report.htm , accessed on October 28, 2011 Attfield, Simon, Gabriella Kazai, Mounia Lalmas, and Benjamin Piwowarski. 2011. “Towards a Science of User Engagement (Position Paper).” Presented at the WSDM Workshop on User Modelling for Web Applications, Hong Kong, China. Bailey, Justin and Lorelle Vanno. 2011. “Market Research Association Current Mobile Research Trends.” Webinar for the Florida Chapter of Marketing Research Association. Bosnjak, Michael, Gottfried Metzger, and Lorenz Gräf. 2010. “Understanding the Willingness to Participate in Mobile Surveys: Exploring the Role of Utilitarian, Affective, Hedonic, Social, Self- Expressive, and Trust-Related Factors,” Social Science Computer Review 28: 350-370. Brick, J. M., P. D. Brick, S. Dipko, S. Presser, C. Tucker, and Y. Yuan. 2007. Cell phone survey feasibility in the U.S.: Sampling and calling cell numbers versus landline numbers. Public Opinion Quarterly 71 (1): 23–39. Buskirk, T.D., Callegaro, M. and Rao, K. (2010) ‘N the Network'? Using Internet Resources for Predicting Cell Phone Number Status. Social Science Computer Review: Special Issue on Mobile Surveys, Vol. 28, No. 3, 271-286. Buskirk, T.D. and C. Andrus. (2012a). Smart surveys for smartphone: exploring various approaches for conducting online mobile surveys via smartphones. Survey Practice. Available at: http://surveypractice.wordpress.com/2012/02/21/smart-surveys-for-smart-phones/ . Buskirk, T. D. and Callegaro, M. “Cell Phone Sampling,” in Encyclopedia of Survey Research Methods, Lavrakas, P.J. (editor), Sage, Los Angeles, © 2008, pp. 84-89. Buskirk, T.D., Gaynor, M., Andrus, C. and Gorrell, C. (2011) “An App a Day Could Keep The Doctor Away: Comparing Mode Effects for a iPhone Survey related to Health App Use.” American Association of Public Opinion Research, Phoenix, AZ.

222

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 References (con’t)

Buskirk, T.D. and Andrus, C. (2014) “Making Mobile Surveys Smarter: Results from a randomized experiment comparing online smartphone and desktop/laptop computers,” Field Methods, forthcoming. See http://fmx.sagepub.com/content/early/recent Callegaro, Mario. 2010. “Do You Know Which Device Your Respondent Has Used to Take Your Online Survey?” Survey Practice, December: www.surveypractice.org. Callegaro, M. and Macer, T. (2011) “Designing Surveys for Mobile Devices: Pocket-sized Surveys that Yield Powerful Results,“ Shortcourse Presented at the 66th Annual Meeting of the Americaon Association of Public Opinion Research, Phoenix, AZ. Cazes, J., Townsend, L., Rios, H., & Ehler-James, J. (2010). The mobile survey landscape - Today and Tomorrow. Impacts of mobile devices usage on current and future market research practices. Retrieved from http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf comScore (2012) 2012 Mobile Future in Focus. Retrieved from http://www.comscore.com/Press_Events/Presentations_Whitepapers/2012/2012_Mobile_Future_in_Fo cus Couper, M. P. (2010). Visual design in online surveys: Learnings for the mobile world. Presented at the Mobile Research Conference 2010, London. Retrieved from http://www.mobileresearchconference.com/uploads/files/MRC2010_Couper_Keynote.pdf Cunningham, JA et al. (2013) “Use of mobile devices to answer online surveys: implications for research,” BMC Research Notes. Vol 6:258 available online: http://www.biomedcentral.com/1756- 0500/6/258 Duan, Shu, Justin Bailey, and Michael Link. 2011. “Data Collection Method Innovation: A Crowdsourcing Application to Collect Factual Information in Africa”. Paper presented at the Midwest Association for Public Opinion Research Conference, Chicago, IL.

223

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

References (con’t)

Fuchs, Mark, 2008. “Mobile Web Surveys: A Preliminary Discussion of Methodological Implications”. In F. Conrad & M. Schober (Eds.), Envisioning the Survey Interview of the Future (pp. 77-94). Hoboken, NJ: Wiley. Graham, P, and C. Cobb (2013). “Comparison of Instantaneous Mobile Time Use Data Collection Methods to Traditional Time Diary Methods.” Paper presented at the Annual Conference of the American Association for Public Opinion Research, Boston, MA. Gregoski, Mathew, Martina Mueller, Alexey Vertegel, et al. (2012). “Development and Validation of a Smartphone Heart Rate Acquisition Application for Health Promotion and Wellness Telehealth Applications.” International Journal of Telemedicine and Applications, doi:10.1155/2012/696324. http://www.hindawi.com/journals/ijta/2012/696324/ Kennedy, C. (2007), Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys, Public Opinion Quarterly, Vol 71 / Num 5: 750-771. Koch, P-P. (2010) Smartphone Browser Landscape, A List Apart Article No. 320, available at http://www.alistapart.com/articles/smartphone-browser-landscape/ (accessed on 10/30/2011) Kumar, A. (2012) “Pilot study tests wearable sensors and smartphone-based data collection for diabetes,” iMedicalApps, accessed at http://www.imedicalapps.com/2012/11/pilot-study-wearable-sensors-smartphone- diabetes/ Kuntsche, Emmanuel and Benjamin Robert. 2009. “Short Message Service (SMS) Technology in Alcohol Research – A Feasability Study.” Alcohol &Alcoholism 44: 423-428. Lai, Jennie W., Lorelle Vanno, Michael Link, Jennie Pearson, Hala Makowska, Karen Benezra, Mark Green. 2010. “Life360: Usability of Mobile Devices for Time Use Surveys” Survey Practice, February: www.surveypractice.org

224 JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15 2 2 4 References (con’t)

Lai, Jennie, Michael Link, and Lorelle Vanno. 2011. “Like? The Effect of Social Networking in the Digital Age of Mobile Application Research.” Paper presented at the Midwest Association for Public Opinion Research Conference, Chicago, IL. Maxl, E., Döring, N., & Wallisch, A. (Eds.). (2009). Mobile market research. Köln: Herbert Von Halem Verlag. Nitsche, P, et al. (2012) “A strategy on how to utilize smartphones for automatically reconstructing trips in travel surveys,” Procedia-Social and Behavioral Sciences, Vol. 48, pp. 1033-1046. Olson, K, and J. Wagner (2013). “A Field Experiment Using GPS devices to measure Interviewer Travel Behavior.” Paper presented at the annual conference of the American Association for Public Opinion Research, Boston (May). Paudela, D., M. Ahmedb, A. Pradhanc, R. Lal Dangold (2013) “Successful use of tablet personal computers and wireless technologies for the 2011 Nepal Demographic and Health Survey,” Glob Health Sci Pract August 12, 2013 vol. 1 no. 2 p. 277-284; http://www.ghspjournal.org/content/1/2/277.full.pdf+html Perkins,J, R. Granger, and E. Saleska. 2009. “Data Security Considerations When Using Social Networking Wesites for Locating and Contacting Sample Members.” Presented at International Field Directors & Technology Conference. Peytchev, Andy and Craig A. Hill. 2010. “Experiments in Mobile Web Survey Design: Similarities to Other Modes and Unique Considerations,” Social Science Computer Review 28: 319-335. Pew Research Center. 2011. “How People Use Tablets and What It Means for the Future of News.” Report. Available at: www.pewresearch.org/pubs/2119/tablet-news Scagnelli, Jeff, Justin Bailey, Michael Link, Hala Moakowska, and Karen Benezra (2012). “On the Run: In the Moment Smartphone Data Collection.” Paper presented at the annual conference of the American Association for Public Opinion Research (Orlando, FL).

225

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15

References (con’t)

Schober, Michael, Frederick Conrad, Christopher Antoun, Alison Bowers, Andrew Hupp, Huiying Yan (2013). “Conversational Interaction and Survey Data Quality in SMS Text Interviews.” Paper presented at the Annual Conference of the American Association for Public Opinion Research, Boston, MA. Smith, Aaron. 2011. “Smartphone Adoption and Usage.” Report available at: www.pewinternet.org/reports/2011/Samartphones.aspx Steeh, C., Buskirk, T. D., & Callegaro, M. (2007). Using text messages in U.S. mobile phone surveys. Field Methods, 19, 59–75. van Heerden, A. Shane A. Norris, Stephen M. Tollman, and Linda M. Richter, “Collecting Health Research Data: Comparing Mobile Phone-Assisted Personal Interviewing to Paper-and-pen Data Collection,” in press, Field Methods, http://fmx.sagepub.com/content/early/recent Vanno, Lorelle, Jennie Lai, and Michael Link. 2011. “Want to Collect Behavioral Data in an Engaging Way? There’s an App for That!” Paper presented at the Southern Association for Public Opinion Research Conference, Raleigh, NC. Zahariev, M., Ferneyhough, C., Ryan, C., & Bishop, S. (2009). Best practices in mobile research. Presented at the ESOMAR Online Research, Chicago, IL: ESOMAR. Zickuhr, K. (2011). Generations and their gadgets. Pew Internet & the American Life Project. Retrieved from http://pewinternet.org/Reports/2011/Generations-and-gadgets.aspx Zuwallack, R. (2009) Piloting Data Collection via Cell Phones: Results, Experiences, and Lessons Learned, Field Methods, Vol. 21, No. 4, pp. 388–406 Zichermann and Cunningham (2011) Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps available at: http://www.amazon.com/Gamification-Design-Implementing-Mechanics- Mobile/dp/1449397670 Andbergen, P. A. (2012). Comparison of WiFi positioning on two mobile devices. Journal Of Location Based Services, 6(1), 35-50. doi:10.1080/17489725.2011.630038

226

JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies Trent D. Buskirk, Ph.D. 6/10/15