Proceedings of the Sawtooth Software Conference
Total Page：16
File Type：pdf, Size：1020Kb
PROCEEDINGS OF THE SAWTOOTH SOFTWARE CONFERENCE May 2020 Copyright 2020 All rights reserved. No part of this volume may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from Sawtooth Software, Inc. FOREWORD These proceedings are a written report of the twenty-first Sawtooth Software Conference, held in San Diego, California, September 25-27, 2019. Two-hundred attendees participated. The focus of the Sawtooth Software Conference continues to be quantitative methods in marketing research. The authors were charged with delivering presentations of value to both the most sophisticated and least sophisticated attendees. Topics included conjoint analysis, surveying on mobile platforms, MaxDiff, market segmentation and classification, experimental design, and the perils of establishing causality in observational data. The papers and discussant comments are in the words of the authors and very little copyediting was performed. At the end of each of the papers are photographs of the authors and co-authors. We appreciate their cooperation for these photos! It lends a personal touch and makes it easier for readers to recognize them at the next conference. We are grateful to these authors for continuing to make this conference such a valuable event. We feel that the Sawtooth Software conference fulfills a multi-part mission: a) It advances our collective knowledge and skills, b) Independent authors regularly challenge the existing assumptions, research methods, and our software, c) It provides an opportunity for the group to renew friendships and network. We are also especially grateful to the efforts of our steering committee who for many years now have helped this conference be such a success: Christopher Chapman, Keith Chrzan, Eleanor Feit, Joel Huber, and David Lyon. Sawtooth Software May, 2020 CONTENTS A COMPARISON OF PC AND MOBILE INTERVIEWING MODALITIES ............................................... 1 Deb Ploskonka and Karlan Witt, Cambia Information Group A RESEARCHER’S GUIDE TO STUDYING LARGE ATTRIBUTE SETS IN CHOICE-BASED CONJOINT ........ 17 Megan Peitz, Numerious; Mike Serpetti and Dan Yardley, Gongos WHAT SOUTH AFRICAN MEDICAL STUDENTS VALUE IN A RURAL INTERNSHIP: A DISCRETE CHOICE EXPERIMENT ........................................................................................ 47 Maria Jose, University of Cape Town LEADERSHIP QUALITIES: PREFERENCES OF THE MILLENNIAL GENERATION ..................................... 59 Ronald Mellado Miller, UVU; Christina A. Hubner, Sawtooth Software; Cray Daniel Rawlings, UVU; and Maureen Andrade, UVU VIRTUAL REALITY MEETS TRADITIONAL RESEARCH: OR THE REALITY BEHIND VIRTUAL REALITY ENHANCED INTERVIEWS .............................................. 71 Alexandra Chirilov, GfK TOO MUCH INFORMATION?: THE CURIOUS CASE OF AUGMENTED MAXDIFF .............................. 87 Jackie Guthart, Curtis Frazier, and Raman Saini, Radius Global Market Research CAN WE USE RLH TO ASSESS RESPONDENT QUALITY? .......................................................... 105 Marco Hoogerbrugge and Menno de Jong, SKIM BANDIT MAXDIFF: THE EFFECTS OF DESIGN PARAMETERS ON HIT RATES IN DIVERSE DATASETS ..... 113 Nico Peruzzi, elucidate TREES, FORESTS, AND SITUATIONAL CHOICE EXPERIMENTS ...................................................... 121 Keith Chrzan, Sawtooth Software and Joseph Retzer, ACT-Solutions EXAMINING THE NO-CHOICE OPTION IN CONJOINT ANALYSIS .............................................. 131 Maggie Chwalek, Roger A. Bailey, and Greg M. Allenby, Ohio State University MODELLING STOCKPILABLE PRODUCT PURCHASE DECISIONS USING VOLUMETRIC CHOICE EXPERIMENTS ................................................................................... 147 Richard T Carson, University of California, San Diego; Towhidul Islam, University of Guleph, Canada; and Jordan J. Louviere, University of South Australia CONJOINT MEETS AI ....................................................................................................... 165 Peter Kurz and Stefan Binner, bms marketing research + strategy i PREDICTING THE (UNOBSERVED) PREDICTABLE: THE USE OF DEEP LEARNING IN WAVE STUDIES FOR MARKET RESEARCH .......................................................................................................... 185 Tom Gardner and Michelle McNamara, Adelphi Research CAN WE REDUCE THE NUMBER OF TASKS AND STILL GET GOOD QUALITY RESULTS? ................... 197 Chris Moore and Ioannia Tsalamanis, Ipsos MORI UK COMBINING CHOICE-BASED CONJOINT AND DYNAMIC CHOICE MODELS FOR MORE ACCURATE FORECASTING ....................................................................................... 215 Faina Shmulyian, SKIM USA DATA FUSION: A FLEXIBLE HB TEMPLATE FOR MODELING STRUCTURES ACROSS MULTIPLE DATA SETS ........................................................................................................ 225 Kevin Lattery, SKIM Group SEGMENTING CHOICE AND NON-CHOICE DATA SIMULTANEOUSLY: PART DEUX ....................... 247 Thomas C. Eagle, Eagle Analytics of California, Inc.; and Jay Magidson, Statistical Innovations, Inc. COMMENTS ON “SEGMENTING CHOICE AND NON-CHOICE DATA SIMULTANEOUSLY: PART DEUX” ................................................................................................................... 281 David W. Lyon, Aurora Market Modeling, LLC UNDERSTANDING CONSUMER PREFERENCES: A COMPARISON OF SURVEY- AND PURCHASE-BASED APPROACHES ................................................................................ 289 James Pitcher, Bradley Taylor, and Dan Kelly, GfK MAXIMIZING THE IMPACT OF OOH (OUTDOOR) ADVERTISEMENT USING DISCRETE CHOICE MODELING AND TEXT ANALYTICS ...................................................................................... 307 Rajat Goel and Rachin Gupta, StatWorld Research Solutions USING ADAPTIVE CHOICE-BASED CONJOINT ANALYSIS TO UNRAVEL THE DETERMINANTS OF VOTER CHOICES ............................................................................................................. 319 David Bakken, Foreseeable Futures Group; Gretchen Helmke, University of Rochester; and Mitch Sanders, Meliora Research THE CHALLENGE OF IDENTIFYING CAUSALITY IN OBSERVATIONAL DATA ................................... 339 Ray Poynter, The Future Place/Nottingham Trent University ii SUMMARY OF FINDINGS The twenty-first Sawtooth Software Conference was held in San Diego, California, September 25-27, 2019. The summaries below capture some of the main points of the presentations and provide a quick overview of the articles available within the 2019 Sawtooth Software Conference Proceedings. A Comparison of PC and Mobile Interviewing Modalities (Deb Ploskonka, Karlan Witt, Cambia Information Group): Deb and Karlan noted that the increase in mobile survey completions has been accompanied by higher rates of breakoff rates for mobile survey takers. To help reduce breakoff rates among mobile survey takers, the authors tested two question types among US and Japanese respondents where they see concerning breakoff rates: unaided brand awareness and grid-style brand rating questions. For unaided brand awareness, they tested a version of the survey that showed 15 blank entry boxes versus five blank entry boxes that expanded with extra entry boxes as respondents filled in the blanks. There was very low abandonment among the US-based respondents, with no difference between the versions. Japanese respondents had higher abandonment for the 15 blank entry box approach. For the grid portion of their experiment, Deb and Karlan tested three approaches: standard grid, scroll approach with items broken into separate ratings questions, and a “freeze” version which was the same as scrolling but left the header portion of the question frozen at the top of the screen. The results were inconclusive regarding which method resulted in the lowest dropouts and most consistency between laptop and mobile data. For future research, Deb and Karlan noted that there are other styles of mobile grids (accordion and progressive grids) that could be tested. *A Researcher's Guide to Studying Large Attribute Sets in Choice-Based Conjoint (Megan Peitz, Numerious, Mike Serpetti, Dan Yardley, Gongos): Choice-based methods have become dominant in our industry, yet no clear answer has emerged regarding which of the many choice-based approaches a researcher should use as the number of attributes increases beyond about six. Megan and co-authors designed an experiment involving choice of smartphones by real respondents to compare Partial Profile CBC, Adaptive Choice-Based conjoint (both partial-profile and full-profile variants), and Full-Profile CBC for attribute lists involving 10, 15, and 20 attributes. They found that for studies involving 10 attributes, the methods were on parity with each other in all respects (holdout predictability, length of survey, data quality, and respondent perception). As the number of attributes increased to 15+, the results suggested that partial-profile ACBC has a number of advantages over the other techniques. The authors also pointed out the large difference in the predicted None rate between ACBC and CBC. *Best Presentation based on audience voting. What Do South African Medical Students Value in a Rural Internship (Maria Jose, University of Cape Town, South Africa): To address inequality in access to healthcare among rural populations in South Africa, effective health worker