IDEAS & TOOLS FOR QUALITATIVE RESEARCH
SPRING 2007 VOLUME 5 • NUMBER 3
• TABLE OF CONTENTS •
SPRING 2007
THE OFFICIAL PUBLICATION OF THE QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION
14 SCHOOLS OF THOUGHT 24 Quit Apologizing: Rethinking the Limitations of “Limitations” • George Balch and Christopher Herbert state the case that qualitative research is not inferior to quantitative research and that, therefore, nothing requires QRCs to be apologetic.
24 QUALITATIVE TOOLBOX What I Learned from Hanging Chads and Exuberant X’s: Testing a Voting System for Usability • QRC Dona Vitale describes how a new application of usability testing is making Chicago- 8 FROM THE EDITOR-IN-CHIEF area elections more effective Hidden Potential • Lana Limpert examines the prospects for finding fresh thoughts, uncommon and more accurate. views and new perspectives, especially in this issue of QRCA VIEWS. 34 QUALITATIVE TOOLBOX Best Practices for Real-Time Online Focus 10 FROM THE PRESIDENT Groups • Summarizing eight years of observations The Silver Lining • Joel Reish discusses the from online qualitative research, Jonathan Hilland emerging and unexpected benefits of recent criticism offers his recommended practices for conducting of qualitative research and focus groups. real-time (or “chat”) focus groups.
Editor-in-Chief: Lana Limpert, [email protected] • Managing Editors: Monica Zinchiak, [email protected], and Susan Sweet, [email protected] • Design/Art Direction & Publishing: Leading Edge Communications, LLC (615) 790.3718 [email protected]
FEATURE EDITORS Book Reviews: Kay Corry Aubrey, [email protected] • Business Matters: Abby Leafe, [email protected] • International Research: Angela Paura, [email protected] • Industry Calendar: open • Qualitative Toolbox: Sharon Livingston, Ph.D., [email protected] • Schools of Thought: open • Targeted Marketing: Judy Langer, [email protected] • Tech Talk: David Van Nuys, [email protected] • Travel & Leisure: Mary Beth Solomon, [email protected]
4 QRCA VIEWS SPRING 2007 www.qrca.org
• TABLE OF CONTENTS •
CONTINUED
SPRING 2007
THE OFFICIAL PUBLICATION OF THE QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION
54 TARGETED MARKETING 76 The Vitality Boom: Marketing to Active, Boomer Women • No longer the “youth generation,” baby-boomer women are helping redefine the meaning of fitness, as this article on the ultimate “power consumer” illustrates.
60 TECH TALK Can Software Help? A Look at Software Tools for Qualitative Research Analysis • Jean Nordgren outlines a brief overview of several software tools that can help with qualitative research analysis.
66 BUSINESS MATTERS To Hire or Not to Hire? Considerations for Expanding Your Business • Husband-and-wife business owners David Kalmar and Abby Leafe provide a framework for evaluating the staffing solution that will best meet your business needs.
76 TRAVEL & LEISURE Free Money (Well, Almost) • A wide variety of reward programs have popped up over the past several years, and most of them are for activities, products and services you are already buying.
82 BOOK REVIEW 42 QUALITATIVE TOOLBOX Valuable Reading… Three Titles that Can Help You Beyond the Data Dump: Provide Qualitative Expand Your Capabilities • Here are two excellent books Deliverables that Spur Implementation • to get you started on usability testing, plus a brand new book Melinda Kizer provides the critical success factors that is an excellent primer on customer-insight research, and considerations that QRCs should address to outlining many techniques. ensure that their qualitative reports are used to make strategic business decisions. 89 EDITORIAL GUIDELINES
48 QUALITATIVE TOOLBOX A Picture Is Worth a Thousand Words… 90 INDEX OF ADVERTISERS Sometimes • Dr. Bruce Eckman offers a qualitative/ quantitative improvement on photo-sort methodologies.
visit QRCA online at
6 QRCA VIEWS SPRING 2007 www.qrca.org www.qrca.org
• FROM THE EDITOR-IN-CHIEF •
Hidden Potential “The most erroneous stories are those we think we know best — and therefore never scrutinize or question.” Stephen Jay Gould (1941–2002)
potential for obtaining a new perspective is everywhere. It is possible to think of qualitative research as a means to unlock the potential of Lana Limpert The perspective that resides in the lives of our respondents. Without a qualitative researcher’s curious mind and probing questions, many clients would not TECHNICLARITY [email protected] have the benefit of understanding their customers’ opinions, attitudes, perspectives and preferences. Business decisions would be constrained by having to imagine in isolation which message is most persuasive, which product concept fills a critical unmet need or which brand image most touches the customer’s emotions. Qualitative research findings enrich the client’s perspective of their customer’s worlds, which direct cogent and informed choices. Without a qualitative Another way to stumble upon new perspectives is to reconsider information already in hand. We have all had the experience of turning data this way and that, researcher’s curious only to uncover yet another layer of meaning. Examining raw material from several perspectives allows us to mine buried gems from the stacks of transcripts and heaps mind and probing of tapes that litter our desks. Arriving in this new place that yields a richer questions, many perspective can help us to better clarify our conclusions and to generate creative recommendations, as well as to unearth questions that need further exploration. clients would not The potential for new perspectives lies within the covers of this issue of QRCA VIEWS as well. You may find yourself considering surprising new twists on well- have the benefit established, but tired outlooks. In “What I Learned From Hanging Chads and of understanding Exuberant X’s,” Dona Vitale shares a spanking new application for usability testing. George Balch and Christopher Herbert, in their article “Quit Apologizing,” offer their customers’ specifics for rethinking the limitations of the “limitations” of qualitative research. opinions, attitudes, Jonathan Hilland, in “Best Practices for Real-Time Online Groups,” explores the strengths and weaknesses of chat groups, which are different today than they were perspectives and when the methodology first hit the streets. In sharing their perspectives, all of the QRCA VIEWS’ authors provide potential for fresh thoughts and uncommon views. preferences. If you find yourself considering some aspect of qualitative research anew as result of this reading, please let us know. Our editorial staff is always interested in gaining new perspectives about and from our readers!
Acknowledgement and apology Apologies to Doris Walsh, editorial director at Paramount Market Publishing. In the Fall 2006 issue of QRCA VIEWS, we failed to acknowledge that the article “Boomer Grandparents: A Hidden Market — Until Now,” by Christine Crosby and Richard Anthony Sr., was excerpted from a book called After Sixty (co-edited by Leslie M. Harris and Michelle Edelman) published by Paramount Market Publishing in Ithaca, NY. Paramount has been very helpful to QRCA VIEWS on this and many issues in the past, and we regret our oversight. To purchase this book and others on market segments and qualitative research, please visit their website at www.paramountbooks.com.
8 QRCA VIEWS SPRING 2007 www.qrca.org QRCA 2006–2007 Officers Qualitative Research Consultants Association and Board of Directors (QRCA) serves its members in the industry through education, promotion and representation. The Joel Reish statements and opinions expressed herein are those of PRESIDENT the individual authors and do not necessarily represent the views of the association, its staff, or its board of Sharon Livingston, Ph.D. directors, QRCA Views, or its editors. Likewise, the VICE PRESIDENT appearance of advertisers, or QRCA members, does Abby Leafe not constitute an endorsement of the products or TREASURER services featured in this, past or subsequent issues of this quarterly publication. Copyright ©2007 by the Peter Lovett Qualitative Research Consultants Association. QRCA SECRETARY Views is published quarterly. Subscriptions are Martha Guidry complimentary to members of QRCA. DIRECTOR POSTMASTER: Send change of address notification 1000 Westgate Drive, Suite 252 to QRCA, 1000 Westgate Drive, Suite 252, St. Paul, Tonya Harper MN 55114 USA. Postage guaranteed. Presort St. Paul, MN 55114 USA DIRECTOR standard postage is paid at Franklin, TN. Printed in Christine Kann the U.S.A. Reprints and Submissions: QRCA Views Tel (toll-free in N. America): DIRECTOR allows reprinting of material published here, upon 888-ORG-QRCA request. Permission requests should be directed to Ilka Kuhagen QRCA. We are not responsible for unsolicited (888-674-7722) DIRECTOR freelance manuscripts and photographs. Contact the Tel (International inbound calls): Nancy Ulrich managing editor for contribution information. 731-584-8080 DIRECTOR Advertising: For display and classified advertising rates and insertions, please contact Leading Edge Shannon Pfarr Thompson Phone: 651-290-7491 Communications, LLC, P.O. Box 680142, Franklin, EXECUTIVE DIRECTOR TN 37068-0142, (615) 790-3718, Fax (615) 794- Fax: 651-290-2266 Darrin Hubbard 4524. Deadlines are the first of the month prior to the ASSISTANT EXECUTIVE DIRECTOR following month’s publication. (Example: August 1 Email: [email protected] for the September issue.) Subscriptions are free to David Ewald www.qrca.org members and are available to buyers upon request. SENIOR ADVISOR
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 9 • FROM THE PRESIDENT •
The Silver Lining
the last few years, it has become quite fashionable to criticize qualitative research. We certainly have seen a steady flow of articles and books that In “bash” qualitative research generally or, in many cases, focus groups specifically. At first, such articles appeared in specialized research industry publications. Then they began to show up in the marketing industry press, then in business and management publications, and now increasingly in major mainstream, general-population newspapers and magazines. Focus groups are even becoming fodder for movies, television shows and advertisements. Invariably, the critics are far from objective — it turns out they have a jaded perspective or even a vested interest in putting down “traditional” qualitative research. Some of these critics are seeking to create a sensation in order to sell more Joel Reish books. Others offer an “alternative” research methodology that, upon closer scrutiny, NEXT LEVEL RESEARCH looks suspiciously like the very focus group methodology that they are putting down. Atlanta, GA Many in the research industry are concerned by these criticisms, as we should be. [email protected] But I believe that there is also something good and valuable to be found underlying these barbs: the silver lining shining through. To be fair, we should recognize that there is some truth to the common criticisms — but only when looking at poorly conducted, badly run qualitative research. I agree to criticize unprofessional qualitative research that only explores rational thought processes without probing deeper, that allows dominators to take over and that takes The silver lining is no precautions against cheaters and repeaters. However, these criticisms are simply that the concept of not valid for the vast majority of qualitative research being conducted today by the dedicated professionals in our industry. And therein lies the problem: these criticisms focus groups is now have been expanded and generalized to all qualitative research. Just why is qualitative research such a popular target of criticism? First off, for widely understood someone who has never done it, moderating focus groups looks deceptively easy. by the public — They think, “Hey, anyone can talk to a group of people for a couple of hours, reading off a list of questions. What’s the skill in that?” As we all know, however, focus groups have moderating focus groups professionally takes tremendous skill and discipline. True “broken through” professionals always make it look easy. Another reason qualitative research is such a popular target of criticism is that, as a cultural in general, people do not like to think that they can be “figured out.” Critics of qualitative research can score a cheap shot by playing into this fear when they phenomenon. The portray focus groups as a tool for misrepresenting or misleading people. By current criticism of portraying qualitative research as some sort of instrument of manipulation, the critics can strike a chord with readers. Yet, properly conducted qualitative research is best qualitative research for understanding what people need and want so that companies can serve them better — it is part of a process by which products and services are pulled to market only works because by consumers, not pushed onto them. the average person While we should be concerned about this unwarranted criticism, we should also look deeper at what this criticism signals. The silver lining in all of this is that the now knows concept of focus groups is now widely understood by the public — focus groups have “broken through” as a cultural phenomenon. The current criticism of qualitative generally what a research only works because the average person now knows generally what a focus focus group is. group is. So, we have seen focus groups go from an obscure, little-understood, specialized business activity to a common, widely known part of the general consumer landscape. The popular media would never consider bashing focus groups unless they were commonly understood. Qualitative research has become an easy target, not because it is deserving of criticism, but because it is one of the few “insider” business tools to reach a mass awareness and understanding, a corporate world secret that everyone knows about. I see this as a very positive development for the industry as a whole.
(continued on page 12)
10 QRCA VIEWS SPRING 2007 www.qrca.org
The Silver Lining CONTINUED
I believe the increase in awareness and knowledge of information for clients to act upon. We embrace these qualitative research in general, and of focus groups in new ideas as additional options that, on a case-by-case particular, could bode well for our industry. What an basis, might represent the best balance of productive opportunity we have now that the term “focus group” research and economical efficiency. And in many cases, has joined the popular lexicon! As more people become that best balance continues to fall to “traditional” familiar with the idea of qualitative research, they may focus groups and depth interviews. be more likely to participate, leading to improvements Any endeavor has its potential for abuse and inept in response rates and recruiting productivity. This increased application. QRCA has always recognized the potential familiarity should also help gain even greater acceptance pitfalls of poorly conducted qualitative research and among clients, not just in marketing departments, but has worked diligently to enhance the professionalism also in the boardroom and in the cubicles. And that is of our members. QRCA members are dedicated to already happening — in spite of the criticisms, qualitative advancing the qualitative research profession, and research is growing. in so doing, we are becoming more than just qual- Nonetheless, we know that qualitative research is itative research practitioners to our clients — we are not above reproach and that we should listen closely becoming true, trusted advisors and consultants. One to the critics; they just might have something valuable of the ways we provide value to clients is to conduct they are trying to tell us. For example, we are well qualitative research professionally, according to codes aware that there is definitely a trend by both research of ethics and individual dedication. We are ready to tell consultants and clients to look for ways to conduct our clients whether “traditional” methods or a new research that gets closer to the respondent’s experience. approach will be best for their needs. And we will There also is a trend to utilize technology more in the listen to, but not be distracted by, the critics because, qualitative research process. We welcome these trends, deep down, we understand that all the hoopla means as well as any other developments that lead to better we have “made it.”
A RESEARCH FACILITY
unlike any other... Nestled in the Midwestern heartland of Northeast Ohio, DecisionPoint Marketing & Research offers the marketing research community a facility that stands a cut above the rest. DecisionPoint offers: • Spacious floor plan which features two focus group rooms served by a single, integrated client viewing room. • Recruiting service with a participant database over 35,000 names • Technologically advanced marketing research facility with a full range of technological capabilities • Professionalism From our printed name cards to our trained, skilled staff, we make your experience comfortable, professional and successful.
3634 W Market Street, Suite 104 Fairlawn, Ohio 44333 Toll Free: 1-800-369-5394 email: [email protected] www.decisionpointmarketing.com Formerly Data for Decisions in Marketing, Inc.
12 QRCA VIEWS SPRING 2007 www.qrca.org
• SCHOOLS OF THOUGHT • QUIT APOLOGIZING: Rethinking the Limitations of “Limitations”
B Y G EORGE I. BALCH, PH .D. Balch Associates • Oak Park, IL • [email protected]
A ND C HRISTOPHER J. HERBERT, MBA The Insight Group Inc./Focused Thinking® • Tempe, AZ • [email protected]
have probably heard them, if not made them yourself — statements suggesting that qualitative research is somehow inferior to You quantitative research. We have collected “disclaimer statements” in qualitative research reports from QRCA colleagues in the U.S. Many show a distinctively negative tone, both in titles (such as, “A note of caution,” “Caveat on interpretation of One thing we find results” or “Limitations”) and in how they describe qualitative research (as helpful is that ancient in, “not projectable,” “not valid in the statistical sense,” “not scientific,” dictum from the Oracle at Delphi — “Know thyself” — which we’ve modified to “Know thyself, know thy clients, and know thy work.”
14 QRCA VIEWS SPRING 2007 www.qrca.org “directional, not definitive,” “small sample size” thyself, know thy clients, and know thy work.” or “limited sample”). Clearly, these statements As consultants, we should have a very good warn readers and even apologize that the grasp of our own strengths and limitations that research is not… QUANTITATIVE. might have an impact on what we can do for a But we have never found a quantitative client, in addition to how well. The competencies research statement warning that the research also require that we know our methods and that is not QUALITATIVE! Both researchers and we know when qualitative and quantitative research buyers seem to consider qualitative work best. research inferior. How warranted is this view? We must think about our clients: Who will see research results? How research savvy are they? Professionalism, Warnings and Apologies What might they misinterpret or even misuse? Although some researchers avoid speaking of And we must know our work well enough to “limits,” either because they do not want to consider how the methods we used relate to the appear to belittle the research or because they project, subject matter and respondents at hand. think their clients have the knowledge to apply study findings properly, we find apologetic Communicating without Apologizing language all too common. Some researchers Think of a common warning or limitations label “apologize” because it is seen to be common — for instance, the kind seen on most any over- practice. Some believe clients (or others who the-counter medication. This information is most may see the report) need to be made specifically helpful when it answers questions the consumer aware of how to use findings appropriately. does or should have: We have even heard that clients require such • Is this drug for my symptoms? statements. We must ask, however, do they • Does it have any contra-indications require specific apologetic language or just relevant to me? a warning against misapplication? • How and when should I take it The QRCA Professional Competencies of (or not take it)? Qualitative Researchers considers it an example • What else should I consider? of “Professional Practices” if a QRC “understands No apologies appear on such labels, no abstract and acknowledges the strengths and limits of her “limits,” just facts to help the consumer with his or his knowledge and capabilities, as well as the or her condition. strengths and limitations of qualitative research…” We should do at least that much for our Further, the “Expert” level of that competency is clients, and we can tailor the information to reserved for the QRC who “knows when to use the specific client and project (unlike the man or not use particular methods; what and whom reading a label telling him he should not take to tell about projects.” the medication if he’s pregnant!). Clients are So, as research professionals, we must know likely to wonder: and acknowledge the limitations both of • Why is this study method suited for my our own knowledge and capabilities and of information need? qualitative research itself. However, nothing • Does it have contra-indications relevant requires us to be apologetic! The “Research” to my project? competency of the QRCA professional • How and when should I use it (or not)? competencies requires professionals to • What additional research, if any, should I understand and apply “the social science consider for the information I need? research process, including the differences So, as you write or read reports and make and appropriate conditions for application or hear presentations, consider the value of of qualitative and quantitative research posing or answering likely questions about [emphasis added]…” In other words, we how the chosen method is relevant to a should do qualitative research where it is particular study, as well as relevant potential appropriate and not do it when it is not. misapplication of the results. A professional Thus, we believe it is responsible to consider must have that competency. appropriate applications and possible misapplic- ations. Also, it best serves the client to communicate Apologize for What? about applications and misapplications in ways The “apologies” we have seen talk about that inform the client’s decisions at hand, qualitative as inferior in “projectability,” without belittling the research. “significance” and the like. It’s as if statistical One thing we find helpful is that ancient inference was the only kind of inference or, dictum from the Oracle at Delphi — “Know at least, the best kind. Yet, it is a remarkably thyself” — which we’ve modified to “Know limited kind of inference.
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 15 Quit Apologizing CONTINUED
The Significance of Statistical Significance use of statistical inference in marketing and The logic of statistical inference and the “signif- public opinion research. Statistical inference in icance” level that it produces are based on a typical surveys deals only with “sampling error” “thought experiment”: If we repeated exactly — the probability that a sample with a result the same procedure (e.g., flip a coin) an infinite like the one obtained (or even more extreme) number of times, we would get any particular comes about by chance alone and, hence, is an numerical result a certain percentage of the time “error” of sorts. Such statistical inference says or less. nothing about what other sources of error might Of course, it is understood that we cannot do account for the numbers found! exactly the same thing again and again forever. In thinking about potential sources of error, But, if we could do that, some important math- consider the seemingly simple issues in the ematics (that you can find in any basic statistics following questions: text) show how probable it is to obtain a particular numerical result (such as eight heads in a row) • Are they relevant to respondents? Have and, more interestingly, how probable it is to respondents ever heard or thought about obtain that result or any of the possible results the product, service or topic? Do they care even more extreme (such as eight, nine or ten heads enough to have an opinion? Sure, they may in a row) by chance alone. That interesting answer; but is the answer an opinion or probability (p) is called the significance level, something else (like an attempt to get the which tells us that the result is more or less interview done and get off the phone)? likely to have been a chance result. That is ALL it tells us. • Are the questions reliable? Would respondents If the resulting probability is very small, say less give the same answer five minutes later? than .05 (which is 5 times out of 100), scientific Would they answer a slightly differently convention calls it “statistically significant at the worded question in the same direction? .05 level.” And if it is more than .05, scientific convention calls it “not (statistically) significant.” • Are the questions clear? Do respondents This means that scientists are inclined to believe know what they mean? Do they mean the that the result happened by something other than same thing to different respondents? chance, a weighted coin perhaps. But maybe it did happen by chance… unlikely events do happen, • Are the questions valid? Do respondents after all. Neither “maybe” nor “maybe not” is understand the questions to mean what the definitive. Statistical inference does not provide researcher intends? Are they phrased so as “definitive” conclusions. It just sizes up the to make a socially desirable answer too easy? different “maybes.” • Obviously, clients usually have only one study, Are the questions representative? Do they not an infinite number of them. It is possible that represent the full range of frames or choices study may yield one of the few atypical results that in people’s minds, or do they oversimplify a large number of studies would have produced. (e.g., give only TV/radio news or newspapers So, it may be a chance result, or it may not. Even as potential sources of news, omitting if it is not, the result may be too small or irrelevant internet and word-of-mouth)? to be “significant” in any non-statistical sense. For • Are the questions biased? Do they omit frames example, a small difference in product preference or choices relevant to predicting behavior based on a large sample may be statistically sig- (e.g., focusing war-in-Iraq questions on nificant, but it is nothing to bet big bucks on. protecting against terrorists or on death The client who wholeheartedly relies on one of soldiers, without balancing the two)? study without weighing other factors may be jumping to conclusions from shaky grounds and We must also consider how the characteristics perhaps plunging into a chasm. Scientists don’t of the questionnaire itself affect the work. Is there do that; they make inferences from a number a bias of order in the questioning? Have different and variety of studies, not from a single one of orders been found to produce similar results? a single kind. Is the questionnaire so long that it induces inattention, non-responses or random responses? The Significance of Non-Statistical Does the questionnaire format induce acquiescence Error: Hidden Challenges (people say “yes” or “agree” because they think To illustrate some of what might lurk in the that is what the interviewer wants to hear, or chasm of relying on only statistical significance, they say “no” or “disagree” to avoid yet let’s stick to sample surveys — the most typical another long series of follow-up questions)?
16 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED Quit Apologizing
And consider, too, how the survey situation itself that many types of error can exist: "Margin affects responses. Do people normally have to of sampling error is plus or minus 3 percentage make decisions on a subject in isolation, alone, points overall. Sampling error is, however, only without any discussion to clarify? Hardly ever, one of many potential sources of error in this except when responding to a survey! Typically, or any other public opinion poll." But, do you surveys offer little or no way for respondents to sense an apology in even this level of disclaimer? express their reality. If they try, the interviewer We don’t. has no way to record it, and the data analyst So, does any of this mean that quantitative has no way to analyze it. Much quantitative research is inherently invalid, unreliable or research (surveys, polls, experiments) tends to irrelevant? Not at all. All generalization — be standardized and artificial, far removed from statistical or not — requires non-statistical situations for showing real thoughts, feelings and inference from other factors. Theory, prior behavior in natural settings of interest to clients. research, experience with the particular research One may be choosing numeric comparisons at methods (including ways to mitigate some of the the expense of validity and relevance. challenges above) and the nature of the client’s With so many different possible “limitations” business all must be weighed and combined to to quantitative research — and we have barely make interpretations. THE NUMBERS NEVER begun to list them all — it is no wonder they are SPEAK FOR THEMSELVES. rarely mentioned in reports. At most, we see some simple comment about a “margin of error” with Some Best Uses of Quantitative Research a number that gives us supposed confidence in Though they do not speak for themselves, the the accuracy of the results. Compare a statement numbers — the quantities that quantitative research from The Washington Post, which acknowledges gives us — can be valuable to:
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 17 Quit Apologizing CONTINUED
• Measure how likely a finding is to be due to chance • Build statistical measures to assess the reliability of survey questions • Compare quantitative measures of interest across categories, segments and time • Make precise estimates and forecasts for share of market, mind or ballot box • Measure similarity and dissimilarity of individual attitudes, opinions and behaviors to form segments • Estimate the relative weights that consumers, voters or others attribute to various issues, attributes, benefits and risks in their choices • Reach large segments and populations cost-efficiently Properly understood and applied — and wisely interpreted — these numbers can be valuable aids to understanding consumer and voting behavior and to make business and policy decisions. We would not want to be without them.
Some Best Uses of Qualitative Research Qualitative research gives us information on qualities to help learn and understand: • What people think, feel, do and talk about… and don’t think, feel, do and talk about
18 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED
• How they think, feel, talk about and do it OR avoid thinking about or doing it and what else they think about or do instead • When and where (times, places and circumstances in which they are most open to encountering or engaging their minds, pocketbooks or ballots with the product or idea) • Why (motives) • Context (how and why the product, service or topic relates to them) • Understanding and insights (the reasons that connect what people do or say about topics, products and services to client actions that may benefit their business and people’s needs) • Emergent issues (things people think, feel and do that we might never have considered had they not brought them up) Truth and Significance in Qualitative Research Clients and researchers may still point to small, selective samples as barriers to the credibility, generalizability and true significance of qualitative studies. Quite to the contrary, qualitative research sampling practices are essential to the value of qualitative research. How so? Whom We Talk To: Purposive Samples Qualitative samples of participants are usually purposive: the people the client most needs to hear from first, in order to address the research question(s). They are people in the target audience who can communicate about the topic and are predisposed to act and/or are those who influence others about the topic (and you can watch that influence take place among group respondents). Why start with a “random” or “representative” sample when a “targeted” sample will provide more useful information? (To be sure, quantitative studies can also use purposive samples. Many do, but qualitative research studies tend to be far more likely to sample purposively, and to do so in depth.) Quit Apologizing CONTINUED
Right-Size Samples perceptions and possibilities for innovative Qualitative samples are the right size: groups are planning and action. small enough for people to recognize and interact Qualitative research samples are the right with each other. Most natural social groups size to hear or see the unexpected and the (“primary groups”) that affect thoughts, feelings unsaid (but perhaps hinted at) and other and actions are also small. And this size allows seemingly small things that might go unnoticed more in-depth study of group interactive process in a large sample. in focus groups. We can see people influencing Qualitative research samples are the right one another — or not — on different issues with size to provide the flexibility needed to change different frames or positioning. IDIs and ethno- the next IDI, group discussion or ethnographic graphic research can explore the individual and encounter, based on what was learned in the familial contexts of opinions and behavior in last one, and to stop when we have enough unprecedented depth. information. Qualitative research samples are the right size In sum, qualitative researchers really observe to get real meanings. We can probe apparent and listen to our clients’ target audiences and inconsistencies to clarify meanings, reinforce stakeholders so that we and our clients can what is truly consistent, and identify what is understand and act on what they are trying to not consistent (or not meant at all). We can communicate to us. So, qualitative research summarize participants’ thoughts for their does allow us to generalize from the research review, elaboration and confirmation — a to the world beyond — not in the statistical potent form of validation. sense, but in the more relevant sense of Qualitative research samples are the right participants’ reality and clients’ need for size to engage the client in the probing and research-based guidance that has significance understanding process and to transform client for their objectives.
Challenges to Qualitative Quality The inherent strengths of qualitative can also provide some challenges. We need to make sure that:
• The interviewing, research, analysis and consulting skills of the qualitative research team are at needed levels of their craft. Qualitative research depends especially heavily on these skills. • Moderating, observing, reports and presentations do not give undue weight to what is vivid (appearance) or memorable (sound bites). • The depth and texture of respondent perceptions, opinions, feelings, needs and actions are reported clearly yet concisely. • Observers, moderators and respondents have adequate opportunity to express and consider different interpretations of what was said and done (versus groupthink).
20 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED Quit Apologizing
Like Numbers, the Verbatims Don’t At best, we humans can never know everything, Always Speak for Themselves Either or even know everything that we might be missing. As we said, qualitative best delivers qualities, not Since we humans are “limited” or “finite” — not quantities. So, if we want only a count of something “definitive” — then our research, too, is always we know is important to consumers (or voters “limited” or “finite,” not “definitive.” Research or other decision-makers) and we know how to professionals understand that, and they make measure it, then we should NOT do qualitative judgments accordingly. Those judgments include research. Nor should we pretend to ourselves or respect for the ubiquitous and countless possibilities to clients that qualitative research can count as of error and risk in their findings, interpretations well as quantitative research can. And if we and conclusions and the applications thereof. So, don’t know what to count or how to count it, as long as we are all doing our best professional then we should NOT do quantitative research. work — qualitative, quantitative or both — let’s QUIT APOLOGIZING! Conclusion: Limitations of “Limitations” So, where does this all leave us? Ideally, we use both qualitative and quantitative research methods This article is based on a presentation on behalf and findings together in various ways, to reinforce of the QRCA Professionalism Committee at the what they share (“triangulation”) and to com- 2004 QRCA Annual Conference in Phoenix, plement what each does uniquely. The result is AZ (with Robert W. Kahle, Ph.D.) and a major an alliance of methods. As we all know, salt isn’t revision of it for presentation in 2005 to the “limited” simply because it isn’t pepper… and Joint Meeting of the Marketing Research vice versa. Each does its own thing best. And the Intelligence Association (MRIA) and QRCA meal is usually better with both condiments. in Toronto by Balch and Herbert.
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 21
• QUALITATIVE TOOLBOX •
What I Learned from Hanging Chads and Exuberant X’s: Testing a Voting System for Usability
B Y D ONA V ITALE Strategic Focus, Inc. • Chicago, IL • [email protected]
began with the butterfly ballot, an error that caused 19,000 Floridians intending to vote It for Al Gore to punch ballots for another candidate as well, invalidating both choices. Then we discovered hanging chads, and we watched as design flaws and voter mistakes changed history. In the 2000 election, Florida was the focus of media attention and legal wrangling, but when officials in Chicago and Cook County looked at their own election system, they made a disturbing discovery. While Florida was still counting votes, Cook County officials realized that, had it not been for Illinois’ solid blue election results, they too might be facing the unflattering glare of the national spotlight. The result was a six-years-and-counting effort to examine every aspect of running an election. Under Cook County Clerk David Orr, election staff hired graphic designers to make ballots more readable and less confusing. They purchased new voting equipment to replace hanging-chad-prone punch cards. And, they asked me to conduct
24 QRCA VIEWS SPRING 2007 www.qrca.org QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 25 What I Learned from Hanging Chads CONTINUED
FIRST LESSON LEARNED: Expect the Unexpected and Assume Nothing In Cook County, two different systems are present in each precinct: an optical reader that scans paper ballots (used by most voters) and a touch-screen computer system available for those who need special help (such as the blind or those who vote in another language). Both systems came from the same vendor, and both had been used previously in other jurisdictions. In-house tests done before purchasing the equipment went well, and both systems seemed to work easily and accurately. So, going into the first round of tests, we expected to find only minor problems that could be addressed with instructions to voters or precinct election judges. To our dismay, we found out that we were wrong. Almost anything is easy to use when it is tested by people who know how it is supposed to work. An uninitiated user, on the other hand, may have serious problems. Although both systems had been used in previous elections elsewhere, neither had been systematically tested for usability, and no one really knew how many previous voters had usability tests to help make the new system a experienced problems. Based on our tests, the more trusted and satisfying experience for voters. answer is: probably quite a few. Usability testing is common in the high-tech world, but it is a new idea among public officials LESSON TWO: You Can’t Test Everything, responsible for elections. When I first suggested usability testing in 2000, the response was, “What’s and What You Don’t Test Can that?” But, when the Cook County Clerk’s office Come Back to Haunt You acquired new equipment and began developing Because everything was new, a big challenge was procedures for the 2006 elections, the staff deciding exactly what to test. Elections involve decided to give usability testing a try. What they two separate groups: individual voters and the are learning about the voting system will have election judges who set up the polling place, an impact on elections for years to come. What I oversee voting and tally and transmit results to learned about testing usability of a complex system Election Central. Both voters and precinct workers may help others with their usability projects, no would be using the new equipment for the first matter what they are testing. time, with completely new materials and procedures Running an election is a complicated undertaking. that we could have tested. Suburban Cook County has over 2,500 polling To keep the project within the limits of available places, staffed by more than 12,500 election judges, time, money and resources, however, we chose serving almost 1.4 million registered voters. to test only the parts of the system that touch Running an election is like opening a chain of voters directly — the machines, instructions, 2,500 retail stores, all on the same day, getting each ballots and peripheral equipment used to cast one fully equipped, with trained staff in place, ready votes. We decided to assume that procedures for to serve 1.4 million potential customers in one setting up the equipment, counting votes and thirteen-hour shift. Then, at the end of the day, transmitting results could be developed without everything shuts down. Customers and staff go testing, and that election judges could be trained away, and the store closes until the next election, to use these procedures correctly. On Primary which might be in a few months or might be a Election Day, we discovered we had been year or two away. In 2006, this already complex overly optimistic. scenario was made even more challenging by the When the polls close, election judges use a switch to new equipment. Established routines machine that combines votes from both systems and procedures had to be scrapped, and new ones and transmits the results electronically to Election developed, to cover every aspect of the operation. Central. In the March 2006 primary, most precincts
26 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED What I Learned from Hanging Chads
completed this operation with only minor process into its individual parts and finds flaws problems. Where major difficulties occurred, the “experts” have forgotten were there. however, the reporting delay became the focus of attention for nervous candidates and media LESSON FIVE: Use a Combination of people waiting for results of close races. Had Direct Observation and Interviewing we tested the ability of typical election judges to In the first test, voters used both systems with follow the instructions for tally and transmittal, sample ballots containing names of celebrities some of the most disturbing election-day mistakes rather than political candidates (this removed might have been avoided. any fear that we were secretly testing political In a complex system, look for the elements preferences, and it let respondents concentrate that will have the most impact on a successful on the voting process). To increase voter comfort, outcome, and start working on the most critical we also decided against video or audio recording, issues first. knowing it was unlikely that anyone would ever LESSON THREE: Usability Testing Is Amazingly Efficient When the Cook County In June 2005, the initial round of testing was conducted in two suburban Chicago focus-group Clerk’s office acquired facilities selected to provide geographic dispersion. We recruited 25 registered voters in each location, new equipment and began setting quotas for age, gender and ethnicity to developing procedures for mirror the population of voters who would use the machines at the polls. Respondents were the 2006 elections, the staff scheduled for individual tests every 20 minutes. decided to give usability With a couple of breaks and time for set-up in the morning, we worked two very long days. testing a try. What they are After it was over, we realized we could have learned everything we needed to know in less learning about the voting than half the time. system will have an impact Only a few testers are needed to identify the most serious problems, and after eight or ten on elections for years to tests by typical users, it is unlikely you will uncover anything new. Small samples, even come. What I learned about smaller than for most other qualitative research, testing the usability of a are all you need. complex system may help LESSON FOUR: Be Aware of others with their usability Pitfalls Disguised by Expertise projects, no matter what (Your Own and Your Client’s) At our test locations, we had people from the they are testing. county election staff and the equipment vendor on hand to set up and to troubleshoot problems during the day. Given that we were concentrating review the tapes. Instead, we positioned the equip- on the voter experience, it seemed like a reasonable ment so that observers could see what was going idea. In retrospect, we would have learned more on by looking over the respondents’ shoulders. had we brought in two or three typical election After several respondents used the touch-screen judges to set up and operate the equipment, giving system, we needed insight into some of the actions us some advance warning of the problems and we were observing. For the last several respondents questions that cropped up at the polls on that each day, we made the touch-screen voting a first election day. “think aloud” experience, where the moderator When you know how to do something, some stood next to the machine and asked voters to steps become so automatic that you don’t even describe what they were doing as they went along. think about them. “Workarounds” initially This helped us learn that some of the scroll buttons developed to cope with a problem (instead and other elements needed improvement, and it of correcting it) become standard procedure. pointed out a number of places where a slight Someone who has never done the task before change to the screen layout would make all the brings a naive perspective that separates the difference in user understanding.
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 27 What I Learned from Hanging Chads CONTINUED
Asking respondents to “think out loud” as If we had not timed the voting and had relied they complete the task is a good way to understand only on respondent perceptions, we would have the thought process behind their actions, but be drawn the wrong conclusion as to the efficiency careful that this request does not change what of each system. If we had not used written ratings the respondent does. Ask questions only when and conducted post-use interviews, we would you really need more explanation. not have realized that poor design made ballot- marking more difficult than it should have been. LESSON SIX: Use Multiple Measures to Provide a Rounded View LESSON SEVEN: Even an Established To get comparative feedback on the two systems, System Has Room for Improvement we had everyone use both, starting half the The first round of testing — of ballot designs and respondents with the touch-screen computer screen layouts used for years in other jurisdictions and half with the optical-scan ballot. To provide — identified 44 specific problems and produced directly comparable measures, we kept track of 28 specific recommendations for change. Some how long it took to use each system, and we had were minor problems, but others could cause a everyone complete written scales evaluating each Florida-style disaster in a close race. method. After respondents finished voting on both In this case, since no testing with “naive” machines, we asked for reactions and comments users had ever been conducted, our findings in short post-use interviews. were a loud wake-up call. County Clerk David We found that the completion times and the Orr, who is committed to making the voting rating scales added to what we saw and what system more accurate and more trusted, quickly approved the recommended changes and wisely asked for more testing to make sure that “fixes” Only a few testers are needed to the system represented actual improvements.
to identify the most serious LESSON EIGHT: Use Iterative Testing problems, and after eight or to Make Sure that Changes Are ten tests by typical users, it Indeed Improvements After a complete redesign of the optical-scan is unlikely you will uncover ballot by an experienced graphic designer, we anything new. Small samples, did more testing to get feedback on the changes. Once again, we set up a mock polling place, even smaller than for most complete with scanning equipment and voting booths, but this time we chose a downtown other qualitative research, Chicago location that could attract people from are all you need. all over the area. Using the same quotas as before, we had 15 registered voters test the new ballot design and give feedback on several respondents said in the interviews. For instance, alternatives under consideration. As before, we most voters said they felt the touch-screen made timed the vote and had respondents complete voting faster. In reality, average voting time on rating scales, so we could compare results to the touch-screen was two minutes longer than the earlier findings. for the paper ballot. The oversized paper ballot, The new design resulted in better ratings filled on two sides with candidates for every and more favorable comments from the voters, office, seemed more tedious and overwhelming indicating that we were making progress. to voters, but their selections actually went However, there were still some opportunities more quickly. for improvement. When asked directly, most respondents said In the original test, voters were told to make they found the touch-screen system easy to use, a check mark or an X in the circle next to their but scale ratings revealed a strong feeling that candidate. This caused problems with what we “others” might need extra help getting used to the called “exuberant X-ing.” When voters make computer. On the optical-scan ballot, the rating large marks that extend into the sensing area for scales and post-use interviews identified problems the next candidate, the scanner rejects the ballot with legibility and clarity — ballot-design flaws as a double vote. To guard against exuberant that respondents were able to work around while X’s, the redesigned ballot changed the instruction marking their choices, but which made them to “Darken the oval” next to the chosen candidate, less certain they had marked the ballots correctly. and it added warnings about staying within the
28 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED What I Learned from Hanging Chads
circle to avoid spoiling the ballot. The result? Testers were so careful to completely fill in the designated oval that completion time went up by almost 50 percent. When we identified the problem, we changed the instruction to “make a mark within the oval,” and time for completion went back down to an acceptable level. A change that seems like an easy fix can cause new problems of its own. LESSON NINE: Look for Problems Where They Are Most Likely to Occur After the March primary, the ballot design was changed again. A consultant hired to evaluate the system pointed out that this particular optical- scan machine had been designed to use a “connect the arrow” mark, rather than an X, check or darkened oval. The County had moved away from “connect the arrow” because other options seemed easier for voters.
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 29 What I Learned from Hanging Chads CONTINUED
Make sure you test with people most likely to encounter trouble, not those who are almost certainly able to complete the task.
LESSON TEN: Users Want to Trust Their Intuition and Ignore Instructions When we tested the arrow ballot, our objective was to make sure the instructions were clear. On the first day, we interviewed twelve voters. We presented them with a ballot using the instruction from other jurisdictions: “Make a line to connect the arrow.” We asked them to mark the ballots as they would in the polling place, with no further instruction. The results were disturbing. Of the twelve voters we tested, only one marked the ballot correctly. This person was also the only one who claimed to have read the instructions. All the other voters looked at the ballot and did what they intuitively thought was right. They marked an X or made a check or circled the name of the candidate of their choice. After marking, they told us they didn’t read the instruction because they “knew what to do.” With less than two weeks before the deadline for printing November ballots, we went into panic mode. After the first few voters did the task incorrectly, our goal for the day became finding a way to make the arrow ballot work because it was far too late to change to another method. So, after respondents marked the ballot incorrectly, we told them what they were supposed to do, and we asked how we should explain the system. We found that “Make a line to connect the arrow” was too vague, especially since what we saw as one arrow cut in half, with space to draw a At the urging of the consultant, it was decided connecting line, was not what voters saw. They to revert to “connect the arrow” for the looked at our arrow and saw an arrow, a space November election. and another shape they called a peg or a stub. Luckily, the practice of usability testing was So, when told to “connect the arrow,” some beginning to establish itself, and election officials were baffled, and others drew a line from the tip asked for another round to check “connect the of the arrow to the candidate’s name. No one arrow” for potential problems. Time and budget understood that they were to connect the two were tight, so we kept the test simple. Instead of parts of the arrow to make a single shape. using a complete precinct setup, we asked respond- Maybe we are lazy or maybe we just think ents to mark the ballots and give some quick we are smarter than we are, but in our culture, verbal feedback. Interviews were scheduled people like to think they can figure things out on fifteen minutes apart, and most took even their own, especially when the task is as simple less time than that. as marking choices on a ballot. A system that We also made a change in recruiting for this needs detailed instructions to counteract user round. Because the question was whether voters intuition is doomed to failure. would understand the written “connect the arrow” instruction, we recruited people with less education After some experimentation, we realized we than the average Cook County voter. We reasoned had a two-pronged challenge. One was to make that if the least-educated voters could follow the voters see a single “arrow” shape to be filled in, instruction, more-literate voters with lots of ex- rather than three separate visual objects. The perience taking standardized tests and interpreting second was to make the instructions so visual instructions would have no problem. that voters could get the idea with a quick scan,
30 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED
before spoiling ballots with X’s and check marks. The solution was to make three ballot changes. First, we added a label. Calling it the “Connect-the-Arrow Ballot” sends a signal of something unusual, not a typical bubble or check-mark system. We also changed the instruction to “Draw a line to connect the arrow’s head to its tail, like this.” Then, we added graphics that clearly identified the head, the tail and the connecting line, so that even
Asking respondents to “think out loud” as they complete the task is a good way to understand the thought process behind their actions. “Great Experience! Be careful, though, You will be my first that this request choice for future • Impeccable research.” does not change Service Client comment what the respondent does. Ask questions • High Quality only when you Recruiting really need more • Superior explanation. Facilities someone who merely glanced at the instructions could see what to do. Need anything else? Will it work? Our latest round of testing says it will, but this is Just ask. being written two weeks before the November election, so we do not know for sure. We do NATIONAL DATA RESEARCH, INC. know more than we did before Capabilities beyond expectation. testing, and we learned in time CHICAGO & NORTHFIELD, ILLINOIS to make improvements. That is the purpose of usability testing, Tel: 847-501-3200 • www.national-data.net and when I cast my vote, I will be very glad we did it. Rated “World’s Best” by Impulse Survey
• QUALITATIVE TOOLBOX •
34 QRCA VIEWS SPRING 2007 www.qrca.org BEST PRACTICES for Real-Time Online Focus Groups
B Y J ONATHAN H ILLAND Mindwave Research • Austin, TX • [email protected]
have all seen the recent press bashing qualitative research. Many of us have also heard (and even been a part of) We some of the rebuttals to those viewpoints. The objective of this “view” is not to convince or sell anyone on the merits of online focus groups, let alone on qualitative research in general. The intent, The initial benefits based on a belief that all research and sampling methodologies have their place, their strengths and their weaknesses, is instead to offer of online qualitative insights on how to best leverage real-time chat discussions as a viable methodology in concert with other research tools that we were similar to online (as marketers and/or practitioners) have at our disposal to gather surveys: they were market-driven feedback that will best support business decision- making for our clients. cheaper, faster and less intrusive on the Online Pros and Cons Online focus groups have existed almost as long as online quantitative respondent. While survey research, but they did not gather momentum as quickly and these benefits still thus took several years longer to gain critical mass as a “known” methodology. The initial benefits that were used to sell online exist today, others have qualitative were similar to online surveys: they were cheaper, faster and less intrusive on the respondent. While these benefits still exist today since emerged that (more so than for online surveys), others have since emerged that may may be equally — if be equally (if not more) important. To best leverage these benefits, QRCs must also understand the problems inherent with online groups, not more — important. as well as the different “flavors” of online currently in use today. Online qualitative typically takes one of two forms: real-time chat- based discussions and bulletin-board (or “threaded chat”) discussions. Bulletin-board groups typically take place over several days or longer, giving the respondents more flexibility as to when they choose to “post” their responses. Most bulletin-board studies also tend to take place in multiple phases: as a respondent, you answer the first series of questions; those responses are reviewed; and you then come back for the next series of questions. This format also allows the respondent to spend time away from the discussion to do research (if necessary) to answer some of the questions with more detailed responses (for example, about budget allocation or configuration specs for a piece of equipment or software). In such cases, time away from the discussion can be a benefit to this methodology, but it can also have a downside. One of the advantages of qualitative research in general is the methodology’s spontaneous
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 35 Best Practices CONTINUED
There are few cities Due to the project time span, bulletin-board discussions can also be a little more difficult to in the world where you manage from the standpoint of keeping all of the respondents engaged and on task during can gather eight to ten the life of the study. participants from eight to My team tends to conduct the majority of our online qualitative studies using the second ten unique companies in approach — a real-time chat method — due in part to some of the challenges/concerns that industry; however, the mentioned above, as well as the ability to execute online forum opens us up studies more quickly than when using an in-person methodology. We can usually conduct most online to the entire market. chat studies in three to three-and-a-half weeks (seven to nine business days for design and nature, which provides a forum for respondents recruiting, two to three days for conducting to answer questions with their top-of-mind the groups, and the remaining time for reporting, answers. The descriptors they chose in their which varies based on the number of groups initial responses or reactions to stimuli are conducted and the complexity of analysis required where we most often discover the truest market in the report). Most platforms, including real-time perceptions around a particular issue. An unlimited chat platforms, can generate automatic/electronic response time may diminish the value of the transcripts of the discussion. If your reports are responses because they may not be top of mind “quote heavy” or at least largely driven by OR they may be based on additional sought-out transcripts and not just the moderator’s notes, information that may not have been part of the then this can shorten your reporting window respondent’s original thinking. by two to three days alone.
36 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED Best Practices
Another advantage of online qualitative methods is that you can cover more ground in less time. Think about the in-person studies you have conducted in the past: you probably rarely asked every single participant to answer every single question in a given session (except, perhaps, in triads or mini- groups). It is just not feasible, given the length of most discussion guides that clients push for these days. You would never finish the group on time because you must wait for each person to respond. In an online discussion (even real-time chat), when you send your question into the (virtual) room, all respondents type their answers at the same time, significantly shortening the amount of time spent on each point, even with probing after the responses come in. Most of our sessions are ninety minutes in length. Respondents more readily commit to a session that is less than two hours, and we can typically cover more material online in ninety minutes than we can in two hours in person). Another benefit from everyone answering at the same time is that you tend to see less cross-respondent bias. In an in-person setting with eight to ten respondents, you may get two or three unique answers to a given question as you go around the table, and the rest typically either don’t answer or say, “Yeah, what Bob said is true for me, too.” Of course, the group-dynamic aspect of that type of discussion is part of what you look for from qualitative research; however, you also want to make sure that you are getting original thinking that is not influenced by the other respondents. Seeing this difference has impacted how I conduct a lot of my in-person groups. For questions that deal with initial reactions to concepts, brand awareness or perception, I usually make the respondents write down their answers first, and then I have each person read from his or her note page as we go around the table in the room. Then I drive the group aspect of the discussion from that.
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 37 Best Practice CONTINUED
One of the differences that the discussion. While we may be the freest nation on the earth from a legal point of view, we place we notice (at least in the many restrictions on ourselves as a society in terms of what is considered to be politically or socially U.S.) is that participants correct thinking or behavior. If you put ten people in online studies tend who do not know each other in a room with a two-way mirror where they know they are being to be more truthful in their watched, in a discussion led by a moderator — and you ask them something like “Do you believe responses because of the in God?” or “What do you think about X political physical distance and greater figure?” — the response you get in that room may be quite different than the one you would perceived anonymity that get in an online forum. For most of our clients, exists between themselves the online methodology gives them more honest, realistic feedback about their brand, their products, and the other participants, their competition, etc. To the extent that we pay attention (in a the moderator and the viewers verbal discussion with in-person focus groups) of the discussion. to body language or other non-spoken cues such as intonation or emphasis on specific words, there are obviously some real deficits Other Online Considerations in conducting online discussions. If this type of So, what else should you think about when con- emotional feedback is an important part of the ducting online focus groups? What are some of learning for a given study, or if you want to the other benefits, and what are some of the other make sure that you do not get false positives concerns you should be prepared to deal with? in reaction to a particular concept or issue, Cost savings are an obvious first benefit. You the online moderator must be careful to ask no longer have the cost of a facility or food (for questions that replace this lack of cues. Such you, the clients or the respondents), and you do questions might include: “When you see or not pay for recording of any kind or transcription. read this, what words would you use to describe Also, you do not incur travel costs for the the emotional response/emotions you have to moderator OR for the clients (which is not this?” or “Why does it make you feel that way?” something they typically factor into the project costs up front because travel costs do not come Putting Clients at Ease with from the vendor). Online Qualitative Research This method is particularly useful for geograph- Clients who have no experience with online ically dispersed targets. We conduct a lot of B-to-B methodology are almost always reluctant to technology studies, and we have several clients give it a chance. Typical questions are: “How who target service providers (the phone companies). do I know that you really have the actual/right There are few cities in the world where you can people there if the discussion is online?” and “How gather eight to ten participants from eight to ten do I know if this will be as good as the feedback unique companies in that industry; however, the I would get from traditional focus groups?” online forum opens us up to the entire market. This hesitancy could be just because the Another issue that can arise in in-person studies methodology is “new,” and it is easier to just is local competition. If you target companies go with what they know. It could be because that compete with each other on a local scale, it they cannot imagine the idea of not hearing may not be realistic to put them into a room and and seeing the respondents. It could be that your expect them speak openly about their goals, client uses focus group trips as boondoggles to strategy, problems, etc. However, if the discussion get to go places he or she wants to visit or just takes place in a virtual, dynamic place that to get out of the office! offers confidentiality, then you can set their A variety of solutions can be provided here. minds at ease, and they will be more forthcoming. For instance, you can conduct a demonstration One of the differences that we notice (at least in of your online chat platform so that clients can the U.S.) is that participants in online studies tend get a feel for what they and the respondents will to be more truthful in their responses because of the see. You can share reports or transcripts from physical distance and greater perceived anonymity other non-proprietary sessions to show examples that exists between themselves and the other of the types of output they can expect. You participants, the moderator and the viewers of can also offer to manage the project with a
38 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED Best Practices
mixed-method approach — conducting some part of the chat/text discussion. This can range groups in person (what clients are comfortable from showing creative, messaging or concept with) and some online — so that they can statements to physically driving the respondent compare the quality, quantity and types of to websites for review. It is important to chose feedback they get from each method. a platform that is rich in features but yet not Another approach to alleviate some of the too far beyond the lowest common denominator lack of “human element” is use a follow-up in browser requirements (or the need to install conference call with the respondents after the additional software or plug-ins, which most chat-room discussion. You can first ask all of participants are reluctant to do). the questions online (so that you are sure to If you conduct groups with participants get the quantity of feedback you expect) but who have high-speed connections, you can also then leverage the phone call for probing and incorporate gathering feedback-to-video-based clarifying points (to establish the credibility content. Obviously even large monitors cannot of the respondents, as well as pick up on any replace the quality of viewing large, detailed emotional or other cues). Be aware, however, storyboards or other items in person. Some that the conference call re-introduces costs (of products or concepts truly require hands-on transcription) and cross-respondent bias (because testing/feedback, and some things simply must only one person can answer a question at a time) be seen in-person to get the “full effect.” into your study. For these reasons and the others mentioned in Most chat platforms built for the purpose of this discussion, the online methodology should conducting focus groups have “push technology” not be pitched as the answer for all qualitative that gives the moderator the ability to drive studies. It does, however, provide some significant content to the respondent’s desktop that is not benefits when used where appropriate.
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 39
• QUALITATIVE TOOLBOX •
Beyond the Data Dump: Provide Qualitative Deliverables that Spur Implementation
B Y M ELINDA K IZER G & S Research, Inc. • Carmel, IN • [email protected]
ualitative and quantitative researchers face similar challenges in converting a plethora of “data” into coherent deliverables that the client can use. For Q qualitative research, there is the additional charge of coalescing non-numeric data into meaningful, directional content. When done well, the research — however complicated — simplifies and clarifies the strategic business decisions that need to be made. When done poorly, though, the study can further muddy the waters. When following Proven techniques and “critical success factors” can ensure that your deliverables these tenets, you are clearly and effectively address the client’s strategic business decisions. When more likely to provide recommendations that will be implemented, which — at the end of the day — is the sign of a successful engagement.
42 QRCA VIEWS SPRING 2007 www.qrca.org following these tenets, you are more likely to can be addressed. Do not rely on the client to provide recommendations that will be imple- consider all the ramifications associated with mented, which — at the end of the day — is changing the discussion topic order, removing the sign of a successful engagement. a topic or substantively changing the discussion points. It is your responsibility to act as guardian Getting Started for the research, review the request(s) and assess The first critical success factors are addressed whether these changes will prevent the project before a proposal or statement of work is from addressing all research and/or business provided. Questions you should ask yourself objectives identified. and your client, before recommending a study design, include: Writing the Report • What is driving the client’s need So, you’ve carefully completed each step in the for information? qualitative research process — designing and • What decision(s) will the client make conducting qualitative research focused on both based on this information? project and strategic business objectives. How • Who will be involved in making these does all of the information come together in a decisions? Who will be key influencers? cohesive, actionable and relevant report? • What internal or external business factors The first step in analyzing and reporting the could influence the implementation of information goes back to reviewing strategic recommendations? business decisions. With these in mind, consider While the client will provide essential insight, the following questions: you should also arm yourself with information • What information will most directly affect about the marketplace, the client company and the client’s decision-making process? any current or developing issues. A thorough • What information clearly provides direction understanding of both the market and the client in regards to business decisions? company sets the stage for an actionable, • What information is inconclusive or relevant final report. contradictory? Once the project is awarded, confirm the • What outstanding questions, if any, remain strategic business objectives, as well as the that could impact the client’s ability to research objectives. If the business objectives do make an informed decision? not correspond to the research objectives, an in- A number of qualitative researchers follow depth pre-project review with the client should be the order of topics as outlined in the discussion initiated. All client constituencies involved should guide. However, this order may or may not agree on the strategic business objectives and reflect how the information will be prioritized the corresponding research objectives prior to when business decisions are made. Thus, the developing any research materials. client will have to scan the entire report to Other points that should be discussed prior to identify those key findings that have the most the project kick-off relate to the actual delivery immediacy and relevance to his or her constituency. and presentation of the results. These include: To alleviate the need for that extra step, the • Report distribution (widely disseminated most effective organization structure should versus limited distribution) follow the hierarchy of the most relevant infor- • Executive summary format (detail mation affecting business decisions to the least versus presentation) relevant information. Content organization for • Format preference (Microsoft Word, each topic area should also be prioritized PowerPoint, etc.) according to its relevance. • Client template versus researcher template Once the report structure has been organized, • Depth of information to be provided in the the next step is to identify how to best highlight main body of report key findings for each area. Several factors can influence how the information will be “displayed” During the Project in a report, including: Once the project commences, the qualitative • Client preference (text-driven versus researcher should review the project objectives visual/graphic representation) and the strategic business decisions at each • Format preference (Microsoft Word, subsequent stage of the research. Clients will PowerPoint, etc.) frequently request adding discussion content, • Technical/information technology limitations changing participant qualifications or otherwise • Intended use (presentation deck versus modifying the research plan in ways that may internally distributed written executive affect whether the strategic business objectives summary)
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 43 Beyond the Data Dump CONTINUED
The technical options that PowerPoint offers executive summary to mimic the “uninvolved client allow qualitative researchers to be creative with the constituent.” If you are an independent researcher, set inclusion of animation, as well as video and audio the executive summary aside and review it after a few clips. However, you need to consider how the content hours have elapsed. will appear to those receiving a hard copy of the report Conclusions and recommendations present the — and if the content will be “lost in translation.” most challenging aspect of writing a compelling, Use of flow charts, tables, colors, shapes and other actionable report. Many qualitative researchers graphic enhancements are tools for the qualitative hesitate to provide strong conclusions and researcher to creatively display key findings, irrespective recommendations, couching the text by using words of an electronic medium. But, be wary of providing like “may” or “perhaps.” For some clients, this type a “dressed-up” data recitation that fails to deliver insight and clear direction. Carefully evaluate whether using these options detracts from communicating the Once the project is awarded, key findings. confirm the strategic business Ultimately, the executive summary content should be developed so that client constituents not directly objectives, as well as the research involved in the research can quickly: objectives. If the business • Understand the key findings • Correlate findings, conclusions and recommenda- objectives do not correspond to tions to the stated research and business objectives • Apply findings, conclusions and recommen- the research objectives, an in- dations to the strategic business decisions depth pre-project review with the If applicable and appropriate, request that a colleague not involved in the research review the client should be initiated.
44 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED Beyond the Data Dump of phrasing is most appropriate. When writing the project report, the most effective However, if you fully understand the client’s business decisions, organization structure should follow the hierarchy the environment in which these decisions will be made and the of the most relevant information affecting business marketplace, then demonstrate decisions to the least relevant information. this knowledge with targeted and insightful conclusions and recommendations. This will differentiate you from those who “state the obvious” or provide non-committal summarization. For example, if the client is con- templating the development of a product and the research indicates that the “need” for the product does not exist, say so. Then provide recommendations that delineate potential steps for developing that need, or recommend that they recon- sider developing the product. Your client could agree or disagree with the recommendations, but they will generate discussions that will likely lead to better recommendations. The detailed findings should follow the same organization as the executive summary so that the client can easily reference details to support the key findings. The level of detail included should be sufficient to be supportive and illustrative, including quotations, but not so finely detailed as to lose the meaning in the minutiae.
In Summary The reason that your clients engage your research services is always linked to their strategic business decisions, even if the research objectives are exploratory. Ultimately, you should ensure that your research deliverables effectively address these business decisions. You can do this by clearly identify- ing the business issues upfront and constantly revisiting these throughout the project’s lifecycle. You should always define how the project findings will be used and by whom. Also, analyze and filter the information according to the client’s decision-making framework, and use the best reporting tactics to clearly convey the findings. Do these, and your client will consider you a true business partner.
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 45 Simplifying the Research Process Videostreaming Products and Services for Marketing Researchers
Mark it. Review it. Clip it. ActiveNotes seamlessly links text notes with the associated point in the video. With the click of a mouse, review notes, download clips for presentations, and export notes (with hyperlinks to the video) to Microsoft Office applications. www.activegroup.net/AN
Traditional transcripts with a twist Every line of an ActiveTranscript is linked to the associated point in the video. Available in Edited-Summarized and Verbatim format 48-72 hours after your session. www.ActiveGroup.net/AT
ActiveGroup's exclusive phonetic search ActiveSearch requires no speech-to-text conversion or transcript to return search results instantly! Search by keyword, phrase or word groupings across your research event. ActiveSearch is the perfect solution for a quick overview of the day’s research or a search for specific responses across a research project. www.ActiveGroup.net/AS
The intuitive portal to your ActiveGroups Researchers find the user-friendly design of ClientLounge increases their productivity with a host of easy-to-use features. With so many tools for researchers, it's more of a research application than a viewing interface for accessing research events remotely. ClientLounge 5.0 is the gateway to a researcher's ActiveGroup sessions, both live and on-demand.
ACTIVEGROUP 3720 DaVinci Court Norcross, GA 30092 800.793.3126 www.activegroup.net Book your next project with one of the ActiveGroup facility partners below
UNITED STATES Jacksonville MASSACHUSETTS NORTH CAROLINA VIRGINIA Concepts in Focus Boston - Downtown Charlotte Alexandria ARIZONA Miami Bernett Research Services 20/20 Research Metro Research Services, Inc. 20/20 Research Phoenix Boston Field & Focus Performance Leibowitz Market Fairfax Ask Miami Plus - GroupNetTM Research Assoc., Inc.-GroupNetTM Delve Phoenix Metro Research Services, Inc. Miami Market Research FocusPointe MarketWise Fieldwork Phoenix, Inc.-South Mountain National Opinion Research Services Schlesinger Associates Raleigh WASHINGTON Plaza Research Spire Audio Visual--Focus on Miami Schlesinger Phoenix Boston - Suburbs Carolina Focus Seattle Spire Audio Visual--WAC L&E Research, Inc. Time N Talent / Strictly Medical Orlando Fieldwork Boston, Inc. Delve Seattle West Group Research Informa Research Services Insight Orlando Scottsdale MICHIGAN OHIO Fieldwork Seattle Product Insights, Inc. Cincinnati Fieldwork Phoenix, Inc.-Scottsdale Schlesinger Associates Detroit Assistance In Marketing Focus Market Research, Inc.-GroupNetTM AccuData Market Research, Inc. MORPACE International, Inc-GroupNetTM WASHINGTON D.C. SIRS Market Research Needs About Orlando Market Research Crimmins & Forman Market Research Washington, DC Cleveland CALIFORNIA (Northern) Palm Beach -Lathrup Village Olchak Market Research Focus Groups of Cleveland Survey Berkeley Field & Focus Crimmins & Forman Market Research Shugoll Research, Inc. -GroupNetTM Opinion Centers America Greenberg Qualitative Research Tampa -Southfield RDA Group Opinions Ltd San Francisco Plaza Research WISCONSIN Shifrin Hayworth Columbus Focus Pointe San Francisco Schwartz Research Services, Inc. Appleton Stander Research Associates, Inc. Delve Greenberg Studios TAi - Tampa Bay, Inc. Delve Appleton Midwest Video The Herron Group-GroupNetTM Nichols Research, Inc.-GroupNetTM OREGON Milwaukee Plaza Research MINNESOTA Mazur / Zachow Interviewing Schlesinger Associates GEORGIA Portland Milwaukee Market Research San Francisco -Suburban Atlanta Minneapolis Focus Portland Ascendancy Research Nichols Research, Inc.-GroupNetTM Atlanta Focus GCR Custom Research Delve Minneapolis Q & A Research, Inc. Compass Marketing Research Market Decisions Corporation Fieldwork Minneapolis, Inc. San Jose Delve Atlanta EUROPE Market Resource Associates, Inc. Fieldwork Atlanta, Inc. PENNSYLVANIA Nichols Research, Inc.-GroupNetTM Focus Market Research, Inc.-GroupNetTM Field Dynamics Focus Pointe Atlanta Philadelphia - Downtown HOLLAND IMAGES Market Research MISSOURI Focus Pointe Amsterdam CALIFORNIA (Southern) Jackson Associates, Inc.-GroupNetTM JRA Van Hellemond & Co Murray Hill Center Kansas City Schlesinger Philadelphia Costa Mesa Plaza Research Delve Philadelphia - Suburban SPAIN Assistance in Marketing-Costa Mesa Schlesinger Associates St. Louis Focus Pointe Bala Cynwyd Madrid Los Angeles User Insight Delve - St. Louis Group Dynamics in Focus, Block de Ideas, S.L . Adept Consumer Testing, Inc Lucas Market Research Inc.-GroupNetTM -Beverly Hills-GroupNetTM ILLINOIS Peters Marketing Research AdCare Schlesinger Associates Bala Cynwyd Barcelona Adept Consumer Testing, Inc Chicago - Downtown Pittsburgh -Encino-GroupNetTM Block de Ideas, S.L. Fieldwork Chicago, Lab NEBRASKA Direct Feedback Adler-Weiner Research-Westwood Focus Pointe Omaha Atkins Research Group Focuscope, Inc. The MSR Group Delve RHODE ISLAND MEXICO Home Arts Guild Research Center Providence Focus Pointe Market Ease/Urban Focus NEVADA Analitica Marketing S.A. Plaza Research Accurate Focus Soluciones Cualitativas, S. C. MedQuest - Chicago Las Vegas Savitz Field And Focus National Data Research, Inc.-GroupNetTM MRC Research Institute Schlesinger Associates National Qualitative Centers, Inc. TENNESSEE Plaza Research Trotta Associates Marina Del Rey Plaza Research Memphis CANADA L.A. Orange County Savitz Field And Focus NEW JERSEY AccuData Market Research, Inc. Mississauga Fieldwork Los Angeles, Inc. Schlesinger Associates Insights Research Group Group Dynamics Trotta Associates Irvine Smith Research Inc. Fieldwork East, Inc-Ft. Lee Nashville Focus Pointe North Jersey-Teaneck Montreal Long Beach Survey Center Focus 20/20 Research Nashville Meadowlands Consumer Center Contemporary Research Centre Assistance In Marketing Inc. Chicago Suburbs Secaucus-GroupNetTM Opinion Search Montreal Riverside Accurate Data Marketing, Inc. Plaza Research-Marlton TEXAS Ottawa Assistance In Marketing Athena Research Group Plaza Research-Paramus Opinion Search ClearView Research Austin Sacramento Q Research Solutions, Inc. Toronto Delve - Oak Brook Austin Market Research Opinions… of Sacramento Q10 Marketing Dallas Consumer Vision Ltd. San Diego Precision Research Schlesinger Associates-Edison Ask Dallas Opinion Search Inc. Flagship Research Fieldwork Chicago, Inc. - North TAi New Jersey-Teaneck Dallas Focus Vancouver Luth Research, Inc. Fieldwork Chicago, Inc - O'Hare Taurus Marketing Research Delve Consumer Research Center Vancouver Plaza Research Fieldwork Chicago, Inc - Schaumburg Field & Focus Inc. Fieldwork Dallas, Inc. Strategic Visions, Inc. Focuscope, Inc. - Oak Park Suburban Associates-Ridgewood Focus on Dallas, Inc. Taylor Research, Inc-GroupNetTM National Data Research, Inc.-GroupNetTM Focus Pointe Dallas Sherman Oaks Schlesinger Chicago - O'Hare NEW MEXICO Northfield Murray Hill Center Facts n Figures Smith Research Inc. - Deerfield Albuquerque Opinions Unlimited-GroupNet Focus & Testing, Inc. Sandia Market Research Plaza Research Qualitative Insights INDIANA Savitz Field and Focus NEW YORK Schlesinger Associates COLORADO Indianapolis El Paso Herron Associates, Inc.-GroupNetTM Buffalo Denver Survey Service AIM Research Houston AccuData Market Research, Inc. KENTUCKY New York City Fieldwork Denver, Inc. FocusPointe Medical Market Research Plaza Research Louisville Focus Room Marketing Research Services TAi Horizons in Focus Focus Plus, Inc. Opinions Unlimited, Inc.-GroupNetTM Personal Opinion, Inc. MBC Research Plaza Research CONNECTICUT Sachs Insights Savitz Field And Focus Houston Stamford MARYLAND Schlesinger Associates Pharr The Focus Room Baltimore New York - Suburban Hispanic Focus Unlimited Hartford Research Baltimore Research-GroupNetTM Fieldwork New York, Inc- Westchester San Antonio Bay Area Research Focus Room-White Plains CCR Florida Maryland Marketing Source, Inc. Innovative Concepts-Carle Place Galloway Research Service-GroupNetTM Fort Lauderdale Observation Baltimore Long Island Groups in Focus MARS Research Ft. Lauderdale Bethesda Plaza Research Shugoll Research, Inc.
For a complete facilities contact list please log on to www.activegroup.net/facilities • QUALITATIVE TOOLBOX •
48 QRCA VIEWS SPRING 2007 www.qrca.org A Picture Is Worth a Thousand Words…SOMETIMES
B Y B RUCE E CKMAN, PH .D. Creative Insights, Inc. • Sherborn, MA • [email protected]
ver wonder why the adage “a picture is worth a thousand words” rings true? Qualitative researchers have used pictures to define E brand images ever since advertising agencies started using focus groups to understand consumer needs and motivations. For generations, people have learned how to speak and critique This technique’s language in school. We all have a history of learning the right and wrong way to write and speak. It is interesting to note that the left end result provides brain is where the “right” way to speak is processed. Visual imagery has escaped much of that critiquing process. The a complete profile right brain is where visual information is processed. This information of personality is processed in a simultaneous way, moving from the whole to the details within the whole. School never focused on the “right way” to look attributes particular at the world. Instead, schoolteachers spent a lot of time stimulating to the brand, the left, analytical side of the brain, explaining how to spell words, construct grammar and how to write and verbalize properly. market and When people interact with products, they learn from what has attracted them about that product, and their first impressions often consumer being inform their judgments, rather than the other way around. This investigated, and pattern of recognition and learning reflects our right-brain biology more accurately than our cognitive side. the inclusion of any Visual stimulation in the form of picture sorts, therefore, made sense to use as stimuli for product and advertising assessments. two variables allows Picture sorting has been done for years. It is a staple of marketing for differential and advertising research. comparisons to The Impact of Recognition be made. What happens, though, when the pictures being exposed have been seen before? Recognition then becomes a combination of what has already been seen with the emotionality of the imagery. Familiarity is confounded with the visual elements, and the result is noise in the system, a muddying of the imagery. A child picks a picture because he thinks he has seen it before, rather than because it is more represen- tative of the new product he is drinking. His “liking” of a picture is based as much on what he has been trained to like (by the media weight of the advertiser) as on what the picture represents. The signage aspect of the familiar has compromised the symbolic value of the visual. There can also be a bias that creeps in at the end of a long day of focus groups when some of the pictures have been torn earlier from
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 49 A Picture Is Worth a Thousand Words... Sometimes CONTINUED
A respondent’s “liking” of a The original conceptual work about right and left brain, as first postulated by psychobiologist picture is based as much on Roger Sperry in the late 1960s, has been modified because the relationship between the what he has been trained to hemispheres of the brain is not as polarized as like (by the media weight of initially thought. Likewise, understanding the intuitive centers of the brain by using visuals the advertiser) as on what is not as simple as many have practiced it. The balancing of the intuitive and cognitive sides the picture represents. The of the brain during the filtering process is signage aspect of the familiar complex, and care must be utilized to minimize interference with understanding the visuals’ has compromised the symbolic meaning to the consumers being tested. Simple value of the visual. things about how a methodology is carried out can influence the ultimate interpretation of the visuals' meaning and can lead to costly a magazine and, so, are no longer offered as marketing mistakes. part of an evoked set. A second concern is when Although well-known products enjoy a picture has been altered in some way that considerable equity in the minds of consumers, informs the respondent of former respondents' there is often limited awareness of the multi- selection of that picture. Another bias comes dimensional factors that define loyal users’ from cultural associations to visuals that may connection to the brand. Exploration of such mean different things to people from different factors allows a brand to emerge from its cultural backgrounds. category with a powerful and solitary voice.
50 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED
Alternative Process Tested Consider a different approach, using the assumption that if pictures could be tested before conducting a qualitative research project, a baseline measurement could be obtained for the pictures. This baseline would allow any movement in the meaning of the pictures in the consumers' minds to reflect the brand being assessed. The technique uses evocative pictures to target specific dimen- sions unique to the formulation of a brand persona. Qualitative inferences are supported by quantitative norms. The end result provides a complete profile of personality attributes particular to the brand, market and consumer being investigated, and the inclusion of any two variables allows for differential comparisons to be made. Starting with several thousand non-advertising pictures from Europe, the Far East and Australia, these pictures were shown to hundreds of consumers who rated the pictures as to their meaning on Likert adjectival scales. These ratings were factor analyzed, and the pictures that statistically measured similar dimensions were selected. This led to a set of just over 700 pictures that measured 13 discrete dimensions: authenticity, desirability, emotionality, quality, security, health, approachability, status, familiarity, gender, maturity, activity and dependability. Normative data was obtained to see how the scales shifted when a brand was being imaged, rather than just the picture. A baseline meaning for the pictures was established. Each picture is set in plastic so that no consumer handling will show. Likewise, each assessment uses the complete set of pictures to select from, so there is no depletion of the potential evoked set. In each study, the brand meaning is assessed by how the picture selections do or do not deviate from the a priori norms established for the pictures. A Picture Is Worth a Thousand Words... Sometimes CONTINUED
The difference from the baseline in the ratings Unfortunately, many consumers are unaware reflects the consumers' perception of the product of the disparate factors that cause them to feel being examined. This becomes particularly clear or experience a brand in certain ways. They when competitive brands are shown. The subtlety cannot tease apart their reaction to discrete of the distinctions between brands is greater than qualities of the brand (e.g., clean taste) from that of verbal measures because of the simulta- those based on integrated perceptions of brand neous, intuitive nature of right-brain perception. character (e.g., “There’s just something about What is measured is more accurate and informing it.”). This makes the process of delineating viable because a more sensitive instrument is being used brand attributes and exploring brand personality to assess how consumers make judgments. nearly impossible based solely on measures of Over the years, this technique has been tested verbal self-report. Given that each picture’s in England, France, Germany, Spain, Japan, meaning has already been ascertained (and Korea, Canada and the United States. Where an open-ended question helps to catch special, differences existed between cultures, adjustments idiosyncratic meanings) when consumers select a were made in the visual sort to account for the picture of a brand without being able to articulate problem being studied. While differences were why, this procedure allows the qualitative greatest in the Far East, once the picture sort was researcher to understand what dimension was enriched to include imagery and symbols more influencing that person's selection. appropriate to those cultures, the methodology was robust enough to account for the differences. Preparation and Control Considerations Thus, a brand character or personality — In order to better understand consumers’ defined by the integration of many distinct, albeit perceptions of brand personality, being prepared consistent, attributes — is dimensionalized and ahead of time allows the qualitative consultant measured. Qualities such as unique taste, price to reach consumers on a deeper, less controlled and healthfulness, when reflected time and time level — where distinct product attributes are again, serve to organize consumers’ perceptions organized, based upon consistency, into working into a fundamental brand identity. opinions of the brand itself. However, the more uncontrolled the level of access is, the more controlled the method- ology needs to be to allow for a more reliable data set and clearer, more consistent interpretation. Not only does this research methodology not distort the outcomes, it functions on that deeper right-brain process of seeing the whole and the parts of the whole simultaneously. This gives the marketer a clearer, more reliable understanding of what aspects of the brand is (or is not) working and how the competition is faring along those dimensions at the same time. This allows for effective market planning and moving products along specific desired dimensions. In summary, while tapping into the less controlled areas of the consumer's brain makes excellent sense to get an unvar- nished sense of what his or her motivations truly are, traps surround the consumer’s more vulnerable sides. Protecting the less controlled aspects of the person has had evolutionary benefit to the species (e.g., children
52 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED
are treated differently from adults); however, as good qualitative researchers, we need to be aware of the pitfalls when entering the emotional stream of consciousness, and we should avoid them where possible. Every clinician knows that when therapeutic discussion gets inside the patient's safe zone, compensation occurs. It is critical to the outcome to know which compensations (defenses)
Given that each picture’s meaning has already been ascertained when consumers select a picture of a brand without being able to articulate why, this procedure allows the qualitative researcher to understand what dimension was influencing that person's selection.
can be moved and which are parts of the structure of the person's personality and need to be left alone. The inter- vention cannot intrude on the basic structure of personality or until the patient has sufficient self-esteem to support a change in that structure; otherwise, the clinician has done a disservice to the patient. Likewise, when moving into the less controlled aspects of consumer behavior, the qualitative researcher needs to have a thorough understand- ing of the methodology being used and should be consistent with the execution of that methodology to enhance the reliability and validity of the consumers' and the researchers' non-verbal judgments and conclusions. • TARGETED MARKETING • The Vitality Boom: Marketing to Active, Boomer Women
B Y M ARY B ROWN JWT Mature Market Group • Portland, ME • [email protected]
A ND C AROL O RSBORN, PH .D. Fleishman-Hillard Inc. • Washington, D.C. • [email protected]
Redefining Fitness Sprinting to the next staff meeting, then out to a fundraiser or home to prepare dinner for the family isn’t the only way a busy baby-boom woman in her 40s gets her heart rate up. Having gratefully emerged from the era of Army-style gym classes and five-times-a-week studio aerobics, she is now embracing more The bottom line? Make soulful fitness activities, including Pilates, yoga and active walking. In the spirit of her modus operandi — multitasking — these pursuits are not only easier on it easy for a boomer her body, but they are also emotionally soothing and empowering at the same time. Stretch and meditate simultaneously? Work her heart while learning self- woman to fit exercise defense techniques? She’s in. into her life — and give it a multitasking purpose — and you will capture the affection of this resourceful, fitness- oriented consumer.
54 QRCA VIEWS SPRING 2007 www.qrca.org Most importantly, she looks for situations 40 percent of women say they always feel where she can leverage a reduced schedule of rushed, versus 28 percent in 1975. (Changing classes with at-home or lunch-hour practice, Work and Family Demographics, 2003.) on her own terms and at her own pace. Maybe it's the “Madonna” factor, too, but between Fitness as the Foundation of the 2000 and 2004, yoga participation increased Well-Lived Life 122 percent, with the largest group of female As she blows out more candles on her (low-fat, practitioners being 45 to 54 years old. (“The low-carb) birthday cake, this woman is embracing Evolving Summer Marketplace,” Leisure Trak the physical aspects of middle age and beyond Report, 2004; Yoga Journal Survey, 2005.) with a whole new mindset. With her kids grown If she has children, chances are they are no (or growing up) and more time to invest on longer infants. She is free to leave the house, getting herself, she is focused on getting healthier, feeling out on her own or with friends to enjoy some fresh stronger and renewing her body and spirit with air. Outdoor sports challenge her muscles and give each passing year. In fact, 7 in 10 boomer women her an emotional shot of physical accomplishment. feel a lot younger than their actual age, and 62 Adventure sports such as hiking and mountain percent say they work at trying to maintain a biking are drawing her to the trails and boosting youthful appearance. (“Baby Boomer Women: her sense of strength and achievement. Coming Reshaping Themselves and Their Future,” up fast also, according to Fran Philip, chief W2W, 2005.) merchandising officer of L.L. Bean, is snowshoeing. She is still busy, however, and in some ways — sandwiched as she is between the conflicting Marketing to Her demands of grown children and aging parents This energetic group has redefined the meaning — she is more in need of serenity than ever. Her of fitness. Talk to her about healthy activity that medicine cabinet and cosmetics table are stocked can take place outside of the gym and in her with beauty and skin products that simplify, living room, on a mountainside trail or spa-like rather than complicate, her everyday routine. setting, combining spirituality and physical conditioning. Give her comfort, performance and style. Recognize her as a serious athlete who needs innovative fitness gear, apparel and “Embrace yourself, not facilities to enhance her physical accomplishments. your age” may be the boomer Tailor products to fit her body and her lifestyle, and she will respond with loyalty and enthusiasm. woman's mantra, but she's Brands such as Title 9 Sports, Nuala yoga wear and Terry Bicycles have thrived with her by still struggling with the creating high performance, style-savvy merchandise changes she sees in the designed by women for women. And Curves — the women-only fitness center — has become mirror and the physical side the world's largest fitness-center franchise almost of aging. While 61 percent exclusively through word-of-mouth referrals. Curves offers an on-demand, 30-minute workout of women believe that older in a barebones, yet encouraging, environment. The busy woman can grab a half-hour workout women can be more attractive on her way to work, during her lunch hour, or than younger ones, 48 whenever the moment can be seized. Capitalizing on this formula for success, Curves has expanded percent think they looked its franchises by 560 percent since 1999 and started an explosive movement of women-centric their best in their 20s. fitness centers. (Entrepreneur magazine, 2005.) The bottom line? Make it easy for her to fit exercise into her life — and give it a multitasking Her kitchen cabinet tells a similar story. purpose — and you will capture the affection of Because of her responsiveness to the benefits this resourceful, fitness-oriented woman. of health foods, she can find items that were Trailing-edge boomers are exercising far more once relegated to the specialty stores but are than past generations at age 40+. But it is still now on her favorite supermarket’s shelves. an uphill struggle for them to find the time and This is a one-stop shopping boon for the busy motivation. Faced with a massive to-do list, exercise woman grabbing something fast and now healthy quickly slips out of the picture, as today nearly on the way home from work. She will even pay
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 55 The Vitality Boom CONTINUED
a higher price for products that are healthful, but Many of the women on these trips have become she is not willing to sacrifice taste in the process. personal friends and have traveled with me multiple She also has the cash and will make the time to times over the years. On each of these adventures, visit the spa, enjoy the latest beauty treatments and we spend the better part of 16 hours a day together purchase alternative and allopathic health products. and often for 9 to 21 days at a time. We have shared life’s transitions: the dilemma Marketing to Her of being single and not wanting to be; dating, marriage, children, divorce, death and illness of Smart marketers can take heed of several tips in children, husbands and parents; 40th, 50th and targeting the boomer woman. Embrace her passion 60th birthdays; their concerns about growing for youthfulness and vitality, but market to her older and expectations of staying younger; the wisdom and her well-honed sense of self. No search for passion and fulfillment in work, Hollywood diets or miracle creams will fly here. relationships and all parts of life; the fear of Develop ongoing relationships that authentically failure and the overwhelming excitement of address her health and beauty maintenance concerns. accomplishing a goal, especially one that could Kiehl's has created an almost cultish following not have been imagined 20 years earlier. for its prestigious hair and skincare products with When you spend such intense time moving relationship marketing and product sampling, about unfamiliar and often uncomfortable areas barebones packaging and no advertising whatso- of the world with a group of women, you talk ever. O, the Oprah Magazine and Oprah.com are about everything (and laugh a lot). Think about two other wildly successful brands for this boomer it. From shuttle services to parking, airports woman. With the “live your best life” slogan, to airlines, hotels to restaurants, boots, packs, Oprah uses her multimedia channels to provide scarves, moisturizer, socks, makeup, cameras, feel-good emotions, information and insights about telephones and carriers, nothing is “usual.” This body and soul in a hip, interactive format. is the beauty of travel. Going out in the evening, The marketing message is clear. Focus on how with a small set of clothes to choose from, becomes your product or service helps her quest for inner a creative fashion show with a captive audience. wisdom, health and beauty, and you may develop One of the most important things that I have a cult-like following of your own. learned from the active boomer women I have traveled with is that women notice the details — The Flip Side before anyone else — whether it is appreciating “Embrace yourself, not your age” may be this the quality of a towel or discussing the variety woman’s mantra, but she is still struggling with of cultural opportunities of a particular day trip. the changes she sees in the mirror and the physical When the travel brochure arrives in the mail, side of aging. While 61 percent of women believe they notice how the envelope was addressed. that older women can be more attractive than Was it addressed to Mr. and Mrs., when there younger ones, 48 percent think they looked is no Mr.? Was it handwritten or mass mailed? their best in their 20s. (Avon’s Global Women’s Was there a personal note for returning clients? Survey, 2003.) They notice the feel of the paper and whether Simply look at the span from the late 1990s it was printed on recycled stock, the emotional to the turn of the century, when the number of impact of the imagery and the size of the font Americans undergoing plastic surgery had increased (often it is too small for bifocaled boomers). by a whopping 1,125 percent. (American Society When they call or email the company, they of Plastic Surgeons, 2002.) care how the phone was answered (hopefully, Statistics don’t lie. But recall Cheryl Tiegs’ directly by a real person). They want to talk comment to Katie Couric on The Today Show: to someone who listens to them and asks “I think the days of plastic surgery, pulling questions, calms their fears, encourages them everything back, getting rid of every single line and speaks informatively. In general, they do and wrinkle, are over.” not want 15 itineraries to choose from in Tuscany or Nepal. They want three to four On Paying Attention to Details that are clearly different for specific reasons I have spent the last two decades of my life (e.g., classic route, more remote/cultural, designing, marketing, selling and leading active boutique properties, comfortable camping). vacations for women to places around the world. Because of their busy schedules and lives, Over the years, I have personally led approxi- they want programs offered in modules (nine- mately 3,000 people on small-group biking, day core trip with a two- to three-day pre-city walking, trekking, snowshoeing, multi-sport and escape, post-jungle option, etc.) so they can cultural trips, family trips and student programs. choose what fits their interest and availability.
56 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED
When they receive the confir- mation materials, the destination information and packing list are extremely important. They want the packing list to be as specific as possible, with recommended brands, as well as descriptions on what the items will be used for. They read all the "cultural" details of what is and is not appropriate and safe in the area. Once on the trip, their first few days are most important. Staying on schedule, the social aspect of introductions and putting anxieties to rest are all key. The first day, they want assistance in fitting packs, checking footwear and reminders to drink water. Small luxuries are important: offering simple things like hot water and mini-spa treatments on a trek, impromptu pre-dinner happy-hour gatherings and assisted shopping adventures; purchasing foreign stamps in advance; offering massages when possible; and assisting with little details that will make her experience more comfortable while still keeping it an adventure. After the tour, boomer women are busy. The importance of a big trip to them can be life- changing. Once the "group" is created, a new community is formed. No one at home will be able to share or understand what has transpired on the trip. Keeping these relationships going by way of a yearly calendar, mini get-away reunions, shared photo websites and email lists enhances the group members’ commitment to each other and maintains your connection to them.
Reprinted from Boom: Marketing to the Ultimate Power Consumer — The Baby Boomer Woman, by Mary Brown and Carol Orsborn, Ph.D. Copyright ©2006 Mary Brown and Carol Orsborn. Published by AMACOM Books, a division of American Management Association, New York. NY. Used with permission. All rights reserved. http://www.amacombook.org.
• TECH TALK • Can Software Help? A Look at Software Tools for Qualitative Research Analysis
B Y J EAN N ORDGREN Market Works Inc. • Savage, MN • [email protected]
are operating in crisis mode as the deadline for the research report you are writing looms closer and closer. As you sip your sixth cup You of caffeine, you wonder if there is a magical elixir to help you churn through your dozen focus group transcripts. Or perhaps as you hit “send” to email your client the draft of your research findings, you have a nagging suspicion We did find software that there is an elusive magic wand out there that could enhance your analysis. Could qualitative analysis software provide the magic to make our analysis tools with the potential easier and improve our results? I set out to explore the mystique behind qualitative research analysis software tools with Jeff Walkowski of Qualcore.com in Minneapolis. to make the work For reference, Jeff and I are both “pencil pushers” when it comes to analyzing easier and the end the qualitative research our firms conduct. Like many of our colleagues, we use results better, as long as the researcher is willing to overcome the steep learning curves we experienced.
60 QRCA VIEWS SPRING 2007 www.qrca.org paper and pen to note key findings and track The second issue preventing software tools significant quotes before jumping onto the from achieving magic status is the large amount computer to write our reports. of work required from the user. Admittedly, we We experimented with various software options knew the software would not do all of the work. and presented our findings at the recent QRCA However, we were somewhat surprised at the conference in Atlanta. As “newbies” to using high level of user involvement needed to code software as part of the analysis process, Jeff transcripts or “slice and dice” content before and I provided conference attendees with first the information could be analyzed. impressions of nine different software tools so that fellow consultants could consider what software might work best in their own analyses. New software has the potential Covering nine different products, we opted for for yielding better end results a broad overview of all, not a deep exploration of any single product. by providing the consultant Although we secretly hoped to discover that software could magically do the work for us, with a new way of looking there really is no way to make the work disappear. at the information. By However, we did find software tools with the potential to make the work easier and the end demonstrating relationships results better, as long as the researcher is willing to overcome the steep learning curves we experienced. between pieces of information, The potential for better analysis stems from the the researcher is given the software’s ability to keep all relevant information in one place. In other words, the researcher can opportunity to investigate retrieve all respondent quotes addressing a specific topic and view them at the same time on the and report findings from computer screen. Assuming the transcripts have a new perspective. been thoroughly and accurately coded, this allows the consultant to conduct a more complete analysis with all pertinent information easily To determine which software to explore, we accessible. Additionally, the software has the conducted a Google search and asked fellow potential for yielding better end results by QRCA members for recommendations. We providing the consultant with a new way of also sought to include a range of cost options, looking at the information. By demonstrating including freeware. Finally, we purposely looked relationships between pieces of information, the for options for PC and Mac platforms. researcher is given the opportunity to investigate We test-drove the following software, listed and report findings from a new perspective. alphabetically: The potential for easier analysis comes from • ATLAS.ti the software’s ability to manage the flow of • Ethnograph v5.0 information, eliminating the manual shuffling • KDA/Revelation through page after page of transcript to find the • HyperRESEARCH perfect quote that you recall hearing in the third • MAXqda2 focus group. Instead, the researcher can call up • QDA Miner all respondent comments on a specific topic and • Qualmetrix easily choose the verbatim that best represents • Transana a particular viewpoint. • XSight However, these software tools are not magic. First, some of the products have very steep Code and Go learning curves. Often, we found the user Most of the software we tested works in a similar interface to be intimidating. Jeff and I “code and go” fashion. The researcher begins by agree we would need to commit to a lot of uploading a transcript document into the software, learning time (and patience) before we would which means a transcript must first be completed feel comfortable putting the software to work for the qualitative sessions. The user then “codes” in our own analysis. Additionally, the software the transcript by assigning main themes to relevant does not speak our language as qualitative respondent quotes. In most cases, we found research consultants. Software terms such as the coding processes fairly intuitive. (For pencil- “nodes,” “analysis frameworks” and “identifier pushing report analyzers like us, think of coding sheet templates” are foreign to our vocabulary. as highlighting text with a different color pen
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 61 Can Software Help? CONTINUED
for each topic.) Once transcripts are coded, the researcher can look at all relevant information by specific code or main idea. The software tools offer different outputs, ranging from a simple frequency count by code to the preliminary framework for a Word or PowerPoint report. Here are our brief, initial thoughts based on our testing of these “code and go” type software options. • XSight. Jeff test drove the XSight software and described it as having “all the bells and whistles.” This software is the only one tested that includes “tags” that provide reminders to the researcher of particularly good or humorous quotes to include in the report. This also is the only software we found that can deliver a preliminary report framework in Word or PowerPoint. • HyperRESEARCH. This is the only software of this “code and go” type that offers versions for both PCs and Macs. I like the way that graphics can be imported into the software to be easily coded and incorporated into the analysis. • MAX qda2. This software offers teamwork capability so that different users can work on the same master version. Also, it can be used with add-on visual modeling software for improved graphic presentations. • QDA Miner. Jeff found this software to be a “poor man’s” version of XSight (which is not meant to be disparaging). The software has fewer bells and whistles and does not
62 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED Can Software Help?
pretend to be a project manager, The software tools offer different outputs, so it is priced accordingly. Jeff found this product to be less ranging from a simple frequency count by complicated, as portrayed in its friendly attitude toward code to the preliminary framework for a new users. Word or PowerPoint report. • ATLAS.ti. I found this software a good tool for visual thinkers, as the networks between codes are visually represented in the analysis process. Beyond this, I was very comfortable coding with this tool. • Ethnograph v5.0. With this software, I found the filters and screens to be a bit daunting. Moreover, it had a very “quantitative” feel with High Quality both the analysis process nationwide data collection and the look of the output. to support moderators and consultants Different Twists A few of the software tools CATI TELEPHONE SURVEYS: Local, statewide and nationwide telephone we viewed offer different twists. surveys. Any size. General public, targeted populations and hard-to-get Transana is freeware that helps respondents. CATI programming. Multi-cultural and bi-lingual interviewers. researchers organize video and Training, interviewing, monitoring and supervision all designed to provide audio data. While the user still a high completion rate. works from a written transcript, the software also contains a window FOCUS GROUPS: Professional focus group facility in Gainesville, Florida so the researcher can simultaneously and professional focus group environment in Ocala, Florida. Also arrange, watch the video. The researcher recruit and coordinate focus groups anywhere in the United States. marks “keywords” and then can set Note-taking, written and typed transcriptions, audiotaping, videotaping searches for “keyword” matches. (VCR tape of DVD), real time video streaming to off-site computers, Jeff noted that Transana offers and translation into English from any language. Professional research a simple, easy-to-use interface in environments created in focus group facilities with one-way mirrors or both Mac and PC versions. His simulated at hotels, conference centers and community rooms with closed only wish was that the software circuit TV in areas where no professional research facilities are available. incorporated Video Marker capabil- ities, which enable the researcher FIELD SERVICE: Nationwide network of field interviewers for intercepts, to easily access video “bookmarks” exit/entrance interviews, mystery shops, observations, testing, evaluations, noted during qualitative sessions. enumeration, auditing, ethnography and mapping. Researchers use these “bookmarks” to easily locate pertinent video clips for analysis or presentation. “Anywhere in the United States KDA/Revelation is another soft- “Anywhere in the United States ware tool that offers an additional -- Rural,Rural, Urban,Urban, SmallSmall Cities,Cities, MetroMetro Areas”Areas” variation — it is an online collection tool, as well as an analysis tool. The researcher gathers information CONTACT: Ken Lyons, VICE PRESIDENT online from blog-like in-depth PERCEPTIVE MARKET RESEARCH interviewing formats. In addition 3615 SW 13th Street, Suite 6 to answering questions, respondents Gainesville, Florida 32608-3540 can also fill out forms and upload 1-800-749-6760, ext. 4012 documents or images. The researcher E-mail: [email protected] “tags” or codes the different Instant on-line quote request form on our WEBSITE at responses. These “tags” are then www.pmrresearch.com filtered by pre-set criteria. The software gathers responses into data
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 63 Can Software Help? CONTINUED
sets to download into reports. Jeff noted that a drawback software in addition to his or her standard analysis. The to this tool is that it cannot be used as an analysis software works by comparing text to an “emotion tool for interpreting transcripts from other sources. dictionary” based on many years of English language Qualmetrix offers researchers yet another twist. This research. Although coding is not required, the text must software is positioned as a consultant’s supplemental be prepped by cutting the transcripts into relevant analysis tool that detects the strength of emotions in analytical chunks (e.g., responses to a particular text. The intent is that the researcher would use this question, divided by market segments). This enables the software to provide a “cleaner” emotional viewpoint on a specific area. The researcher uploads the text into the software, and Qualmetrix does the rest. The process feels somewhat like sending transcripts into a “black box” to see what results will come out the other side. The output is a graphic representation of the intensity of emotions. Jeff thought this would be an interesting tool to help researchers detect subtle emotional nuances that might not be noticed by the researcher simply taking responses at face value. What’s Next? The potential for easier and better analysis certainly intrigues both of us, which merits our further investigation of these software tools. Perhaps you might explore these tools to choose the best option for your own analytical process. Maybe it is time to learn a new bag of tricks! Try It Yourself Except for Qualmetrix, all of the tools we tested offer free down- loads for limited use. For further information and demo downloads, visit the following websites.
SOFTWARE WEBSITE
ATLAS.ti www.atlasti.com
Ethnograph v5.0 www.qualisresearch.com
KDA/Revelation www.kdaresearch.com
HyperRESEARCH www.researchware.com
MAXqda2 www.maxqda.com
QDA Miner www.provalisresearch.com
Qualmetrix Still in development
Transana www.transana.org
XSight www.qsrinternational.com
64 QRCA VIEWS SPRING 2007 www.qrca.org WHY DID 250 COMPANIES MAKE CCR THEIR RESEARCH FIRM OF CHOICE FOR OVER 1500 PROJECTS IN 2006? TRUST.
Give us a call today for:
Telephone Interviews • Focus Groups • One-On-One Interviews Spanish Language Interviews • Mall Intercepts • Coding/Data Entry Mock Jury Panels • In-Store Interviewing • Taste Test Interviews Mystery Shops • CATI Programming
In Houston In San Antonio In Phoenix Miguel Pantoja Jason Davis Y-Vette Cave Sajan Pillai Kevin Hagelstein June Allen 877.530.9646 877.599.7025 877.906.0666 [email protected] [email protected] [email protected] • BUSINESS MATTERS •
To Hire or Not to Hire? Considerations for Expanding Your Business
B Y D AVID A. KALMAR, PH .D. Customer Strategy Consulting, Inc. • Princeton, NJ • [email protected]
A ND A BBY L EAFE Customer Strategy Consulting, Inc. • Princeton, NJ • [email protected]
aybe this has happened to you: One day, the phone is ringing off the hook, the projects are pouring in, and work is piling up faster than M you can manage. “If only I had some help,” you moan, as you pour yourself the 14th cup of coffee of the day. “Maybe I should hire someone…” For many entrepreneurs, the decision to go from being a one-person shop or husband-and-wife team to an employer of others can be one of the biggest steps in Successfully bringing the evolution of their business. When hiring decisions are made well, they can enable resources on board you to expand your business, increase profits and manage the inevitable ups and requires answering two distinct questions: What do I need? Who do I want? Answering each question will require a bit of self- assessment — you will have to act as your own management consultant to decide what will be best for your business.
66 QRCA VIEWS SPRING 2007 www.qrca.org downs of the marketing research industry. When As with investments, decisions are made poorly, however, the fallout can cost you time, sleep and even clients. Some maintaining flexibility has would argue that, in our service industry, hiring decisions are all the more critical because, unlike a cost. The more flexibility makers of widgets, all we have to offer our clients you want, the more you will are our expertise, knowledge and skills. Essentially, our brains represent our only assets and our core have to pay. equity, so we had better make sure we have the best brains on staff. least amenable to “outsourcing” or obtaining Successfully bringing resources on board requires additional resources to help execute. But we do answering two distinct questions: What do I need? know qualitative researchers who outsource both, Who do I want? Answering each question will due to personal preference, time considerations require a bit of self-assessment — you will have and/or the availability of low-cost alternatives to act as your own management consultant to for executing them. decide what will be best for your business. Six Critical Dimensions to Evaluate Determining Your Needs As a husband-and-wife team, we have owned If you have gotten this far, you have already our own marketing research company since decided that you need more help. The first 2002. Our intent was never to be a two-person question is, what kind of help? To answer this, consultancy — we both like the camaraderie you will need to break down your business and, frankly, the financial gains to be realized process into its components. Evaluate each from building a mid-size business. Yet, as our process component for three factors: workload grew, we were faced with what we • Do I like doing it, or do I wish somebody thought would be an easy task — hire our first else were doing it? employees — until we decided to do it. Over the • Do I have time to do it, or is my time course of four years, we have hired temps, interns, better spent elsewhere? contractors, part-time and full-time employees, • Can I make my margins (the professional all in the name of expanding our business, and services fees I need and want above and we learned a lot of lessons along the way. beyond my out-of-pocket expenses) doing We have come to realize that there are six key it, or is it really a poor use of my time? dimensions that must be evaluated when making Just because you like something or are good a staffing decision. at it doesn’t mean that it makes good business • Cost. The most visible and easy-to-evaluate sense for you to be doing it. In our own business, dimension is cost. You know what the client David likes handling IT around the office, and is paying, and you know how many dollars although he is an expert at it, it really is a poor use per hour bringing in staff can cost. So, the of his time, because we are not in the technology math seems pretty simple. Oh, if it were services business. Abby likes managing the finances only that easy. We usually think of “cost” and would happily do it all day, but her time is as the out-of-pocket expense, whether an better spent managing client relationships and hourly rate, a day rate or a flat fee. This conducting qualitative research, so we have side of “cost” is the most visible, but the retained a financial advisor to help us. hidden costs can easily make the difference Qualitative researchers are commonly drawn to between a profitable engagement and losing the field because they enjoy moderating in one or money. Hidden costs can include your lost more of its many versions. But liking one aspect of management time, additional costs such as the research project does not excuse us from being agency fees, “loaded costs” (federal and state responsible for others. A typical engagement might taxes, FICA, unemployment insurance, etc.), contain the following components, all of which overhead (health care, additional computers are candidates for handing off to someone else: and office space, etc.) and, perhaps most • General project management unpredictable, productivity. • Developing the discussion guide • Productivity. You have a pretty good idea • Recruitment of how long it will take you to moderate a • Moderating two-hour focus group. You probably even • Writing the report have a pretty good idea of how long it would Abby enjoys moderating the most but likes take you to prepare the report. But how long reporting the least. Of the research components would it take someone else to do the same outlined above, those may be the two that are thing? How do you weigh the productivity
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 67 To Hire or Not to Hire? CONTINUED
term, when temporary staff goes, so goes their acquired knowledge, both of the project and your institution. How important is this to you? How much do you value this type of intellectual capital? • Intellectual capital. Intellectual capital is the sum of the knowledge acquired by your company that is used to deliver value to your customers. In businesses whose functioning depends upon a comparatively simpler skill set, such as transcribing, you may not find it necessary to pay a premium for staff to obtain the level of services you need. Similarly, when tasks are fundamentally the same after ten years as after one year, there is little incremental knowledge about the category or the organization; hence, there is little accumulated intellectual capital. In custom marketing research, however, those types of knowledge can quickly become a significant part of an employee’s value. Consider, for example, the rapid escalation in salaries that can be observed as a worker gains experience in this field: According to Sunita Manwai of Forum Personnel in New York (a specialist in placing marketing researchers), a of a $20/hr. temp versus the productivity of typical new employee, straight out of college, a $40/hr. temp versus a $150/hr. contractor? can expect to earn around $40,000 a year To do so, you will need to put together your and, within two years, can be making between historical project knowledge with a best guess $60,000 and $65,000. Clearly, in the field on your prospective staff member’s efficiency of marketing research, acquired knowledge to come up with the actual value of each hire. is valuable. As an employer, then, should you With an estimate of productivity in hand, wish to capture the value of the intellectual you can now focus on when you will need capital acquired by your staff while on your the help. If you are like most consultants, team, what you will look for is a commitment. you will need it when projects hit, not all • Commitment. When you tie up your money the time. What you will want from your in a certificate of deposit, you generally get a prospective staff, then, will be flexibility. better rate of return from a longer commit- • Flexibility. As with investments, maintaining ment. The same principle applies to hiring flexibility has a cost. The more flexibility staff. So long as you have the work for them you want, the more you will have to pay. to do, your labor cost per work unit will be You pay not just for employees’ skills but lowest if you hire permanent employees. But also for their willingness to take a job with doing so raises a whole specter of issues, virtually no promise for the future. However, ranging from loaded costs (taxes, FICA, etc.) in the field of marketing research, and to overhead to human resources issues to especially with a small company, flexibility legal and regulatory compliance. is critical: work ebbs and flows, and you While those sound daunting enough, one want to avoid (at all costs, almost) running issue rises above the others — meeting payroll your business in the red during slow times. on a bimonthly or monthly basis. With a small Temporary staffing gives you that flexibility, business, adding one or two projects can boost but it comes at a price — continuity. the workload and keep everybody at the office • Continuity. It is great to be able to throw late, but taking away one or two projects can short-term resources against a short-term turn an otherwise profitable enterprise into a project. But it can become a logistical money-losing one. Hence, before you decide to challenge when your Tuesday staff is not bring on permanent staff, consider carefully the available for questions on Wednesday, or commitment you are making to both yourself your morning staff is not available when and to the new employee who will be depending the client calls in the afternoon. In the longer upon you for his or her livelihood.
68 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED To Hire or Not to Hire?
Hiring Options — Date, with Excel to run simulations on a model he had Go Steady or Get Hitched? developed. The task required someone with solid Once you have a good idea of what your true (although basic) Excel skills, but since Abby had needs are and what dimensions are most and just had a baby, she was unavailable to help out. least important to your business, you can move We used a local temporary agency to hire someone ahead and begin looking for the right person. who we were assured was an Excel expert — he While the “right” person will vary by business had achieved perfect scores on both the basic and (based on the required skill set, the level of commitment and even the personalities of the people involved), a basic review of the options available demonstrates their relative strengths and weaknesses. As you are consider moving ahead, think about approaching the project as if you were marketing your company to potential employees. You will need to consider channels (where to look for people), messaging (what to say — and not say — to them) and price (salary or fees). The approach can and will differ based on the type of employee you decide to search for. Temporary Workers At the very low end of both the cost and commitment scales, small-business owners can hire temporary workers to manage overflow during particularly busy times. Temporary workers come in all shapes and sizes, which can also be a boon to a qualitative researcher. Today’s staffing firms provide people who are versed not only in answering phones and typing the occasional letter but also in accounting systems and software, Excel, PowerPoint and any other computer application imaginable. Hiring someone to pay the bills or put slides together based on your notes can free up your time for things like moderation or proposal writing that demand your expertise. There are also drawbacks to a general temporary worker, as we learned last winter. David was working on a particularly demanding quantitative study, and he needed someone to work
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 69 To Hire or Not to Hire? CONTINUED
advanced Excel skills tests commonly administered (our local Greek diner), we also wanted to hire by employment agencies to assess a candidate’s several Princeton University students as interns technical skills. He lasted one day. Once he arrived, to help us in times of overflow and, during the he admitted to David that he had not used Excel slower times, to work on those projects we never in a few years and was not really all that proficient. seem to get to ourselves. Our intern experience If you decide to hire a temporary worker, the has been a smashing success. By tapping one of best channels are usually local temporary agencies, the greatest universities in the country, we have so a simple internet or phone-book search should hired some of the smartest, most educated and point you in the right direction. Temp agencies well-traveled young people we have ever met. will certainly craft your message for you, but In fact, one freshman recently spent the summer remember to be very specific about what you need. with us and, after only three weeks, was doing For example, should the person have computer the work of a senior research associate. experience? If so, with what programs and what There are also drawbacks to hiring interns level of skill? Expect to pay by the hour, at a that you should take into account. One major rate that varies depending on your area of the drawback is scheduling. Interns must fit their country or world. working hours around their classes, other school activities and the academic calendar. As a result, your intern may not be available to work when you need him or her most. Also, this may be your So long as you have the work intern’s first experience in an office environment; for them to do, your labor so, for better or worse, you may be training him or her on the most basic tasks, such as how to cost per work unit will be take a phone message or file reports. lowest if you hire permanent To locate the best college students, you need to hunt for them in their native habitat — the employees. But doing so internet. Job-hunting has changed a lot since we were in college, and most colleges — if not all raises a whole specter of — utilize their own website to list part-time jobs issues, ranging from loaded and internships. Start with the websites of the schools near you, and look for information about costs (taxes, FICA, etc.) to the student employment office. When you do post your position, make sure that you stress overhead to human resources that the position is one where the intern will be issues to legal and regulatory learning, not just doing a “job,” in order to get the best students. One trick for attracting the compliance. best candidates is to figure out what the highest- paying jobs at the school pay, and then offer $1 or $2 more per hour. To you, it will be a relatively Interns small amount of money, but it will definitely What could be better than smart, motivated college compensate the student for the inconvenience students who are willing to work for free (or close of leaving campus to come to your office. to it)? Hiring interns is another option that allows a small business to expand capacity without the Contractors cost or commitment of full-time employees. It also Contractors generally have more relevant skills allows an employer to “test drive” students while and experience than a temporary employee. They they are still in school and to determine whether can be a wonderful solution if you decide that you to hire them upon graduation. Students can also be need help with components of the actual research eager to learn and enthusiastic, and they provide process (requiring both intellectual capital and you with a rewarding interaction as you train the productivity) but you do not want the commitment next generation of researchers. The intern arrange- of a full-time employee. Contractors can take many ment also benefits the students, who often earn shapes and forms and fill many different roles, college credit and gain valuable experience that depending on your business needs. People who will help them land their first job after graduation. are transitioning away from full-time qualitative When we decided to leave the streets of Jersey research (such as moms who are taking a child- City for the bucolic country lanes of Princeton, rearing break or practitioners looking toward we definitely knew that we wanted office space retirement) can be terrific sources, as can fellow within walking distance of the university. Along QRCA colleagues who may be experiencing a with proximity to Starbucks and Zorba’s Brother slow time just as your calendar is filling up.
70 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED To Hire or Not to Hire?
Other than the task of identifying the right skills set, give as much information as you possibly person, the main drawback to the contractor can. The rate you will be charged will also vary relationship is one of cost. It will be critical to widely; it may, in fact, be commensurate with find the person who can perform work to your what you charge clients yourself. standards without you sacrificing any or all of the profit you would make on a particular Part-Time Employees project. Furthermore, although you can gain When they are experienced researchers, part-time additional intellectual capital by working with employees provide many of the same benefits a contractor on a repeat basis, if this person and drawbacks of contractors, with the added has direct access to your clients, you will want complication of an additional person on your a signed agreement preventing him or her from payroll. If you hire a part-time bookkeeper or soliciting or accepting business from this client administrative assistant, however, you are not for a defined period in the future. (Of course, necessarily adding intellectual capital; however, we are not attorneys, so you will want to check you may be increasing your overall productivity with an expert on this type of non-solicitation by shifting these types of duties onto someone or non-competition agreement.) other than yourself. While your costs will also Contractor arrangements can be more difficult go up by hiring this type of employee, you may to come by, since there is no exclusive outlet for end with higher profits since you are freed up people searching for these types of arrangements. to do the kind of work for which you are most Your own list of contacts is a good place to start, highly compensated, such as moderation. as are the professional associations you belong Any employee working less than 35 hours per to. Your message will need to be tailored to week usually qualifies as a part-time employee. your specific needs, and be upfront about your This provides for a number of options when expectations. If you need someone with a specific hiring part-time employees, ranging from someone
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 71 To Hire or Not to Hire? CONTINUED
who comes in for only a few hours per week or you know this already, which is why you are in month, to an employee who functions much like the position of needing to hire in the first place!) a full-time staff member but works only four days Lustig has two recommendations for businesses per week. Usually, you are not required to provide looking to add qualitative researchers to their the same benefits for part-time employees as for organization. First, consider hiring someone full-time employees, which can also represent a with minimal skills who is willing to train. Such cost savings beyond simply the smaller salary employees can be more affordable in the short point of a full-time employee. term, and you have the luxury of training them to To go about actually hiring part-time employees, do things exactly your way. For some, however, the steps can be very similar to those that you take training is less a luxury than a chore. In this for hiring a temp or contractor, except that you are case, says Lustig, “If I were running a qualitative instead looking to make a longer commitment. If business, I would tag-team with another you need someone with office or bookkeeping independent [consultant] and have him or her skills, an employment agency may be the way do my overflow.” This arrangement allows for to go. If you need a researcher, break out your flexibility and intellectual capital — and sounds contacts and start networking. Again, the cost for a lot like a “contractor” arrangement! hiring these types of employees can vary widely. When you are ready to take the step of hiring a full-time employee, be prepared for the process to Full-Time Employees eat up a lot of your own time. Between crafting When you are ready to make an investment in your job description, posting it in the appropriate your company’s productivity, continuity and places and reviewing the resumes that come in, intellectual capital, you are ready to hire a full- you may be exhausted before you have even time employee. While this decision may represent conducted the first interview. When casting your a big initial jump in your operating expenses, if net, keep in mind that wider is not always better. you have the work to occupy a full-time employee, Sites like Hotjobs and Monster will definitely your productivity will go up, and your cost per result in dozens, if not hundreds, of resumes in work unit is likely to drop. your inbox, but they may not attract the most Hiring a full-time qualitative research consultant qualified candidates. On the other hand, more may not be as easy as it sounds. Susan Lustig at specialized sites or publications will result in Forum Personnel notes that experienced qualitative fewer applicants, but those who do apply may researchers who are interested in working for indeed be better qualified. Don’t forget about local someone else’s business rather than going out on universities, whose graduates might be looking their own can be difficult to recruit. Although to stay local, or even your own alma mater. Susan notes that she works with “a lot of suppliers In your job posting, keep in mind that your role who would love to have a moderator on staff,” is not just to detail your list of needs — you also the qualitative research consultant is often likely need to sell the position to job hunters. To that to believe that it is more lucrative for him or her end, we always include a list of “what we offer” to work as an independent consultant. (Of course, in our job listing and describe not only training
Evaluating Your Options Part-time Full-time Temps Interns Contractors Employees Employees
Cost
Productivity
Flexibility
Continuity
Intellectual Capital
Commitment
72 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED To Hire or Not to Hire? opportunities but also our Our intern experience has been a smashing success. By company culture and philosophy. Including salary is optional, and tapping one of the greatest universities in the country, given the wide range of salaries in the market (depending on the we have hired some of the smartest, most educated experience level of the employee), and well-traveled young people we have ever met. leaving it out gives you the most flexibility if and when you decide to extend an offer.
Other Options The options above and beyond what we’ve outlined here could fill another article. One partic- ular source of manpower — outsourcing companies — bears a brief mention here. Nowadays, there are many suppliers to the marketing research industry who will serve as outsourced contractors for field management (including multi-city engagements), tran- scription, video editing and other services. They can provide a reasonable solution for many researchers who simply aren’t sure of their long-term needs but would like help in the short term without embarking on an exhaustive search process.
A Few Final Thoughts While a reasoned, systematic and business-based analysis of your needs and the available options is incredibly valuable in helping you make the right staffing decision, don’t forget to trust your instincts, too. The right employee or contractor not only has to be “right” on paper; he or she needs to feel like a fit with your company. For us, that meant getting someone who is enthusiastic, can learn independently without a lot of hand-holding and, above all, can have patience with two bosses who can be both easygoing and cantankerous, often in the span of one afternoon. As we learn from our hiring decisions over the years, we are getting better at identifying the people who best fit our ideal profile, and so will you.
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 73
• TRAVEL & LEISURE • Free Money (Well, Almost)
B Y M ARY B ETH S OLOMON Solomon Solutions • Jersey City, NJ • [email protected]
today’s business traveler, there is simply no reason not to join a travel rewards program… or several programs. For They are free, they are easy to join (see www.webflyer.com for dozens of online signup forms), and nowadays they offer a world of options beyond the old reliable first-class flight upgrades. With the money you spend on travel, the least you can do for yourself is With the money get a few perks! But given the myriad of programs out there, which you spend on ones are right for you? Generally, reward systems start at a rate of one-tenth of one percent: travel, the least Spending $1,000 gets you 1,000 points, which you can redeem for you can do for $1 cash value. If you are careful, though, there are lots of ways to build up your points faster and redeem them in ways that fit your yourself is get a lifestyle. Are you looking for exotic getaways or just to sleep in a nicer bed in your hotel chain of choice? Would you like to get into few perks! But that sold-out show or to the front of the movie line? Jonesing for given the myriad that hot new face cream or video-game controller? Or would you rather just have some extra money in your pocket? Whatever your of travel rewards preference, you can make the money you spend on travel work programs out harder for you. there, which ones Take the Cash The simplest and most straightforward benefit is cash back — you are right for you? spend money, you get money, and the more you spend, the more you get! Sound too good to be true? Well, like long-distance phone plans, cash-back systems tend to have complicated rules that can be difficult to figure out, but it is worth the trouble if you can find the right one for you. To take the heavy lifting out of the calculation, there is an online tool (www.creditcardtuneup.com) that can take your estimated monthly expenditures in a variety of areas and tell you which credit card is the best fit for your spending habits. If you are not in a hurry to see the money come back to you, check out www.NestEggz.com, which deposits your savings directly into an IRA or other retirement plan. Rebates in the program can be substantial — up to eight percent for in-store purchases or thirty percent for online purchases, with vendors ranging from Staples and Home Depot to Eddie Bauer and The Sharper Image. As always, make sure you read the fine print. With MBNA’s WorldPoints, for example, you get a more favorable rate by trading in your points for widely accepted Visa or MasterCard gift certificates than for cold hard cash.
76 QRCA VIEWS SPRING 2007 www.qrca.org QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 77 Free Money (Well, Almost) CONTINUED
If you subscribe to the principle “the more you give, Before you fill up your rental car, see if your credit the more you have,” reward programs make it easy card gives you a rebate on gasoline purchases. For for you to donate your points or miles to charity. Most example, the Discover Gas Card offers rebates of five networks are linked up to a variety of worthy recipients. percent on gas and auto-maintenance spending, while They don’t go out of their way to maximize the exchange the Citi Diamond Preferred Rewards Card gives five rate for charitable giving, however. Oh, and if you points for every dollar spent in supermarkets and drug were thinking about getting a tax deduction for your stores, as well as at gas stations. (One caveat here: The charitable contribution, unthink about it. You may Diamond card is limited to 75,000 points in a calendar both (you and the charity) be better off if you keep the year, so it may not be the right card for high rollers.) points for yourself and just write the charity a check. If you are thinking about travel that is more for pleasure than business, MBNA’s WorldPoints can be The Journey Is the Reward redeemed on Celebrity Cruises for everything from a (Chinese Proverb) terry cloth robe to a Caribbean trip for two. For those who prefer the tried and true, frequent- If you don’t already know about Starpoints from flier programs usually require about 25,000 miles Starwood Hotels, you should. It is one of the most before you can redeem them for a free domestic round- well-known travel reward programs around. In trip flight in coach class. For hotel-loyalty plans, the addition to the standard benefits like free nights in rule of thumb is that a $1,000 expenditure leads to Sheraton, Westin and W Hotels (with no blackout a free night's stay. It is worth noting that you can redeem dates) and spa treatments, you can get exclusive access American Express Membership Rewards points through to PGA tickets and merchandise. You can also rack their site (www.americanexpress.com/travel), and since it up the Starpoints more quickly with a Starwood is affiliated with Travelocity, you can fly on the airline Preferred Guest American Express card or MasterCard. of your choice. The best source of up-to-date information The same goes for Hilton HHonors, which encom- on all frequent-flier programs is on the user-maintained passes over 2,700 Hilton Family hotels worldwide. FlyerTalk Wikipedia (www.FlyerTalk.com/wiki).
78 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED Free Money (Well, Almost)
Keep in mind that some airlines are organized 1960s. If you remember Green Stamps, you into alliances that honor each other's frequent- probably remember how much time and trouble flier programs. For example, oneworld Alliance it took to lick and paste them into stamp books, (www.oneworld.com) covers eight major airlines, which is why S&H has gone digital with S&H including American Airlines and British Airways. GreenPoints. You can earn them from all kinds of And do not forget that different programs have businesses, although travelers will be particularly different expiration rules. There is nothing like interested in the multitude of restaurants accessible saving up 90,000 miles toward a free trip to New Zealand only to discover that the airline has reset your odometer to zero! (Not that I’m bitter or anything.)
Let Them Entertain You Even the most dedicated road warrior comes home once in a while, and if that home is in a major metropolitan area, you might like the benefits of the new INSIDE Rewards cards from American Express. They started off with the IN:NYC card and have expanded their offerings to include IN:CHICAGO and IN:LA. These cards focus their rewards on the finest things each city can offer — restaurants, shows and special events. For example, last year I was able to cash in points to get excellent seats to a sold-out performance of Henrik Ibsen’s Hedda Gabler, starring Cate Blanchett. Sometimes points can get you something money cannot buy! If you are a Broadway buff, you can also earn VIP treatment and access to special events and memo- rabilia through Audience Rewards (www.audiencerewards.com), a loyalty program dedicated to active theatergoers. No matter where you live, you are probably not too far from a Blockbuster, and you can redeem points from your eligible Visa Extras card (the only major rewards program for debit cards) for a Blockbuster gift card or an iPod (in case you are the last person on your block without one).
Retail Therapy The grandparent of shopper rebate programs is S&H Green Stamps, which started in 1896 and reached its peak of popularity during the
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 79 Free Money (Well, Almost) CONTINUED
through the S&H Dining Connection. S&H’s Building Up Points Faster best-known rewards were mostly consumer products, and that is still true today. For 56,900 Point and Purchase GreenPoints, you can pick up a Samsonite Business One Mobile Office, while 65,900 Increasingly, you can rack up points for online GreenPoints will get you a George Foreman purchases from a wide swath of vendors, from The Next Generation Grill. A NETGEAR Wireless Gap to Barnes & Noble to Circuit City. Often, Router will cost you 108,900 points. you can earn ten or twenty points per dollar spent, If you already drink Starbucks coffee, you on purchases you would make anyway! Just make might as well use a Starbucks card to pay for sure you start from the website of your rewards it. It is quick and easy, and although there is program, enter your membership number, and then no formal point reward system, sometimes they click from there to the merchant’s site. CitiBank’s do throw a few extra dollars on the card. But ThankYou Network (www.thankyoumerchants.com) if Starbucks has gone beyond just a source is a good example. of morning caffeine for you and has become a way of life, you will be interested in their Don’t Delay! Duetto Visa card, which gets you coffee perks You can get big bonuses through American Express on regular purchases, and vice versa. This Membership Rewards if you keep abreast of their card also generates charitable donations via limited-time promotions. At time of writing, the the Starbucks Foundation. program offered double points for one-of-a-kind The major reward programs have their own adventures from Signature Days and triple points for merchandise catalogs, most featuring flashy stays at Hilton Hawaii and purchases from online consumer items like Bose speakers and Movado retailers such as Red Envelope and Toys ‘R’ Us. watches. But how about your bread-and-butter business purchases? A couple of loyalty Too Much Is Never Enough programs specialize in home- or small office If you have a teenager, you already know that goods. Staples Business Rewards offers two MTV wants his or her heart and mind. But you levels of service: regular membership and may not know that MTV wants to give your a Gold tier that can save you five percent college-age child a Citi mtvUTM card, which on each purchase, plus qualify you for free pays five points to the dollar for entertainment shipping and members-only sales events. The expenses at movie theaters and bookstores and catch is, you can’t get more than thirty dollars gives a 25-point bonus each month just for back in any given quarter. The Office Depot paying the bill on time. Can bribery promote Platinum Business Visa, which supports multiple fiscal responsibility? It just might! cards per company and has a wider range of Remember, though, reward points are no bargain benefits, is a better solution if your office-supply if you are paying too much for the merchandise to spending is more than minimal. begin with. As with frequent-flier programs, you Meanwhile, for all-around tools to support or will get the best value for your money if you first grow your small business, you are unlikely to do select the right product from the right merchant and better than the American Express Open Small then see what reward system matches that purchase, Business Network, which has partnerships with rather than the other way around. suppliers like FedEx, information on business loans and much more. Be careful: joining these clubs can be addictive. When the rewards start rolling in, you may keep signing up until you start having trouble keeping track of them all. But, don’t fret. You can open up a free account at Points.com, a reward-system aggregator that keeps track of all your current points balances. Just make sure you are not having so much fun getting points that you forget to earn the money to pay for all the spending!
80 QRCA VIEWS SPRING 2007 www.qrca.org
• BOOK REVIEW •
Valuable Reading… Three Titles to Help You ExpandYour Capabilities
B Y K AY C ORRY A UBREY Usability Resources, Inc. • Bedford, Massachusetts • [email protected]
Handbook of Usability Testing Practical Guide to Usability Testing By Jeffrey Rubin By Joe Dumas and Janice Redish
nterested in expanding your services to isolate the specific aspects of a product that include usability testing? Need a quick study might mislead or confuse a target user. Usability I to get up to speed? Either of these books will testing is slow, quiet and focused on details; help you get started. during the session, you need to get inside the Both offer details on the nuts and bolts of how participant’s cognition. to plan, run and execute usability tests. You As with running focus groups, you must pay will learn how usability testers employ many a lot of attention to the care and feeding of your of the same skills as the qualitative researcher: participants. Both books give excellent tips for empathy, listening and observation, working creating a testing environment where people feel with clients to establish study objectives, safe, respected and free to speak their minds. research design, study sample definition and The Rubin book cautions you to speak with recruiting, as well as managing observers in participants before the session, if you have recruited the back room. them through an agency, to make sure they under- You will also learn about the unique skills stand what to expect. The author also suggests involved in usability testing, where the goal is strategies on how to interact with participants to to determine how well people understand and elicit complete and objective responses. can perform critical tasks with a product. Rather Both books go into detail on the need to train than gathering insights on the participant’s observers so they do not embarrass the partic- perceptions, opinions, beliefs and attitudes of ipants or otherwise derail the session. With a product, usability testers design a study to usability testing, the observers are often in the
82 QRCA VIEWS SPRING 2007 www.qrca.org same room as the moderator and the participants, As with focus groups, if so observers should “fade into the background” to keep the test productive. you do a good job setting Both books offer perspective on where usability testing fits into the qualitative researcher’s toolkit up the study, you will get (which also includes focus group moderation, in- valid results, although often depth interviews, task analysis and observations) and when it is appropriate to use each technique. you can’t predict what these You will learn the thought process and tactics involved in working with engineering teams to results will be. define study objectives and carry out sessions that produce reliable data. Both books also provide templates and checklists that will save inferential statistics. Use inferential statistics you time and ensure that you have the many only when appropriate and when you and your bases covered. client understand how to apply and interpret them. Instead of using a discussion guide, usability As with interviews and focus groups, usability test moderators give the participants a task list, tests produce huge amounts of data and insight. which is designed to explore the usability issues Rubin advises you to ignore politics when you of specific aspects of the product. As the partic- write your first draft and to report the raw truth ipants work through the list, they narrate their on how the sessions went. Dumas and Redish thoughts and impressions. These narrations, advise you to hold off on writing the report until combined with how well each participant succeeds you and your client have reached a consensus on or fails at a task, are used to determine the the product’s usability problems. They provide a product’s usability. Both books do an excellent very detailed step-by-step approach for consoli- job of describing approaches to creating a task dating, understanding and prioritizing the results. list, as well as providing examples. The task list As with other types of qualitative research, should be self-explanatory, as well as easy to the results from your sessions can be the first understand and follow: it should guide the outside feedback your client has received, and participants so they use particular product sometimes the news is shocking and upsetting. features without giving explicit instructions. It is not uncommon for a client to ignore or As with focus groups, if you do a good job reject your results, even if your study methods setting up the study, you will get valid results, were sound and your participants had the same although often you can’t predict what these results confusions. The Dumas and Redish book offers will be. Sometimes people breeze through tasks excellent advice on how to deal with the you thought would be difficult but stumble over organizational dynamics around usability unexpected areas. Both books advise you to be feedback. They describe techniques to minimize methodical (i.e., treat each participant the same resistance, such as having the client identify and have them cover the same product areas), potential usability issues and waiting to write while still allowing the participants to veer off your report until you have agreement among the task list when their exploration serves the the stakeholders on the problems the sessions study objectives. To keep each session uniform, surfaced. Another distinguishing feature of the authors recommend using a script for each usability testing is that it is most often done planned interaction with the participants. The with technology products, so learning how authors suggest holding your interview questions to communicate with engineers and get their until the end of a session, during a “post test” respect is a critical determinant of your success. where you and the observers review the session If you are interested in learning more about with the participants. usability testing, either of these books will provide Both books recommend collecting quantitative you with a solid background. Read both books and qualitative data. Usability studies most often to get a complete understanding of the usability involve a small number of sessions, generally eight testing process and skills needed. While neither to ten. The quantitative data you collect (e.g., book covers usability testing for websites, you time to complete a task, success or failure on a can easily adapt these techniques for this purpose. task, number of errors and number of times the The Rubin book, while older and more participants referenced help) can lend another expensive, is shorter and easy to read. The dimension to support the overall feedback, testing techniques detailed still hold true. The providing simple descriptive statistics. Both formality and staffing roles he recommends, authors advise you to stick with descriptive however, no longer apply because usability statistics and avoid using quantitative data for testing is more commonplace. Most companies
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 83 Valuable Reading ... CONTINUED
are unwilling to pay for a videographer and timekeeper. Unless there is a good reason to spend To make sense of your money on production, sessions are commonly target audience, Vitale not recorded and are run in a conference room with client observers sitting at the table with you advises researchers to read and the participant. what your customers read, The Dumas book is more comprehensive and contains many more templates and checklists to visit their stores, read their guide you through your first sessions. This book also contains valuable sections on how to handle magazines, watch their TV the politics of usability testing, as well as strategies shows and basically do for writing reports that your clients will read and appreciate. everything you can to understand their world.
range of research approaches from diverse disciplines such as anthropology, data mining, psychology and linguistics. These firms’ goal is to use the most effective techniques to learn about the consumer and to put consumer insights in the center of everyone’s thinking. As a case study, the author presents Dunkin Donuts, which was losing ground to Starbucks in the earlier part of this decade. They reor- ganized their company so that all strategic decisions — from menu options to store design and employee uniforms — were to be driven by customer feedback. Vitale outlines how Dunkin Donuts’ market researchers approached thousands of coffee purchasers through interviews, focus groups and anthropological techniques. They learned that customers loyal to Dunkin Donuts are fundamentally different from people who go to Starbucks. Loyal Dunkin Donuts’ drinkers see themselves as average people who “sweat for Consumer Insights 2.0 — How Smart their money” and want “a regular cup of coffee for a regular guy,” as opposed to Starbucks Companies Apply Customer Knowledge drinkers, whom they viewed as ambitious, to the Bottom Line pretentious, self- and work-obsessed yuppies. By Dona Vitale The researchers learned that Dunkin Donuts’ customers appreciate the shop’s utilitarian and Forces such as globalization, immigration, efficient atmosphere. As a result of this knowledge, the influence of boomers, gen-Xers and new they focused their energies on increasing efficiency; communication technologies have created a society they got rid of the espresso, added fresh food of unique subcultures. Successful companies know to the menu, increased the store’s comfortable they must go beyond traditional market research seating and kept the familiar pink-and-orange to really understand how to create products that color scheme. So far, the strategy has worked well. will appeal to very distinct niches, each with its Because consumers process marketing infor- own world view and identity. mation in a variety of ways, Vitale recommends Vitale describes how insights-driven companies employing multiple models to make sense of have a culture that encourages information sharing consumer behavior. One approach — the social and collaboration. Using a “bricolage” approach psychological model — links behavior to a to research, they “tinker around and build from consumer’s aspirations. In this framework, you whatever materials may be available” to follow focus on the consumers’ values, the people they the cultural and environmental trends affecting admire, how they see themselves and how a target audiences. This means borrowing a wide product either enhances or detracts from this
84 QRCA VIEWS SPRING 2007 www.qrca.org CONTINUED Valuable Reading ...
self-image. The Economic Utility Model looks people explore sensitive topics and negative at the relationship between what a consumer is emotions. ”Personification” is a projective willing to pay versus the item’s perceived value. technique where you ask a consumer to draw The Freudian model examines how our deeply a picture of a person who embodies a brand or rooted needs find expression in purchasing idea, such as Mr. Budweiser. Ask the customer to decisions (e.g., the middle-aged man who buys start with specific physical features and then dig a red convertible instead of taking a mistress). deeper with questions about the character’s Throughout her book, Vitale describes the interests, ambitions, values and dreams. Then, need to get into your customers’ shoes to really place the character in a scenario with opposing understand them. To make sense of your target brands to see what happens. audience, she advises researchers to read what Much of a customer’s emotional attachment to your customers read, visit their stores, read their a product can be detected through the words a magazines, watch their TV shows and basically consumer uses — daddy’s “toy” versus “favorite do everything you can to understand their world. tool.” Listening closely can expose indirect It is also important to understand everything criticism, such as when teens tell you a product there is to know about the product you are might appeal to their parents. studying. Visit the factory where the product Metaphor and projection are two additional is made, and learn about the supply chain it methods to elicit information from your con- follows on its way to the marketplace. Talk with sumer. Use metaphor to understand a consumer’s product salespeople, trainers, quality-assurance feelings towards concepts that are more sensory people and anyone who can round out your and that have not yet been put into words. understanding of how customers use the product. Vitale’s new book is an excellent primer on In the latter part of her book, Vitale outlines the emerging area of customer insight research. techniques to explore the feelings and needs of It would be a very good starting point for people your target audience. Ask a consumer to tell a new to the field, as well as a gift you might give story around how he or she uses the product, clients to help them understand your work. and then listen for the social and emotional triggers, as well as structural elements such as plot, main characters, conflict, themes, heroes, villains, problems and struggles. Ask whether the brand satisfied the customer’s needs, caused a favorable outcome or disappointed him or her. Vitale describes the use of projective techniques to help
Much of a customer’s emotional attachment to a Amy R. Riker (510) 978-2911 product can be TRANSCRIPTION SERVICES [email protected] detected through SERVICES AVAILABLE INCLUDE: the words a Live transcription consumer uses From cassette tape CD/DVD and video tape — daddy’s “toy” Audio file server upload versus his “favorite tool.” Call or email for a quote.
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 85
• EDITORIAL GUIDELINES •
Call for Authors: Publishing Opportunities ditorial content for QRCA VIEWS is managed FAQs about Article Submissions by an editorial team that includes the editor-in- Will I see my article before it is published in QRCA E chief, managing editors, copy editor, contributing VIEWS? Not always, as our tight publication schedule editor, and the features editors for each of VIEWS’ may not allow for author review of edited manuscripts. regular columns such as Qualitative Toolbox and Tech Talk. Will I be paid for the time and effort I put into writing VIEWS’ editors welcome QRCA members and the article? Like most professional association members of the marketing research community to publications, VIEWS does not pay contributors. submit article ideas or manuscripts for consideration. However, since the magazine is distributed broadly in We review each manuscript on an individual basis to the research community, you will gain a good deal of ensure that the article conforms to VIEWS’ mission and visibility as a result of being published in VIEWS. goals, as well as to the topic mix needed for each issue. Occasionally, we may save a manuscript to use in a Can I submit an outline for consideration before future issue. We reserve the right to edit any manuscript I submit the completed article? Yes, you may submit or to change the title. a 50-100 word description of your story idea to the Submissions should be objectively written and appropriate features editor. supported by case-study examples. Self-serving articles or those that promote a moderator’s or a research company’s expertise will not be published in VIEWS. My article was published in another journal or Please remember that it takes a good deal of time magazine. Can I send it to VIEWS? We do not have for our editors to read through all the submissions. the resources to deal with the complicated copyright We will let you know as quickly as possible whether or issues raised by articles that were published elsewhere. not your article has been accepted for publication in An article is considered “previously published” if it QRCA VIEWS. was published before in another print or online trade journal, magazine or newspaper. However, if you Submitting a Manuscript have substantially rewritten the article before submitting it to VIEWS, we will consider it. Should Please send articles via email as an attached Word file to this be the case with a manuscript that you are the attention of Lana Limpert at [email protected] submitting to VIEWS, please inform us. Along with • The preferred article length is 1,500–2,500 words. the rewritten manuscript, please enclose a copy of • To make sure that your manuscript can be easily the article as it was first published. identified and retrieved once it has been down- loaded in our “Article Submissions Folder,” your Word document/file attachments should be labeled Writing Style Guidelines as follows: • Write in the present tense and use the active voice as much as possible. Avoid the use of contractions, Brief title.Last name of author.doc i.e., don’t, wouldn’t, can’t, etc. • Please be sure to tag each page of your manuscript • Footnotes, endnotes or lists of references are not with a left header that identifies the article title necessary in a trade magazine such as VIEWS. and author’s last name and a right header with • Make liberal use of topic subheads to help readers the page number. scan your article and follow your main • In addition to the title of the article, the front page of points easily. your manuscript should include the author’s full name, • Please direct your queries and/or submit your full company name and address, phone number and manuscript to Editor-in-Chief Lana Limpert, email address. [email protected] • Please use only simple formats in your Word documents. Avoid using unusual indentations or Electronic Article Reprints tabulations as well as outline-style paragraphs with Authors who publish in VIEWS now have the subsets, boxes or other page graphics. When the opportunity to obtain a PDF file of their article as editing process has been completed, VIEWS’ graphic it appeared in the magazine. The cost to authors designers will format your manuscript so that it will for an electronic article reprint in PDF format is be attractive and easy to read. $50. Please direct your request by email to Eddie • Include your full name, company name and mailing Coutras at Leading Edge Communications, address at the bottom of your manuscript. If your article [email protected]. is published, in appreciation of your contribution, we will send you three complimentary copies via U.S. mail. We look forward to working with you!
QUALITATIVE RESEARCH CONSULTANTS ASSOCIATION 89 Index of Advertisers
123 Transcriptions/Compudex ...... 19 Fox Research Inc...... 78 www.123transcriptions.com www.foxresearchinc.com
ActiveGroup ...... 46, 47 & Back Cover Group Dynamics in Focus, Inc...... 17 www.activegroup.net www.groupdynamics.com
Advanced Focus ...... 23 GroupNet ...... 41 www.advancedfocus.com www.group-net.com
Amy Riker Transcription Services ...... 85 Home Arts Guild Research Center ...... 12 www.hagrc.com Atkins Research Group ...... 90 www.atkinsresearchinc.com InterClipper ...... 69 www.viccd.com Baltimore Research Agency ...... 5 www.baltimoreresearch.com Jay Martin ...... 80 www.jsmartintranscription.com Bernett Research Services ...... 59 www.bernett.com L&E Research...... 21 www.leresearch.com C&M Research Services ...... 50 www.cmresearchservices.com Leibowitz Market Research Assoc., Inc...... 11 www.leibowitz-research.com Carmen Degidio ...... 52 www.cdqual.com Meadowlands Consumer Center, Inc...... 51 www.meadowcc.com Chamberlain Research ...... 79 www.chamberlainresearch.com MedQuest...... 75 www.medquest-chicago.com Consumer Opinion Services, Inc...... 58 www.cosvc.com Morpace International, Inc...... 64 www.morpace.com Creative Consumer Research ...... 65 www.ccrsurveys.com MRCGroup Research Institute ...... 18 www.mrcgroup.com DecisionPoint Marketing and Research, Inc...... 12 www.decisionpointconsulting.com Murray Hill Center ...... 44 www.murrayhillcenter.com Delve ...... 87 www.delve.com National Data Research, Inc...... 31 www.national-data.net Doyle Research Associates, Inc...... 78 www.doyleresearch.com National Field & Focus...... 29 www.nff-inc.com Ebony Marketing Research ...... 86 www.ebonymktg.com Observation Baltimore ...... 9 www.observationbaltimore.com Ecker & Associates ...... 20 www.eckersf.com Opinions of Sacramento ...... 37 www.opinionsofsac.com Field & Focus ...... 85 www.field-n-focus.com Perceptive Market Research, Inc...... 63 Fieldwork ...... 40 www.pmrresearch.com www.fieldwork.com PFI Research...... 52 First Choice Facilities ...... 81 www.PFI-Research.com www.firstchoicefacilities.net Precision Research, Inc...... 19 Fleischman Field Research, Inc...... 73 www.preres.com www.ffrsf.com Qfact Marketing Research, LLC ...... 39 Focus Forward ...... 62 www.qfact.com www.focusfwd.com RDD VuPoint Portland ...... 36 Focus Groups of Cleveland Survey Center ...... 29 www.rdd.info www.idirectdata.com Riva Training Institute ...... 57 Focus Market Research - Minneapolis & Phoenix ...... 3 www.rivainc.com www.focusmarketresearch.com Sabena Qualitative Research Services ...... 51 Focus Pointe Global ...... 33 www.qual.com www.focuspointeglobal.com Schlesinger Associates, Inc...... Inside Back Cover Focus Solutions Alliance ...... 13 www.schlesingerassociates.com
Focus Suites ...... 71 Taylor Research, Inc...... 45 www.focussuites.com www.taylorresearch.com
Focuscope, Inc...... 58 The Focus Network ...... 7 www.focuscope.com www.thefocusnetwork.com
FocusVision Worldwide, Inc ...... Inside Front Cover, 22 Tragon Corporation ...... 31 www.focusvision.com www.tragon.com