How ‘Behavioural Recruitment’ is leading research into the social age
Tom Woodnutt – Founder, Feeling Mutual Hugh Carling – Founder, Liveminds
1 If a participant were a fish…
...then recruiters are fishermen Responsible for sourcing the right participants.
...researchers are chefs They interpret participants’ feedback and serve up well-balanced insights that nourish their clients businesses.
...clients are diners Earnestly paying their bills on time, in return for delicious dishes of research (that help them to grow their bottom line, so to speak).
2 Researchers are preoccupied with method
As researchers (or chefs) we are preoccupied with the research method and spend a lot of time thinking about research techniques, to ensure the meals we serve are well balanced and not overcooked. Behavioural science inspires new ways to research Methods informed by behavioural science, have inspired lots of innovation in recent years such as implicit measures and eye-tracking.
These methods are powerful because they help researchers minimise the impact of cognitive biases, whereas traditional methods - which rely on direct questioning and ‘claimed behaviour’ - are prone to error.
3 But what about the quality of the ingredients?
What about the quality of the participants we recruit? What happens when you apply behavioural science principles to recruitment? Traditional recruitment relies entirely on what people claim
Traditional methods of recruitment, can start to look artificial because they rely entirely on what people say rather than what they do. Ultimately your results can’t be right if your recruitment is wrong.
In this paper we’ll reveal the results of a detailed study with researchers commissioned by Liveminds, show how Behavioural Recruitment tackles the issues raised in the study and share a case study showing how it works in practice.
4 We have a problem
It’s clear that recruitment is a huge concern for the industry. ‘Accessing a quality and representative sample is the single biggest challenge’
And the problem is getting worse. The latest GRIT Report also revealed a ‘General decline in the quality of sample, with falling participation rates’.
5 Research on Recruitment 2018
Liveminds wanted to understand this huge, but rarely discussed problem in more detail, so we commissioned SketchBook Consulting to carry out research benchmarking the views of qualitative researchers on the state of research participant recruitment.
The aim of this research was to recognise and understand the challenges faced by researchers, to explore opportunities for improving recruitment methods within the industry, and to improve Liveminds Behavioural Recruitment service.
To engage respondents, the survey was promoted on LinkedIn and publicised by key media partners like GreenBook.
Click here to download the Research on Recruitment 2018 report
6 Quality of participants is #1 challenge
Q: Which, if any, of these do you consider to be challenges facing the qualitative research industry in your market today? Base: All participants - 100
100 qualitative researchers in the UK responded and the findings strongly echoed the GRIT report, with ‘quality of participants’ considered to be the biggest challenge facing the industry today.
Although the study focused on qualitative research recruitment, many of the concerns apply equally to quant research - since panels also use databases of repeat respondents.
7 Horror stories…
During the Research on Recruitment 2018 study, researchers shared examples of the poor recruitment they’d experienced including:
● The bald man in a shampoo group ● The lady with the cushion under her jumper pretending to be pregnant ● And the man who politely excused himself to go to the toilet, having taken the incentive, never to return
These are extreme examples, but the credibility and therefore integrity of what researchers do is threatened if recruitment is not effective.
Read more research respondent horror stories
8 Quality matters more than speed and cost
95 93
85 84 78 76
60
37
12
quickly
Recruitment is cheap & recruitment criteria
Participants are reliable by recruitment partner
Participants are articulate Fewer repeat participants Recruitment is completed Participants are motivated
Accurate fit between sample Receive regularupdates and accurate on project progress
Responsive project management
Q: How important do you think each of these factors are for “effective recruitment”? Figures shown are the NET rating 4 or 5 out of 5 for importance Base: All participants - 100 But what is effective recruitment? In terms of participants, most researchers in the study see it as:
● accurate fit between sample & recruitment criteria ● reliability ● motivation
Interestingly speed and cost of recruitment – often assumed to be vital today – were deemed relatively less relevant to effectiveness.
9 Three systemic factors encourage problems
But why are researchers facing so many issues with their recruitment? We believe there are three key systemic factors which increase the risk of low quality participants.
Firstly, traditional recruitment relies entirely on what people claim
In the past we’ve had to rely entirely on what the participants say, not what they actually do. As Behavioural Science has shown us, memory is highly fallible. 1 in 2 researchers believe that too many participants lie to get recruited
As revealed in our research. When you consider GreenBook’s finding that participants primary motivation is 5X more likely to be money than any other factor, the incentive for participants to lie is very clear. 1 in 2 researchers believe some recruiters encourage participants to lie
We aren’t saying all recruiters are guilty of this. But we are saying that the current system allows it. There is no transparency in these interactions and recruiters can be under considerable pressure.
10 Secondly, we’re limiting ourselves to the same small pools of participants…
…rather than fishing in the vast oceans beyond them. This gives us far less chance of finding people that genuinely match the criteria. Thirdly, participants become more professional and less naturally representative
Repeat respondents become increasingly professional, are conditioned to the research process and no longer represent the views of real consumers. Psychologist’s advice is that doing repeat groups will change what participants say in a group. This is because certain types of response become unconsciously reinforced by a prior experience.
Participants can also start to second guess the moderator - like the example we heard of a participant who interrupted the moderator to suggest they really should be doing a personification exercise.
11 Professional participants are a problem
Q: Overall, do you think “professional participants” are a problem in qualitative research? Base: All participants - 100
Most researchers in the survey think that professional participants are a problem in qualitative research. Although, perhaps worryingly some don’t.
More than half of researchers said they had recognised the same people in different projects within the last 12 months.
51 19 30
Q: I recognise the same participants taking part in my qualitative research projects
Base: All participants - 100 Agree Neither agree or disagree Disagree
12 The pros and cons of professional participants
76 73
64 64 59
21 17 16
More reliable
More motivated
Insufficiently motivated
Too familiar with research Quicker and easier to find Understand research well
More likely to lie when recruitedMore likely to lie in the research
Q: Which, if any, of these statements do you think apply to "professional participants”? Base: All participants - 100
Professional participants are too familiar with the research process. Also they’re more likely to lie during recruitment (because they’re more likely to say whatever they need to to get in). Those who don’t see it as a problem may say so because of the practical benefits. They’re seen as easier to find and because they’re so research-literate it’s easier to have a constructive, affirmative discussion. But does it mean they should be there just because they make the research process easier?
13
To us, the solution is simple…
Use Facebook’s ocean of people, rather than fishing in the small finite pools of repeat respondents and professional participants. Facebook has over 2.2 billion users around the world, in 190 countries. This is well over 100X the size of even the biggest quantitative panels.
14 Accurate fit based on real behavioural data
Cast your line exactly where it’s needed, based on the network’s incomparable behavioural data, rather than relying on what fallible humans have said. Fresh to research The greater reach means that participants are typically fresh to research, so you get the views of real consumers rather than conditioned responses.
15 What Facebook knows about us…
On average we spend 55 minutes a day on Facebook reading, watching, sharing, commenting, liking, reacting, clicking and posting, and every single one of those behaviours goes into building Facebook’s understanding of us.
In a recent study, Stanford professor Michael Kosinski revealed that a computer model based simply on Facebook likes, knew more about its subjects than their closest friends and family. And also this… ‘Likes’ are just 1 of 98 different data points collected on users, from the TV shows we watch to our international traveling habits.
16 From what we do on Facebook…
And 10 million websites with this button…
17 How does Behavioural Recruitment work?
Reach Screen Manage Research Pay
Clients
Via Ad Networks
First, we place our hyper-targeted adverts into the News Feed of any user whose demographics, interests and behaviours closely match the project’s criteria.
If they want to take part, they click through to take a detailed screener, inside our app. If it’s quant survey, they’ll take it there and then.
If it’s qual, we’ll contact all the people who’ve passed the screener, to further qualify them, and choose the best ones for the project. When the research is done, they receive their reward. They are not added to a database to be contacted about future projects.
We’ve recruited this way in more than 70 countries already, for every kind of research from face-to-face focus groups, to quant studies, to placing automated cameras around family breakfast tables.
18 Case study
How, who, when and why?
How do young people consume music video?
In the context of these recruitment challenges, The Box Plus Network (who are the UK’s biggest broadcaster of music video), challenged Feeling Mutual to go the extra mile and understand ‘real youth’ and their relationship with music videos.
Feeling Mutual dove deep into the needs that drive music video consumption, and measured them across contexts, devices and audiences.
19 We went beyond claimed behaviour
Psychology 01 review
Stakeholder 02 Workshop
Online & Mobile 03 Qualitative
Quantitative Validation
To go beyond claimed behaviour, Feeling Mutual:
● Commissioned a music psychologist to challenge our assumptions about why young people love music video ● Ran stakeholder interviews and a workshop to clarify the business decisions that would be made after the research ● Used online and mobile qual to develop our hypotheses - allowing us to get into their natural environments. Participants: ○ completed mobile diaries of music video occasions ○ conducted mobile deprivation exercises ○ took part in a ‘Simon Cowell record label manager’ game ● Used Behavioural Recruitment to find a sample of 1000 16-24s for a quant study measuring their behaviour and needs in order to validate our model ● Ran a debrief workshop to brainstorm implications for the business
20 Finding fresh, authentic fans
Feeling Mutual set Liveminds - The Behavioural Recruitment company - the challenge of finding authentic, Generation Z music lovers who were:
● Actively viewing music videos regularly ● Real fans of particular genres of pop music ● Fresh to research, rather than people who were regularly taking surveys and doing group discussions
In addition, some participants had to be actively engaged in the Box Plus Network’s niche channels (including 4music, Kerrang and Kiss).
It was thought that traditional recruitment approaches might struggle because of the underrepresentation of youth in databases and the fact that this group can be less reliable for qual research.
21 Access to a huge youth audience
Behavioural Recruitment gave us access to a huge youth audience. To put this in context, Facebook has 7-8 million monthly active users aged between 18-24 in the UK.
It is true that this is dipping slightly, some reports say by 3% last year, but the number of active users remains absolutely huge.
22 Based on actual behaviour
Through the hyper targeting of Facebook’s advertising system, Liveminds was able to narrow that huge audience down to those people:
● whose online behaviour on both Facebook, and 10 million other connected websites, had clearly demonstrated that they regularly watched music videos, and ● a proportion of music video lovers who were also fans of the relevant music genres and actively engaged with niche The Box Plus Network channels like Kerrang.
23 Better quality participants
To make confident decisions The Box Plus Network needed to talk to people who were engaged and reliable.
The participants Liveminds found on demand were fresh to research, so were naturally curious, enthusiastic, and keen to discuss a subject we knew they were genuinely interested in.
In fact, a recent study of Liveminds qualitative research platform, analysing more than 35 million English words, found that Behavioural Recruitment participants give 47% more data in research projects than those recruited through traditional methods.
There is less chance of participants ‘gaming recruitment’ because there’s no human interface between them and the screener. And they’ve only been invited to screen, based on hugely in-depth data about them.
24 (Some) of what we discovered
TERTIARY SECONDARY PRIMARY
Through the psychology review Feeling Mutual identified and ranked a number of needs that music video addresses in young people.
These were then refined through the online and mobile qualitative research.
The findings were finally validated using a survey with 1000 16-24 year olds before being workshopped with the client team.
All of this was recruited via Behavioural Recruitment.
The psychology review gave us a big headstart when it came to designing and interpreting the study.
25 ‘Teen Narcissistic Vulnerability’
Psychology studies tell us that young people are becoming more preoccupied with their own image. Campbell (2009) found that each generation is more interested in their own image than their predecessor. Gentile (2012) tied this to the rise of social media.
Narcissism and the ability to style identity online in order to generate attention across networks is a hugely useful skill in today’s hyper connected world.
One theory takes an evolutionary psychology perspective, suggesting that adolescents are hardwired to prepare to leave their families to mate with someone from a different community, to maintain genetic diversity. Therefore it’s adaptive to be able to develop an identity different from our parents’.
These insights helped Feeling Mutual focus on the role music plays in developing and maintaining identity.
26 Music helps people manage their moods
“This song never fails to cheer me up. The video makes me want to join Taylor and dance, not caring what anyone else thinks; It’s that carefree attitude you need in life”
(Emma-Louise, Belfast, 17)
We also found that music video plays a vital role in managing emotion. The importance of mood regulation was shown in a study by Zilan (2002) which found that in bad moods, people elect to listen to uplifting music for longer than when in good moods.
In this example, the participant explained how Taylor Swift picks them up and gets them dancing. Not just making them happy but also giving them a confidence boost.
The music video diaries consistently highlighted just how important music video is to mood management.
27 Many who watch to lift their mood, dance
Another thing we found, that was not present in the psychology literature, was the high number of young people who watch music video and dance to it. Most people who watch music video to escape from the here and now will dance along.
28 This validated investment in dancing content
The research helped validate a stream of content already in production which teaches the audience dance moves. For example, this show called ‘Show me the moves’. The research was able to validate further investment in this direction.
29 Emotional benefits become rituals
“Usually I watch in the morning after my shower as I dry my hair! I would say it is now part of my wake up routine!”
(Emma, Dundee)
The qual uncovered a number of rituals involving music video.
For example, watching music video has become embedded in people’s going out and getting up routines.
When a significant number follow a certain ritual, it suggests there’s potential to inspire others to follow it.
30 The needs behind rituals can inform scheduling
“You mentioned that often watch music videos for the following reasons. What time do you usually do it?” | Base: Respondents who ‘Always’ or ‘Often’ watch music videos for each reason (n=158-864)
The quant then allowed Feeling Mutual to measure the emotional needs behind rituals and the context in which they’re active, which in turn can inform scheduling.
For example, we found that people tend to watch music video for confidence in the mornings, for motivation in the afternoon and for nostalgia in the evenings.
This could have implications for scheduling particular types of music video playlists at particular times of the day.
31 Summary
Traditional Behavioural
Reach 10 million 2 billion
Claimed Real Accuracy Behaviour Behaviour
Repeat Fresh Freshness Respondents Participants
Since the advent of social media most researchers have focused on social listening. However, we believe it’s true potential lies in recruiting for primary research.
The benefits of utilising social network data are the same:
1. The vast reach gives you access to more than 200X the number of people in even the world’s biggest panels. 2. You’re recruiting based on real behaviour, not just what people claim in a screener. 3. You get fresh participants for every project, not the same conditioned repeat respondents.
Surely the time has come to use behavioural data, to put an end to the industry’s fish farming and find better participants in their natural environment.
32
Please get in touch for further information:
Hugh Carling Tom Woodnutt liveminds.com feelingmutual.com
33