Planning for a Media Evalution

PLANNING FOR A MEDIA EVALUATION

PLANNING FOR A MEDIA EVALUTION W. D. Evans, K.C. Davis, M.C. Farrelly

A Practical Guide to Program Evaluation D. Holden & M. Zimmerman, 2009

PA 522 Program Evaluation Dea Rankin Fall 2011

1 PLANNING FOR A MEDIA EVALUATION

The truth® media campaign is the largest tobacco cessation social marketing campaign outside of the tobacco industry to have been undertaken in the U.S. Launched in 2000 by the American Legacy

Foundation, with a budget of over $100 million in the first year and media buys placed in every major

DMA in the country, the campaign utilized a number of marketing and communications platforms with the aim of reaching two primary target markets.

The truth® campaign used a mixture of mass media and secondary messages through a multitude of platforms to reach their target audiences. The campaign and the message was primarily dissseminated through a mix of national media commercials that were edgy and contained an “in-your- face” message to capture the attention of the adolescents they were attempting to reach. The use of secondary messages was helpful because the of the target population who are typically identified to have rather short attention spans.

The overall goal of the campaign was to reduce the incidence of smoking among teenagers and young adults. The primary demographic age group was identified to be 12-17 years and the secondary target market were individuals in the 18-24 year age range. Primary consideration for the major selection criteria for the target audiences was not specifically based on age, race, income nor gender.

However, there was an underlying intent to reach specific racial and ethnic groups.

Health communication, as cited in the Holden and Zimmerman text, “encompasses the study and use of communication strategies to inform and influence individual and community decisions that

2 PLANNING FOR A MEDIA EVALUATION enhance health” (Freimuth, Cole, & Kirby, 2001) Effective health communication has been identified as a national health priority because the majority of relevant health related messages can be disseminated to target markets using mass communication methods. The goal for many health communication campaigns is to modify unhealthy behaviors among specific segments of society and is typically referred to as social marketing. The truth® campaign is one such example of social marketing.

The combination of social marketing and health communication is frequently referred to as

“health communication and marketing” and typically follows standard marketing practices that build campaigns based upon place, price, product and promotion (Borden, 1964). Both health communication and marketing campaigns are typically designed and evaluated based on, “social marketing theory and behavioral theory, including theories of exposure, messaging, communication, and behavior change

(Hornik, 2002).”

The American Legacy Foundation (Legacy) was able to adapt much of the campaign based on the Florida TRUTH campaign, which successfully reduced teen smoking rates throughout the state of

Florida (Bauer, Johnson, Hopkins, & Brooks, 2000). Both the Legacy campaign and the Florida campaign were based on the efforts of the Columbia School of Public Health youth marketing experts which began in 1996 with funding from the Centers for Disease Control and Prevention (CDC)

(Columbia Marketing Panel, 1996; Mckenna, Gutierrez, & McCall, 2000).

In addition to recognizing that there is an extreme brand consciousness among adolescents, the

Columbia Public Health marketing panel also recognized that the “voice” of the campaign would be essential to its success. Campaign designers recognized early on that the campaign message had to be delivered in the “voice” of a teen. This “voice” was critical to spur behavior modification. The developers realized that if the message was not delivered using a teen “voice”, there was a high probability that the campaign would be perceived by the target market as disciplinary or reprimanding.

The panel also recommended that the campaign highlight the tobacco industry's failure to be truthful and developed a “counter-marketing brand” with which adolescents could relate. (Columbia Marketing

Panel, 1996; Mckenna, Gutierrez, & McCall, 2000).

Three primary objectives were identified which included exposing youth to the truth® messages and promoting positive reactions to those messages. The second objective was to modify attitudes and beliefs toward tobacco use and the final objective was to reduce tobacco use among youth. (Holden,

Zimmerman, 2009, p. 126). These three objectives formed the basis for the media evaluation plan, which included three particular aims.

The first goal of the media evaluation was to determine the level of truth® campaign exposure among the target markets. The second goal was ascertain, “the relationship between campaign exposure and related knowledge, attitudes and beliefs” (Holden, Zimmerman, 2009, p. 126). The final aim of the media evaluation was to evaluate whether there was a measurable decline in smoking among the target demographics.

While the goals of the media evaluation were simple enough in theory, there were several contributing external, contextual factors which complicated the evaluation process. Evaluation planners were tasked with developing an evaluation that was capable of measuring each of the aims identified and tangible campaign effects but had to also plan for external trends in tobacco use and other environmental factors that could have potentially affected evaluation results.

While there are numerous differences between a media campaign evaluation and other types of program evaluations, there is one notable major difference. In most cases, media evaluation planners are not responsible for identifying the target audiences/population nor the medium by which the campaign (program) is delivered. For media evaluation planners, they operate independent of message developers, market researchers who identify the target demographic and the best message for reaching the target demographic, creative designers who develop the campaign and finally, the media buyers who determine how much to spend and in which DMAs and on which outlets or platforms.

Media evaluations are more complex because they often require that the implementation process is measured at the same time that outcome data is being gathered. This is a best methods practice because it allows the feedback received to alter and potentially improve the campaign messages and delivery. Process and outcome evaluations must work together so that media messages and outcomes are correlated. Conducting evaluations immediately and on an ongoing basis provides immediate feedback for campaign enhancement.

Evaluation planners for the truth® campaign modeled the evaluation based on other commonly used media evaluations that are based on four key process and outcome dimensions of campaign effectiveness (Holden, Zimmerman, 2009, p. 125). These have been identified as exposure and recall, message and receptivity, behavior determinants which include knowledge, attitudes and beliefs and finally, behavioral outcomes (Holden, Zimmerman, 2009, p. 125).

Media evaluation planners for the truth® campaign decided that the best method for assessing context would be to determine, how media evaluation measurement and evaluation design strategies could best be tailored to the three major truth® campaign objectives (Holden Zimmerman, 2009, p.126).” This process included measuring exposure and message reactions, assessing attitudinal changes and assessing behavior impact.

Beginning in 2000 shortly after the media launch of the truth® campaign, evaluators began conducting phone surveys. At the time, home telephone surveys were still an effective means for reaching large population segments. In keeping with the first objective of the campaign, evaluators sought to measure exposure to truth® messages and receptivity to those messages (Holden,

Zimmerman, 2009, p. 126).

Evaluators designed and engaged the use of the Legacy Media Tracking Survey (LMTS).

However, it is important to note that the first LMTS surveys were conducted in 1999 prior to the campaign launch. This was crucial because for the first time, a health communications campaign was

6 PLANNING FOR A MEDIA EVALUATION attempting to market a brand rather than just serve as a public service announcement.

The LMTS was an ideal measurement instrument because it allowed evaluators to quickly measure responses to the quickly changing messages and because they could easily continue to survey on an ongoing basis. This method also proved to be fairly inexpensive and could be conducted on a national-level from one location.

Although the marketing planners did not clearly specify race as an identifier for determining their primary demographics, the media evaluation planners were able to measure whether the message reached the Hispanic and African American populations and how effectively it did so. There were several measures employed by the evaluators for the telephone surveys. These not only accounted for message recall, but also identified the number of members of the target audience actually exposed to the truth® message and required that, “respondents identify specific features of the ads to confirm that they had actually seen” ads that were currently airing (Holden, Zimmerman, 2009, p. 127).” The LMTS also asked respondents about other elements of the campaign to confirm or measure overall awareness of the truth® campaign brand.

The LMTS also incorporated measures based on the Elaboration Likelihood Model (ELM), which examine changes in attitude. This was done by asking series of questions that ascertained whether respondents in the target demographics found advertisements believable or if they thought the ads presented messages that offered solid motivation for not smoking. The responses offered insight

7 PLANNING FOR A MEDIA EVALUATION that allowed evaluators to examine, “the level of cognitive elaboration and information-processing activity” which is the essential to ELM (Petty & Cacioppo, 19986).

Because the evaluation occurred simultaneously with the truth® campaign, the campaign's implementation team and media planners received rapid feedback on whether youth were receiving messages and could alter messages and content as needed. Of critical importance was also the ability of the LMTS to ask questions that served as indicators for the likelihood of smoking initiation.

The LMTS evaluation occurred in phases throughout the campaign. Each phase serves a very specific purpose from message development spurred by baseline knowledge, attitudes and beliefs about tobacco to attitudes about specific message content presented in ads (Holden, Zimmerman, 2009, p.

128). The first evaluation results published demonstrated significant movement toward the campaign's first two goals. The data presented showed that 75 percent of 12 to 17 year-olds in United States had been exposed to specific ads (Holden & Zimmerman, 2009, p.128). Another important finding were the, “significant changes in knowledge, attitudes, and beliefs related to truth® messages 10 months after the launch of the campaign, and these changes were statistically associated with exposure to the campaign (Farrelly et at., 2002) (Holden & Zimmerman, 2009, p. 128).”

Although the LMTS served as the primary evaluation tool for measuring awareness of reactions throughout the campaign, it underwent several iterations as the campaign developed and implemented new campaigns and secondary campaigns. However, the LMTS could not reliably assess the final

8 PLANNING FOR A MEDIA EVALUATION objective for the truth® campaign. It was difficult via phone survey to determine how truthful respondents could be when asked if they used tobacco, thus a different evaluation tool was needed to make a final determination about the reduction in smoking among the adolescent population.

The evaluation team decided to use in-school surveys to gather more data regarding tobacco use and ultimately decided to use information collected from the National Youth Tobacco Survey (NYTS) and Monitoring the Future (MTF) (Holden & Zimmerman, 2009, p. 129). Both of these surveys are self-administered and nationally distributed. The importance of self-administration was crucially important to the collection process because it meant that students were more likely to be truthful about tobacco use. The MTF survey actually became the, “cornerstone of evaluation planning efforts to assess the campaign's impact on youth smoking behaviors,” (Holden & Zimmerman, 2009, p. 129) because of its coverage in similar major media markets where truth® was already airing and because of its national sample size.

The major issue encountered with collection of data regarding behavioral changes with regarding to smoking reduction among youth was that Legacy implemented the campaign in all major media markets on a national scope without identifying or excluding comparison markets. Every media market in the U.S. Received some exposure to the truth® campaign and sub-campaigns. This resulted in the absence of a control or comparison group. Fortunately for evaluators, the media buy and resulting exposure on a market to market basis “produced a natural pattern of variability” in campaign delivery (Holden & Zimmerman, 2009, p. 130).

Certain markets lacked particular networks which in turn created differences in visibility and therefore exposure among specific national markets. Evaluators were able to use gross rating points

(GRPs) to obtain information about viewership on networks that the truth® campaign was airing on and then correlated this data with the MTF surveys to, “estimate the association between market-level exposure to the truth® campaign and youth smoking prevalence,” (Holden & Zimmerman, 2009, p.

The last important consideration essential to the success of the evaluators in determining the overall success of the campaign were the external environmental factors. During the implementation phase of the truth® campaign, several states developed and implemented their own tobacco cessation media campaigns with funding from the 1998 Master Settlement Agreement. Other external factors that had to be considered stemmed from increased state funding for tobacco control programs, both increasing cigarette prices and tobacco taxes.

Also occurring at this time was the tobacco industry's own youth tobacco cessation campaigns like the Philip Morris, “Think. Don't Smoke” campaign as pointed out by Holden & Zimmerman, 2009.

While it was impossible to plan and account for every external factor that might contribute to reduced smoking, evaluation planners attempted to identify as many of these as possible during the planning stage of the evaluation. Evaluation planners used LMTS in an attempt to track which messages were specific to other tobacco industry or state youth anti-smoking campaigns.

It is important to note that the media evaluation conducted for the truth® campaign utilized an evaluation model based on the Elaboration Likelihood Model and not a standard conceptual model like the Evaluation, Planning, Incorporating, Context (EPIC) model. There are three key differentiators between these two models for program evaluation: media evaluators are not directly involved in assessing context; the relationship between the evaluators and media campaign developers is integral to the success of the evaluation; and finally, the process evaluation becomes equally important to the outcome evaluation because, “the media campaign developer acts as an intermediary between evaluators and campaign audience,” (Holden & Zimmerman, 2009, p. 140).

A great deal of the success of the truth® campaign can be attributed to the vast budget that was allocated to campaign research, planning, implementation including the national media buy, initial and ongoing evaluation. The budget was not, nor is it, typical of most health communications and marketing campaigns. The sizable budget resulted in the evaluation component's ability to develop and refine the message and brand throughout the campaign to better reach the intended demographic thereby having a greater impact on the truth® audience. The final result was was reduced smoking among the adolescent target market.

While the truth® campaign's outcome was determined largely successful there were also several identifiable challenges and lessons learned based on the media campaign evaluation. One such lesson was the decision about the timing for the national launch. This significantly impacted evaluators' decisions about the design of the evaluation including the methods and measures used. Additionally, evaluator input would have also been beneficial to media campaign developers and implementers early

11 PLANNING FOR A MEDIA EVALUATION on in campaign development. The lack of a control group meant that evaluators were unable to evaluate dose-response and required that evaluators creatively adapt evaluation measures to include both surveys and media reports like GRPs in lieu of dose-response analyses.

Last and as noted previously, it is difficult to account for every external factor that may impact a media campaign's outcome, it is difficult to extract how much of the success in reduced tobacco use among adolescents in the target demographic age range is directly attributable to the truth® campaign and how much was spurred by other campaigns airing simultaneously. Numerous tobacco cessation campaigns were launched during truth® implementation and it is highly probable that similar messages being disseminated to the same population coupled with the downward trend of tobacco use among teens, increased prices and taxes on tobacco and the uncertainty of GRP data reliability were contributing factors to the overall reduction measure of 22 percent. A more accurate reduction based on the truth® campaign might lend itself to the 5 – 9 percent reduction rate range.