Station to Station

Home , CINF, CINW, CJAD, CJMS, CJWI, CKAC

STATION TO STATION

Measuring radio audiences with a PPM panel in Québec

Pasquale (Pat) A. Pellegrini Ken Purdye

The authors report findings from a comprehensive analysis of radio audience data captured by Arbitron’s Portable People Meter (PPM). The paper compares the diary and PPM results for a common area and time period and list of stations. Comparisons, not unexpectedly, show differences in overall amount of radio use, the distribution of radio use by time periods, the relative importance of radio listening for different subgroups of the population, the reach of radio, and shares and reach of the different stations. There are also indications that the structure of radio listening (e.g., relative weight of heavy-light listening, demographic profiles, etc.) is different. The reasons for these substantive differences are explored. It becomes evident that the tasks of writing a daily diary and carrying a PPM attract different populations, as measured by standard demographics, socio-economic and lifestyle variables, but account for relatively small differences between the methodologies. The paper also examines differences in the definition and measurement of radio listening as the reason for differences between diary and PPM measurement. This analysis points to the definition of radio listening as the predominant reason for the different radio audiences observed.

INTRODUCTION BBM has been operating a commercial Personal People Meter (PPM) panel for the measurement of French-speaking television audiences in Québec for almost two years now. Our extensive experience in setting up and enhancing this panel is covered fully in an earlier paper (Pellegrini and Purdye, 2004). In addition to measuring TV audiences, 22 radio stations (both English and French, Canadian and U.S.) have allowed their signals to be encoded to facilitate the testing of Radio measurement by this same PPM panel. The radio PPM data for these stations has been produced and analysed, and are the subject of this paper. Background The Personal People Meter is an audience measurement system developed by the Arbitron Company in the United States. Co-operating broadcasters place an identifying audio code and a time stamp in their output using a special encoder. A sample of the population is equipped with a small carry-around device similar to a pager, which “listens” for, detects and stores these codes. The PPM therefore provides a near-passive record of the sample’s exposure to encoded media (Patchen and Webb, 2002; Pellegrini and Purdye, 2004). The BBM Canada PPM Television panel in Québec currently has an installed base of about 550 homes, of which about 375 are in the Montreal Extended Market. Data from the panel have been used for two years as the basis of trading television time in Quebec. The panel is selected in a two-step procedure. A large Establishment Survey is used as the master sample from which the actual audience panel is selected, using stratified random procedures. Viewing data from respondents is collected and edited on a daily basis: respondents are accepted into tabulation provided they meet fixed qualification rules, the most important of which is that the PPM has shown at least four hours of carry time during the preceding 24 hour period, as indicated by a built-in motion detector. On an average day, about 90% of the installed panel is accepted into production. The Québec PPM panel is a French- language panel, comprising those who speak French most often at home. Although now operating solely as a television measurement panel, the majority of radio stations in Montreal have also allowed their signals to be encoded. Currently, the encoded stations account for over 92% of all radio listening in the Montreal extended market. The data is currently processed and edited, using the television compliance and qualification rules, although it is not released to BBM members so as not to disrupt the market which obtains its currency through a diary methodology.

The standard measurement instrument for radio in Canada is a personal diary. The diary is a one-week record of radio listening with a fresh sample selected for each week of an eight-week “sweep” period. There are two such sweep periods per year. The sample of homes is selected at random and contacted by telephone. Those homes co-operating in the telephone survey are sent radio diaries to complete. All members of contacted households are sent a personal diary covering a one-week period (Monday through Sunday). Diaries are edited on a weekly basis – the diary must be acceptably completed for each of its seven days in order to be accepted into tabulation. The diary has a time grid for each day, marked into quarter hours. Respondents are asked to keep a record of their radio listening in this diary by entering into the grid the call letters or frequencies of stations listened to, together with the location of listening. It is important to note that this diary format is significantly different from that used by Arbitron (which is much more open- ended), or indeed the RAJAR diary in the U.K. (which is more closed-ended). As the diary is the currency for radio audience measurement, any movement to electronic measurement necessitates a careful comparison and analysis of the competing methodologies. We now have two diary sweep periods during the life of the TV PPM panel within which we can compare radio audience measurements. The Spring 2004 sweep extended from February 16 through April 11, 2004 while the Fall 2004 sweep was from September 6 through October 31, 2004. The standard geographic area used for analysis by the radio industry in Canada is the Montréal Census Metropolitan Area (CMA) as defined by Statistics Canada. However, the comparisons in this paper are made on a somewhat larger area, the Montreal Extended Television Market (EM). This was dictated by the sampling and weighting structure of the PPM panel, which, as noted earlier, is specifically designed for television measurement. Diary data were also processed for this larger area. The Montréal CMA has about 80% of the population of the EM. For purposes of the comparisons, “Radio Listening” is defined as the sum of listening to the 20 (Spring) or 22 (Fall) encoded radio stations, for both PPM and the diary survey. For the Spring 2004 survey, the weekly diary sample was on average 576 people aged 12 years and over, for a total of 4,609 respondents over the eight weeks of the survey. The PPM panel averaged around 548 people aged 12+ years per week. It would be easy to make the comparison of 4609 vs. 548 diaries and dismiss the PPM sample as “too small”. However, we need to keep in mind that the PPM sample is a panel: theoretically, the same sample is used for eight weeks. This does not mean that it is worth 8 x 548 = 4384, but the “re-use” factor means that the effective base is considerably

more than 548. For the Fall 2004 sample, equivalent figures were 794 for the diary and 641 for the PPM panel

COMPARATIVE ANALYSIS RESULTS During the Fall 2004 sweep survey, the average diary respondent listened to the radio for 2.6 hours per day. The equivalent figure for the PPM panel at the same time was 2.0 hours per day. On the face of it, PPM shows about 20% less radio listening in total than the diary. However, this is to ignore an important difference in definition. The diary collects information for each quarter hour during the week. Respondents are asked to count themselves as listening to radio for a specific quarter hour if they listen to at least five minutes within it. They are then counted as having listened for the full fifteen minutes. Radio listening, according to the diary, is essentially an agglomeration of quarter- hour cumulative audience estimates. The PPM rules used for these comparisons are identical to those used for the commercial TV panel: exposure is recorded on a continuous basis and then summarized by minute. A minute is counted as listening if at least 31 seconds of exposure occurred. Each minute is counted as a full minute of listening. We can simulate the diary definition of listening with the PPM data by counting a quarter hour as “listened to” if the respondent listened to at least five minutes of radio within it. In all such cases, the full quarter hour is then credited as listening, as with the diary. When this is done, the PPM and diary data become much closer: The average PPM panel respondent listened to 2.6 hours of radio per day, about the same level as the diary. Our comparison of BBM data for the Spring 2004 sweep period shows similar results. The listening estimates were 2.6 hours per day for the diary and 2.5 hours per day for PPM. Arbitron’s comparison of diary and PPM in the first year of the Philadelphia test showed almost equal quarter hour ratings for the two methods, when adjustments were made for the definition of radio listening (Patchen and Webb, 2002). However, the two measurement systems (PPM and diary) differ in how they arrive at the common 2.6 hours of listening per capita, per day. With the diary system, 69% of respondents listened to radio at all on an average day, and they listened for 3.7 hours each. PPM shows more people listening to radio, but they listen less. According to PPM, more people listened to radio on an average day (81% of respondents) but they listened for a shorter period of time (3.1 hours).

Differences by Age and Gender There are significant differences between the Diary and PPM with respect to the amount of radio listening reported by gender and age group. There is a consistent tendency for PPM to estimate larger radio audiences for men than the diary (2.9 hours per day per capita for the PPM, 2.7 hours per day per capita for the diary). Conversely, there is a tendency for the diary to estimate larger radio audiences for women than the PPM (2.8 vs. 2.4 hours, respectively). There is also a tendency for the diary to show larger audiences than PPM for older respondents, while the reverse is true for younger respondents. Figure 1 shows the average hours of radio listening per day per capita for different age groups for the two measurement systems. Among teens, the PPM shows radio listening about 50% higher than the diary. The difference decreases as the age of the respondent increases. Among those 65 years and over, the PPM shows radio listening at about 80% of the diary levels.

Figure 1

3.5 DIARY PPM 3.0 3.0 2.9 2.8 2.7 2.7 2.6

2.5 2.2 2.3

2.0 1.8 1.7

1.5 1.5

1.0 1.0 AVERAGE HOURS HOURS OFLISTENING AVERAGE

0.5

0.0 12-17 18-24 25-34 35-49 50-64 65+

AGE GROUP

Radio with the PPM is a younger and more masculine medium than it is with the diary. These findings are consistent with Arbitron’s findings in Philadelphia (Patchen and Webb, 2000). Given the importance of the age/sex variables in both programming and the trading of radio time, this difference is an important one. Differences by Daypart Figure 2 shows the audience to radio on a quarter-hour basis, throughout the average weekday (Monday to Friday). The variables are time (x axis) and the percent of the population using radio (y axis). The data is from the Fall 2004 comparison and is quite similar to the findings for the Spring of 2004. Monday to Friday has been chosen for the comparison, since listening patterns are somewhat different from this on the weekends for both the PPM and the Diary.

Figure 2

30.0 DIARY PPM

25.0

20.0

15.0

10.0 PEOPLE USING RADIO RADIO USING (%) PEOPLE

5.0

0.0 05:00 05:15 - 05:45 06:00 - 06:30 - 06:45 07:15 07:30 - 08:00 - 08:15 08:45 09:00 - 09:45 - 09:30 10:15 10:30 - 11:15 - 11:00 11:45 - 12:00 12:30 12:45 - 13:30 - 13:15 14:00 14:15 - 15:00 - 14:45 15:30 15:45 - 16:30 - 16:15 17:00 17:15 - 17:45 18:00 - 18:45 - 18:30 19:15 19:30 - 20:15 - 20:00 20:45 21:00 - 21:30 21:45 - 22:15 - 22:30 23:00 23:15 - 24:00 - 23:45 24:30 24:45 -

The two series show similar audience levels in the early morning hours, but then diverge sharply, with the maximum difference appearing between 7:30 am and 8:00 am. The difference then declines until the two series converge at around 10:30-11:00 am. The series follow each other closely for the rest of the day, although there is an increase in audience for the diary, unmatched by the PPM, between 4:30 and 5:30 pm. (This blip was more pronounced during the

Spring 2004 survey). During the evening hours, the PPM curve is consistently higher than the diary curve. Similar patterns exist for adults only (18+ years), for adult men and women and for different age groups. There are differences between the diary and the PPM with respect to the overall amount of radio listening, as we have seen. However, the patterns shown in figure 2 persist for each of the groups: there is a gap during the morning drive period, with convergence by 10:30-11:00 am, after which time the two series parallel each other closely. As a consequence, the morning drive period is relatively less important in a PPM world than in a diary world (Poesmans, 2004; Futsaeter, 2004). Conversely, daytime and evening hours are relatively more important. This is illustrated in figure 3, which shows the distribution of weekday radio listening between the four broad time periods of morning drive, daytime, afternoon drive and evening, for the two systems. While daytime hours (Mo-Fr 10:00 am - 4:00 pm) are the most important according to both measurement systems, they are more important for the PPM than for the diary. The morning drive period is relatively more important for the diary; evening hours are relatively more important for PPM.

Figure 3

100%

MF7-1 9.6 12.6 MF4-7 MF10-4

MF 5-10 17.0 80% 17.3

60%

40.8 42.9

40% PERCENT OF WEEKDAY OF LISTENING PERCENT

20% 32.5 27.2

0% DIARY PPM

Recall from above that the difference between the Diary and PPM results are less pronounced for men than for women. This observation, combined with the greater importance of the daytime time block in the PPM data, produces an interesting result: in the PPM data, male listeners are more numerous than female listeners during the daytime period. With the PPM, the average People Using Radio (PUR) for men is 23% and for women 19%. The equivalent diary estimates are 20% for men and 23% for women. The most important difference between the two time series in figure 2 is the discrepancy in the morning drive period, and to a much lesser extent in the afternoon drive period. Why does the diary produce relatively higher estimates of radio listening in these periods? We know that there is no such issue with television. Until Fall 2004, BBM operated a push button meter panel in Québec, in parallel with the PPM panel. The two panels showed very similar amounts of television viewing in the early morning hours with parallel viewing curves. In addition, we know that by 7.30 am, half of the PPMs in the sample have been undocked and are showing motion (Pellegrini and Purdye, 2004). If morning television produces similar figures, why is this not the same for morning radio? The morning drive period is radio’s prime time, the equivalent of the 8:00- 10:00 pm period for television. When BBM replaced a television diary with a push button meter system, the largest differences between the two systems were found in prime time (BBM Canada, 1999). The diary had relatively larger audiences for prime time programming, the meter for off-prime periods; the meter showed smaller variance across time. The same effect is evident here for radio’s prime time (morning drive) compared to its off-prime periods (evening and weekend). To some extent, we may be observing here the same phenomenon – the tendency of a diary to accentuate the positive and vice versa. One tentative hypothesis is that this difference in listening estimates is related to the capture of out of home listening, and specifically in car radio listening. The version of the PPM used in the Québec PPM panel does not provide information on the location of the radio listening detected (in home, at work, in a car or other). However, the diary does collect such information on a quarter- hour by quarter-hour basis. From this diary research we know that the eponymous morning and afternoon drive periods have a larger percentage of the total radio audience in cars than other time periods. Is there an association between the percent of the total radio audience that is in cars and the relationship between the diary and PPM estimates of total radio listening? We have tried to test this by correlating the difference between the diary and PPM estimates with the percentage of the radio audience that is in car (according to the diary).

It is difficult to specify a simple regression model of the type Yt = a + bXt + et, where Y is the relationship between PPM and the Diary, X is in-car radio listening and e is the error term. The error term is likely to be highly autocorrelated, violating one of the preconditions for a simple regression analysis. We therefore took first differences and estimated

Yt - Yt-1 = b(Xt - Xt-1) where Y is the difference between the two estimates of PUR and X is the percent of the diary audience located in cars. The results of this analysis are mildly supportive of the hypothesis that in-car listening may be (to some extent) the culprit. The regression is “significant” beyond the 99% level, as measured by an F test) and the value for b is significant beyond the 99% level (t = 6.3). The fit is poor, with an R2 of just .34 and there is some evidence that autocorrelation in the error term persists in spite of the differencing. (There seems to be a tendency to underestimate morning drive and overestimate evening drive and the Durbin-Watson statistic is uncomfortably high). Perhaps this result is simply due to the wide array of unmeasured variables in the model. Clearly, the difference between the two series is due to many more things than the simple percentage of the audience listening in cars. Figure 4 shows the observed difference in PUR (Diary minus PPM) plotted against the predicted difference, by quarter hour. It shows the actual difference and the difference that is predicted with knowledge only of the percentage of audience in the car. Knowing that indexes are the preferred tool of media analysts, we have repeated this analysis using relative differences (Diary PUR: PPM PUR) as the dependent variable. The fit is still a rough one with a small R2 and, again, there is some evidence of autocorrelation in the error term.

Figure 4

10 OBSERVED PREDICTED 8

0 DIFFERENCE IN PUR (PCT PTS) (PCT PUR IN DIFFERENCE

-2

-4 05:00 05:15 - 05:45 - 06:00 06:45 - 06:30 07:15 07:30 - 08:00 08:15 - 08:45 09:00 - 09:30 - 09:45 10:30 - 10:15 11:15 - 11:00 11:45 12:00 - 12:30 12:45 - 13:15 - 13:30 14:15 - 14:00 14:45 15:00 - 15:30 15:45 - 16:15 16:30 - 17:00 17:15 - 17:45 - 18:00 18:45 - 18:30 19:15 19:30 - 20:00 20:15 - 20:45 - 21:00 21:45 - 21:30 22:30 - 22:15 23:15 - 23:00 23:45 24:00 - 24:30 24:45 - M ONDAY-FRIDAY 5:00 AM - 1:00 AM 12+

Nonetheless, this analysis does give some food for thought. There is obviously some relationship here through the fog of unmeasured variables and autocorrelated errors. One can speculate on why there may be such an issue with PPM measurement in cars. There may be a technical issue with capturing the encoded signals if the PPM is carried in a pocket under several layers of clothing during a typical Montreal winter. There may be a compliance issue with those who do not habitually listen to radio in the work place and therefore do not see the need to take the PPM with them, although compliance statistics argue against this. One final consideration is that the difference between the diary and the PPM in the morning (and evening) drive periods is greater for the Spring survey (February - March when the weather is cooler) than in the Fall (September - October, when the weather is warmer). Alternatively, there may be an issue with the way in which recording of in-car radio listening takes place with the diary. Obviously such listening is not recorded contemporaneously, but rather on a recall basis, long after the event. There is scope here for over-estimation of actual listening – a point we return to later in this paper. To illustrate how this applies to the morning drive period, we can consider the reach differences between diary and PPM. For most time

© Copyright by ESOMAR® / The ARF Station to station 11 periods, PPM is considerably higher than the diary. However, this is not the case during the morning drive period. Here, the diary reports average daily reach of 60 vs. 44 (morning drive versus day), while PPM reports 57 and 63, respectively. There is certainly an issue here that is associated with in-car listening that merits further investigation. Differences by Day of the Week When we look at radio listening by day, the diary shows a consistent pattern: Monday levels are greater than Tuesday levels; Tuesday levels are greater than Wednesday levels, and so on throughout the week. People listen to less radio as the week progresses. This is true for both average PUR and for daily reach of radio and for different age groups and men and women. However, it is not the case for the PPM data. Listening is at about the same level for each weekday; if anything there seems to be a small increase at the end of the week on Thursday and Friday. It has been known for some time that the drop off in recorded listening from day to day in the diary is a consequence of respondent fatigue. (Experiments starting the diary with a different day show that the new start day invariably becomes the highest listening day. For this reason, BBM publishes only average Monday-Friday figures). We have been unable to detect fatigue in the PPM panel. Both the Diary and the PPM data show weekend listening less than weekday listening, although the drop-off is markedly less with PPM. The average diary respondent listens to the radio for 14.6 hours during the Monday-Friday week; the average PPM respondent listens for an hour less (13.6 hours). The average diary respondent listens to the radio for 3.4 hours on the weekend (Saturday and Sunday). The average PPM respondent listens to an hour more (4.4 hours). Again, there may be a relationship here with diary fatigue. Station Shares of Audience and Reach Audience shares for the different stations encoded are quite similar for the Diary and for PPM (see table 1). The correlation coefficient is .95 (Spring) and .98 (Fall), which clearly shows significant agreement. The rank order of stations is also well preserved. Within this overall high level of agreement, it is clear that AM stations as a group tend to have slightly lower audience shares with the PPM system than with the diary system. (A difference of 5 percentage points in the Spring and 2 percentage points in the Fall). There seems also to be a slight tendency for English language stations to have a higher share of audience, as a group, with PPM than with the diary. In addition, it would appear that the talk stations, as a group, lose audience with PPM (down 7 percentage points in the Spring and 2 percentage points in the Fall). The two classical music stations consistently lose audience share with PPM, which

leaves the large group of “adult contemporary, hits, and rock” to pick up the difference. These stations are oriented to younger people and have slightly higher audience shares with PPM than with the diary. This is consistent with the change in the audience composition of radio as a whole, noted earlier.

Table 1 STATION AUDIENCE SHARES, MO-SU, 5:00 AM – 1:00 AM, 12+

PPM Diary CBF-FM 10.1 11.3 CBFX-FM 1.2 2.3 CBME-FM 0.0 0.3 CBM-FM 0.0 0.2 CFGL-FM 15.0 12.0 CFQR-FM 2.5 2.8 CHMP-FM 5.0 5.7 CHOM-FM 6.0 5.3 CIME-FM 0.9 1.2 CINF 1.0 1.3 CINW 0.1 0.1 CITE-FM 13.2 12.5 CJAD 0.1 0.2 CJFM-FM 4.3 3.4 CJMS 0.6 1.0 CJPX-FM 2.3 4.6 CJWI 0.4 0.4 CKAC 6.7 7.7 CKGM 0.3 0.1 CKMF-FM 15.9 13.3 CKOI-FM 14.2 13.4 WVNV-FM 0.0 0.3 WYUL-FM 0.2 0.9 100.00 100.00

The daily reach levels for individual radio stations produced by the PPM sample are, in general, higher than those of the diary sample. In some cases, the difference is quite pronounced, especially for the adult contemporary music stations, where PPM daily reach levels are consistently double those shown by the diary. Weekly reach levels by station are also much higher with PPM than with the diary and, again, it is generally the higher audience stations (according to the diary) that show the most increase in weekly reach (according to PPM). In figure 5, the weekly reach levels have been converted to logarithms to simplify the visual comparison and to adjust for heteroscedasticity in the error term. As can be seen, the larger the weekly reach, according to the diary, the larger the increase in reach shown by PPM.

Figure 5

y = 1.1218x + 0.4155 R2 = 0.9111

1 LN PPM WEEKLY REACH

0 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4

-1

-2 LN DIARY WEEKLY REACH

The higher reach levels with PPM have the same implication as the higher reach levels for radio listening as a whole, discussed earlier. For most stations, PPM discovers more listeners than the diary, but they listen less to the station than indicated by the diary. Restricting the analysis to the twelve radio stations with significant audiences, the average listener spends 9.4 hours with the station according to the diary and 3.6 hours according to the PPM. A corollary of this is that PPM listeners listen to more stations per week (4.7 on average) than diary listeners (2.2 on average). In our comparisons of a push-button television and a PPM television panel, we noticed a tendency for the audience composition of certain television programs to lose some of their age skew with PPM (Pellegrini and Purdye, 2004). The same tendency appears to take place with radio, although it is much less pronounced. Here is one example, for CFGL-FM, an easy-listening music station, which the diary shows as appealing predominantly to a 35-54 age group with 78% of its audience in this category. However, according to the PPM, 63% of the audience is in this same group. The difference is dispersed among younger and older age groups.

Light Listeners and Heavy Listeners We have seen that, compared to the diary, PPM shows radio (and individual stations) with more reach, but less average hours of listening per listener. There is a related difference in the structure of radio listening in the two measurement systems. The Diary system classifies all listeners into quintiles and we have done the same thing with the PPM sample for the average week of the eight-week period. In this case, the analysis uses average minute audiences. Each sample clearly has its own definition of the limits for each quintile, but we can compare the two.

Figure 6

100%

90%

80% 46 54 70%

60%

50%

40% 26

25 30%

Q5-Heavy 20% Q4-Med/Heavy 16 14 Q3-Medium 10% Q2-Light/Med 9 Q1-Light 6 3 0% 1 DIARY PPM PERCENT OF TOTAL LISTENING (MONDAY-SUNDAY 5:00 AM-1:00 AM)

The distribution of total radio listening hours between the five quintiles is shown in figure 6 for the diary and PPM data. Clearly the heavy-listening quintile (5) is of paramount importance for radio listening in both systems. However, it is not quite as important for PPM as for the diary. Quintile 5 accounts for 54% of all diary radio listening and 46% of all PPM radio listening. With PPM, heavy listeners listen less than with the diary, light listeners listen more.

DISCUSSION We have, of course, speculated on possible reasons for the differences discussed above. Clearly, there are two possible sources: the sample and the measurement instrument. Are different types of people attracted to complete a radio diary compared to those who find carrying around a PPM appealing? Are there inherent properties of the diary and the PPM which contribute to the observed differences? The speculation and research is ongoing at BBM but we present here our current findings and thinking. Differences in Sample Composition The comparisons in this paper used the (BBM-defined) “francophone” radio diaries from the Spring 2004 and Fall 2004 radio sweep surveys in the Montreal Extended Television Market. They also used the Montreal francophone TV panel as it existed during the weeks of the radio sweeps. Although the surveys measure the same population, the same geographical area, the same radio stations and are conducted at the same time, there are significant differences in sampling and survey methodology. The question is: are the resulting samples similar? Each methodology collects “background” demographic, socio-economic and other data on participants using questionnaires. The two samples were compared and 25 items identified where the questions were either identical or roughly equivalent in wording. The profiles of the “weighted” samples were then compared for each of these 25 items. Weighted samples were used, since they were also used in the initial comparison of radio listening. The radio diary sample is weighted by geography and age/sex. The PPM sample is weighted by geography, age/sex, household size and TV reception type. The profile comparisons show differences that remain between the two samples after these weighting processes have been applied. As with all surveys, some of the questions contained “no answers”. In these cases, the profiles were reprocessed excluding the no answers. The incidence of “no answers” varied by question and was roughly the same between the two surveys. The Diary and PPM results were then compared for each of the 25 items, using either a chi-square or a t-test. This procedure was followed for both the Spring and the Fall survey comparisons. The results of the comparisons are shown in table 2. Of the 25 items listed in the table, 11 showed a “significant” difference at the 5% level or less in both of the comparisons. Three more showed a “significant” difference in one of the two comparisons. The other 11 variables showed roughly similar distributions in the two samples.

Table 2 COMPARISON OF DIARY AND PPM SAMPLES ON COMPARABLE VARIABLES

Variable Spring Fall Direction of difference Official Language p<.001 p<.001 Diary has more bilinguals. Visit Museum/art gallery p<.001 p<.001 Diary has more who visit. Go to Movie in past year p<.001 p<.001 Diary has more who go to movies. Sporting events in past year p<.001 p<.001 Diary has more who attend. Access internet in past week p<.001 p<.01 Diary has more internauts. Diary has more graduates, fewer Education p<.01 p<.001 CEGEP graduates. Household Income p<.01 p<.001 Diary has higher household income. Household Size p<.05 p<.001 Diary has fewer one-person hholds. GICs/Term deposits p<.05 p<.05 Diary has more who own. Mutual funds p<.05 p<.05 Diary has more who own. Visit Bar/pub p<.05 NS Diary has fewer who patronize. Gardening p<.05 p<.05 Diary has more who garden. Diary has more sole principal Principal shopper p<.05 p<.05 shoppers. Diary has more MOPS, fewer blue Occupation, (Full time) p<.1 NS collar Computer at home p<.1 NS Diary has more who own Age Group NS NS (Weighting Variable for both) Visit Coffee/donut shop NS NS Fine dining restaurant NS NS Hours Worked NS NS Kids<12 in home NS p<.05 Diary has fewer homes with kids<12 Loan NS NS Mortgage NS NS Mother Tongue NS NS RSPs NS NS Sex NS NS (Weighting variable for both)

Anyone who looks at this list could find commonalties and multivariate statistical analysis might well shed further insight. However, it is clear (at least to the authors) that several threads do indeed exist. Here are four. As might be expected, given the universe definition, over 95% of each sample have a mother tongue French. (Mother tongue is defined as the language first learned in childhood and still understood.) However, there is a significant difference between the two in their knowledge of Canada's other official language, English. The diary has a consistently greater percentage of bilingual francophones than the PPM. Secondly, the diary sample is significantly more “upscale” than the PPM. The diary sample has a higher percentage of university graduates and high-income households. In one of the comparisons, it also had a higher percentage of MOPs (Managers, Owners, Professionals), although not in the other. The diary sample is more likely to own Guaranteed Investment Certificates or term deposits and mutual funds. It is (marginally) more likely to have a computer at home, and has a higher proportion of internet users. This picture is not completely one sided. Several items that would be a priori expected to belong to this cluster of items do not show a difference between the two samples: fine dining, for example. Thirdly, the diary sample has fewer people living in one-person households than the PPM panel. This is not surprising, since it is recruited via a telephone survey and any adult at home is an eligible respondent. Unless special efforts are made, a bias in favour of larger households is almost inevitable. Note that the PPM panel uses household size as a weighting rim, while the diary sample does not. The difference may be, to some extent, a simple artifact of this. Nonetheless, the difference in sample profiles, whether natural or forced, is evident in the comparison and could have an impact on the listening data. Finally, the diary sample is more “active” than the PPM sample. Diarists go to the movies, to art galleries and museums, to professional sporting events and spend more time on the internet than do PPM carriers. They even do more gardening. (Or, as an alternative explanation, they are more prone to “yea- saying” when answering long questionnaires). So, there do appear to be at least these four differences in the profiles of the two samples. The logical next question is: are the differences associated with differences in radio listening? For example, do “bilinguals” (represented more in the diary sample than in the PPM sample) listen to more radio, to more breakfast radio, choose “talk” radio stations like Radio-Canada and CKAC more than “unilinguals” (represented more in the PPM sample than in the diary sample)? In other words, can these differences in sample characteristics

© Copyright by ESOMAR® / The ARF Station to station 19 explain to at least some extent the behavioural differences noted above between the two samples? To test this, we constructed four summary measures of radio listening, and tested whether the measures were correlated with the groups of significant variables. Here are the measures of radio listening that have been used for each of the sample subgroups and a summary of the results. 1. Total amount of radio listening. Measure: The average use of radio per quarter hour for the sample subgroups, compared to the average use of radio for all adults 18+ years (Monday-Friday 5:00 am to 1:00 am). The average for all adults is 15.6%. If differences in sample composition can be used to explain differences in radio listening between the diary and PPM, then we should expect average listening for the over-represented groups to be higher than this. In fact, they are not. Bilingual respondents listen to roughly the same amount of radio as the total sample. “Up-scale” respondents (graduates, MOPs, high income) listen, if anything, to less radio than the total sample. Those living in households of one person or more listen to about the same extent as the total sample. Respondents taking part in various activities (museum visits, etc.) listen to about the same amount of radio as the total sample. 2. The relative importance of “morning drive listening”. We have used the percentage of all radio listening (Mo-Fr 5:00 am - 1: 00 am) that is accounted for by the breakfast period (Mo-Fr 5:00 am - 1: 00 am). For all adults, this is 32%. If differences in sample composition can explain the difference between the diary and the PPM with respect to the relative importance of morning drive radio listening, then we should expect to see higher figures for the over-represented groups. The results are mixed. Bilingual respondents show no abnormal tendency to listen more in breakfast than the rest of the day. The same is true for those living in households consisting of more than one person, and for the various measures of participation in activities. There is nothing here to suggest that their over-representation in the sample compared to PPM is responsible for the observed difference. However, for the “upscale” groups (MOPs, Graduates and high-income groups), this tendency does exist, although much more in the Spring than in the Fall survey. This group does tend to listen relatively more in the morning drive than the rest of the day, compared to the total sample of adults. Therefore, their over-representation in the diary may possibly be contributing to the observed tendency for the diary to show relatively more

listening in the morning drive than the PPM. Quantifying this factor is not possible from this analysis but in all likelihood it is not enough to account for the size of the observed difference between the two systems. 3. The relative importance of weekend, compared to weekday, listening. The measure used is similar to that used in the previous test: the percentage of all radio listening (Mo-Su 5:00 am – 1:00 am) that is accounted for by weekend listening (Saturday and Sunday). For all adults, this is about 19%. If differences in sample composition can explain the difference between the diary and the PPM with respect to the relative importance of weekend radio listening, then we should expect to see a higher ratio for the groups over-represented in the diary survey. There are no such differences. The proportion of radio listening accounted for by the weekend is about 19% for all the sample subgroups where the diary sample is over-represented compared to PPM. 4. Differences in audience share. We have compared the audience share for all adults 18+ years to the audience share for each of the groups over- represented in the radio diary survey. If there are differences in share in the predicted direction, we can take this as evidence that sample composition is affecting audience shares. The “predicted direction” is higher for “talk” stations and lower for “music stations”: PPM yields lower audience shares for “talk” stations: and shows higher audience shares for stations that may be termed “music” stations. If the observed differences in sample composition are influencing this result, then we should expect sample groups over-represented in the diary sample to show higher audience shares for the two talk stations and lower audience shares for the four music stations. Is this the case? The results for talk stations are mixed, but tend to support the hypothesis. MOPs, University graduates, high-income households, those who go to galleries and museums, and to a lesser extent bilingual respondents, are all groups that are over-represented (relatively) in the diary survey. The audience share for the talk stations taken as a group is also higher among these audience categories, although not consistently so for both surveys or for all stations in the group. For the other over-represented groups, the share of talk stations is about on a par with the average for all respondents 18+. Results are also mixed for the music stations taken as a group. For MOPS, university graduates and art gallery hounds, the audience share of these four stations is lower than the average for 18+ years. However, the share for the other groups is either essentially the same as or higher than average.

In sum, there are significant differences in the composition of the PPM and the Diary samples on a wide range of variables. The diary sample tends to have a relatively greater proportion of bilingual respondents, “upscale” respondents (as measured by occupation, education and household income, larger households (i.e. households of more than one person) and more “active” respondents (as measured by a small list of self-reported activities). However, the practical importance of these differences as an explanation of differences in estimates of radio listening between PPM and the diary is modest. This was tested by comparing groups relatively over-represented in the diary sample with the total for various measures of radio listening. The diary shows slightly more listening to radio than PPM, even when corrections are made for differences in the strict definition of listening. However, sample subgroups over-represented in the diary sample do not show more radio listening than average – indeed for some of them the reverse is the case. On this evidence, differences in sample cannot explain the small difference in reported listening between PPM and the diary. The diary shows significantly greater listening to radio in the breakfast, or morning drive period, than PPM. “Upscale” respondents (MOPS, graduates and high-income homes) also show relatively more radio listening in this period. This is not true for other groups over-represented in the diary survey. On this evidence, differences in sample could explain part of the difference in the importance of the morning drive period between the diary and the PPM. However, there is no evidence that differences in sample composition account for the relatively greater importance of weekend listening in the PPM results. Audience shares are different for PPM and the diary. The diary shows relatively higher audience shares for “talk” stations (taken as a group) and relatively lower audience shares for background music stations (taken as a group). Again, evidence for the possible impact of differences in sample composition on this result is mixed. It is possible that the relative over- representation of “upscale” respondents in the diary sample is contributing to this difference. Taken together, the results suggest that the relative skew to bilingual respondents, larger households and more active respondents in the diary survey seems to have no effect on the results. However, the “upscale” skew may be contributing to differences in estimates for the importance of morning drive, and in audience shares.

Differences in the Definition and Measurement of Radio Listening We have already discussed differences in the formal definition of radio listening between the diary and PPM. We noted that the diary counts a listener for a whole quarter hour if he/she listened to at least five minutes within it. This is different from the normal unit of measure for TV electronic meter panels, like the PPM, which use units of one minute. In the comparisons made above, the PPM data was recalculated using the diary definition of listening, which approximates a 15-minute reach measure with a 5-minute non- consecutive minimum. However, the measurement issue is much more complex than this (Pellegrini, 2005). It is relatively clear what the PPM measures: exposure to an encoded radio signal. If the PPM is being carried or worn by the respondent, it will pick up any codes to which it is exposed, with known tolerances. “Listening” equals exposure, and the recording of exposure is immediate, with no recall required. Exposure is recorded in home and out of home quite automatically. This exposure is verified by a motion detector – an integral feature of the PPM. Motion data is downloaded nightly with the exposure data, and is used for daily qualification of respondents. Currently, respondents are rejected for a day if there is less than four hours of motion. The great majority of panel members have more motion than this: the PPM is out of its dock for a median 15.3 hours per day and motion is detected for a median 14.4 hours per day or about 95% of the time it is out of the dock. These results are comparable with those reported by Arbitron during its Philadelphia test (Patchen and Webb, 2002) as well as other tests around the world (Pellegrini, 2004; Pellegrini and Purdye, 2004; Futsaeter, 2004; Poesmans, 2004). Although there is a high degree of carry-around compliance, it does show a (weak) association with radio listening: the more somebody carries around a PPM, the more they are exposed to radio. This could expose the PPM listening estimates to the possibility of an upward bias. This is illustrated in figure 8 which shows average hours of radio listening (y-axis) for different ranges of carry-around time. Ignoring the first two data points which are based on very small sample sizes (and are non-compliant, out of tab respondents), there is a slight positive slope to the curve. With this small caveat, the definition of radio listening inherent in the PPM is clean and clear: exposure to a radio station, verified by data from a motion detector.

Carry Around vs. Listening The diary is a different instrument, however. The written instructions given to the respondent ask him/her to record his/her “listening” without attempting to define what is meant more closely than the “five minute rule” discussed earlier. Other than that, the definition is left to the respondent (some examples include: paying attention, radio on or off, radio in the background, vaguely aware, etc.). It is evident from the structure of the diary that the respondent must know the name of the station, its dial position, a slogan, or some other identifying information. The only way to indicate listening is to write down that a station was listened to within the quarter hour. The respondent must also have some external time reference; he/she must know the particular quarter hours within which listening took place. To recapitulate, to record his/her “listening” the respondent must know the quarter hour, the identity of the station listened to, and whether the listening was for at least five minutes within the quarter hour. Respondents are exhorted to “carry your diary with you at all times, so you can record your listening wherever it occurs during the survey week”. However, it is clear from debriefing interviews that this is more honoured in the breach than in the observance. Most respondents complete their diaries a couple of times a day or once a day. Problems of recall and memory cast a fog over the respondent’s ability to answer the three questions: which quarter hour, which station, at least five minutes? And this problem is compounded by not knowing exactly what personal definition respondents are applying to “listening”. We can illustrate the problem by looking at the PPM exposure data. In the comparisons provided earlier, we paralleled the formal diary definition by qualifying PPM listening on a quarter hour basis using a five-minute minimum. For example, in figure 2 we showed quarter-hour by quarter-hour estimates of PUR for the diary and the PPM for an average weekday. How does the PPM data change if we use a qualification other than five minutes? The answer is in figure 7 which shows the original listening curves from figure 2 (Diary and 5 minute PPM) plus additional curves from PPM for 1, 3 and 7 minutes). All the PPM curves move in parallel, but there is a wide spread between them (the stricter the qualification rule, the lower the curve). How accurately diary respondents are interpreting the 5-minute rule is clearly crucial in any comparison with the PPM. (If they are actually using a 1-minute rule, for example, the gap between diary and PPM in the morning drive period is attenuated, but the difference increases for the rest of the day).

Figure 7

15 PUR (%) PUR

1MIN 3 MINS 5 5 MINS 7 MINS DIARY 0 05:00 05:15 - - 06:00 05:45 06:45 - 06:30 07:15 07:30 - 08:00 08:15 - 08:45 09:00 - 09:45 - 09:30 10:15 10:30 - 11:00 11:15 - 11:45 12:00 - 12:45 - 12:30 13:30 - 13:15 14:00 14:15 - 14:45 15:00 - 15:30 - 15:45 16:30 - 16:15 17:15 - 17:00 17:45 18:00 - 18:30 18:45 - 19:15 - 19:30 20:15 - 20:00 20:45 21:00 - 21:30 21:45 - 22:15 - 22:30 23:15 - 23:00 23:45 24:00 - 24:30 24:45 - M O-FR 5:00 AM - 1:00 AM

Recording of out-of-home radio is known to present special problems for the radio diary since entries are made ex post facto. Even if the diary is carried around, it is difficult to believe that entries are made while driving, or at the office. Not surprisingly, the PPM shows relatively more radio listening for groups for whom out-of-home listening is important. For example, PPM shows more listening for men (60% of listening away from home) than for women (44% of listening away from home). Similar correlations apply to different age groups: younger respondents show more listening with PPM than the diary. They also do proportionately more of their listening out of home. The problem of interpreting exactly what respondents have in their heads when they make an entry in a radio is exacerbated by a further problem. As indicated earlier, BBM uses a “household-flooding” approach to sampling: samples of households are drawn and then each member of selected households is asked to complete a personal radio listening diary. It is known that a certain proportion of returned diaries are “kept diaries”, that is, they are completed by one household member on behalf of another. In most cases, this is done on a co-operative basis. (The diary minder asks the diary recipient what he or she listened to during the day). Nonetheless, this adds to the problems of deciphering what “listening” is being reported and how it is being filtered and recalled.

As indicated earlier, PPM estimates radio listening with more listeners and less average time spent listening than the diary. Radio reaches more people, but they listen less, thus standing conventional wisdom on its head: radio is, after all, supposed to be a “frequency” medium. Again, this appears to be a logical consequence of the different definitions of listening used by the two methodologies – passive measurement of exposure and conscious recall of “listening.” The diary has fewer listening events, but they are longer; the PPM has more, but shorter listening events (Patchen and Webb, 2002).

SUMMARY AND CONCLUSIONS We have compared parallel systems for the measurement of radio in Francophone Montréal. The comparisons use the same time, the same universe, the same list of radio stations and the same geographical area. The only differences are the system of measurement (diary vs. PPM) and the sample. Once corrected for a difference in the definition of radio listening, the two systems yield almost identical estimates of the total amount of radio listening. However, the two do produce different pictures of the distribution and structure of radio listening. Compared to the diary, PPM shows radio with a higher daily reach and less listening per listener; the two drive periods are of less importance; evening and weekend radio are of greater importance. In the world of PPM, radio is more masculine and younger. Its heavy listeners listen less while its light listeners listen more. Although the shares of audience produced by the different systems are very similar, it seems that PPM yields slightly larger shares of listening for popular music stations, slightly smaller shares of listening for talk radio stations. These differences can trace to differences in sample composition or to differences inherent to the two methodologies. We have demonstrated that, although the diary and the PPM attract different types of people to their respective samples, they are generally uncorrelated with the substantive differences found in the comparisons. Our discussion on the definitions of radio listening used by the PPM and the diary leads us to believe that they are measuring, at times, different things and that this is the predominant reason for the two different pictures of radio listening that we outlined earlier. To test this, we are currently conducting two simple but valuable experiments. Respondents to a radio diary survey are being asked to carry around a PPM for a few weeks. In addition, rotated members of the PPM panel are being asked to complete a one-week diary of their radio listening. Our prediction is that each of these paired comparisons will show differences in the directions indicated in this paper. The PPM will show less listening to radio in the

morning drive period, more listening in the evenings and on weekends, more reach and less average listening per listener. We await the results anxiously.

REFERENCES BBM Canada (1999). Fall ’98 Meter vs. Diary Data: A comparison of apples and oranges. BBM Canada. Futsaeter, Knut-Arne. (2004). Testing of PPM for Radio in Norway, The 2004 European Radio Symposium, Berlin. Patchen, Robert H. and Beth M. Webb. (2000). A Full Year of Audience Research with PPM. What we’ve learned so far. ARF-ESOMAR WAM Proceedings, Los Angeles. Patchen, Robert H. and Beth M. Webb. (2002). The Future is Now: The Very Latest Finings from the US Market Launch of the Portable People Meter in Philadelphia. ARF- ESOMAR WAM Proceedings. Cannes. Pellegrini, P.A. (2005). Listen without Prejudice: Passive measurement for Radio, TV and beyond. VUE: the magazine of the Marketing Research and Intelligence Association (forthcoming). Pellegrini, P.A. (2004). Personal Meters: Tests, Trials and Comparative Studies. The 2004 European Radio Symposium, Berlin. ASI, UK. Pellegrini, P.A. and Ken Purdye (2004). Passive vs. Button Pushing: A Comprehensive Comparison from Parallel TV Meter Panels in Quebec. ARF-ESOMAR WAM Proceedings, Geneva. Poesmans, Daniel. (2004). One year PPM Panel – a confrontation between expectation and reality. The 2004 European Radio Symposium, Berlin.

THE AUTHORS Pasquale (Pat) A. Pellegrini is Vice President Research, BBM Canada, Canada. Ken Purdye is Meter Consultant, BBM Canada, Canada.