The paradox of measuring success of nations in elite sport
Veerle De Bosscher, Bruno Heyndels, Paul De Knop Vrije Universiteit Brussel Maarten van Bottenburg Utrecht University Simon Shibli Sheffield Hallam University
ABSTRACT The achievement of international and especially Olympic sporting success is increas- ingly important to a growing number of countries. It is however not clear how success is defined and can be measured. The number of medals won in Olympics Games and other international sport competitions offers the most self-evident and transparent mea- sure of success in high performance sport. In this article different methods to measure success of nations are compared. Market share was identified as the best measure of absolute success which enables meaningful time series analysis to be conducted. A Linear regression analysis is used to introduce relative success as a measurement of success when controlling for macro determinants such as population and wealth. This method allows comparing nations on more equal grounds, which is necessary if one wants to measure effectiveness of elite sport policies. Similar analysis is done for Olympic Summer and Winter Sports. It is concluded that conflicting results can be given on nations' success. Defining success therefore depends on the purpose wherefore it is used and on the priorities of individual nations.
KEY WORDS: elite sport, international success, measuring success, Olympic success
RÉSUMÉ MESURER LES PERFORMANCES DES NATIONS EN SPORT DE HAUT NIVEAU: UN PROBLÈME PARADOXAL Les performances sportives au niveau international, et particulièrement au niveau olym- pique, prennent de plus en plus d’importance pour un nombre croissant de nations. Mais il n’est pas évident de définir ni de mesurer le succès. Le nombre de médailles gagnées lors de Jeux olympiques ou autres compétitions internationales représente la mesure la plus objectivable et la plus transparente du succès dans le sport de haut niveau. Dans cet article, nous comparons différentes méthodes permettant de mesurer ce genre de performances. La part de marché a été identifiée comme la meilleure mesu- re de succès absolu permettant de procéder à une analyse de séries temporelles signi- ficative. Une analyse en régression linéaire est utilisée pour présenter le succès relatif comme mesure du succès lorsqu’on maîtrise des facteurs macro-déterminants tels que la population et la richesse. Cette méthode permet de comparer les différents pays sur
BELGEO • 2008 • 2 217 des bases plus égales, ce qui est indispensable si l’on veut mesurer l’efficacité des poli- tiques en matière de sport d’élite. Nous procédons ensuite à une analyse similaire pour les Jeux olympiques d’été et d’hiver, avant de conclure que nous parvenons à des résul- tats contradictoires quant aux performances des différents pays. En effet, le succès dépend du but dans lequel on l’utilise ainsi que des priorités de chaque nation.
MOTS-CLÉS: sport d’élite, succès international, mesure du succès, succès olympique
INTRODUCTION
long with the FIFA Football World Cup, in both the literature and media with Athe Summer Olympic Games is the respect to international comparisons. most high profile event in the global sport- Success is typically expressed in ing calendar. In recent Games, the absolute terms, such as the total number International Olympic Committee (IOC) of medals that a country wins during the has sought to make the event truly global. Olympic Games or other championships. This point can in part be appreciated by Such comparisons of countries, however, looking at the number of nations taking give a biased view of how successful part in the summer Olympic Games: an countries actually are in using their avail- increase from 14 in 1896, over 59 in 1948 able resources. These accounts do not to 201 in 2004. The number of athletes take into consideration a number of socio- competing in the Games started with 241 economic and political variables that play in Athens 1896 and has more than dou- an important part in determining each bled in the period 1980 (5,217 competi- country's success. The Olympic Games tors) to 2004 (11,099 competitors). It is are still dominated by a small number of thus reasonable to conclude that the capitalist core and (formerly) socialist summer Olympic Games have become countries (Green & Houlihan, 2005; increasingly competitive and medals Stamm & Lamprecht, 2001). One should have become relatively harder to win. As hardly be surprised that large and rich the supply of medals (success) will countries like the United States tend to remain essentially fixed in the future (the win more medals than Zimbabwe or IOC has indicated that it would like the Ecuador. Yet there are exceptions, such number of events to be capped at around as Cuba, Ethiopia and Kenya. The ques- 300), and the demand for success is tion is how external factors should be increasing (more nations taking part and taken into account when defining suc- more nations winning medals), the «mar- cess. The aim of this paper is to examine ket» adjusts by raising the «price of suc- various methods by which the outputs of cess» (Shibli, 2003). Nations will have to an athlete production system can be mea- invest even more just to maintain their sured. The definition of absolute national success. It is implicit that the «produc- success using the medal tables from the tion» of successful elite athletes by 2004 Athens Games will be taken as our nations is an output from a strategic plan- point of departure. We shall then proceed ning process (De Bosscher, 2007). It fol- to examine relative success, or rather lows that if nations are adopting a strate- success controlling for economic, socio- gic approach to the production of elite logical and political determinants. athletes, then part of that process must be Furthermore we also examine the limita- to evaluate the results achieved (outputs). tions of the analysis and propose some The definition of sporting success is a alternative measures using other events. theme which has been widely discussed
218 The paradox of measuring success of nations in elite sport MEASURING ABSOLUTE SUCCESS OF NATIONS
Although the IOC does not recognise the dimensions of «quality» (for example the Olympic medal table as an order of merit, value of a medal in athletics may be dif- it is widely accepted outside of the IOC ferent from a medal in archery). A method that the final medal table for each games which allows for the relative values of is an order of merit. This finding is per- gold, silver and bronze medals is a haps best demonstrated by the fact that «points» system which makes use of a many nations invest heavily in sport pre- weighting system to convert a nation's cisely to climb the medal ranking. Several medal haul into a points equivalent. This authors used different methods of mea- methodology is standard practice when suring absolute success, i.e. without tak- analysing Olympic results (see for exam- ing into account socio-economic vari- ple Ball, 1972; De Koning & Olieman, ables. Some methods will be further 1996; Den Butter & Van der Tak, 1995; explored in the next part. Van Bottenburg, 2000). These are usually In its most simple form, the total number weighted as follows: a gold medal gets 3 of medals that a country wins serves as «points» (or 4), silver receives 2 and a the standard measure for absolute suc- bronze just 1. Applying this point system cess (see for example Hoffmann, Ging & to the Athens 2004 medal table for the top Ramasamy, 2002a; Gärtner, 1989; 10 nations means that the United States Grimes, Kelly & Rubin, 1974; Kiviaho & was the most successful country, followed Mäkelä, 1978; Levine, 1974; Novikov & by Russia and China (see Table 1). Maximenko, 1972). The argument for Table 1 shows that the ranking positions using absolute scores is based on the for the top ten nations is the same for total fact that modern competitive sport at an medals or weighted points. Exceptions elite level is absolute in nature. Victories can be found if one looks exclusively at and point scores are what counts (Kiviaho the number of gold medals that have & Mäkelä, 1978). A key weakness with been won. For example, ranked on the this absolute medal table is that it does basis of gold medals, China with 32 gold not account for the «quality» of medals medals (total medals 63) would be ranked won (gold, silver and bronze) nor other ahead of Russia which won 27 gold
Points Total Rank Country Cumulative 3-2-1 Medals percentage
1 United States 212 103 11,6% 2 Russia 173 92 21,0% 3 China 144 63 28,8% 4 Australia 99 49 34,2% 5 Germany 92 48 39,3% 6 Japan 78 37 43,5% 7 France 64 33 47,0% 8 Italy 63 32 50,4% 9 Korea (South) 60 30 53,7% 10 Great Britain 57 30 56,8%
Table 1. Absolute success of the 10 best performing countries in Athens 2004, ranked according to weighted medals. Source: IOC, 2004
BELGEO • 2008 • 2 219 medals (total medals 92) (Shibli & (2003 a & b) also found the number of par- Bingham, 2005). There is a difference of ticipants in the Games per country to be an 67 medals (103-33) and 155 medal points indicator for success. between the first ranked nations, the When these methods are compared -as a United States and the tenth nation (Great- general rule- it does not make much of a Britain). The United States won 11.6% of difference if and how the number of the medal points on a total of 1832. The medals are weighted or not, nor whether top ten nations together win more than the first three or the first eight places are 50% of the medals, of which the first three taken into account (see Table 2) (De nations allready take 29%. The remainder Bosscher, 2007). 43% of the medals are divided among 65 In Table 2, high (Pearson) correlations can nations. be observed for all methods. This means Whilst medal based measures of perfor- that, in general, a country that gets a high mance are easily understood measures of score using one method will also have a success, they still ignore the totality of high score using another. Even the num- achievement of an elite sport programme. ber of participants in the Games from As has already been demonstrated, com- each country is significantly correlated to petition for medals is increasing as more the number of medals won (r>0,8). In nations take part in the Olympic Games. It other words, the selection for achieving is quite possible for Performance Directors success has already taken place, that is in individual sports to make considerable before participation in the Games with progress in developing a sport without this norms that are, on the one hand, set by progress being translated into medals in the IOC and, on the other, by each elite competition. Therefore, a number of National Olympic Committee (NOC). studies examine the first six (Shaw & Nevertheless, these correlations are only Pooley, 1976; Gillis, 1980) or eight places statistical methods that never give a cor- (Condon, Golden & Wasil, 1999; Kiviaho & relation of 100%, which means that there Mäkelä, 1978; Stamm & Lamprecht, 2001), will always be exceptions to this generali- using a weighting or 6-5-4-3-2-1 or 10-8-6- sation of methods, like in the example of 5-4-3-2-1 respectively. Kuper and Sterken China and Russia given above. Great
Med total Med gold Med weighted First 8 Participants (3–2–1) (10–8–6–5–4-3-2-1)
Total medals 1,000 0,971** 0,997** 0,985** 0,886**
Gold medals 1,000 0,984** 0,964** 0,849**
Medal, weighted 1,000 0,984** 0,882**
First 8, weighted 1,000 0,901**
Total participants 1,000
Table 2. Correlations (Pearson) of various indicators of absolute success during the Olympic Games in Athens 2004: total number of medals and gold medals, weighted medals, first 8 places and number of participants (N = 201). ** correlation is significant at 0,01 level
220 The paradox of measuring success of nations in elite sport Britain is more successful than Italy in increasing trend was noticed in France terms of top eight places whereas Italy and Italy until 1996 and in Australia until won two more medals (of which one was 2000 with slightly decreasing perfor- gold). mances thereafter. A decreasing trend Using one Olympic Games may result in can be observed for Great Britain until coincidental rankings. For example the 1996 and Korea until 2000; both nations Netherlands won only 4 gold medals (22 increased their performances thereafter. in total) in Athens but they won 12 gold Although the UK won fewer medals in (25 in total) in Sydney 2000. To overcome Sydney (28 medals) than they did in the problem of success by coincidence, Athens (30 medals), their market share in countries must be evaluated over time. Sydney was higher (3.28%). After the uni- Whilst the points system is a more useful fication of East and West Germany in measure of performance than position in 1989 and the break up of the former the medal table or total medals won, it has Union of Soviet Republics (USSR), the one major limitation. As the number of United States is constantly outperforming events contested at each Games has var- other nations with a market share above ied considerably over time and to a less- 11%. Another point that can be observed er extent the number of points per event from Figure 1 is the narrowing gap in mar- has also varied (for example two or more ket share between the ten best perform- nations «tying» for the same medal), the ing nations in Athens 2004 compared to number of points available at each the ten best performing nations 1988 (of Olympic Games has also varied. In order which 4 were not in the top ten list in to convert points won into a standardised Athens). While the difference in market measure, SIRC (2002) offers the principle share between the first and tenth nation of computing «market share», that is, was 16,8% in Seoul (not in figure 1), it is points won as a proportion of points avail- only 8.5% in Athens. This finding demon- able to win. Using market share it is pos- strates how «traditional market leaders» sible to make a more accurate time series like x, y and z are confronted with an diagnosis in standardised terms. Figure 1 increased competition from new entrants illustrates the temporal evolution in the in the Olympic market. This may reflect share of medals for a selection of the ten the more general tendency of increased best performing countries (in medal competitiveness in sports. points in Athens 2004) over the period A key question arising from the data pre- 1988-2004. There is a logic to starting sented in Figure 1 and Table 2, is how with Seoul in 1988 as this was the first might nations be expected to perform games since 1972 that had not been con- given the resources at their disposal? In taminated by some form of boycott. The the previous methods, success continues market share shows for example that in to be defined in «absolute terms»; exter- the 301 events contested in Athens, a nal (socio-economic) influences are not total of 1,832 medal points was awarded, taken into account. The medal success of which the United States won 212. This (output) is not presented in relation to its is a market share of 11.6%, which in turn determinants (input). Bernard and Busse is slightly more than the 203 points won (2004) raise the question of how many out of 1,829 (300 events) in Sydney 2000, medals a country would need in order to a market share of 11.1%. be classified as successful. In other In terms of market share, Figure 1 reveals words, what can a country expect in that only four of the top ten nations (in terms of medal outputs, given its inputs Athens) have increased their perfor- (resources). This is the definition of rela- mances from Sydney to Athens: USA tive success (De Bosscher, 2007). We (increase of 0.5%), China (a remarkable contend that when establishing indicators increase of 2.1%), South Korea (0.3%) for national sporting success, one must and Japan (a remarkable increase of take the socio-economic and political dif- 2.3%). Over a longer period a steadily ferences between countries into account.
BELGEO • 2008 • 2 221 18,0%
16.3% 16,0%
15.6% 15.1% 14,0%
13.6% 13.4% 12,0% 11.2% 11.6% 10.4% 11.1% 10,0% 10.0% 9.4%
8.2% 8,0% 7.9%
6.8% 7.5% 6.4%
5.8% 6,0% 5.8% 5.6% 5.4% 5.0%
4,0% 3.6% 4.1%
3.0% 2,0% 1.5%
0,0% 1988 1992 1996 2000 2004
United States Russia (1988: USSR/1992: CIS) China Australia Germany (east+west)
5,0%
4,5% 4.5% 4.3% 4.3% 4.3% 4,0% 3.8% 3.7% 3.7% 3.5% 3,5% 3.4% 3.3% 3.4% 3.3% 3.1% 3.1% 3,0% 3.0% 3.0%
2,5% 2.3% 2.1% 2,0% 2.1% 2.0% 1.8% 1.6% 1,5% 1.5% 1.4%
1,0%
0,5%
0,0% 1988 1992 1996 2000 2004
Japan Korea (south) Great Britain France Italy
Figure 1. Market share of ten best performing countries in Athens during the Olympic Games between 1988-2004.
222 The paradox of measuring success of nations in elite sport MEASURING RELATIVE SUCCESS OF NATIONS
The aim of this part is to present a method index. When we divide the number of whereby some of the differences between medal points in Athens by the per Gross countries on the macro-level, which are of Domestic Product per head of the popu- major significance in international suc- lation China becomes the most success- cess and cannot be influenced by poli- ful nation. cies, are controlled. This allows us to con- Taking into account just one determinant, struct indicators for relative success, i.e. population size or wealth, is rudimentary success-indicators that control for exoge- in two respects. Firstly, it disregards other nous macro-influences. This approach potentially important determinants. looks at the efficiency of sports systems Secondly, it assumes an implicit linear when the national characteristics of relationship between these two factors Olympic success are isolated. and success. By dividing medal points by Numerous empirical studies show that population or wealth (criteria that are population and wealth are the most impor- sometimes used by the media) the tant socio-economic determinants of suc- degree to which these factors can influ- cess (see, for example, Bernard & Busse, ence success is not taken into account. 2004; De Bosscher, De Knop & Heyndels, This creates a potentially biased view. 2003 a & b; Jokl, 1964; Johnson & Ali, After all, a country that has twice as many 2002; Kiviaho & Mäkelä, 1978; Levine, inhabitants cannot win twice as many 1974; Morton, 2002; Novikov & Maxi- Olympic medals. This is the principle of menko, 1972; Suen, 1992; Van Botten- decreasing returns to scale (Glejser burg, 2000). These two variables fre- 2002). This derives from, among others, quently explain over 50% of total medals institutional characteristics mentioned or medal points. For this reason, success above such as the fact that countries are has also sometimes been expressed in allowed to send to the Games only a lim- terms of medals per head of population or ited number of athletes who have met the in terms of per capita GDP, as a measure- IOC criteria. ment for wealth. To assess whether a particular country or It is obvious that the size of a country's group of countries does «well» at the population will be a determining factor for Olympic Games, the literature offers a sporting success (De Bosscher, 2007). number of methods. These take both of The bigger the population, the larger the the aforementioned criticisms into pool from which talent may be recruited account: (a) several determinants of suc- and the greater the opportunities to cess at the same time and (b) possible organise training and competitions. non-linear effects. Several studies Divided by population the Bahamas, with explored the relationship between inter- two medals was the most successful national sporting success and the country in Athens. (macro) economic, sociological and polit- There are reasonable explanations for the ical context within which sporting talent fact that wealthy countries perform better. thrives. Most studies used simple correla- Richer countries can invest more in sport tions and regression analysis. During the and elite sport, individuals may partici- last decade, some authors have tried to pate in a broader number of sports and a improve the methodology of these studies higher living standard may improve their (see for example Baimbridge, 1998; general fitness and ability to perform at Bernard & Busse, 2004; De Bosscher, De top level. Den Butter and Van der Tak Knop & Heyndels, 2003 a & b; De Koning (1995) have found that the number of & Olieman, 1996; Den Butter & Van der medals won correlates strongly with Tak, 1995; Johnson & Ali, 2002; Tcha & income (GDP) as well as with more gen- Perchin, 2003). These studies revealed eral welfare indicators, such as the human that aside from the «key» environmental development index or the quality of life factors for success (wealth and popula-
BELGEO • 2008 • 2 223 tion), which have already been addressed This dummy is equal to 1 for (former) above, the main explanatory factors for communist countries and 0 for other success are: area, degree of urbanisa- countries. β1 to β6 are the regression tion, religion and political system. These coefficients to be estimated. ε is the error variables are the inputs in the production term for the regression model, which is of sporting success that cannot be con- the unknown variation (the vertical devia- trolled by sports policies. tion from the unknown true regression The starting point for our empirical work is line)(2). a simple OLS (Ordinary Least Squares) Initial analysis examined the fitting of the estimation of a reduced form model that model described by the equation above captures the main macro-determinants of to the entire data set. After diagnostic absolute Olympic success. The «outputs» tests to see if there were any outlying val- are the weighted number of Olympic ues (with respect to their Y-values and/or medals that a country has won. their X values) that could influence the appropriateness of the fitted regression LINEAR REGRESSION ANALYSIS function (Kutner, Nachtsheim, Neter et al., 2005), Jamaica was omitted and 74 With the Athens 2004 Olympic Games as cases remained in the sample(3). In the a point of departure a linear regression final model three independent variables analysis (OLS) of socio-economic factors were significant: (Ln (population), Ln that influence (absolute) sporting success (GDPCAP) and communism (dummy). was carried out using the Statistical The results are shown in table 3. Package for the Social Sciences (SPSS). A stepwise regression (whereby signifi- Here the logarithm of the weighted cant explanatory variables are added one medals (gold=3, silver=2, bronze=1) for by one and deleted one by one), as the Olympic Games in Athens in 2004 are shown in Table 3 indicates that the popu- taken as the absolute success measure- lation size is responsible for 19.3% of the ment. international success. Wealth adds anoth- Taking into account the aforementioned er 19.5% and together with the political factors, the functional form to be estimat- system for (former) communist countries, ed is as follows: we end up with a model where 52.4% of the international success is explained.
Ln MED (weighted points) = E0 + E1 Ln (POP) + E2 Ln (GDPCAP) + This model was used for further analysis
E3 Ln (DENS) + E4 MUSL + E5 PROT + E6 COMM + H of the residuals. The residuals are normal- ly distributed and constant and no sys- In the above(1), Ln (POP) is the (logarithm tematic pattern is present (homoscedas- of the) number of inhabitants, that were ticity). This was confirmed with a recorded in millions. Ln (GDPCAP) is the Kolmogorov Smirnov Test for normality, (logarithm of the) Gross Domestic Product which was not significant (sign. = 0.200) per head (recorded in US dollars). Ln and a Shapiro Wilk (sign. = 0.488). The (DENS) corresponds with (logarithm of following section covers in greater depth the) population density (population/ area). this residual analysis as a way to deter- Given that we control for the number of mine success of nations controlling for inhabitants, this variable takes the possi- macro-level determinants. ble influence of the area of a country into account. These three independent vari- IDENTIFYING RELATIVE SUCCESS OF ables and the dependent variable have NATIONS been transformed into logarithms, to account for non-linear effects in the spec- A regression analysis serves two purpos- ification. MUSL and PROT are related to es. First it identifies the determinants for the «religious» structure of a country and international success on the macro-level. includes the percentage of Muslims, and Second, under ceteris paribus conditions, Protestants respectively. COMM is a an analysis of residuals allows the com- dummy for (former) communist countries. parison of countries and thus also the
224 The paradox of measuring success of nations in elite sport
Dependent Independent Coefficients t-value Adjusted R² variable variables Ln(Population) .193 Ln(Population) .388 Ln(GDP/cap) Athens 2004 (Constant) -5.532 -5.405 Ln (Medal points) Ln(Population) .511 7.215 .524 Ln(GDP/cap) .669 6.534 Comm 1.068 4.620
Table 3. Stepwise Ordinary Least Squares for weighted number of medals (gold = 3, silver = 2, bronze = 1) in Athens 2004.
Note: The stepwise estimation showed exactly the same results as a normal «enter» method.
Residual: variation that cannot be explained by socio-economic variables = relative success of countries Absolute Success
Population, GDP/head or communism
Figure 2. Graphic representation of a linear regression and the policy as part of the residual.
determination of their relative success. the basis of macro-economic determi- The analysis of residuals compares this nants. The degree to which the country «prediction» with the weighted number of performs «better" is reflected in the size of medals, actually won. Using a «case-by- the residual. case» analysis, we can answer the ques- Applying this statistical technique to our tion: «which countries are successful, if study, the starting point of our analysis is one accounts for socio-economic vari- now that the residual represents (partly) ables?» Figure 3 illustrates, in a two- the effects of elite sport policies. In other dimensional space, (that is for a situation words, the positive residual of nations is in which we only analyse one explanatory among others the result of effective elite variable), the regression line that is the sport policies, which is part of the unex- best fitting line of a topographical point plained variance. It should be noted that system (Ottoy, Van Vooren & Hughe, some other factors will, to an unknown 1993). The points in the figure show the extent, also influence the size of the resid- positions of the respective countries. The ual. Examples are the elite sport culture, regression line divides the points (coun- the tradition of sport and sporting success tries) into two groups. A successful coun- in a nation and maybe just «coincidence». try is one (above the regression line) that However, these factors cannot be fash- performs better than one would expect on ioned by policies. Although it is not known
BELGEO • 2008 • 2 225 to what extent the residual can be these nations would won respectively explained by elite sport policies, these are 1.70 and 1.03 times more medals than the only factors that can be fashioned. expected. Moreover, the results showed Table 4 offers an overview of the ten coun- that the United States is ranked 10th, still tries with the greatest relative success in with a positive residual (0.93). Athens, taking the differences in popula- Some rich Western European countries tion size, wealth and, where relevant, the perform less than predicted, for example: (former) communist character of the Belgium (65th), Sweden (44th), Austria country into consideration. This ranking (52th), Finland (59th), Norway (64th), reflects the regression residuals [residual Switzerland (68th). = absolute success - predicted success] When Table 4 is compared to the top ten in the multidimensional model with only of nations in terms of absolute success the significant variables. (Table 1), it can be seen that only three Table 4 shows that Cuba was the most nations are mentioned in both tables: the successful country in Athens in terms of USA (ranked first in absolute success), relative success, followed by Australia, Russia (ranked second) and Australia Kenya, Ethiopia and Greece. Kenya's (ranked fourth). Hence, these nations are third and Bahamas' seventh position is highly successful when both absolute and noteworthy. On the basis of the socio-eco- relative measurements are used. nomic context it was anticipated that
Rank Country Predicted Residual 1 Cuba 2.11 1.84 2 Australia 2.87 1.73 3 Kenya 0.86 1.70 4 Ethiopia 1.01 1.63 5 Greece 2.30 1.22 6 Korea (South) 3.00 1.09 7 Bahamas 0.36 1.03 8 Russia 4.16 0.99 9 Zimbabwe 0.82 0.98 10 United States 4.42 0.93
Table 4. Ten best performing nations during the Olympic Games in Athens according to OLS method (relative success), with Log (points of medals) as the dependent variable and Log (pop), Log (GDP/head and communism as independent variables.
226 The paradox of measuring success of nations in elite sport OTHER MEASUREMENTS OF PERFORMANCE: THE OLYMPIC WINTER GAMES
The preceding analysis has focused pri- MARKET SHARE DURING THE WINTER marily on the Summer Olympic Games, OLYMPIC GAMES illustrating different methods using the Athens Games as a case study. The For the last five Winter Olympic Games, we Olympic Games are a truly global event have replicated the market share figures containing a portfolio of different sports for the ten best performing nations in Turin that are popular and recognised in a num- 2006 and these are shown in Figure 3. ber of nations. Moreover, Olympic perfor- From Figure 3 three general conclusions mances are seen as the ultimate perfor- can be drawn. First, the market share of mance in many sports. In this regard the Winter Games for the top ten nations is Olympic Summer Games were indicated higher than for Summer Games. This find- to be the best measure of the overall ing may indicate that competition is less sporting performances of nations. in Winter Games and for participating However, some nations may prefer to nations it is easier to win medals com- invest in Winter sports because their geo- pared to Summer Games. Second, in graphic environment makes this likely. Turin the medals have been divided more This was not included in the preceding equally among the top ten nations than in analysis. To illustrate the point, Canada is Salt Lake City, resulting in a range of 8.3% particularly successful in speed skating between the first and tenth nation in Turin and in Short track whilst Norway is suc- versus 15.3% among the top ten nations cessful in cross country skiing and alpine in Salt Lake City (of which 2 nations differ skiing. These nations do not belong to the from the top ten in Turin; not represented top 20 in summer Olympic sports. Thus in figure). The market share figures for there is the danger that the success of Korea (5.2%), Switzerland (5.6%), nations who particularly value success in Sweden (6.0%) and Norway (6.2%) are the Winter Olympic Games is underesti- very close to each other in Turin. Third, mated when only summer Olympic sports five nations in figure 3 are not in the top are taken into account. We will therefore ten list of summer sports: Norway, apply the same analysis for Winter Games Sweden, Switzerland, Austria and as that presented earlier in this paper, Canada. using market share as a measurement of Figure 3 also shows that Germany has success in absolute terms (i.e. without always played a leading role in winter controlling for socio- and macro-econom- sports, with the exception of Lillehammer ic determinants) and residual analysis as 1994 where the Russian federation heads a measurement of relative success. In the other nations, despite the break up of terms of efficiency of elite sport policies the former USSR. Russian performances an analysis for winter sports, comparable decreased until Salt Lake City 2002 with a with the preceding analysis, may help to remarkable increase in 2006 (Turin). understand a broader interpretation of Germany's market share decreased with success with regard to nations' elite sport 3.5% in Turin. Austria begins the time investments. However, the Winter Games series with a third place in Albertville 1992 are less globalized and have different and then has a remarkable decrease of characteristics compared to the Summer 7.3% in the run up to Lillehammer (1994). Games and their size is a little less than Afterwards a steady increasing trend can 20 percent of the Summer Games in be noticed with in Turin an equal score to terms of participants and events (Kuper & Canada that also improved its market Sterken, 2003b). Only 77 National share from 4.1% in Albertville (1992) to Olympic Committees participated in the 9.5% in Turin. The USA was the host Games in Salt Lake City and of these 25 nation in Salt Lake City where it improved won medals. its market share from a fourth in Albertville
BELGEO • 2008 • 2 227 18,0%
16,3% 16,0% 16,0% 15,1%
14,5% 14,0% 14,1% 13,7% 13,4% 12,5% 12,0% 11,7%
10,3% 10,0% 10,2% 9,5%
8,2% 8,7% 8,0% 7,8% 7,3% 7,2%
6,8% 6,8% 6,0% 5,7%
4,0% 4,1% 4,4%
2,0%
0,0% 1992 Albertville 1994 Lillehammer 1998 Nagano 2002 Salt Lake City 2006 Turin
Germany United States Canada Austria Russia (1992: CIS)
18,0%
16,0% 15,6%
14,0% 13,4%
12,8% 12,0% 11,9%
10,7% 10,0%
8,0% 8,2%
6,2% 6,0% 6,0% 5,6% 5,2% 4,9% 5,3% 5,2%
4,0% 4,0% 4,2% 2,6% 4,1% 3,2% 2,1% 2,0% 1,7% 2,2% 1,9% 1,5% 1,2% 0,0% 1992 Albertville 1994 Lillehammer 1998 Nagano 2002 Salt Lake City 2006 Turin
Norway Sweden Switzerland Korea Italy
Figure 3. Market share in Winter Olympics 1992-2006 of the ten best performing nations (in Turin 2006).
228 The paradox of measuring success of nations in elite sport to a second in the last two Olympic of GDP/CAP (<10.000$), are former com- Games, despite falling market share in munist nations (Russia, Croatia, Belarus, Turin. Norway also gains its highest mar- Bulgaria, Poland and Estonia), which may ket share in Lillehammer when it hosted explain the significant output of commu- the Olympic Games. nism in the regression. The estimated function, with only signifi- RELATIVE SUCCESS IN WINTER cant variables, resulted in a model with a OLYMPIC GAMES determination coefficient of 54.6%(5):
Contrary to the Summer Games only a few MED (weighted points) = -51.623 + 0.003 GDPCAP + 35.904 COMM + H authors have estimated success on Winter t-values (-3.619) (5.351) (3.605) Games (see Balmer, Nevill & Williams, 2001; Kuper & Sterken, 2003b). Adjusted R²: 54.6% Making an OLS estimation on one edition of the Winter Games is less plausible than The residual analysis was normally dis- for Summer Games because of the small tributed(6) and collinearity statistics were data set. Without going into as much satisfactory (VIF = 3.184). The position of detail as with Summer Games, an OLS the ten best performing nations ranked estimation was made for the 25 medal according to their residuals is shown in winning nations in Salt Lake City 2002. Table 5. The same independent parameters as for The residual analysis shows that Norway the Summer Games were entered with was the most successful nation in Salt one additional variable: mountain eleva- Lake City. Germany, first in absolute suc- tion for Alpine Skiing (MOUNT). The vari- cess measurements has become sixth able was inserted as a dummy equal to and the USA fourth. Three nations appear «1» when mountains with a minimum in Table 5 that did not belong to the top height are available in the country and ten in terms of market share: China (3rd), «0» when they were not(4). Croatia (9th) and Spain (10th). Interestingly, the results revealed that only The analysis thus far has focused on per- wealth and communism are significant formance in the Olympic Games, which variables in this estimation for Winter although a truly global event, is not a Games. Population is not significant (t = measure of success in all events. The 1.134), nor is mountain elevation (t = Olympic medal table is but a snapshot of 1.094). The latter may partly be explained global sporting prowess at a given point by the fact that our analysis is confined to in time and does not necessarily repre- medal winning nations. Nevertheless, this sent global sporting achievement for the finding corresponds with earlier findings duration of each Olympiad. In some of Kuper and Sterken (2003b) who esti- sports, success in the Olympic Games is mated time series models for each not recognised as the pinnacle of Olympic Games from 1924-1998. achievement, notably tennis where Grand Apparently, contrary to the Summer Slam tournaments are widely regarded as Games, a large basis for recruitment of being the pinnacle of achievement; and in young talent is less important in Winter football the FIFA World Cup. Therefore, for Games. A logical explanation for the nations which view themselves as «sport- wealth of nations as the main precondition ing» nations, there needs to be another for success may be found in the fact that measure to contextualise overall sporting Winter Sports are generally more expen- achievement. This measure needs to be sive sports and require more expensive able to change more frequently than once equipment. Poor nations can send fewer every four years and also needs to rate athletes to the Games. The poorest medal recent success more highly than historical winning nations in Salt Lake City, in terms success.
BELGEO • 2008 • 2 229 Rank Country Predicted Residual 1 Norway 33.01 22.99 2 Russia 7.81 19.19 3 China -4.72 18.72 4 United States 58.98 17.02 5 Korea (south) -2.43 12.43 6 Germany 55.77 11.23 7 Canada 24.15 9.85 8 Italy 15.90 9.10 9 Croatia 2.00 9.00 10 Spain 3.37 2.63
Table 5. Ten best performing nations during the Olympic Winter Games in Salt Lake City according to OLS method (relative success), with «points of medals» as the dependent variable and «GDP/head and communism» as independent variables.
CONCLUSION
The key purpose of this article has been using the sources (at macro-level) avail- to demonstrate a methodology by which able), then differences in population, an objective assessment of performance wealth and other economic, political and can be made using techniques that have sociological circumstances must be con- both an absolute and relative nature. sidered. Policy determinants are part of Each of these methods - in their own spe- the residual and countries that perform cific way - allows countries to be classi- better than predicted may thus have more fied with respect to their absolute and rel- efficient policies in producing medal win- ative sporting success. One key point ning capability. It is self-evident that noted from this paper is that defining suc- establishing such criteria is impossible cess of nations is a tremendously difficult albeit for the simple reason that we are exercise and from this perspective it can not even aware of all the factors that influ- be assumed that ranking nations in terms ence sporting success. Analogous to of their success may raise more questions findings in literature it can be concluded than answers when it comes to the rela- from our methodology that international tion with effectiveness of policies. An sporting success, both in summer and objective performance ranking of nations, winter sports is determined for over 50% in order to identify a link between perfor- by these determinants. These factors mance and policy, has proven to be an cannot be controlled by politics and this ambitious task because the sporting pri- may be an argument to use relative suc- orities of individual nations vary along cess measurements. On the other hand winter or summer sports and because the there are some arguments to support ranking differs along the purpose of the absolute measurement. One is that top success measurements. level sport is absolute by definition (Van If we wish to identify which country is the Bottenburg, 2000) and this makes relative most efficient (i.e. which country suc- success measurements irrelevant. What ceeds in achieving sporting success finally counts is the absolute number of
230 The paradox of measuring success of nations in elite sport medals won, no matter how rich or large quantifying and ranking the performance nations are. The best performing nations of nations, other measurements of suc- are then the nations with the highest num- cess can be discussed. The analysis has ber of medals. Traditional measures such focused on performance in only Olympic as medal tables have a function in terms sports. Some 28 sports and 35 disciplines of being easily understood rankings of are contested at the Olympic Summer actual performance. However, as outlined Games and in many nations over 50 in this paper, it is possible for medal sports are formally recognised by the based measures of performance to give national agencies of sport. Therefore, the conflicting results concerning a nation's Olympics are but a subset of all sports performance. To overcome this problem, and inevitably for some nations culturally market share is identified as being a stan- important sports do not figure in the dardised measure of absolute success Olympic programme. Furthermore other which enables meaningful time series success measurements could be intro- analysis to be conducted. duced, such as the number of top eight The key point noted from an application of places or the number of world level ath- these two methods (absolute and relative letes. success) to Olympic Summer and Winter Finally this paper also highlighted an Games in this paper is that all these suc- issue that may have an impact on how cess measurements give different results policies towards elite sport development in terms of ranking nations on the basis of systems are monitored and evaluated in performances. Determining success the future. Competition for success in elite therefore depends on the purpose for sport is increasing. More nations are which the ranking is made. We therefore adopting strategic approaches towards state in this paper that a broad perspec- the development of elite athletes and as a tive of success measurements must be result an increasing number of nations considered in order to draw any meaning- have developed genuine medal winning ful conclusions regarding the relation capability. Therefore a further measure of between elite sport policies (input and performance is the efficiency of the pro- throughput) and international sporting duction process in terms of input-output success (output). analysis and the processes that lead to a In addition to raising the issues linked to certain output(7).
REFERENCES
• BAIMBRIDGE M. (1998), «Outcome • CONDON E.M., GOLDEN B.L. & WASIL uncertainty in sporting competition: The E.A. (1999), «Predicting the success of Olympic Games 1896-1996», Applied nations at the summer Olympics using Economics Letters, 5, pp. 161-164. neural networks», Computers & opera- • BALL D.W. (1972), «Olympic games tions Research, 26, pp. 1243-1265. competition. Structural correlates of • DE BOSSCHER V., DE KNOP P. & national success», International Journal of HEYNDELS B. (2003a), «Comparing rela- Comparative sociology, 13, pp. 186-200. tive sporting success among countries: • BALMER N.J., NEVILL A.M. & WILLIAMS Create equal opportunities in sport, A.M. (2001), «Home advantage in the Journal of Comparative Physical Winter Olympics (1908-1998)», Journal of Education and Sport, 3, 3, pp. 109-120. Sports Sciences, 19, pp. 129-139. • DE BOSSCHER V., DE KNOP P. & • BERNARD A. & BUSSE M. (2004), «Who HEYNDELS B. (2003b), «Comparing ten- wins the Olympic Games? Economic nis success among countries», resources and medal totals», Review of International sport studies, 1, pp. 49-69. Economics and Statistics, 86, pp. 413- • DE BOSSCHER V. (2007), Sports Policy 417. Factors Leading to International Sporting
BELGEO • 2008 • 2 231 success, Published Doctoral thesis, Vrije • KUPER G.H. & STERKEN E. (2003a), Universiteit Brussel, Brussel, VUBPRESS. Olympic Participation and Performance • DE BOSSCHER V., BINGHAM J., SHIBLI Since 1896, Graduate School/Research S., VAN BOTTENBURG M., DE KNOP P. Institute Systems, Organisations and (2008), A Global sporting arms race. An Management, University of Groningen international comparative study on sports Report, No. 03C19, Retrieved September policy factors Leading to international 3, 2003, from http://som.eldoc.ub.rug.nl/ sporting success, Aachen (Germany), reports/themeC/2003/03C19. Meyer & Meyer. • KUPER G. & STERKEN E. (2003b), The • DE KONING J. & OLIEMAN R. (1996), Olympic Winter Games: Participation and «Twijfel over medaille-euforie [Doubts Performance, Retrieved April 20, 2006 about medal euphoria]», Economische from Departement economy, University of Statistische Berichten, 2, pp. 813 - 815. Groningen: http://www.eco.rug.nl/medew- • DEN BUTTER F.A.G. & VAN DER TAK erk/kuper/download/olwinter.pdf. C.M. (1995), «Olympic medals as an indi- • KUTNER M.H., NACHTSHEIM C.J., cator of social welfare», Social Indicators NETER J. & LI W. (2005), Applied Linear Research, 35, pp. 27-37. Statistical Models (5th edition), New York, • GÄRTNER M. (1989), «Socialist coun- McGrawHill. tries’ sporting success before perestroi- • LEVINE N. (1974), «Why do countries ka-and after?», International Review for win olympic medals. Some structural cor- the Sociology of Sport, 24, 4, pp. 283-297. relates of Olympic Games success», • GILLIS J. (1980), «Olympic success Sociology and Social Research, 58, pp. and national religious orientation», 353-360. Review of Sport and Leisure, 5, pp. 1-20. • MORTON R.H. (2002), «Who won the • GLEJSER H. (2002), «Verdere bewijzen Sydney 2000 Olympics? An allometric over het comparatief nadeel van kleine approach», The Statistician, 51, pp. 147- landen op het gebied van sport [further 155. prove on the comparative disadvantage • NOVIKOV A.D. & MAXIMENKO A.M. in sport of small countries]», Nieuw (1972), «The influence of selected socio- Tijdschrift van de Vrije Universiteit economic factors on the level of sports Brussel, 15, 1, pp. 82-91. achievements in the various countries», • GREEN M. & HOULIHAN B. (2005), Elite International Review of Sport sociology, 7, sport development. Policy learning and pp. 22-44. political priorities, London and New York, • OTTOY L., VAN VOOREN M. & HUGHE Routledge. V. (1993), Inleiding tot SPSS voor • GRIMES A., KELLY W. & RUBIN P. Windows, Ghent, University Press. (1974), «A socio-economic model of • ROGGE J. (2002), The challenges of the national Olympic performance», Social third millennium. Opening speech made science quarterly, 55, pp. 777-783. at the International Conference on Sports • HOFFMANN R., GING L.C. & Events and Economic Impact, RAMASAMY B. (2002a), «Public policy Copenhagen, Denmark. and Olympic success», Applied • SHIBLI S. (2003), Analysing perfor- Economic Letters, 9, pp. 545-548. mance at the Olympic Games: Beyond • JOHNSON K.N., & ALI A. (2002), A tale the final medal table. Paper presented at of two seasons: participation and medal the 11th Congress of the European counts at the summer and winter Olympic Association for Sport Management, Games, Retrieved February 15, 2003, Stockholm, Sweden. from Wellesley college, Massachusetts • SHIBLI S. & BINGHAM J. (2005), «An website: http://www.wellesley.edu/eco- evaluation of medal-based measures of nomics/wkpapers/wellwp_0010.pdf. performance in the Summer Olympic • KIVIAHO P. & MÄKELÄ P (1978), Games [Abstract]», Proceedings of the «Olympic Success: A sum of non-materi- 13th Congress of the European Association al and material factors», International for Sport Management, Power in sport, Review of Sport sociology, 2, pp. 5-17. Newcastle, England, pp. 246-247.
232 The paradox of measuring success of nations in elite sport • SIRC (2002), European sporting suc- of Sport, Seoul, Korea, Retrieved March, cess. A study of the development of 2002, from http://www.lssfb.ch/down- medal winning elites in five European load/ISSA_Seoul.pdf. countries. Final Report, Sheffield, Sport • SUEN W. (1992), Men, money and Industry Research Centre. medals: an econometric analysis of the • SHAW S. & POOLEY J. (1976), Olympic Games, Discussion Paper from «National success at the Olympics: An the University of Hong Kong. explanation», in LESSARD C., MASSI- • TCHA M. & PERCHIN V. (2003), COTTE J.P. & LEDUC E. (Eds.), «Reconsidering performance at the sum- Proceedings of the 6th international mer Olympics and revealed comparative Seminar: History of Physical Education advantage», Journal of Sports econom- and Sport, Québec, Canada, Trois ics, 4, pp. 216-239. Rivières, pp. 1-27. • VAN BOTTENBURG M. (2000), Het top- • STAMM H. & LAMPRECHT M. (2001), sportklimaat in Nederland [The elite Sydney 2000, the best games ever? sports climate in the Netherlands], ‘s Her- World Sport and Relationships of togenbosch, The Netherlands, Diopter- Structural Dependency. Paper presented Janssens and van Bottenburg bv. at the 1st World Congress of the Sociology
(1) As a point of departure, a correlation-matrix nated. A major reason for discarding an was taken for the selection of these vari- outlier is that «under the least squares ables and to avoid problems of multi- method, a fitted line may be pulled dispro- collinearity. In this respect the percentage portionately toward an outlying observation of Catholic inhabitants was omitted, due to because the sum of squared deviations is the high correlation with percentage of minimized» (Kutner et al., 2005, p. 23). Muslims (r = -0.435; sign. 0.000) and more- Further analysis showed that this was the over only a low correlation with medal case with Jamaica, mainly because of its points. Urbanisation, which was highly cor- Protestant nature. Indeed, when Jamaica related with GDP/head (r = 0.605; sign. was omitted from the regression model, 0.000), was left out for the same reason; Protestantism was not a significant variable area, which correlates highly with popula- anymore. Refined measures, to ascertain tion (r = 0.558; sign. 0.000) was replaced whether nations are influential, showed that by density (= population/ area); the number for Jamaica the «DfFITS» (= 0.475) (which of protestant inhabitants in a nation corre- measures its influence on the fitted value) lated significant at the 0.05 level with was too high and so was the DFBETA value GDP/capita (r = 0,281) but was included (= 0.4) for the influence on the intercept. because of the significant correlation with Although these measures are not problem- medal points (r = 0,317; sign. 0.000). atic it was decided to omit Jamaica from (2) The source of our data is the World the sample mainly because of its influence Factbook 2004 (http://www.cIa.gov/cia/ on the regression parameters (Protestant- publications/factbook/index.html) for the ism) and thus on the other residuals. variables of population, wealth, religion and Cook's distance was only 0.117 for density. The data relating to the former Jamaica, the largest but one (after India) communist countries derives from Encarta but within the required critical values. (http://encarta.msn.com/relat- (4) We thank Mr Kuper and Mr Sterken for ed_761572241/Communism.html). The securing this data, obtained from the information on the medals comes from the Encarta encyclopaedia. Medal data are IOC official report 2004, available at: obtained from the official IOC report avail- http://www.olympic.org/uk/games/index_uk able at: http://www.olympic.org/uk/games/ .asp index_uk.asp (3) A boxplot with the Mahalanobis distance (5) Via Diagnostic tests two outlying cases in X- indicated that Jamaica was an extreme direction were identified: Germany and the value. After diagnostic tests , to ascertain USA, with a Mahalanobis distance of how influential this case is in the fitting of 92,000 and 66,000 respectively, far above the regression function, Jamaica was elimi- the other nations. These nations are also
BELGEO • 2008 • 2 233 the most successful nations in absolute therefore decided to keep Germany and number of medals and repeating the OLS the USA in the sample. Nonetheless pru- without either, or both, of these two nations dence is needed for the interpretation of lead to a serious decrease of the determi- the coefficients and residuals of these two nation coefficients: 43.0% (excluding nations. Germany), 37.8% (excluding USA) and (6) Shapiro-Wilk: sig. = .475 15.7% (excluding both). Further analysis (7) This was the subject of an international revealed that these nations influenced the comparative study on elite sport policies regression coefficients (DFBeta) and their that has recently been finished and where single fitted Value (DFFits was 8.28 for GER the methods presented in this paper were and 7.92 for USA). Cook's distances, which used as measurement of outputs in six measures how much the residual of all nations. For more information about this cases would change if a particular case study: see De Bosscher, 2007 and De were excluded, are all right (0.288 for Bosscher, Bingham, Shibli et al., 2008). Germany and 0.352 for the USA). It was
Institutional address: Vrije Universiteit Brussel Faculty of Physical Education and Physiotherapy Drs. Veerle De Bosscher Pleinlaan 2, 1050 Brussels, Belgium Tel: +32 2 629 27 72/ +32 16 40 22 08 Fax:+32 2 629 28 99
manuscript submitted in November 2007; revised in March 2008
234 The paradox of measuring success of nations in elite sport