Statistical Methodology

Statistical Methodology

171 Arab Knowledge Index 2015

172 Preamble The results of the in-depth correlation analysis and alpha Cronbach coefÀcient The Arab Knowledge Index (AKI) consists conÀrmed the validity of the selection and of six composite thematic indices, expressing classiÀcation of the variables, in which the six key developmental sectors, namely: Pre- alpha Cronbach coefÀcient exceeded 0.70 University Education, Technical Vocational in more than 80 per cent of cases. Weak Education and Training (TVET), Higher consistency of certain variables could be Education, the Economy, Information attributed to lack of and/or the nature and Communications Technology (ICT) of the correlation between these variables.3 and Research & Development (R&D) and Innovation. Each of these six indices Data Used was constructed in accordance with the standard international methodologies for The 304 variables incorporated into the the construction of composite indicators.1 construction of the six Arab Knowledge The following is a presentation of the steps indices can be classiÀed into three types. used in constructing these indicators. The Àrst consists of the hard data obtained from various resources such as the World Variable Selection Bank, the UNESCO, other Uniterd Nations agencies and others.4 The second type includes

Selecting the individual variables included composite indicators computed by some Statistical Methodology in the construction of each of the six of the international institutions such as the knowledge indicators is relying on a well- International Telecommunication Union deÀned, clear and scientiÀc methodology (ITU), the European Union (EU) and the based on a thorough review of relevant Organisation for Economic Cooperation international and local literature. In and Development (OECD). The third type addition, the concepts and experience of consists of survey data which are used when international organisations and agencies variables have no data or data are incomplete. are explored. In each of the six sectors, it also relied on consultations with a large For the sake of transparency, simplicity and the number of specialists in various countries possibility of replicating the results, no attempts of the world such as Egypt, Jordan, United were made to impute missing values to the Arab Emirates, Canada, United Kingdom various variables. The use of the arithmetical and United States through a special formula in computing an indicator is . The participants were able to equivalent of imputing each of the missing express their agreement with the selected list values of the indicator to the mean value of of variables and their various aggregations; the variable. The missing values, indicated reject or propose additions or amendments by the symbol “n/a”, were not entered in to it. In accordance with the feedback from the composite sub-indicators which were the participants and the rest of the core computed, as is customary in similar cases, by team who prepared the Index, a Ànal list of just using the available data for each country.5 the variables was compiled. Data-processing was performed on the Principal Components Analysis was assumption that the data were error-free, the used to conÀrm the consistency of the team having reviewed it more than once to selected variables and the structure of their ensure there were no errors in the data entry. classiÀcation into the various sub-indicators. The variables that might lead to a biased These results supported the consistency indicator value were processed using suitable of the conceptual context in selecting the statistical methods.6 It had been observed that variables and their classiÀcation in the some indicators were linked to other, size- various sub groups, in which the explained dependent indicators, such as population or ratio in most cases exceeded 50 per GDP; these indicators were therefore rescaled cent.2 using the size.

173 indicate the largest and smallest value of the Normalisation available indicator values respectively. The normalisation criterion depends on whether The values of variables were normalised the variable is good i.e. has a positive relation in the of 1-100, in which the higher with the overall index, or bad i.e. has a values indicated better results. The rescaling negative relation with the overall index. The or the ‘maximum–minimum’ method was good indicators were normalised using the

used, in which the maximum and minimum following formula:

Normalised indicator value of the country = raw indicator value of the country - raw minimum value of the indicator across countries ( 99X +1 (raw maximum value of the indicator across countries - raw minimum value of the indicator across countries

In the case of the bad indicators i.e. indicators with an inversely correlated relation, this formula

should be adjusted as follows:

Normalised indicator value of the country = raw maximum value of the indicator across countries - raw indicator value of the country ( 99X +1 ( raw maximum value of the indicator across countries - raw minimum value of the indicator across countries

linked sub-indicators to form a single factor Weighting containing as much information as possible that is shared between these linked indicators. The methods of estimating weights used in The resulting estimated weights using both the the construction of the six knowledge indices budget allocation and methods range from equal weighting, budget allocation were extremely consistent with each other, process and the factor analysis method. Equal as well as being consistent with the initial Arab Knowledge Index 2015 weights are used in the absence of clear weights estimates, based on the intellectual evidence of the diversity of signiÀcance of and conceptual framework. each Index, as well as in the absence of sound and complete information concerning the Index Computation existence of causal relationships or a lack of consensus on a classical method for estimating The Arab Knowledge Index was calculated weights. for 22 Arab countries in its Àrst version using the most recent and best available data for the The budget allocation process method was various variables of each country, together with also used for weighting, in which a group of the choice of the year 2005 as a cut-off year. specialists, concerned and experienced experts The values of the composite sub-indices for the were invited to attend a workshop for each of Knowledge Index were calculated by applying the six knowledge sectors. Each expert was a series of successive aggregations starting with given a budget consisting of 100 points for the the more detailed level of variables and ending variables or sub-indicators used. If the variable by attaining the overall index. In the case of or sub-indicator was believed to have greater ICT for example, the sub-indicators relating relative importance, it was allocated a greater to infrastructure and digital content were number of points. Subsequently, the weights aggregated to form a single composite indicator were estimated by calculating the of expressing the infrastructure and digital the total points allocated to each variable or content. Similarly, the sub-indicators relating sub-indicator.7 to each of communications affordability, individual, company and government usage The weights were also assessed using factor were aggregated to form a sub-composite analysis, which is based on aggregating the indicator for each. By aggregating these three

174 sub-indicators, a single composite indicator was indicator (CI) takes the following form: created expressing the direct ICT indicators. Using the same method, one composite indicator was created expressing the indirect indicators to ICT.

The overall ICT Index was created by The geometric aggregation formula of sub- aggregating the two composite indicators for indicators (SIj) to compute the composite the directed and indirect indicators. indicator (CI) takes the following form:

Due to an inability to obtain data for all the sub-indicators of each country, some countries have missing values for certain sub-indicators. For example, in the case of infrastructure and digital content, no data was available for the Internet bandwidth (kb/s per user) indicator for Libya and Yemen. Accordingly, and with CI is the proposed composite index to be a desire to calculate the indicators for all computed, wj is the relative weight of the the countries, a maximum number of sub- sub-indicator SIj, n is the number of sub- indicators for each country was calculated indicators aggregated to form the composite using the available data. For example, the indicator and exp and ln are the exponential infrastructure and digital content indicator for and logarithmic functions respectively. The Statistical Methodology Libya and Yemen alone was calculated through arithmetical (or linear) aggregation method the use of four sub-indicators – electricity is employed to compute all knowledge sub- production in terms of kWh per capita, the indicators in this Index. extent of mobile phone network coverage as a percentage of population, ratio of households Index Sensitivity with a computer and access to digital content. There was no sub-indicator concerning Like all composite indices, the construction Internet bandwidth or kb/s per user due to the of the Arab Knowledge Index and its six unavailability of data. sub-indices depends on the choices of the researchers, reÁecting the unavoidable On the other hand, when the data for sub- elements of uncertainty. These elements indicators were available for a single country include the methods for variables selection, the only or for two countries at the most, the structure of the sub-indices, imputation of the decision was taken, however, not to calculate missing values, normalisation, weighting and this indicator, because the data were insufÀcient aggregations. The Index sensitivity study aims for normalisation. Subsequently, this indicator to assess the effect of uncertainty elements was excluded from the calculation of the rest separately or jointly on the performance of of the overall composite indicators, and the the Index. results of the calculation were not presented subsequently. The results of the sensitivity analysis of the knowledge indices for weighting There are two well-known aggregation and normalisation methods showed the methods: arithmetical (or linear) and insensitivity of the indicators to these geometric aggregations and any of them can elements i.e. the indicator performance does be chosen by the researcher. Sometimes, both not differ signiÀcantly due to variation of these methods are used together for comparing elements. For example, to study the sensitivity their outcomes to assess the sensitivity of the of the knowledge indices to the aggregation Index to the aggregation method. method, each index was calculated by using both arithmetic and geometric . The The linear aggregation formula of the sub- geometric mean was used because it offers indicators (SIj) to compute the composite a partial compensability between indicators,

175 contrary to the that assumes mostly unchanged, meaning that the highest full compensability between the indicators ranked countries, according to the results of incorporated into the Index calculation, arithmetic aggregations, maintain the same where both methods – arithmetic and rank according to the results of geometric geometric aggregations – were applied to the aggregation. sub-indicator of infrastructure and digital content. Despite the geometric aggregation It is noteworthy that the construction of formula giving lower values for the indicator the Arab Knowledge indices – despite their than the results of the arithmetic formula, validity in reÁecting the true knowledge the performances of the sub-indicator of situation in the Arab region – is open to future infrastructure, digital content and the overall development, including the completion and ICT indicator do not differ signiÀcantly. improvement of the data quality and global Thus, the of the countries is still sensitivity of the Index.8 Arab Knowledge Index 2015

176 Endnotes

1 OECD 2008b. 2 For more information about Principal Components Analysis refer to Hair et al. (2010). 3 For more information about Cronbach’s coefÀcient alpha refer to Tavakol & Dennick 2011. 4 For more information about the data sources of the Arab Knowledge indices, refer to the annex. 5 Cornell University et al. 2015. 6 Groeneveld and Meeden 1984. 7 For more information about the Budget Allocation Process method, refer to OECD 2008b. 8 Refer to Saltelli et al. 2008. Statistical Methodology

References and Background Papers

177 Arab Knowledge Index 2015

178