International Journal of Management and Applied Science, ISSN: 2394-7926 Volume-3, Issue-3, Mar.-2017 http://iraj.in A META-ANALYSIS IN FINANCE
CHANG-SOO KIM
Department of Finance and Business Economics, University of Washington, Seattle, WA, USA Department of Business Administration, Yonsei University Yonseidae-gil 1, Heungup-myon, Wonju, Gangwon-do 26493, South Korea E-mail: [email protected]
Abstract- A tremendous amount of academic papers have been published in finance over the past several decades. Important topics include internal capital market, corporate social responsibility, socially responsible investment, mergers and acquisitions, and so forth. Many researchers have attempted to figure out whether variables pertaining to these subjects have a positive relationship with corporate financial performance. However, it is not easy to synthesize empirical results of extant research, since papers are so different in terms of methodology used, research period, measurement of key variables, data set, etc. A meta-analysis is an excellent tool to aggregate a variety of heterogeneous results and comprehend the overall picture scientifically. This method is far better than a synthesis paper produced by a single person due to the problem of researcher bias. It is also better than a voting method since it takes care of the size and variance of impacts. This paper investigates the possibility of using meta-analysis in finance area and discusses issues regarding statistical test, bias correction, small sample problems, data collection, and so on. In addition to the discussion of the meta-analysis per se, we also provide an example of meta-analysis applied in one of finance areas, socially responsible investments.
Keywords- Meta-analysis, effect size, fixed-effect model, random-effect model, socially responsible investments
I. INTRODUCTION help managers build up their reputation at the expense of firm resources, SRI will generate a lower return A meta-analysis is an excellent tool to aggregate a than equivalent conventional investments. Portfolio variety of heterogeneous results and comprehend the theory also predicts a negative influence of SRI. overall picture scientifically. This method is far better Since SRI screening imposes constraints in the than a synthesis paper produced by a single person process of portfolio formation, the efficient frontier of due to the problem of researcher bias. It is also better SRI is inferior to that of conventional investments than a voting method since it takes care of the size that have no restrictions at all (Markowitz, 1952; and variance of impacts. We apply a meta-analysis to Girard et al., 2007). However, if diversification costs one of most interesting areas in finance, socially from SRI are not big and the increase in corporate responsible investments (SRI) to figure out whether performance from improved reputation and SRI performs better than conventional investments. governance caused by SRI activities is large, then There have been attempts to synthesize the existing SRI will generate positive impact on average (Boutin- literature, but they had many flaws which can be Dufresne and Savaria, 2004). improved (Orlitzky et al. 2003, Margolis et al. 2007, and Rathner 2013). Since existing papers are very Since theory indicates effects in both directions, the heterogeneous in terms of methodology and empirical impact of SRI has to be delineated by empirical results, scientific evidence needs to be provided by investigation. For this we attempt to synthesize employing a sound and comprehensive statistical existing literature by adopting a meta-analysis, which method. Since meta-analysis on SRI is quite rare, this is an excellent tool to aggregate many different research will contribute to understanding overall papers and results. Meta-analysis follows these steps: performance of SRI. In addition, since we plan to 1) identifying research questions, 2) collecting data, address many research issues in performing meta- 3) evaluating data, 4) analyzing and interpreting data, analysis, this paper can contribute to understanding and 5) reporting. and improving methodology, too. 1. Identifying Research Questions II. METHODOLOGY To achieve the goal of figuring out the impact of SRI on financial performance, we will calculate the We begin our research by investigating theoretical weighted average ESs for both a SRI group (SRI arguments related to the effect of SRI, since it will mutual funds, SRI indices, and SRI portfolios) and a enrich interpretation of empirical results. SRI effects non-SRI group (conventional funds, indices and have both positive and negative dimensions portfolios). Since ESs can vary systematically with (Hamilton et al., 1993). A positive theory argues that dimensions of SRI such as location of markets, SRI can serve as a channel for expressing social financial performance measures, investment horizons, responsibility values, so it will produce a positive SRI thematic approaches, types of researcher, impact on investment performance. A negative theory publishing status, and data comparison method, we emphasizes agency problems. Since CSR activities
A Meta-Analysis in Finance
123 International Journal of Management and Applied Science, ISSN: 2394-7926 Volume-3, Issue-3, Mar.-2017 http://iraj.in will perform various subgroup analyses and meta- 3. Evaluating Data regression using many moderator variables. In the stage of data evaluation, we establish the criteria for inclusion in the analysis and code data for 2. Collecting Data computer-assisted analysis. To be included in the Data collection is very important for the success of final analysis, a study has to satisfy various screening meta-analysis. First, all relevant documents will be criteria. First of all, the study has to have necessary extensively collected to avoid the possibility that final quantitative information to calculate ESs, so all results are driven by the data collected. For this, we qualitative studies will be excluded. Second, studies will search all databases that are considered to without information on control groups will be contain documents about SRI performance such as excluded, since it is impossible to evaluate whether Scopus, ABI Inform/Global, EBSCO, JSTOR, SRI performance is better or not without information Econlit, Science Direct, Wiley-Blackwell, Web of on comparison groups. Third, event studies will be Science and SSRN for full-text articles. We will also excluded since the focus of this paper is on the long- include web search engines such as Google Scholar term performance, not the immediate reaction to SRI and Google Books. When searching, we are going to announcements. Fourth, an overrepresentation bias use a list of phrases that are formed by combining will be controlled for. It is possible that there are basic keywords like ‘SRI’, ‘ethics’, ‘responsible’, multiple studies on the same subject by the same ‘social’, ‘financial’, ‘funds’, and ‘performance’. We author in different forms like dissertation, working will carefully design the combination of keywords to paper, and published article. If all of these studies are reduce redundancies by considering necessary and included, the study will be over-weighted. In this sufficient conditions. case, only the version that has the necessary information and is most reliable will be included. In addition, we will search academic journals that are related to SRI and perform manual searches of the When applying deletion criteria, we use upper level reference lists of selected papers. The list of journals criteria first and then use lower level criteria. We includes Academy of Management Journal, American clearly indicate the number of articles and documents Economic Review, Journal of Finance, Review of that are deleted and the number of documents Financial Studies, Journal of Corporate Finance, included in the final meta-analysis by reporting a Journal of Banking and Finance, Financial Analysts flow chart of deleting process. Journal, Journal of Financial and Quantitative Analysis, Journal of Portfolio Management, Journal This stage is very important since it can have a of Financial Economics, Journal of Business Ethics significant impact on the final results, so we proceed and Corporate Governance. very carefully during the deleting process. After we decide on papers to be included in this analysis, data When collecting documents and data, we will try to entry into a computer is performed. Since there is avoid typical biases that may have a significant always a chance of making errors, two people impact on the results of meta-analysis. First, a independently enter data and the resulting files are publication bias that is caused by collecting only compared to confirm the accuracy of input data. published articles and documents will be taken into account. In general, published materials are better in 4. Analyzing and Interpreting Data quality and more reliable, but it is also true that only In the stage of data analysis and interpretation we papers with a high statistical significance tend to be have to calculate the effect size (ES) of individual published. On the other hands, many good papers studies. For this we use Cohen's (1969) d which is with meaningful empirical contents are rejected from similar to Hedges’ (1981) g that is calculated as academic journals due to a lower statistical follows: significance. To avoid publication bias, we include all articles and documents regardless of publication status and test whether meta-analysis results are systematically different by the status of publication. where is the average performance of SRI A bias caused by an inconsistent use of terms will (conventional) group and σ is the pooled within- also be considered. For example, experiment, group standard deviation which can be computed randomized trial, and RCT have the same meaning using the following formula: but they are used more frequently in one area but not in other areas. In this case, if only one term is used, a significant amount of information can be lost. We make a table of core keywords that summarize all of Here, NT(NC) is the sample size of SRI the variety of SRI related words with the same (conventional) group and VT(VC) is the variance of meaning and check the number of articles and SRI (conventional) group. documents to avoid this bias.
A Meta-Analysis in Finance
124 International Journal of Management and Applied Science, ISSN: 2394-7926 Volume-3, Issue-3, Mar.-2017 http://iraj.in Hedges’ g obtained above is often biased due to a that has become more popular recently. The statistics small sample size. If the sample size is not big for Q and I2 are calculated as follows. The higher the enough, we will adjust it using the following formula statistics, the more heterogeneous the ESs are among to obtain an unbiased estimator as suggested by samples. Hedges & Olkin (1985).
There are studies that do not report statistics required If the test results reject homogeneity, the random to calculate ES. For example, many studies report effect model should be used. The weighted average only t-statistics and correlation coefficients. We can effect size from the random effect model is calculated calculate d even in this case. Rosenthal (1984) as follows. proposes the following formula to convert t-statistic into d.
where is an effect size from individual research i, If sample sizes of SRI group and control group are and the weight ∗ is calculated as follows. not equal, Rosenthal and Rosnow (2008) recommend using the following equation to compute d: 1 Here ∗ is calculated using the following formula.
After calculating the effect size of individual studies, the weighted ES of the total sample will be computed as proposed by Hasselblad and Hedges (1995). The The SRI effects on financial performance can be basic idea of weighting is to give more weight to systematically different according to moderators. In studies with more precise ES estimates. Weights are order to examine this possibility, we will perform obtained by computing the reciprocal of variance of subgroup analysis, meta-regression analysis or each study. When aggregating individual effect sizes, multiple regression analysis (MRA) as developed by two models are considered, one is a fixed effect Lipsey and Wilson (2001). The moderators are model and the other is a random effect model. The generated by considering theoretical determinants and fixed effect model is used when the effect sizes from methodological characteristics. Since the individual studies are relatively homogeneous and so development level of capital markets is different the variability among ESs is mostly caused by depending on economic status, culture and sampling error (Lipsey and Wilson, 2001). The geographical location, we make SRI market weighted average effect size from the fixed-effect moderator of Anglo-Saxon, European and Asian model is calculated as follows: markets. Since there is a possibility that SRI effects can be different among different types of securities, we also create a moderator for investment vehicles such as stocks, bonds, and diversified funds. where is an effect size from individual There could be a difference in ESs among experiment i, and is a weight on experiment i environmental, social, and governance issues, so a that is a reciprocal of variance . Here, the variance moderator for ESG criteria is included. The impact of of experiment i is calculated using the following SRI screening on financial performance could vary formula. with the length of investment, so we consider a moderator for an investment horizon that is either short (less than 10 years) or long (longer than 10 years). It is possible that studies use different risk measures, and ESs can be systematically different by After computing weighted average ES of fixed effect risk measures used. To investigate this effect, we model, we test whether the homogeneity assumption incorporate a performance measure moderator for is satisfied. An intuitive and easy way of checking either no risk adjusted measure (raw returns, means) heterogeneity among individual studies is to examine or risk-adjusted measures (Jensen’s alpha, Sharpe or a forest map that shows confidence intervals of those Treynor ratios). We also differentiate between a studies holistically. For a more rigorous test of single factor risk adjustment like Jensen’s alpha and homogeneity, we will use Cochran’s Q test or I2 test
A Meta-Analysis in Finance
125 International Journal of Management and Applied Science, ISSN: 2394-7926 Volume-3, Issue-3, Mar.-2017 http://iraj.in multi-factor risk adjustment like Fama-French or When reporting final results the overall weighted Carhart models. Portfolios constructed by average of ESs derived from all individual papers are professional managers and academics could show reported first. Then the results from subgroup difference in ESs since these two groups of people analysis will be shown according to various are quite different in terms of constraints they face moderators to show the impact of each moderator such as transaction costs and access to information. A variable on the ESs of SRI. Finally, results from the moderator for this effect is required. We also make meta-regression where we include ESs as a dependent moderators associated with methodological variable and all moderators as independent variables characteristics. For example, we will include a will be reported. moderator for publication bias. III. RESULTS Finally, the independence issue is taken care of. If there are multiple estimates from the same study, the 1. Overall Effect Size independence assumption is not satisfied. In order to Table 1 shows the overall effect size that is computed control for dependence among ESs, a hierarchical by weighting individual effect sizes with a reciprocal linear model will be run to identify both within- and of variances of those effect sizes. Since the effect size between-study variations in the results (Doucouliagos is -0.048 with a very high statistical significance, and Laroche, 2009). investors have to sacrifice returns when investing SRI funds or portfolios. However, apart from statistical 5. Reporting significance the effect size of -0.048 indicates that the When reporting the results of meta-analysis, we sacrifice of investors is quite minimal. As for the describe every step in detail so that subsequent heterogeneity the values of Cochran’s Q and Higgin’s studies can replicate. Particularly, we will provide a I2 suggest that homogeneity assumption is rejected, full explanation on data collection and the exclusion so the random effect model is more appropriate. process, and a table or flow chart that clearly shows Table 2 reports the result from random effect model. the number of papers and documents included after Although the effect size becomes bigger in absolute each deleting criterion is applied. value, it is still a very small effect size.
This table shows the overall effect size, z-value from a hypothesis test of zero effect size and 95% confidence interval. It also shows values of Cochran’s Q and Higgin’s I2 from the test of heterogeneity among experiments. The number of observations is 507 experiments extracted from 99 papers. The figure in the parenthesis is p- value. *** indicates a statistical significance of 1%.
Table 1 Overall Effect Size – Fixed Effect
1 As a rule of thumb, if the sample size is large enough, analysis based on random effect model is desired. However, if the sample size is small, it is recommended to consider both random and fixed effect models and see whether there is a significant difference in the results.
This table shows an overall effect size, z-value from a hypothesis test of zero effect size and 95% confidence interval. It also shows values of Cochran’s Q and Higgin’s I2 from the test of heterogeneity among experiments. The number of observations is 507 experiments extracted from 99 papers. The figure in parenthesis is p-value. *** indicates a statistical significance of 1%.
Table 2 Overall Effect Size – Random Effect
2. Meta-regression In this subsection, we investigate effects of various moderators on the effect size. We begin with the test of publication bias. Since so many papers are submitted, academic journals become very selective and papers with an interesting result with a high statistical significance are more likely to be published than papers with an insignificant results (Song et al., 2000). Table 3 shows a meta-regression result from testing publication bias. A significantly positive sign of publication dummy variable suggests that the effect size tends to become larger if a paper is a published one. In other words, papers with a stronger result are inclined to be published.
A Meta-Analysis in Finance
126 International Journal of Management and Applied Science, ISSN: 2394-7926 Volume-3, Issue-3, Mar.-2017 http://iraj.in This table reports a meta-regression result from testing whether the sample is subject to the publication bias. The dummy variable is defined as Publication = 1 for a published paper, or 0 for a non-published paper. *** indicates a statistical significance of 1%.
Table 3 Publication Bias
When measuring the performance of portfolios, it is important to take risk level into consideration. Since a higher risk leads to a high return in equilibrium, a comparison of raw return or raw excess return over risk-free rate is problematic. Table 4 indicates that if a risk adjusted measure is used, effect size tends to decrease by 0.0298. This is related to the fact that the holdings of SRI funds are composed of many small firms. Since a small firm effect is well documented in the literature, simply generating a higher return is not sufficient to justify SRI fund performance. To earn while doing good, SRI funds have to earn a higher return on risk-adjusted basis.
This table reports a meta-regression result from investigating the effect of risk adjustment on the value of effect size. The dummy variable is defined as Risk_adj = 1 if risk adjusted measure is used, or 0 if non-risk adjusted measure is used. *** indicates a statistical significance of 1%.
Table 4 Risk Adjustment
Table 5 reports the impact of matching on effect size. When comparing two different groups, the characteristics of the two groups should be similar except the variable of interest. About a quarter of total sample examines the impact of SRI on performance using treatment group and control group. The result in Table 5 suggests that experiments with both treatment and control samples tend to generate a higher effect size.
This table reports a meta-regression result from investigating the effect of matching on the value of effect size. The dummy variable is defined as Matching = 1 if there is a control sample, or 0 if there is no control sample. *** indicates a statistical significance of 1%.
Table 5 Matching
When investigating financial returns of mutual funds and portfolios, inclusion of dead funds and portfolios can systematically change the results. If attrition rate of SRI funds is higher (lower) than conventional funds, inclusion of dead funds will lead to a relatively lower (higher) return for SRI funds than conventional funds. In our sample, about a half of experiments takes this aspect into consideration. Table 6 shows that survivorship bias dummy has a negative coefficient with a high statistical significance. It indicates that incorporating survivorship bias lowers effect size suggesting that the attrition rate of SRI funds is higher than conventional funds. This table reports a meta-regression result from investigating the effect of survivorship bias on the value of effect size. The dummy variable is defined as Survivorship = 1 if survivorship bias is controlled, or 0 otherwise. *** indicates a statistical significance of 1%. Table 6 Survivorship Bias
A Meta-Analysis in Finance
127 International Journal of Management and Applied Science, ISSN: 2394-7926 Volume-3, Issue-3, Mar.-2017 http://iraj.in When SRI is implemented, two screening procedures are typically employed. One is a positive screening and the other is a negative screening. According to portfolio theory negative screening will shrink the portfolio frontier, which will result in an investment opportunity that is inferior to the one with the original portfolio frontier. On the other hand, a positive screening can enhance the efficient frontier, which can lead to a better investment performance. The positive coefficient of screening dummy variable in Table 7 is consistent with this argument.
This table reports a meta-regression result from investigating the impact of screening method on the value of effect size. The dummy variable is defined as Screening = 1 if positive screening is employed, or 0 if negative screening is employed. *** indicates a statistical significance of 1%.
Table 7 Positive Screening vs. Negative Screening
CONCLUSIONS REFERENCES
This paper performs a meta-analysis to examine the [1] Girard, E., Rahman, H. and Stone, B. 2007. ‘Socially responsible investment: goody-two-shoes or bad to the effect of SRI on the performance improvement. bone?’ Journal of Investing, 16:1, 96–110. Although the overall effect size is negative with a [2] Hasselblad, V. and Hedges, L.V. 1995. ‘Meta-analysis of high statistical significance, the magnitude of effect screening and diagnostic tests’. Psychological Bulletin, size itself is very small. Therefore, SRI does not seem 117:1, 167–178. [3] Hedges, L.V. and I. Olkin. 1985. Statistical Methods for to have any significant influence on portfolio Meta-Analysis. London: Academic Press. performance. Meta-regression results indicate that [4] Lipsey, M.W. and Wilson, D.B. 2001. Practical Meta- publication bias and survivorship bias significantly Analysis. Thousand Oaks, CA: Sage. change the effect size. In terms of methodology [5] Rosenthal, R. 1984. Meta-Analytic Procedures for Social Research. Beverly Hills, CA: Sage Publications. adjustment of risk and matching procedure have a [6] Rosenthal, R. and Rosnow, R.L. 2008. Essentials of systematic influence on the effect size. As predicted a Behavioral Research: Methods and Data Analysis, 3rd positive screening generates a higher effect size than edition. New York: McGraw-Hill. a negative screening. [7] Song, F., Easterwood, A., Gilbody, S., Duley, L. and Sutton, A.J. 2000. ‘Publication and other selection biases in systematic reviews’. Health Technology Assessment, 4:10, 1–115.
A Meta-Analysis in Finance
128