JOURNAL OF INVESTMENT CONSULTING TRACKING ERROR: An Essential Tool in Evaluating Index Funds By Chris Tobe, CFA As the dollars being indexed grow and portfolio management becon1es more sophisticated, consultants should demand tighter tracking to the index in question from their index n1anagers. In fact, the key to assessiI1g index fund managers is determining if their performance has tracked their benchmarks within reasonable tolerances. The alltl10r reviews what is known about this aspect of performance evaluation. He also sl1ares 11is experie11ce in detern1ining what factors caused a difference in performance for a $3 billion S&P 500 index fund. This article is based on a study written by the author and should be no attempt to outperform the index or Dr. Ken Miller for Kentucky State Auditor Edward B. bencl1mark. William F Sharpe holds that a passive Hatchett Jr. The study was reported in the July 27, 1998, strategy requires the manager issue of Pensions and Investments. Earlier versions of this ... to hold every security from the market, with each article were presented at two conferences on indexing. represented in the same manner as in the n1arket. Thus, if security X represents 3 percent of the value ver $1 trillion is now being managed of the securities in the market, a passive investor's within index funds. Lipper lists the portfolio will have 3 percent of its value invested in Vanguard 500 Index fund as the second X. Equivalently; a passive manager will hold the same percentage of the total outstanding amount of largest stock fllnd in the world with a each security in the market. 1 value of over $91 billio11 as of August 31. Frank Vanguard's website Cwww.vanguard.com) glos­ Russell Company recently reported that its top sary defilles index investing as "an investment strat­ institutional clients had over 46 perce11t of their egy involving the creation of a portfolio tailored as U.s. equity portfolios indexed, with 29 percent of closely as possible to match the perforn1ance of a their non-U.S. stock portfolios and 9 percent of specific stock or bond market index... and thus their fixed income portfolios n1anaged passively expected to provide a highly predictable return rel­ Russell attributes tllis growth ill dollars indexed to ative to the specific benchmarks...." the demand for lower asset management fees and This brings us to the issue of modifying an transaction costs, increased market efficiency, as indexed portfolio. Some indexing managers will well as the development of more benchmark index­ drop issues because it isn't practical to buy all the es. Witl1 many new entrants into index manage­ securities in the index. This practice is acceptable if ment and many plan sponsors themselves indexing, tlle mallager is using optimization or sampling tech­ new standards of performance evaluation need to niques that blind the manager to any security analy­ be established. sis factors. Other managers, however, use this selec­ tion process to enhance the overall return on the What Is an Index Fund? portfolio. The question for consultants is "to what imply put, index funds are paSSively man­ degree does this practice change the management of aged portfolios. This strategy seeks only to tl1e portfolio to an active style?" At the 1998 AIMR match the composition and investment annual conference, Vanguard's ]ol1n Bogle said that return of a specific stock, bond, or other tlle inte11t of the manager is the important distinc­ market index or performance benchmark. There tion. Modification could be acceptable, he 8 THE]OURNAL OF INVESTMENT CONSULTING © 1999 Investment Management Consultants Association Inc. Reprint with permission only. JOURNAL OF INVESTMENT CONSULTING explained, if the intent is to mirror the index. But ties) should have higher tracking error over once a manager taints the intent with any active short periods, but this should revert to zero over decision other th.an n1irroring, then the fund is no longer periods. longer passively managed. According to Bogle, a Since non-mean reverting tracking error is manager who tries to track a 1000 stock index by largely associated with fees and other costs, all of sampling with 300 stocks is still indexil1g. Anot11er the higher fee retail fU11ds tend to have negative manager who holds 497 stocks of the S&P 500 tracking error. Trading costs within the funds would index and substitutes the other thTee with tobacco lead to non-mean reverting tracking error. Always stocks in the hope that they will increase in value is maintaining a cash position creates long-term nega­ actively il1vesting, not indexing. tive tracking error, but the error could be positive over shorter periods. What Is Tracking Error? A11drew England discussed the causes of track­ lthough tracking error is not a new con­ ing error in an article on a United Kingdom website: cept, it is still not widely understood by One-third of index tracker funds fail to deliver ade­ many buyers of index funds. "Trackil1g quate performance according to Reuters error" can be defined as the percentage Hindsights performance differe11ce il1 total return between an index fund n1easurement service. This AmOUGH a11d the benchmarking index the fund was designed is based on the consisten­ to replicate. Even if a self-styled index manager out­ cy and persistency of pas­ TRACKING ERROR performs the deSignated benchmark, the difference sive funds in producing in performance is tracking error. returns that match their In the literature and among investment profes­ benchmarks. Factors that IS NOTA NEW sionals, the concept is sometimes addressed using can create performance differe11t terms. At the IMN-Fabozzi Superbowl of inefficiencies in trackers CONCEPT, IT IS Indexing conference in December 1998, Charles include the methods used Galligan, who writes the Bear Stearns I11dex by managers, dividends, STILL NOT WIDELY Forecast, referred more precisely to "index tracking taxes, and the size of funds. The approach used error." At the April 1999 IRR Global Indexing UNDERSTOOD BY by a passive fund (whether Summit in Bermuda, John Goodwin of Northern sampling, full replication, MANY OF Trust Quantitative Advisors broke tl1e concept or derivative style) and the BUYERS down to mean reverting and non-mean reverting changing composition of tracking error. Mean reverting can be positive or indices can also affect per­ INDEX FUNDS. negative, usually reverting back to zero over longer formance. The 111easure­ periods. Non-mean reverting is always negative, is ment of a tracker is based never recovered, and is best exemplified by fees and on the certainty of performance returns rather than transaction costs. the probability of performance. With an actively In this article, tracking error is defined as the managed one, you attempt to distinguish whether perce11tage difference in total return. between an the performance is due to skill or luck. With an index fund and its benchmark index. index tracker, researchers are trying to judge if a tracking error is due to the investment process or to random factors. There are two elements to this. The The Causes of Tracking Error first being whether the actual returns are in line with oodwin's mean reverting/non-mean benchmarks. The second is calculating the amount reverting categories help us examine the of tracking error, and any relative volatility is lon.g-term effects of tracking error. Mean deemed as negative. 2 reverting tracking errors are primarily Vanguard's website also discusses this issue the result of not holding the exact weighting of each under Myths on IndeXing: security in the index. It is assumed that iI1 the long Some fund sponsors are offering index funds with run all securities in an index will have performances cut-rate expense ratios in an effort to market the similar to the index and revert to the mean. A port­ "lowest cost index funds." Be wary of this practice for two reasons: folio sampling strategy (not holding all the securi­ VOL. 2, NO.1, NOVEMBER 1999 9 © 1999 Investment Management Consultants Association Inc. Reprint with permission only. JOURNAL OF INVESTMENT CONSULTING First, such funds are likely to raise their expense Galligan suggested that there could be 5 to 13 basis ratios-without warning-after a period of time. points inherent in managing an index fund: 3 to 5 Second, the performance of an index fund is from commissions, 2 to 3 from dividends, up to 3 affected not only by its expense ratio, but by from cash flows, and up to 2 basis points from factors such as brokerage costs and fund manage­ changes in the benchmarked index. ment practices. John Rerlkthaler of Mornin.gstar has years of Thus, those funds without adept management experience researching index funds. He has sug­ may have a higher tracking error relative to their tar­ gested some rough rules of thun1b for evaluation: get benchmark. S&P 500, ±50 basis points (including expenses); REIT or MidCap, 100 to 150 basis points; and Different Standards for Different Funds Russell 2000 or another sn1all cap index, 200 he tolerance for tracking error depends on basis points. many factors, but the most important is The reasons for these different standards are the index to be tracked and the size of the easily explained. How expensive is it to purchase fund. The S&P 500 has by far the most the securities contained in the benchmarked index? funds indexed to it. Most investment professionals, What about a manager's experience? Th.ere is a lot including Bogle, agree that funds indexed to it more historical experience in. rUl1.ning an S&P 500 should be held to higher standards with respect index fund than in running a smaller cap or to tracking error. Gus Sauter of Vanguard has this REIT fund.