Is the Efficient Frontier Efficient? by William C. ScheeL William J. Blatcher, Gerald S. Kirschner, John J. Denman Abstract The paperdefines plausible ways to measuresampling error within efficient frontiers, particularly when they are derivedusing dynamicfinancial analysis. The propertiesof an efficient surfaceare measuredboth using historical segmentsof dataand using bootstrapsamples. The surfacewas found to be diverse,and the compositionof assetportfolios for points on the efficient surfacewas highly variable. The paperbaces perfOrn1ance of on-frontier and off-frontier investmentportfolios for different historical periods. Therewas no clear cut superiorityto the on-frontier set of portfolios, althoughlower risk-return on-frontier portfolios were generally found to perform better relative to comparable,off-frontier portfolios than thoseat higher risk levels. It is questionable . whetherpractical deploymentof optimization methodscan occur in the presenceof both high samplingerror and the relatively inconsistenthistorical performanceof on-frontier portfolios. The implications of this paperfor DF A usageof efficient frontiers is that samplingerror may degradethe ability to effectively distinguishoptimal and non-optimalpoints in risk-return~. The analystshould be cautiousregarding the likelihood that points on an efficient frontier are operationallysuperior choices within that space. Thereare many possiblefrontiers that optimally fit different empirical samples. Samplingerror amongthem could causethe frontiers to traversedifferent regions within risk-rebun space,ped)aps at points that are disparatein a decisionsense. What is an efficient point on one frontier may 1 William C. Scheel,Ph.D., Consultant,DFA Technologies,LLC; William J. Blatcher,Financial Actuary, AEGIS Insurance Services,Inc.; Gerald S. Kirschner, FCAS, MAM Vice President,Classic Solutions Risk Management,Inc.; and JohnJ. Denman,Senior Vice President,Finance, AEGIS InsuranceServices, Inc. The authorsgratefully acknowledgethe very constructivecomments of reviewersof the paper. PageI of 45 be inefficient when calculated from a different sample. The paper finds the use of an efficient surface to be helpful in diagnosing the effects of such sampling error. 1.0 Introduction Companieschoose among investments often with the purposeof optimizing somegoal and always limited by constraints. Assetsare divided among competinginvestment alternatives with the hopethat risk will be minimized for a desiredlevd of return,either investmentreturn or overall return. When the allocation fulfills the goalswithin the boundariesof constJaints,it is thoughtto be efficient. The allocation is deemedto be a ~mber of the efficient set at a point on an efficient frontier. It is efficient becauseit dominatesoff-frontier, interior points in the risk-return space. This paperinvestigates this popular investmentallocation strategyin two ways. First. it seeksto detenninewhat the sensitivity of the frontier is to possiblesampling error in risk-return S18:e. Secondly,both on-frontier and off-frontier portfolio allocationsfor actual seriesof returnsare b'ackedfor their respectiveperfonnance. We begin with an apologue;it gives the reader both a rationale and definition of what we mean by an efficient surface. 1.1 A Sampling Error Apologue I walk into a casinowith shakyknees and a rather small~. Betting doesn't come easily for me, and I expectto losethe stake. Ralph told me I would lose it But. I have a bevy of infonnation gleanedfrom experimentsRalph did with a computerizedsimuJation of a Clapstable. One of the items I call "knowledge" is the efficient surfacehe madefor me. Ralph said it would help me understandthe risk/rebJrnproperties of the crapstable and guide me in allocating my stakeamong the various betsthat I can make "There are manybets you can make at the table," Ralph explained. "'Come', 'Big-8' am lots of others. think of the gaming as a multivariate~. Of course,it has probabilitiesthat are objectiveand can be measured Do you want me to figure out the combinatoricsof the crapsgame aM derive analytic solutionsfor optimal bet placement?My consulting fee might be a bit high becausethe math will take awhile, but I could do il .. mentally recalculatedmy meagerstake and replied, "Is there a lessexpensive wayT' Page2 of 4S Ralph shruggedand said, "Sure. I can usea computersimulation I haveand take a sampleof gameoutcomes. I'U usethe sampleto empirically developa covariancematrix for someof the bets. Then. I'll figure out which combinationsof bets have minimum variancefor a particular payoff. You can choosewhich risk/return profile of bets is best for you. You'O be able to more efficiently allocateyour stake. By-the-way, this is called an efficient frontier-it gives a profile of bets that are expected to producea given return with minimum variance. I'll do a sampleof 25 gameseach with a combinationof various bets. This will keep the cost down. "Well, okay," I replied. "But, will this single efficient frontier really work?" "What do you mean, 'single frontier'?" he asked. "What if the sample your computer simulation comes up with is unusual?" Ralph scratched his head, and I continued, "You measure this thing you call a sample covariance matrix. But, what if you took a different sample? You'd get a different sample covariance matrix, right?" "Yes." " And, it might be different?" "Yes. Even materially different." "So, your efficient frontier (Ef) is subjectto samplingelTor-it was empirically derivedfrom the sampleof only 25 games." I then asked, "What if you had a second sampleof 25 games and did another mathematical optimization. So, we now have two different EFs; both do the samething, but the answersare different Which one do I usewhen I walk into the casino?" Ralph exclaimed,"I'll take a sample,and then another,and another. Each will havea different EF. Then, I'll plot eachpoint of the samples'EFs in risk/return ~ce. I'll count the numberof times the variousEFs traversea particular cell in that space. Page3 of 45 Maybe 10 EFs b'aversethe cell at the coordinates(10,15). Maybe only 3 EFs traversedthe cell at (1,3). Don't you see? Just by counting the numberof times the sampleEFs baVersea region in risk/retmn spaceaOO normaliring the count to probabilities, I can measure an efficient surface. I asked,"Why is the surfaceimportantr' Ralph was now animated. He leapedto his feet. "Because,if the various sampleEFs all nversed the samecells, the EFs would all be the ~e-d1ere would no samplingerror. What if the surfaceis spreadout? SUA>Osesome sectors of it are relatively flat? Then the efficiency of the EFs varies. Would you prefer to pick a point on the surface(with a particular combinationof bets) that appearsmost often amongdifferent EFs? Probablyyou would. You want the surfaceto be tightly peaked In 3-dimensions,that's a ridge or very JXJintyhill; in two dimensions,it is a probability distribution with little variance. He then went home to begin the chore of sampling and constructing an efficient surface for me. I began to think, ..A single efficient frontier is measuredfrom data. We often think of the data being a samplefrom a replicableexperiment. If a sample of dice gamesis observed,the n-tuple bet outcomesfor the correlatedbets is the empirical data sourcefor an optirni7J11ioo.It is easyto seehow different samplescan be drawn when talking aboutdice games. But, the world of security returnsis different from a crapstable. What is a samplethere? What is the meaningof samplingerror, and how might it affect the way I measure efficient frontiers? Would an EF for securitiesreally be efficientr Theseare imponant questions-ones addressedin this paper. It is difficult to think of how we'd repeatan experiment involving security returns. Is a seriesof experimentsone that usesdifferent historical periodsof returns? Is it a bootStI3pof a broad segmentof history? Theseare the two approachesthat are equivalentto samplingand measuringsampling error. The result of our measurementsis an efficient surface. 1.2 Roadmap for the Paper Section2 of the ~r lays the groundworkfor measuringsampling error that affectsefficient frontier measurement We examinetwo approachesthat seem~cularly useful for dynamic fmaocial analysis(OF A). We also review the literature P.4of4S relating to EF efficiency. Section3 introducesdie notion of an efficient su1f«e-this is a constructfOf understaOOingand measuringsampling error in EFs. In this sectionwe describethe methodologyand data set usedin our ggdy The main body of results is presentedin Sections4, 5 and 6. We measureforecast perfonnance of efficient frontiers in Section 4. We are particularly concernedabout the performanceof off-frontier portfolios. Are they really inefficient? Do on-frontier portfolios dominateperformance as we might antici~te given that they are billed as "efficient'" The evidencewe presentin Section4 showsinstability in EFs derived both with historical segmentsand bootstrapsamples. This leadsus to concluck later that caution shouldbe exercisedwhen using efficient frontiers in OFA analysis. On the road to this conclusion,we closely examinethe efficient surfacein Section5. It portrayssampling error from two different ~ve&-historical and bootstrap sampling. The efficient surface is a useful COostJ\lctfor visualizing sampling error in EFs. We observethat such~r is particularly large in the high risk/return regionsof the surface This OOservationis reinforcedin Section6 by observingthe diversity of portfolio compositionas we comparedifferent historical segments. The