Is the Efficient Frontier Efficient?

by

William C. ScheeL William J. Blatcher, Gerald S. Kirschner, John J. Denman

Abstract

The paperdefines plausible ways to measuresampling error within efficient frontiers, particularly when they are derivedusing

dynamicfinancial analysis. The propertiesof an efficient surfaceare measuredboth using historical segmentsof dataand

using bootstrapsamples. The surfacewas found to be diverse,and the compositionof assetportfolios for points on the

efficient surfacewas highly variable.

The paperbaces perfOrn1ance of on-frontier and off-frontier investmentportfolios for different historical periods. Therewas

no clear cut superiorityto the on-frontier set of portfolios, althoughlower risk-return on-frontier portfolios were generally

found to perform better relative to comparable,off-frontier portfolios than thoseat higher risk levels. It is questionable . whetherpractical deploymentof optimization methodscan occur in the presenceof both high samplingerror and the relatively

inconsistenthistorical performanceof on-frontier portfolios.

The implications of this paperfor DF A usageof efficient frontiers is that samplingerror may degradethe ability to effectively

distinguishoptimal and non-optimalpoints in risk-return~. The analystshould be cautiousregarding the likelihood that

points on an efficient frontier are operationallysuperior choices within that space. Thereare many possiblefrontiers that

optimally fit different empirical samples. Samplingerror amongthem could causethe frontiers to traversedifferent regions

within risk-rebun space,ped)aps at points that are disparatein a decisionsense. What is an efficient point on one frontier may

1 William C. Scheel,Ph.D., Consultant,DFA Technologies,LLC; William J. Blatcher,Financial Actuary, AEGIS Insurance Services,Inc.; Gerald S. Kirschner, FCAS, MAM Vice President,Classic Solutions Risk Management,Inc.; and JohnJ. Denman,Senior Vice President,Finance, AEGIS InsuranceServices, Inc. The authorsgratefully acknowledgethe very constructivecomments of reviewersof the paper.

PageI of 45 be inefficient when calculated from a different sample. The paper finds the use of an efficient surface to be helpful in diagnosing the effects of such sampling error.

1.0 Introduction

Companieschoose among investments often with the purposeof optimizing somegoal and always limited by constraints.

Assetsare divided among competinginvestment alternatives with the hopethat risk will be minimized for a desiredlevd of return,either investmentreturn or overall return. When the allocation fulfills the goalswithin the boundariesof constJaints,it is thoughtto be efficient. The allocation is deemedto be a ~mber of the efficient set at a point on an efficient frontier. It is efficient becauseit dominatesoff-frontier, interior points in the risk-return space.

This paperinvestigates this popular investmentallocation strategyin two ways. First. it seeksto detenninewhat the sensitivity of the frontier is to possiblesampling error in risk-return S18:e. Secondly,both on-frontier and off-frontier portfolio allocationsfor actual seriesof returnsare b'ackedfor their respectiveperfonnance. We begin with an apologue;it gives the reader both a rationale and definition of what we mean by an efficient surface.

1.1 A Sampling Error Apologue

I walk into a casinowith shakyknees and a rather small~. Betting doesn't come easily for me, and I expectto losethe stake. Ralph told me I would lose it But. I have a bevy of infonnation gleanedfrom experimentsRalph did with a computerizedsimuJation of a Clapstable. One of the items I call "knowledge" is the efficient surfacehe madefor me. Ralph said it would help me understandthe risk/rebJrnproperties of the crapstable and guide me in allocating my stakeamong the various betsthat I can make

"There are manybets you can make at the table," Ralph explained. "'Come', 'Big-8' am lots of others. think of the gaming as a multivariate~. Of course,it has probabilitiesthat are objectiveand can be measured Do you want me to figure out the combinatoricsof the crapsgame aM derive analytic solutionsfor optimal bet placement?My consulting fee might be a bit high becausethe math will take awhile, but I could do il ..

mentally recalculatedmy meagerstake and replied, "Is there a lessexpensive wayT'

Page2 of 4S Ralph shruggedand said, "Sure. I can usea computersimulation I haveand take a sampleof gameoutcomes. I'U usethe sampleto empirically developa covariancematrix for someof the bets. Then. I'll figure out which combinationsof bets have minimum variancefor a particular payoff. You can choosewhich risk/return profile of bets is best for you. You'O be able to more efficiently allocateyour stake. By-the-way, this is called an efficient frontier-it gives a profile of bets that are expected to producea given return with minimum variance. I'll do a sampleof 25 gameseach with a combinationof various bets. This will keep the cost down.

"Well, okay," I replied. "But, will this single efficient frontier really work?"

"What do you mean, 'single frontier'?" he asked.

"What if the sample your computer simulation comes up with is unusual?" Ralph scratched his head, and I continued, "You

measure this thing you call a sample covariance matrix. But, what if you took a different sample? You'd get a different

sample covariance matrix, right?"

"Yes."

" And, it might be different?"

"Yes. Even materially different."

"So, your efficient frontier (Ef) is subjectto samplingelTor-it was empirically derivedfrom the sampleof only 25 games." I

then asked, "What if you had a second sampleof 25 games and did another mathematical optimization. So, we now have two

different EFs; both do the samething, but the answersare different Which one do I usewhen I walk into the casino?"

Ralph exclaimed,"I'll take a sample,and then another,and another. Each will havea different EF. Then, I'll plot eachpoint

of the samples'EFs in risk/return ~ce. I'll count the numberof times the variousEFs traversea particular cell in that space.

Page3 of 45 Maybe 10 EFs b'aversethe cell at the coordinates(10,15). Maybe only 3 EFs traversedthe cell at (1,3). Don't you see? Just by counting the numberof times the sampleEFs baVersea region in risk/retmn spaceaOO normaliring the count to probabilities, I can measure an efficient surface.

I asked,"Why is the surfaceimportantr'

Ralph was now animated. He leapedto his feet. "Because,if the various sampleEFs all nversed the samecells, the EFs would all be the ~e-d1ere would no samplingerror. What if the surfaceis spreadout? SUA>Osesome sectors of it are relatively flat? Then the efficiency of the EFs varies. Would you prefer to pick a point on the surface(with a particular combinationof bets) that appearsmost often amongdifferent EFs? Probablyyou would. You want the surfaceto be tightly peaked In 3-dimensions,that's a ridge or very JXJintyhill; in two dimensions,it is a probability distribution with little variance.

He then went home to begin the chore of sampling and constructing an efficient surface for me. I began to think, ..A single efficient frontier is measuredfrom data. We often think of the data being a samplefrom a replicableexperiment. If a sample of dice gamesis observed,the n-tuple bet outcomesfor the correlatedbets is the empirical data sourcefor an optirni7J11ioo.It is easyto seehow different samplescan be drawn when talking aboutdice games. But, the world of security returnsis different from a crapstable. What is a samplethere? What is the meaningof samplingerror, and how might it affect the way I measure efficient frontiers? Would an EF for securitiesreally be efficientr

Theseare imponant questions-ones addressedin this paper. It is difficult to think of how we'd repeatan experiment involving security returns. Is a seriesof experimentsone that usesdifferent historical periodsof returns? Is it a bootStI3pof a broad segmentof history? Theseare the two approachesthat are equivalentto samplingand measuringsampling error. The result of our measurementsis an efficient surface.

1.2 Roadmap for the Paper

Section2 of the ~r lays the groundworkfor measuringsampling error that affectsefficient frontier measurement We examinetwo approachesthat seem~cularly useful for dynamic fmaocial analysis(OF A). We also review the literature

P.4of4S relating to EF efficiency. Section3 introducesdie notion of an efficient su1f«e-this is a constructfOf understaOOingand measuringsampling error in EFs. In this sectionwe describethe methodologyand data set usedin our ggdy

The main body of results is presentedin Sections4, 5 and 6. We measureforecast perfonnance of efficient frontiers in Section

4. We are particularly concernedabout the performanceof off-frontier portfolios. Are they really inefficient? Do on-frontier portfolios dominateperformance as we might antici~te given that they are billed as "efficient'" The evidencewe presentin

Section4 showsinstability in EFs derived both with historical segmentsand bootstrapsamples. This leadsus to concluck later

that caution shouldbe exercisedwhen using efficient frontiers in OFA analysis.

On the road to this conclusion,we closely examinethe efficient surfacein Section5. It portrayssampling error from two

different ~ve&-historical and bootstrap sampling. The efficient surface is a useful COostJ\lctfor visualizing sampling

error in EFs. We observethat such~r is particularly large in the high risk/return regionsof the surface This OOservationis

reinforcedin Section6 by observingthe diversity of portfolio compositionas we comparedifferent historical segments.

The final sectionis devotedto conclusionsand cautionson the ~ of EFs in OFA work. We concludethat EFs may not

warrant the terD1efficient. Their best use may be as advisory measurements concerning the properties of risk/return space.

2.0 Scenario Generationin DFA

Dynamic financial analysisinvolves scenariogeneration. There are many types of scenariosthat are simulatedso that the

model builder can measurea hypotheticalstate-of-the-world with accountingmetrlcs. Assetgenerators typically createreturns

for investedassets. They model exogenouseconomic conditions. Each modelersees the forces of the financial markets

unfolding accordingto a set of roles. The rule set is almost as diverseas the numberof modelers.

SomeOF A model builders prefer stochasticdifferential equationswith variousdegrees of functional interrelatedness.The

transitionof returnsover time, as wen as the correlationsamong different assetcomponents, always is representedin multiple

simultaneouseqWltions. Other DF A modelersuse multivariate nomlai models.which conjecturea covariancematrix of

investmentreturns. Thesemodels do not havetime-dependent b'ansitiOD modeling information. Suchan efficient frontier, by

Page5 of 45 definition. has no time transition properties. A sampletaken from any sub-periodwithin the time-serieswould contain samplingerror, but otherwise,the investmentallocation would be unaffected

Both approachesbegin with a single instanceof reality. They both pIrport to model it One approach.stochastic equations, useslargely subjectivemethods to ~eterize the process.2Another awroacb to modeling clings to assumptionsthat seemto be or are takento be realistic! Both producescenarios that are deemedsufficiently similar to reality to representit for the pwposeat hand

The efficient frontier calculation can be a constrainedoptimization either basedon a SUDpiefrom a historic seriesof returnsor a derived serieswith smoothingor other ad hoc lkijustment Alternatively. an EF may be createdfrom simuJatedOF A results.

Both the efficient frontier aM OF A asset-based modeling are using the same set of beliefs regarding the manner by which statisticallyac<:eptable parameters are used.4They both stan with a single historic time-seriesof rehJrDSfor various component assets.

2.1 Two Viewpoints on the Use of Efficient Frontiers

The pmctitioner hasa straightforwardobjective: define investmentallocation strdtegygoing forward. Today's portfolio allocation leadsto tomorrow's result The portfolio is then rebalancedrelative to expectations.The new one leOOsto new

results. The cycle repeats. Where doesthe chicken end and the egg begin? In practice,the practitionerhas only one instance of yesterday'sreality and tomorrow's expectationsfrom which to constructa portfolio and a model.

There are at least two approaches to using a OF A model to define an investment allocation. In one, a OF A analyst might set up an initial allocation of assetsusing an efficient frontier obtainedfrom quadIaticoptimi7Jition on a prior historical period. A

OFA model would be repeatedlynm-a different state-of-the-worldwould ensueeach time, and a different ~g obtained

2 The calibration may dependon examinationof stylistic facts, but there seldomis formalized. statisticalhypothesis testing to judge whetherthe facts can be acceptedas suchor whetherthe representationof thesefacts in the model is really a scientific dctermination. .1 Somemodels use multivariate nonnal simulationfor renderinginvestment returns for consecutiveperiods. Thereusua1Iy is an assum~on that the covariancematrix usedfor multivariate nomlal simulation is stationaryfrom period to period in these models. 4 DFA and optimization do havea critical junction. SomeDF A modelersbelieve they understandtime dependencieswithin period-to-periodrates of return. EF attem~ to optimize expectedreturn. If there is a time-

Page6 of 4S for the metric. Thesesimulations produce endpoints in the modeledrisk-return space. In this approach,one beginningasset allocation leads to many different observations about eoopoints. The reason they are different is that, although each starts with the samestate, the model simulatesvarious outcomes.Each hypotheticalone probably leadsto a different endpointfor the planning horizon.

But, anotherviewpoint exists.5 We refer to it as the hybrid approach. Supposethat history servesa valid purposein calibrating a model,but shouldnot be usedto define a beginning allocation. In this viewpoint, the investmentmix is suggestedby the optimizer. DF A servesonly to measurewhat could happenwith somehypothetical starting allocation.

The optimizer dealsthe cardsin this deck, and DFA traceswhere the cardslead.6.7 The optimizer, not the modeler,submits an initial allocationfor review. In this hybrid approach.there is no initial portfolio basedon optimization using prior history. In the hybrid model, the optimizer finds a portfolio, which leadsto an ex post optimal result The metric usedin this optimization is part of the DF A model-it is calculatedby the accountingmethodology of the model as it generatesfuture statesof the world. It may be difficult to reconcile the useof efficient frontiers for investmentswithin hybrid-DFA modeling that, on the one hand,believes there is a historically dependentcomponent that canbe usedfor calibration,but rejectsthe useof datato

define a startingportfolio. Yet. on the other hand,simulations of that model are derived to constructan efficient frontier. It

may appearas though history hasbeen rejected as information for the purposesof decision-making,yet indirectly it is usedto

representthe future. The startingportfolio in the hybrid approachis basedat leastindirectly through modeling and should

representan analyst's expectations.These expectations are in theory built into the model for return scenariogeneration and

that model was calibratedto history in somefashion.

In DFA work. a performancemetric is chosen. This metric is measuredwithin a risk-return space. The metric must be

measurableaccording to the chosenaccounting framework. Risk might be variance,semi-variance or somecbance-constrained

5 Correnti, et ai, review an approachsimilar to the hybrid model describedhere. 6 The optimizer posits a trial solution; it consistsof a certainportfolio allocation. This trial allocation doesnot dependon any prior allocation of assets. Rebalancingthat ensuesduring the optimization period (and underthe control of the OFA model) also is unknown to the optimizer. The objectivevalue that is returnedby the model is driven by the initial triaI solution and model machinationsthat build on the trial solution. 7 Investmentrates are forecastedby the OFA model, which might use multivariate normal simulation. There may be an overlapbetween what the optimizer usesand what the OFA model uses. For example,the covariancematrix usedfor the multivariatenormal simulation is estimatedfrom historical dataand generallyis assumedto be stationaryduring the forecast period. It is usedboth by the optimizer and by the OFA model.

Page7 of 45 function of the metric. In the real world. the corporate manager is rewarded for favorable performance of the metric and often penalized by unwanted risk in the metric. The volume of investment in various stochastic components affects a metric's performance. The operational question is how should an allocation be made in investments so that performance of the metric is optimized.

In the forecastperiod, the modelergenerates a scenarioof unfolding ratesof return using, say, a multivariate, time-dependent assetmodel. An examplewould be any of the multi-factor meanreversion models in usetoday. The simulatedprogression of returnsfor a scenariogenerated by one of thesemodels is affectedby an underlying mechanismthat forces unusualdeviations in the path back towardsan expectedtrajectory of returns. The OFA model typically ties in someway the businessoperations to the simulatedeconomic environment.s This economicscenario typically generatesother economicrates such as the rate of inflation. A scenariothat is generatedby the economicmodel is taken to be exogenous;it is mingled with expectationsabout corporateperfonnance. The company'soperations are tied to the exogenousinfluences of the economicscenario.

In the end, this modeling processis repeatedmany times for the optimizer in the hybrid model. The optimizer requiresan answerto the question:given an initial investmentallocation. what is the end-horizonperfonnance of the metric. The optimizer forcesthe model to measurethe result of a simulation experimentgiven only an initial investmentallocation. The model takesthe allocation and producesan experimentalpoint in risk return space. All that is requiredof the model is its ability to measurethe trajectory of the metric within the company'sbusiness plan and a beginningallocation of assets.In this

regard,the hybrid model is using a sort of dynamic programmingapproach to optimization. The possibleoutcomes are

considered,and the most desirabletraced back to the inputs (initial allocation). The hope is that the optimized feasibleset is

robustrelative to possiblestochastic outcomes in the model trajectory. The efficient frontier tracesthe allocationsnecessary to

achievevarious points in this risk-return space. All of this raisesthe thorny questionof subsequentperformance dominaIK:e of

the on-frontier portfolios in the hybrid model. Do EF points truly dominatethe perfonnanceof off-frontier frontier points-

portfolios that are thought to be inefficient and havehigher risk for the samereturn level?

8 A typical behavioralpattern for businessgrowth is modeling it as a function of inflation, which was generatedby the economic scenario. Another is to tie severity in claims to underlying inflation as unfolded in the economic model simulation.

Page8 of 45 The reason that this is a hybrid approach is that DF A modeling is not deployed on an optimal asset allocation derived directly from the prior time-series. Rather, DF A is combined with optimization to answer the single question: how should the portfolio be immediately rebalanced to achieve an optimal point in risk-return space over the future OF A planning horizon.

Two portfolios can be devised through optiJIIization procedures-one is based on historical results prior to the start of the simulated future time periods. Another one involves allocations that are selected and tried by the optimizer-the OF A model is integral to this second approach. The latter hybrid optimization uses DF A-measured metrics in the optimizer goal function. If applied over the course of the simulated future time periods, and according to the plan of the DF A model, the hybrid approach woald seem to yield optimal results at the end of the simulated time horizon. There is no reason to suppose that these two approaches produce the same initial portfolios. Which one is the real optimwn?

During the planning horizon, the hybrid model may ignore imperfections that. in real life, might have (and, probably would have) been dealt with by on-going decision making. The EF could have been recalculated with realized data and the portfolio

rebalanced. The published state-of-the art in DFA modeling is unclear in this regard; but, it may be that no inua-period

portfolio optimization is done by OF A models between the time the analysis starts with an allocation posited by the optimizer

and when it ends, say, five-years later with a OF A-derived metric. It is inconceivable that an organization would mechanically

cling to an initial, EF-optimal result for an operational period of this length without retesting the waters.

2.2 Limitations of this Study for Use of the Efficient Frontier in DFA

We do not do a completeDF A analysis-there is neithera liability componentnor a conventionalDF A metric suchas

economicvalue of a businessenterprise. Rather,the dataare limited entirely to marketable,financial assets.Nevertheless, we

believe our findings are of value to OFA work. If the efficient frontier producedsolely within a traditional investment

9 There is no reasonother than a few computationalprogramming complexities why intra-periodoptimizations cannot be done within DFA models. The questionis whetherthey are, or they are not, being done. For example,the DF A model can simulate a wide variety of rebalancingstrategies including the real-life one that involves a rebalancingtrigger for simulatedportfolios whoseallocation hasdeviated from a recentEF by someamount Mulvey, etal [1998, p. 160] describean n-period simulation wherein suchrebalancing is triggered. In addition. Mulvey, et ai, describethe useof optimization constraintsin a cleverway to achievean integrationof strategic,long-term optimi7.ationwith short-termtactical objectives. However, a DF A modelthat allows intra-periodoptimization must also capturethe transactionand tax costsassociated with the intra-periodrebalancing and re-optimization. SeeRowland and Conde [1996] regardingthe influence of tax policy on optimal portfolios and the desirability of longer term planning horizons.

Page9 of 45 framework has unstable properties, these instabilities will awly to its use in OF A work were it to be calculated and used in a similar way.

2.3 Other Investigations of the Efficacy of EF Analysis

Michaud basextensively investigated the useof EFs with particular regardits generalefficacy for forecasting. For example,he basshown [1998, pp. 115-126]that inclusion of pensionliabilities can substantiallyalter the statisticalcharacteristics of mean- varianceoptimization for investmentportfolios.

Michaud's book [1998] examinesefficient frontiers both with respectto their inherentuncertainty and what might be doneto

improve their worthiness. He suggeststhat the effectsof samplingerror may be improved using a methodologydescribed as a

resamp/edefficient frontier. The motivation for somekind of improvementover classicalEFs is that "...optimized portfolios

are 'error maximized' and often have little, if any, reliable investmentvalue. Indeed,an equally weighted portfolio may often

be substantiallycloser to b11eMY optimality than an optimizedportfolio." [Michaud, 1998,p. 3].

The determinationof a resampledefficient frontier is complex; Michaud haspatented it Although his book exposesthe core

of the methodthat he believesimproves on forecasterror, there is no empirical evidenceprovided in the book that a resampled

efficient frontier basthis desirableeffect. Interestedreaders are directedto his book. The conceptof an efficient surface

espousedin our paperis built on different constructs. We will readdressthe important work of Michaud at a later point in the

paper. We now turn to the definition and measurementof an efficient surface.

3.0 The Efficient Surlace

An efficient frontier consistsof points within risk -return spacethat haveminimum risk for a return. If there were a time-

stationary,multivariate probability distribution for prior history, then history is a samplefrom it History, therefore,would

have.sampling error.1~

10 If there were conjecture,the multivariate distribution would be subjective,and the efficient frontier would be the subjective frontier. A subjectivelyderived EF hasno samplingerror, but it may lose operational appealwhen representedin this manner, becausesubjectivity requiresdifficult reconciliation within a corporate,decision-making framework.

Page10 of 45 The concept of a conditional marginal probability distribution either for return or risk emerges, and it, too, would have sampling error. We discuss the properties of this marginal distribution. an equi-return slice of the efficient surface, in Section

S.l

Were the instance of reality to be a sample. what is the sampling error?

Figure 1 Comparison of Efficient Frontiers for Different Time Periods

Figure 1 showsefficient frontiers for random5-year blocks of history. The EFs were derived from monthly returnsbeginning

in January,1988. Each curve in Figure 1 requiresoptimizations for a 5-yearhistory of returns. The block of monthly returns

was picked at randomfrom the entire time series. The points along eachEF are obtainedfrom separatepasses through the data

with the optimizer. On eachpass, one of the constraintsdiffers. That constraintis the requirementthat the averageportfolio

returnbe a specifiedvalue in the return domain. The optimizer's objectivefunction is the minimization of varianceassociated

with that portfolio expectedreturn.

Each EF in Figure I consistsof nine points; eachpoint involves a separatequadratic optimization. For example,one of the

optimization constraintsis the portfolio expectedreturn, which is set to an equality condition. There were nine different

expectedreturns used in the study; one was a monthly return of 0.004. An examinationof the figure at this value showsa point

for eachof the four EFs. An empirically derived covariancematrix was detenninedfor eachof the four time seriesillustrated

in Figure I as well as for hundredsof othersthat are not shown. Thejuxtaposition of the EFs displaysa tangle of overlapping,

Page 11 of 45 crisscrossingcurves.!! This illustration canbe viewed as samplingwith replacementfrom a historical sample;it is appropriate, then, to view the figure as illustrative of a probability surface. It is a surfaceshowing the extent of samplingerror provided there basbeen a stationary,multivariate distribution of components'retums.12 Figure 1 indicatesthat it may be hazardousto accept any particular segment of history as the "best estimator. This figure showsonly severalof the EF curvesthat build up an efficient surface. Examplesof efficient surfacesappear later in Figures8 and 10. The distribution of risk in a cross- sectionalslice of this efficient surfacealso is reviewedin Section5.

The position and slopeof the EFs in Figure I are wildly different, and were other historical EFs to be included, the complexity would be greater. This lack of historical stability castsdoubt on the operationalvalidity of a particular efficient portfolio actually producing optimal performance. The figure also hints that off-frontier portfolios may perfonn as well or better than on-frontier portfolios. We examinethis questionof forecastreliability in detail in Section4.

In addition to the positional changesin EFs over time, there is dramaticchange in portfolio compositionalong the curve of

eachEF in Figure An exampleof the changein portfolio compositionfor EFs appearsin Figures2a and 2b. Each chart is

categorical- a tic maIk on the x-axis is associatedwith one of nine optimi~tion points. Each chart showsa stackedarea

renderingof the proportion of an assetcomponent within the efficient set. If the readerviews the chart in either Figure 2a or

2b from left-to-right, the unfolding change,and possiblecollapse, of a particular componentis illustrated. This type of chart is

a useful way to show a component'scontribution to the efficient set moving along the EF from low-fisk-return to high risk-

return portfolios.

11 Somesegments of EFs suchas thoseshown in Figure I canbe indeterminate.This is becausethe quadraticoptimizer could not identify a feasibleset of investmentalternatives for all of the averagereturns chosen in the analysis. There is a small probability of overlap of databecause the 5-yearblocks of returnsused for eachEF could have overlappingsub-periods of time. 12 The populationdistribution is unknown.but it is estimatedfrom the historical recordby calculation of an empirical covariancematrix for eachhistorical block.

Page 12 of 45 Figures 2a Portfolio Compositions for Different Efficient Frontiers

Efficient Frontier Profile

100%

80%

60%

40%

20%

0% 1 2 3 4 5 6 7 8 9 a=Point

Figures 2b Portfolio Compositions for Different Efficient Frontiers

Efficient Frontier Profile

1~ ~ 80% 8~1 70% mLBOOff' «J% 8HYLD 50% 8lST8 40% 8S&F5 30% 20% 8NTlltD 10% aEAFBJ 0% 1 2 3 4 5 6 7 8 8 e=Point

There is faint hopethat the two different EF portfolio compositionsshown in Figures2a and 2b will operationallyprOOuce the

sameresult when put in practice-were this to be a reasonablerepresentation of the effectsof samplingerror, the operational

useof efficient frontiers would be questionable;sampling error swampsoperational usefulness and forecastresponsiveness.

Page13 of 45

~ However, another illustration, Figure 3, indicates that if history is a sample from a multivariate distribution. there should be optimism that the efficient frontier evolves slowly, at least measured in monthly metrics. This figure shows EFs calculated from consecutive, overlapping historical blocks of time. In this case, the time interval between between consecutive EFs is one

month. The stability deteriorates fastest at higher risk-return levels. The result was found to hold for a wide variety of

consecutive historical blocks starting at various points since 1977. This stability may provide an operational basis for

investing in an on-frontier portfolio and seeing its perfOrmalx:e prevail over off-frontier portfolios. at least for relatively short

planninghorizons.

Figure 3 EFs for Consecutive Time Periods

Efficient Frontier 1

L Risk

Thereare other ways to usethe historical record. The papershortly will turn to the useof the bootstrapas a methodof

measuringsampling error. First, the dataand manipulationmethods are describedin more detail.

3.1 Data Manipulation

This studYuses the time seriesdescribed in Appendix A: Review of Data Sources.Except where gapswere presentin the

historical record. the portfolio returns are actual. I 3,14

13 The datarepresent returns for a selectedgroup of investmentcomponents. There was no attemptto filter or smooththe time series in any way. However, a few gaps in the historical record where interpolated.

Page 14 of 45 The datawere usedin two ways: (1) bootstrapsamples were madefrom the original time seriesin an attempt to approximate samplingerror phenomena,and (2) various historical seriesof the datawere usedfor performanceanalysis. The study examinesperiod segmentation,and the performanceof efficient and inefficient portfolios for different forecastdurations.

3.1.1 Historical Performance Analysis

In this sectionof the paper,data for an efficient frontier are extractedfor an historical period and usedto evaluatethe efficient

frontier. The on-frontier portfolios are minimwn varianceportfolios found using quadraticprogramming. I 5 Off-frontier

portfolios also were calculated.16The study is concernedwith whether the performanceof off-frontier portfolios really were

inefficient comparedto the performanceof on-frontier portfolios.

3.2 Bootstrap Sampling

The bootstrapsample of a data set is one with the samenumber of elements.but randomreplacement of every elementby

drnwing with replacementfrom the original set of data. Whenthis processof empirical resamplingis repeatedmany times, the

bootstrapsamples can be usedto estimateparameters for functions of the data. The plug-in principle (Efron and Tibshimni,

1993,p. 35] allows evaluationof complex functional mappingsfrom examinationof the samefunctional mappingon the

bootstrapsamples. The function 8 = t(F: of the probability distribution F is estimatedby the samefunction of the empirical

"" " distribution F, 8 = t(F) , where the empirical distribution is built up from bootstrap samples. This technique often is

deployedfor the derivation of errors of the estimate.

14 One techniquefor deploying efficient frontiers within DF A analysisinvolves removal of actual valuesfrom the data series usedin optimization. Thesepoints in the actualtime seriesmay be deemedabnormalities. The efficient frontier calculation doesnot use all availabledata or usesthem selectively. SeeKirsclmer [2000] for a discussionof the hazardsof historical p;riod segmentation. 5 All optimization was done using Frontline Systems,Inc. PremiumSolver Plus V3.5and Microsoft Excel. 16It is possibleto restatea portfolio optimi~tion problem to produceoff-frontier portfolios. Theseare assetallocations for points in risk-return spacethat are within the concaveregion defmedby the set of efficient points. They are portfolios with variancegreater than the minimum variancepoints for the sameexpected returns. They were found by goal equality calculationusing the sameconstraints as were usedfor minimum varianceoptimization. However,the equality risk condition was set to a higher level than found on the efficient frontier. Non-linear optimization was usedfor this purposewhereas quadraticoptimization was usedfor minimum varianceoptimization.

Page 15 of 45 The plug-in featuresof a bootstrapenable inference from sampleproperties of the distribution of bootstrapsamples. The plug- in propertiesextend to all complex functions of the bootstrap,including standarddeviations, means, medians, confidence intervalsand any other measurablefunction. The EF is one of thesefunctions.

The bootstrap is used in this paper to illustrate the impact of sampling error on the EF .17 EF is a complex function of the historical returnsfrom which it was calculated. If the sampleis from a larger, unknown domain,the bootstrapprinciples apply.

In the caseof correlatedinvestment returns, a segmentof history might be thought of as a sample,but it may not be operationallymeaningful because of samplingerror. Yet, tl¥: useof the historical datain DF A applicationstreats it as though

it were both meaningful, representative and not a sample.

The behaviorof the EFs for our bootstrapsamples is a non-parametrictechnique used to evaluatethe effect of samplingerror.

were history to be properly thought of as a sample. Becauseactuarial scienceis built largely on the preceptthat past history,

even of seeminglyunique phenomena,really is a sample,we proceedalong this slippery slopetoo.

3.2.1 Bootstrapping n- Tuples

The n-tuple observationof correlatedobservations at time t can be sampledwith replacement.This techniquewas usedby

Laster [1998]. The experimentis similar to drawing packagesof colored gum dropsfrom a productionlot. Each package

containsa mixture of different colors that are laid out by machineryin somecorrelated manner. Supposethe lot that hasbeen

sampledoff the production line containsn packages.A bootstrapsample of the lot also containsn observations.It is obtained

by draws.with replacement,from the original sample lot. The n-tuple of investmentreturns at time t is analogousto a package

within the lot of gum drop samples. The historical sequenceof correlatedreturns is analogousto the mix of different colors of

gum dropsin a package. The analogyhalts becausewe know the lot of gum drop packagesis a sample. We neverwill know

whetherthe sequenceof historical, n-tuple-investmentreturns is a samplein a meaningfulsense.

The dataconsist of a matrix of monthly returns;each row is an n-tuple of the returnsduring a common interval of time for the

component assets (columns of the matrix); the value of n was ten and measures thc use of the ten investment categories

describedin Appendix A: Review of Data Sources.The bootstrapmethod involves samplingrows of the original datamatrix.

Page 16 of 45 An n-tuple describingthe actual returnsfor assetcomponents at an interval of time is drawn and recordedas an "observation' in the bootstrapsample. Becausethis n-tuple can appearin anotherdraw, the processinvolves sampling with replacement.

This randomizedchoice of an n-tuple is repeatedfor eachobservation in the original sample. When the original samplehas beenreplaced by a replacementsampling of the sample,the result is referredto as a bootstrapsample. This processof drawing a bootstrapsample can be repeatedmany times, usually in excessof 2,000.

Each bootstrapsample has both a measurablecovariance matrix and an efficient frontier that can be derived using that covariancematrix. It is unlikely that any two bootstrapsamples will necessarilyhave the samecovariance matrix. Each samplecan be subjectedto mathematicaloptimization to producean efficient frontier. The study askswhether this frontier is stableacross the samples. Instability is measuredin two ways. First, the bootstrappedefficient frontier may fluctuate from sampleto sample. This meansthat the distribution of risk for a return point on the EF is not a degeneratedistribution that collapsesto a single point. Rather,there is a rangeof different portfolio risks amongthe bootstrapsamples at a given return.

Thereis a probability distribution associatedwith risk, given a return amongthe bootstrapsamples. In other words, the study

attemptsto measurethe distribution, and the studyviews that distribution as a measureof samplingerror in risk-return spaceas

it impactson the calculationof an efficient frontier.

Second,the portfolio allocationsmay diverge qualitatively amongbootstraps. Were portfolio allocationsto be aboutthe same

in an arbitrnrily small region of risk-return spaceamong different bootstrapsamples, the practical effects of samplingem>f

would be small.

3.2.1 Extension of the Bootstrap Sample as a DFA Scenario

The bootstrapsamples can be usedin the way a DFA model might haveused the original historical data,including their direct

usewithin the calculation of the DF A resultsas a rnndominstance of investmentresults. They are the sourceof DFA

scenarios.This papersuggests how that direct useof the bootstrapmight unfold in a DFA liability-side simulation, but it does

] 7 The bootstraphas been used in connectionwith mean-varianceoptimi~tion by Michaud and othersin an attemptto improve performance of EF portfolios. See Michaud [1998].

Page 17 of 45 not deploy it in that manner. 18.1 9 The authors have a less ambitious objective of examining just the performance of the efficient frontier built from bootstrappinginvestment information.

3.3 Sampling Error within Risk-return Space

There is no clear-cutmethod for estimatingsampling error that may exist in risk-return space. We do not know the underlying distribution generatingan historical sample. We do not know whethera populationdistribution, were it to ex.ist,is stationary

over any time segment. We might, however,view history as an experimentalsample, particularly if we want to useit to forecastcorporate strategic decisions using DFA

Samplingerror can be envisionedand approximatedin different ways for this hypotheticalunfolding of reality. One way is to

break the actualtime seriesinto arbitrary time segmentsand ask whethera random selectionamong the subsetsof time leadsto

different, operationallydisparate results-these would be EFs basedon the sub-segmentof time that haveportfolio allocations

disparateenough to be viewed as operationallydissimilar. If they are dissimilar enoughto warrant different treatment,a

samplingdistribution of interestis the one measuredby the effects of thesetime-period slices.

Another approachis to envision prior history as an instantiation,period-to-period, from an unknown multivariate distribution.

The samplingerror in this processis driven by a multivariate distribution. Dependingon our model, we mayor may not place

dependenciesfrom prior realizationson this period's realization. That is, for DFA investmentreturn generationand intra-

period portfolio rebalancing,the multivariate model may be stationaryor non-stationarywith respectto time.

3.3. 1 Michaud's Efficient Frontier

Michaud [1998] approachesthe measurementof samplingerror effectson EF in a different way. Although his approach

differs, his overall conclusionsare important and consistentwith many of our findings. He notes [1998, p. 33], "The operative

18 Although the n-tuplc usedin this paperis a cross-sectionalobservation of returns,it can be expandedto a cross-sectionof the entirebusiness environment at time t. This includesall economicaggregates, not just ratesof return. Any flow or stock businessaggregate that can be measuredfor interval t is a candidatefor the n-tuple. This would include, inflation, gross domesticproduct or any worldly observationof the businessclimate prevailing at that time. A bootstrapsample can be usedas a componentof a larger simulation requiring simulation of theseworldly events. 19DF A model builders spendtime modeling empirical estimatesof processand parameterrisk [Kirschner and Scheel,1998]. Bootstrappingfrom the dataremoves much of this estimationwork and leavesthe datato speakfor themselves.

Page18 of 45 question is not whether MY optimi~tions are unstable or unintuitive, but rather, how serious is the problem. Unfortunately for many investment applications, it is very serious indeed." Our paper will draw a similar conclusion.

He does not refer to an efficient surface but calculates a "resampled" portfolio that seems to capture some similar properties.

Michaud uses multivariate DOnna! simulations from the same covariance matrix used to calculate EFs. This covariance matrix

is from a sampleof data-the data observedduring somehistorical period. Justwhat definition of samplingerror basbeen

accommodatedin the Michaud resampledportfolio is unclear.

One of the Michaud simulationsis not equivalentto a bootstrapsample used in this study. Michaud's approachdoes not

attemptto adjustfor a primary sourceof sampling error-sampling error in the covariancematrix. In our study,each bootstrap

samplehas an independentlymeasured covariance matrix. Using the DFAjargon of Kirschner and Scheel[1998], Michaud's

approachmay not accountfor parnmeterrisk in the underlyingreturns generation mechanism. The ranking mechanismused by

Michaud to combineEFs derived from various multivariatenonnal simulationsmay distort risk/return spacebecause each EF

is segmentedin somenon-linear fashion to identify equally rankedpoints in risk/return space[Michaud, 1998,p. 46, footnote

11]. The portfolio profiles for identically rankedEF points are averaged.yet it is not clear that equi-rankedpoints fall within

the same definition of risk/return space.

3.4/mportance to DFA Scenario Generation

This papercannot and doesnot attemptto rationalizethe processunderlying investmentyields over time,20 Rather,the model

builder shouldbe careful to designthe DF A model to be in accordancewith perceptionsabout how a samplingmethodology

may apply. The useof the model will invariably mimic that viewpoint

If, for example,one views history in the fashion imaginedby a bootstrapof n-tuples,and if that view doesobserve operational

differences,then one can createscenarios from bootstrapsamples. No more theory is required. Hypothetical investment

returnsare just a bootstrapsample of actual history.

20 What if there were no commonobservable stationary probability measurefor securityprices? Kane [1999, p 174] argueswe must use utility measurements.

Page19 of 45 Similarly, ifEFs for historical periods produce superior performance in forecasting (compared to portfolios constructed from off-frontier portfolios derived from the same data), then the use of an empirically detennined covariance model and multivariate nonna! simulation makes a great deal of sense.

3.5/mporlance to DFA Optimization

Optimization often is used within DF A and cash flow testing models to guide portfolio rebalancing. The DF A model usually grinds through the process of business scenario and liability scenario simulations before the optimizer is deployed. But, accounting within the model often is done while the optimizer seeks a feasible solution.

The sequence of model events runs like this:

Independentlymodel many instancesof exogenousstates of the businessworld (e.g.,asset returns, inflation. measures

of economicactivity, monetaryconversion rates). Nwnber theseinstances, B" ~, B3,...,Bn.Note that eachof these

instances is a vector containing period-specific values for each operating fiscal period in the analysis.

Model many instancesof the company'sperfonnance. Numberthese instances C\, ~ ,Cn. C\ often is dependenton

B. becauseit may usean economicaggregate such as inflation or economicproductivity to influence C1'sbusiness

growth or loss and expenseinflation. Each C is a vector spanningthe samefiscal periodsas B.

Observethat in someD F A modelsneither B nor C is necessarilyscaled to the actual volume of business.They are

unit ratesof changefor underlying volumesthat areyet to be applied.

4. Let the optimizer searchmechanism posit a vector of weights that distribute the volume of assetsat to ' the inception

point for a forecastperiod.

s. Apply the accountingmechanisms uscd by the DFA model to beginning assetsand accountfor the unit activities

expressedinB and c!\ Do this accountingfor eachvector pair {BI,CI}, {~,Cl},...,{BnCn} over the rnngeof its time

span.22

6. Calculatethe metric usedfor the goal and any constraintsas of the end of the fiscal period if it is a metric suchas economicvalue or surplus. If it is a flow-basedmetric suchas portfolio duration or discountedGAAP income,derive

21 At this stage,the derivation of taxeswould occur. As notedby Rowland and Conde [19%], the detemrinationof federal incometaxes is convolutedby the combinedeffect of discountrates, changes in loss reserves,varying wlderwriting results,and tax canyforwardsand canybacks.

Page20 of 45 the metric for the holding period results. This calculation is done for each business/company scenario pair. There are

n resnlts; collectively they constitute a simulated sample.23 7. Return the required metrics for the sample to the optimizer. If the optimizer is deployed for EF calculation, the goal will be a samplestatistic for risk suchas variance,semi-variance, or chance-constrainedpercentile or range. The

sampleaverage for the distribution developedin step(6) for the metric will be usedwithin the constraintset.

8. The optimizer will repeatsteps (4)-(7) Wltil it hasobtained a feasibleset.

The optimizer usesa sample. The optimizer resultshave sampling error. Steps(1) and (2) are experiments. Let dterebe 10 repetitionsof this experiment. Application of steps(1 )-(8) will result in 10 efficient frontiers, eachderived from a different experimentalsample. It is likely that they will havedifferent characteristics.

In a DFA experimentthere are many draws from the urn; eachsimulation is anotherdraw. The modelergets distributional infonnation aboutthe contentsof the urn by the experimentalgrouping of all the simulations. When enoughsimulations within each experiment are nm, convergence of the distribution of results can be achieved. Since it is unlikely for the output distribution to be known, or necessarily capable of being parnmeterized, no a priori estimate is available. Instead, an empirical measureof convergencemust be used.

The allocation of company assets among competing investment alternatives using a single efficient frontier calculation (based on a single experimental result) may seem to be similar to betting on the allocation among balls of different colors within the urn based on a single sample from the urn containing them. One may, or may not, be lucky. But, you improve your luck by increasingthe numberof simulations.

One still may become victimized by a faulty decision while ignoring sampling error. This may arise in calibrating a model to history. The historical record is a single draw from a true underlying probability distribution. We may be lucky that the number of periods in the historical realization contains sufficient infonnation about the underlying process for unfettered decision-making. But, we could be victims of sampling error, which we are unable to control or even limit

22 Somemodels may achievecomputational efficiencies when economicscenarios are paired with E(C) insteadof with direct pairing to C\, C2, , Cu. When this is done,however, the varianceof the metric being optimized will be reduced,and the minimum varianceportfolio is likely to be different.

Page21 of 45 4.0 Historical Performance Comparison

Figure 4 illustratesthe performanceof severalportfolios over increasinglylonger forecastperiods. It showsresults for

24 The multipliers shown in the legendof Figure 4 portfolios, which, a priori, havedifferent levels of risk for the samereturn. are multiples of the minimum variancerisk. The line for Multiplier-l tIacesthe performanceof the on-frontier, EF, portfolio.

Other lines in the figure with multipliers> 1 showperfonnance of portfolios with the sameexpected return, but higher variance.

Figure 4 Comparison of Performancefor On-Frontierand Off-Frontier Portfolios

Performance I

762 ..A~ - Multiplier 1 562 ~ ~ Multiplier 1.25 ,362 A ~ -cO- Multiplier 1.5 162 - Multiplier 1.75

~ Multiplier 2 -.038

-.238 Performance Information Ratio Historical Period: January, 1988 -December, 1992 Forecast: January, 1993- December, 1999 Expected annualized return=.O825

23 If enoughpairs are used,the chancethat the model will convergeimproves. 24Risk in this study is measured as the standard deviation of return.

Page22 of 45 25 It is known as the infonnation ratio. The Figure 4 bares perfonnanceusing a variation of the Sharpeperfonnance measure.

Sharpeperformance ratio, which measuresexcess return to risk, is adjustedin the denominatorof the information ratio. The denominatorof the Sharpeperformance indicator is changedto excessrisk. The information ratio is given by (0.1):

E(rp-rf) SD(rp-rf) (0.1) where,

r p = monthly return on the portfolio,

rf = monthly return on the risk free componentof the portfolio26,

E = expectationoperator,

so = standard deviation operator.

Although the infonnation ratio was computedwith monthly data, it is expressedas an annualmeasure in the paper.

4.1 EF Performance Is Better for Low Risk-return Portfolios

The off-frontier portfolios, so-calledinefficient portfolios, achieveperfonnance that rivals or bettersthat of the EF portfolio.27

There is no conceptof "significance" that can be attachedto the observeddifferences. However, it is clear that the

performancedifferences are great and that.inefficient portfolios out-performthe efficient one in the Figure 4. When

perfonnanceis measuredby geometricreturn. the under-perfonnanceof the EF portfolio can be more than 100basis points as

shownin Figure 5. The underperfonnanceshown in Figure 5 is measuredover a seven-yearholding period, and therewas no

25Laster [1998] createdvarious portfolios by combining two assetcomponents, domestic (represented by S&P 500) and foreign (representedby Morgan StanleyEAFE). His bootstrapsamples of thesetwo componentswere usedto calculate portfolio variance,assmning various mixes. He did not separatehistorical and forecastperiods. Instead.he measuredquantiles from the bootstrapsamples after constructingportfolios. He concludedthat diversification into foreign equitiessubstantially changedand improved the risk/return profiles. 26 The 90-dayTreasure bill index is usedas the proxy for the risk free return. 27 Shortholding periodshave performancemeasures calculated with few observations.The ordinal rnnkingsamong the different multipliers are volatile and shouldbe ignored. The first six monthly periodsare generallyignored in this paper.

Page23 of 45 portfolio rebalancingduring this time. Data for other time periodsand the useof interveningportfolio rebalancingmight

materially affect this evidenceof underperformance.

Figure 5 Comparison of Geometric Return for On-Frontierand Off-Frontier

Portfolios

Performance

,079 ,074 -Multiplier 1

,069 -0- Multiplier 1.25 ,064 -A- Multiplier 1.5 ,059 ~ Multiplier 1.75 ~ Multiplier 2 .054 049 I L I ,--r- 1 21 41 61 81 Geometric return Historical Period: January, 1988 - December, 1992 Forecast: January, 1993 -December, 1999 Expected annualized return=.O825

The performancevaries considerablywith the level of return and historical period. For example,Figure 6 illustrates

perfonnancefor an earlier period and a lower expectedreturn level. Here, the EF portfolio, does,indeed, out-perform tre ofI-

frontier portfolios for about ten years. Thereafter,it reversesand perfonnancefalls below off-frontier portfolios. The Figure

illustratesthat the contemplatedholding period for useof an EF shouldprobably not be as long. The performancevariance

illustrated in Figure 6 is volatile; the differencesin perfOmlancein on- and off-frontier portfolios varies considerablywith the

choiceof historical startingpoint and length of the holding period.

Page24 of 45 Figure 6 EF Portfolio Performance at Low Risk-return Levels

Performance

1.606 -Multiplier 1 -a- Multiplier 1.25 1.106 -.- Multiplier 1.5

-0- Multiplier 1.75 .606 -.- Multiplier 2

106 1 51 101 151 Performance Information Ratio Historical Period: January, 1980- December, 1984 Forecast: January, 1985 -December, 1999 Expected annualized return=.0649

4.2 Overall Behavior of On-Frontier Portfolios for Information Ratio

The historical recordwas examinedfrom severalperspectives to seewhether an EF portfolio continuesto out perform ofJ- frontier portfolios. Equi-retum portfolios were examined. Theseare portfolios whosereturns are the same,but they have higher risk. The forecastperiod immediatelyfollowing the end of the historical segmentwas examinedto detenninehow long the on-frontier portfolio maintainedsuperior performance. This forecasthorizon extendedto the end of the data,December,

1999. Historical segmentsconsist of a 5-yearblock of 60 observations.

Severaladjustments were madefor this analysis. The first six-monthperiod was ignoredbecause the ratio is highly volatile and computedfrom few observations. The exb"emelow return levels also were removedfrom the analysisbecause higher ones shown in the table dominatedthem. 2S

- 28 The extremelow risk-return observationsoccur below where the EF curve hasa positive first derivative. A portfolio with a higher return for the samerisk can be found abovethis changein the curve.

Page25 of 45 Table 1 showsthe relative behavior of the information ratio at the return level indicatedat the top of the column. Each row block includesthe time for subsequentrow blocks. For example,the forecastbeginning January,1980 covers the period endingDecember, 1999. The interval of measurementis a month. All of the other blocks begin at a later point, but all forecast periodsend in December,1999.29

Missing cells in Table I indicate that a feasibleset was not found at that return level for one or more of the on or off-frontier portfolios. There were five portfolios with risk up to two times the risk of the on-frontier point.

Table 1 Information Ratio Behavior

Forecast Period Return Levels

Infonnation Ratio (forecast begins 1/1980) 0.00661 0.008 0.0085 0.009 0.0095 0.01

:leriods until on-frontier point under perfonT1s(max=238)

Number of periods on-frontier point outperforms all others 10 14~ 148 153 154 151

Average on-frontier rank (5 is highest) 3.05 4.3~ 4.34 4.33 4.30 4.271

Information Ratio (forecast begins 1/1985) 0.0066 0.008 0.0085 0.009 0.0095 0.011

'jeriods until on-frontier point under performs (max=178) 11 110 109 119 69

Number of periodson-frontier point outperformsall others 105 104 10~ 11~ 124

Average on-frontier rank (5 is highest) 3.4~ 3.40 3.38 1.92 3.72 3.90

Information Ratio (forecast begins 1/1990) 0.0066 O.OO~ 0.0085 0.009 0.0095 0.01

~eriodsuntil on-frontier point under performs (max=118) 40

Number periods on-frontier point outperforms all others 34 66 83 103 11

Average on-frontier rank (5 is highest) 4.18 4.05 4.57 4.72 4.89 4.96

Infomlation Ratio (forecast begins 1/1993) 0.0066 0.008 0.0085 0.009 0.0095 0.01

"'eriods until on-frontier point under performs (max=82) 19 10

29Each block of rows usesa different set of on and off-frontier portfolios-the respectiveEFs are derivedfrom optimizations on differentperiods. For example,the January,1980 forecast is basedon the perfonnanceof EFs derived from an historical segmentcovering the five-year period, January,1975 - December,1979). However,the January,1995 forecast uses EFs derivedfrom a different period, one covering the five-year ~riod, January,1989 - December,1994. The information in the blocks is not cumulative;the nwnber of periodsthe on-frontier excelsor outperformsoff-frontier portfolios is a separate measurementfor eachrow block. The row blocks show performancefor portfolios constructedat different points in time.

Page26 of 45 Forecast Period Return Levels

~umber of periodson-frontier point outperformsall others 15

IAverage on-frontier rank (5 is highest) 2.10 1.83 1.8~ 1.81! 1.60 1.56

Information Ratio (forecast begins 1/1995) 0.00661 0.008 0.0085 0.009

:)eriods until on-frontier point under perfonns (max=58) 5~ 56 57 Never

Number of periods on-frontier point outperforms all others 4~ 50 5" 53

Average on-frontier rank (5 is highest) 4.8~ 4.94; 4.96 5.00

"Periodsuntil on-frontier point under perfonns" meansthe first period that an off-frontier portfolio beatsthe on-frontier efficient portfolio. "Number of periods on-frontier outperformsall others" meansthe last period where the efficient portfolio wins. Performancetends to hold up better for lower return levels. This effect is reinforcedby the larger valuesshown for the numberof periodsthe on-frontier portfolio doesout rank the off-frontier portfolios. In genernl,the on-frontier portfolio ranks well comparedto the others. The averagerank is generallyhigh, above3 out of 5. But. the performanceis not consistent.The on-frontier portfolio did well during the long forecastperiod starting January,1980 and during the shorterforecast period startingJanuary, 1995. However, the low averageof the on-frontier for the January,1993 showsthat the perfonnanceis greatly influencedby the historical period and perhapsinfluenced by samplingerror.

Therealso is great inconSistencyin the nwnber of periodsbefore an off-frontier portfolio hasa higher infonnation ratio. The scanbegins in period 6 of the forecasthorizon. so the reversalshown in the table will either be never or a numberbetween 6 and n. In most cases,the reversalis early, but not pennanent Thereare many situationswhere the on-frontier portfolio wavers betweenhighest rank and somethingless. This latter fact is found in the rows, "Number periodson-frontier outperfonns. In most casesthis numberis larger than the numberof periodsbefore reversion.indicating that the on-frontier waflles in and out of superiorperformance. This could be anotherindication of samplingerror. The choiceof an on-frontier point may no~ and probablydoes not, imply superiorperfonnance.

4.3 Behavior for Other Performance Measures

The information ratio is believedto be a valid measureof perfonnancebecause it adjustsfor variation in the return series during the period of measurement Were it applied to two consultants'portfolio allocation recommendations,the consultant

Page27 of 45 with lower excessreturns could be rankedhigher than the other consultantbecause of proportionatelylower risk in excess return. This may be small consolationto the bolder of the lower wealth portfolio recommendedby the higher ranked consultant. This is why it is important to assessother characteristicsbeyond the appetitefor risk before making an allocation decision. The managerwith the higher infonnation ratio has the better cost of risk per unit of return; yet. it is not of muchuse if a minimum return level or endingwealth is required.

Thereis considerablehistoric instability in the standarddeviation of returns. This can be seenin Figure 7, which showsthe historic progressionof changesin the standarddeviation of monthly returnsof the portfolio componentsused in this study.

The lines showthe changein standarddeviation for rolling five-year blocks of data.30 Any performancemeasure tl1at is a function of this risk proxy, suchas the infonnation index, will be inherently sensitiveto suchvolatility and, perhaps,exhibit similar historic instability. This volatiljty m risk helps to explmn why WstoricalEFs may lack forecastpower.

30 Therewas significant volatility in the securitiesmarkets in 10/87("Black Monday") and 8/98 (Long Term Capital crisis). These periods are highlighted in the figure.

Page 28 of 45 -- T-t: I ~I ~ Q) ~ ~ N Q) [J \»" c -.., ) I w ~ ~ t ~ ~ "a5 it ~ I" .. $ .0T ~. . . A( -41+; ~ ,1 .., ~ ~ 8 , ~ .!- ~ . cII) !l 0- 0 N ~ .m > i~~ Q) - ~ "'I C . ~ - "C r J ... "0 0- ~ \~ N "C C ~- ~ - ~ - ( \. p., (/) >- i it' :E C ..- 0 - ~ c ... 0 ~ ~ Q) N ~ >- . .***\ \ ~ ca Q) > > .0- Q) ii: C 0' Il) 0) D.. "C ) '- .5 ~ (/) ca "0 "C ~ C , ~- I ca ... 0 cn 0 « c m '- - -.J ~ + + G ... Q) . ft: C c po :I: ~ ~ ! ...~ ~ ~ ca N- . 0 > to.. 1. - Q) § '- ~ ~ ~ 0 ~ ~ ~ ~ ~ ~ . C) ,... <0 uo) . coj N ..- 0" u.. One measureofperfonnance that is not risk-adjustedis geometricreturn during a holding period. Resultsare anayed in Table

2. The layout of this table is similar to Table 1

Table 2 Geometric Return Behavior

Forecast Period Return Levels

Geometric Return (forecast begins 1/1980 0.0066 -0.0081 0.0085 0.009 0.0095 0.01

~eliods until on-frontier point under performs

(max=239)

Number of periodson-frontier point outperforms 107 106 102 102 105

~verage on-frontier rank (5 is highest) 2.4 3.35 3.39 3.3~ 3.38 3.40

Geometric Return (forecast begins 1/1985 0.0066 0.0081 0.0085 O.OO~ 0.0095 0.01

~eriods until on-frontier point under performs l(max=179) Number of periodson-frontier point outperforms fAverage on-frontier rank (5 is highest) 1.00 1.00 1.00 1.00 1.03 1.04

Geometric Return (forecast begins 1/1990) 0.0066 o.ooal 0.0085 o.oo~ 0.0095 0.01

~eriodsuntil on-frontier point under performs

(max=119) 21 74 89 11 119

Number of periods on-frontier point outperforms 15 68 85 105 113

Average on-frontierrank (5 is highest) 4.1 4.06 4.60 4.75 4.92 4.9~

Geometric Return (forecast begins 1/1993) O.OO6~ O.OOal 0.008~ O.OO~O:OO95 0.011

:leriods until on-frontier point under pelforrns (max=83) 15 15 16 16 16 16

Number of periodson-frontier point outperforms 1 16 18 1~ 1- 10

Average on-frontier rank (5 is highest) 1.99 2.1~ 2.22 2.22 1.92 1.73

Geometric Return (forecast begins 1/1995) 0.0066 O.OO~ 0.0085 0.009

':'eriods until on-frontier point under performs (max=59)

Page30 of 45 Forecast Period Return Levels

~umber of periodson-frontier point outperforms 30 fAverage on-frontier rank (5 is highest) 2.44 2.80 2.96 3.3~

The forecastpropensity of the on-frontier allocation is markedlychanged. Wealth growth appearsto be unrelatedto the on or off-frontier portfolio choice, and often is worsefor the on-frontier allocation. The numberof holding periodsthe efficient frontier portfolio dominatesoff-frontier portfolios is generallya lower proportion of the possiblenwnber of holding periodsin

Table 2 than in Table Michaud [1998, pp. 27-29] claims there is a portfolio within the EF, the "critical point," below which single period mean-varianceefficient portfolios are also n-period geometricmean efficient and abovewhich single periodMY efficient portfolios are not n-period geometricmean efficient

Performance Failure within CAPM

Work with betahas led to various criticisms [Malkiel, pp. 271].3] For example,some low risk stocksearn higher returnsthan theory would predict Other attackson beta tend to mirror what we seewith EF:

Capital assetpricing model predictsrisk-free ratesthat do not measureup in practice.. » 2. Beta is unstableand its value changesover time.

), Estimatedbetas are unreliable:4

4. Betas differ according to the market proxy they are measured against.35

5. Averagemonthly return for low and high betasdiffers from predictionsover a wide historical span.36

31 Beta is a measureof systematicrisk either for an individual securityor for a portfolio. High beta portfolios, measuredex ante, in theory should have higher returnsex post than low beta portfolios. 32When ten groupsof securities,ranging from high to low betas,were examinedfor the time period 1931-65,the theoretical risk free rate predictedby CAPM and actual risk free ratessignificantly diverged. Low-risk stocksearned more and high risk stocksearned less than theory predicted. [Malkiel, pp. 256-7] 33 During shortperiods of time, risk and return may be negativelyrelated. During 1957-65,securities with higher risk ~roducedlower returnsthan low beta securities.[MaIkiel, pp. 258-60] The relationshipbetween beta and return is essentiallyflat. Beta is not a good measureof the relationshipbetween risk and return. [MaIkiel, pp. 267-8] 35 Predictionsbased on CAPM about expectedreturns both for individual stocksand for portfolios differ dependingon the chosenmarket proxy. In effect, the CAPM approachis not operationalbecause the true market proxy is unknown. [MaIkiel, pp. 266-7]

Page31 of 45 Malkiel [po270] concludesfrom his surveythat, "One's conclusionsabout the capital-assetpricing model and the usefulnessof

betaas a measureof risk dependvery much on how you measurebeta." This appearsto be true of EFs too. The definition of

efficiency is what is important here-perbaps more important becausecorrect measurementrequires precise definition.

The choiceof an optimization mechanismcouched in tenDSof risk-return trade-off may not lead to wealth maximization.

Under thesepretenses one might wish to deploy a different optimization mechanismsuch as the one mentionedby Mulvey, et

oJ [1999,p. 153] in which the optimiZ3uonseeks to maximize utility. The choiceora particular utility function may be framed in tenns of absoluterisk aversion-negative exponentialutility works in this regard37 And. if the behavior of securityprices

doesnot have an observablestationary probability measure[Kane, 1999],utility approachesseem to be mandatory

The subjectof what is optimal is controversiaLand not apt to go away. The useof optimization within hybrid modelsand

generationof metricsby DFA modelshas many subtlemanifestations. One is the choice of planning horizon. Michaud [1998,

p. 29] arguesthat investorswith long-tenn investmentobjectives can avoid possiblenegative long-tenD consequences of mean-

varianceefficiency by limiting considerationto EF portfolios at or below somecritical point. There is a parallel in our paper,

in what we refer to as samplingerror and its affect on the shapeof the efficient surface. This surfaceappears to have properties

at the lower risk-return areasof both lower dispersion,greater similarity in portfolio composition,and better on-frontier

performanceamong different samples(either bootstrapor historic segment).

5.0 Characteristics of the EF Surface

The bootstrap-generatedEF surfacerises within the risk-return space. Views of this surfacefrom two different anglesarc

shown in Figures 8.

36 Theratio of priceto bookvalue and market capitalization did a betterjob of predictingthe structureof nonfinancial corporateshare returns than beta during a 40-yearperiod. EugeneF. Famaand Kcnncth R. French,"The Cross-Sectionof ExpectedStock Returns," Journal of Finance, June, 1992. 37The recommendationof a utility-decision approachhas great breadth in the insuranceliterature-beyond the useof utility as goal function in optimization, other venuesfind it appropriatewhere stochasticdominance is sought. For example,exponential utility usewas suggestedin rate making by Freifelder. SeeFreifelder LeonardR, A Decision TheoreticApproach to Insurance

Page32 of 45 Figures 8 Views of EF Surface Createdfrom Bootstrap Samples

The surfaceis constructedfrom monthly returns. Looking down on the surfaceof the views, one obtainsa projection on risk- return space. The surfaceis seento curve as the efficient frontier curves. In the low risk-return sector,the surfaceis more

Ratemaking,hwin, 1976,pp. 141. The choice of parametersfor utility functions is perhapsas much an art as the

Page33 of 45 peaked. The surfaceflattens and broadensin the risk-return space. Imagineyourself walking along the ridge startingin the southwestand proceedingnorthward and then northeast. You would first be descendinga steepincline and then a vista of a vast plane would unfold along your right. This canbe interpretedwithin the context of changesin the marginal distributions representingslices through the surfaceeither along the risk or along the return dimensions. We refer to the latter as an equi- return slice, and its propertiesare examinedin more detail at a latter point in the paper. In either case,the visualizationis one of moving from lessdispersed marginal distributionsto oneswith greatervariance as either dimensionis increased

There is an artifact of the intervalization that resultsin a suddenrise in the surfaceat the highestrisk level. This occurs becausehigher risk observationswere lumped into this final interval. Were higher levels of risk intervalized over a broader rnnge,this ridge would flatten.

The surfaceshown in either of the views in Figures8 is built from many efficient frontiers, eachproduced from optimizations doneon a bootstrapsample. We alreadyhave seenin Figure I a subsetofEFs that tangletogether-they can be organizedto producea surface. The surfacedevelops the sameway an empirical probability distribution is built from a sample. Repeated samplingproduces points that are intervalizedand counted

A frequencycount can be madeof observationsfor EFs falling within an arbitrnrily small, two-dimensionalregion of risk- return space. An exampleof this mappingfor 5,000 bootstrap-simulatedEFs appearsin Figure 8. Collectively, this mapping involves the 2-dimensional,intervalization of approximately45,000 quadraticoptimizations constituting the EFs for the underlying bootstrapped samples.38

5.1 Equi-Return Slice of the Efficient Surface

A slice through the efficient surfacealong the return plane producesa histogramof the minimum risk points for a given return in the EFs usedfor the EF Surface. As return increases,this marginal probability distribution becomesmore disperse. An exampleappears in Figure 9.

~eterization of claims generations in DF A models. Equi-return, minimwn variance points for the 5,000 bootstrapped EFs were intervalized based on an overall evaluation of the range of risk among all points on all EFs. If an efficient set could not be identified for a return level, the observation was ignored. The marginal probabilities (risk-return) were normalized to the nwnber of viable observations for that risk level. The nwnber of viable optimizations exceeded 4,500 at each return level.

Page34 of 45 Figure 9 Dispersion of Risk Given a Return Level

Distribution of Risk

0.25

>- 0.2 ~ :s. 0.15 .g 0.1 Q: 0.05

0 M M M "I" "1"10 10 10 <0 <0 ~lOo)M""'~lOo)M"'" ~~~NNMMM"I""I" 0000000000 0000000000 Risk at Expected Return-.OO9

The dispersion increases with return for both surfaces constructed from bootstrap samples and from randomly selected blocks

of history. The distributions are positively skewed. increasingly so as return increases. The inset bars in Figure 9 identify the

intervals containing tile mean and median points of tile distnbutioo. Additional statistics botil for bootstrapped and historical

segment evaluations of sampling error appear in Tables 3 and 4.

Page35 of 45

~ Table 3 Statistics for Equi-Return Slices of the Efficient Surface

Statistic Efficient Surface from Bootstrapped Efficient Frontiers

Return Level 0053 0066 0080 .0085 0090 0095 0100

Mean .0125 0533 3.76 8.86 17.9 33.6 50.7

(times 1.0E4)

Standard .627 3.77 38.2 58.6 81.8 109.7 131.6

Deviation

(times 1.0E4)

Skewness 000123 378 57.8 136 27S 516 779

(times 1.OES)

The statisticsare visually apparentin the EF surfaceshown in Figures8. The surfaceis partially bowl-like-sloping downwardin a concavefashion. Its rim encompassesa planewithin the risk-return domain that is broad in the risk dimension.

As one movesfrom low to high return, the marginal distribution ofEF points measuringoptimized risk (an equi-returnslice through the surfaceas illustrated in Figure 9) becomesmore dispersed.In a visual contextas one movesfrom low to high risk along the EF surfaceand takesequi-return slices through it, one would find higher variancein the distribution of optimizedEF risk points-variance shown in histogramplots suchas Figure 9 is greater.

An efficient surfacealso can be createdfrom EFs calculatedfor historical time periods. An exampleappears in Figure 10. The dataare for 5-year,overlapping blocks calculatedon a monthly basisstarting in 1970. The samegeneral features are found in this representationof sampleerror. However,the surfaceis lessflat than the one developedfrom bootstrapsamples. The reduceddispersion in the surfaceof Figure 10 arisesin part from the useof overlappingfive-year blocks usedto constructthe underlyingEFs from which the surfaceis built. A statisticaltable similar to Table 3 was constructedfor this surface. It appears in Table 4.

Page36 of 45 Figure 10 Efficient Sutface from Historical Samples

Bflclent Surface (based on historical segments)

Table 4 Statistics for Equi-Return Slices of the Surface Shown in Figure 10

Statistic Efficient Surface from Bootstrapped Efficient Frontiers

ReturnLevel 0053 0066 ,0080 0085 0090 0095 0100

Mean 663 5.53 21.0 25.5 28.8 33.2 46.5

(times 1.OE4)

Standard 080 451 861 940 995 1.06 1.23

Deviation

(times 1.OE2)

Skewness .00676 769 2.92 3.54 4.00 4.62 6.46

(times 1.0E6)

Page37 of 45 6.0 Stability ofPol1folio Composition Along an Efficient Frontier

Portfolio allocation amongcomponent securities changes, usually dramatically,along the efficient frontier. A componentmay

enterthe feasibleset at somepoint, increasein weight, decreaseand then drop out at anotherpoint along the EF. This effect

was shown in Figure 2.

The change in composition for an equi-retum level was examined among different EFs, constructed both from historical

segment EFs and bootstrap EFs. We refer to this type of comparison as an avalanche chart because when shown in an

animation, the change in composition is similar to an avalanche. An example appears in Figure II

Figure 11 Avalanche Chart for Historical Segments

The vertical bars are stackedcolumns. Each segmentwithin a column representsa different componentof the portfolio. A bar, therefore,compares the percentagevalue eachcomponent in the feasibleset contributesacross all componentsin the set. All bars are shownfor a constant,equi-return level of an EF; but eachbar is for a different historical segment In Figure 11, each bar representsthe portfolio compositionfor the equi-returnlevel point on the EF, which was caicuiatcdfor a five-year block of monthly observations. The bars are for ten randomly chosenhistorical segments.39Were the blocks within the bars to consist

39 There is a small chance that two or more bars in an avalanche chart could be identical. However, there is a much larger probability that two or more bars have overlapping time periods in the calculation of their respective EFs.

Page38 of 45 of the samecomponents and were they to be about the samesize, the portfolio allocationswould be the sameregardless of the time frame. Examinationof Figure 11 showsthat the compositionof the bars and individual componentallocations varies considerably.

The portfolio compositionis much more stableat lower risk-return levels. This result is in accordancewith other similar findings basedon the EF surface.It. too, showsless disperse results for lower return levels. This approachto measuring samplingerror implies that performanceof efficient frontiers may not be optimal relative to off-frontier portfolios. If the mix and compositionof portfolios fluctuatesconsiderably both with respectto historical aIKibootstrap sampling methods, the performanceexpectations of an ex ante allocation are not apt to hold ex post.

7.0 Conclusion

The behaviorsshown in both Tables 1 and 2 illustrate a markedtendency towards randomness. The efficient surfacebuilt from bootstrapsamples is highly variable within the risk-return domain. There appearsto be sometemporal dominanceof on- frontier portfolios for lower risk-return levels,but the historical recordis mixed. The bootstrappingof the single sampleof assetreturns provided by the historic dataillustrates that samplingerror could materially affect the position and shapeof the efficient frontier.

7.1 Should Efficient Frontiers Be Used in DFA Models?

Thereis no strongsupport in this paperfor the practical deploymentof efficient frontiers in DF A. The risk in DF A models stemsfrom model, processand parameterrisk. It impactsthrough all aspectsofDF A modelsof the insuranceenterprise. The existenceof model and processrisk [Kirschner and Scheel,1997] thwarts the usual convergenceto the true w1derlying distributionsgained by running large numbersof simulations. Whenall of thesenew risk elementsare heapedon top of the samplingerror derived from assetmodel calibration or empirically measuredcovariance matrices, one wonderswhether EFs are really useful in DFA analysis.

The work of Michaud [1998] bears on the issue of improving the perfonnance ofEF portfolios. He defines a measure of statisticalequivalence for mean-varianceefficiency. Any portfolio within the efficient surfacesufficiently closeto the optimal portfolio is consideredequivalent to it. The extensionof his idea to the efficient frontier surfaceis to identify a region on it

Page39 of 45 whoseex ante chance-constrainedprobability both can be measuredand hasdesirable statistical properties in a forecasting

sense. This is analogousto acknowledgingthe existenceof samplingerror and specifying an unknown populationparameter

only to within an interval of statisticalconfidence. Unfortunately,the definition of sufficiently close is constructivebut

difficult to implementin a rigorous manner,particularly within the context of the hybrid DFA model.

Future study will haveto answerthe questionof whether on-frontier assetallocations that are measuredfrom hybrid DF A

modelssuffer a similar unreliability. But, the problemswith on-frontier assetportfolios raisedin this paperare apt to be

exacerbatedby inclusion of known samplingerror in tile liability side of DFA models.

7.2 How Can EFs Be Efficiently Deployed?

Usersof this constructshould be awarethat the tenD"efficient" in efficient frontiers hasa good chanceof being operationally

false. The efficiency of portfolio compositionis unlikely to be manifest in better perfonnanceof the on-frontier portfolio

comparedto other, off-frontier portfolios. The risk/return surfaceis not adequatelymeasured by a single EF, and sampling

error may lead to unwarrantedconclusions about the efficacy of portfolios measuredin suchsingular optimizations.

The userofEFs should probably view them as containingprovisional, useful information about risk/return relationships. But,

any single EF haslimited value in understandingthe risk/return surface. The conceptualbasis of an efficient surfaceis an

organizedresampling of the data so that the decisionprocess benefits from better wlderstandingof uncertaintythat might arise just becausethe EF is operationallyderived from a sample. The misunderstandingof this uncertaintymay lead to erroneous

decisions,and the practitioner must be alert to potential inefficienciesof a single EF measurement.The authorsrecommend

the elicitation of an efficient surfacebecause the surfaceis apt to show a lack of statisticalconfidence in any single frontier on

that surface. Under thesecircumstances, the practitionermust think in tenDSof confidenceranges. The samplingerror shown

in the efficient surfaceemphasizes how careful one must be when drawing inferencesderived from optimization. An

optimizedfrontier is basedon an empirical covariancematrix; one that has samplingerror. That error may be very important.

It is easyto believe that stmtegicor tactical decisionsmotivated by so-call optimized DFA measurementwill effectively move the organizationto a betterposition in risk/return space. Unfortunately,there appearsto be a broad region of "inefficiencY" that may serveas well. An EF may be better than a crystal ball; but there is a good chancethat it should not be takentoo seriously.

Page40 of 45 Appendix A: Review of Data Sources

This paperuses monthly time seriesof assetclass total returns. A selectionof broad assetclasses typical of P&C insurance companyasset portfolios was chosenfor examination. The time seriesall begin January1, 1970. However, certainasset classes (e.g. mortgage backed securities) do Dot have a history that extends back this far. For these classes the time series were backfilled to the January I, 1970 start date by an investment. consultant. The backfill process was based on a consideration of the market conditions of the time (e.g. interest rates, fixed income spreads, inflation expectations) and how the particular sector would have performed given those market conditions. The Start Date in Table 5 refers to the date historical data begins.

Table 5 Asset Components

Class Code Source Start Date

InternationalEquities EAFEU MSCI EAFE Index 1/1970

InternationalFixed Income INTLHDG JP Morgan Non-US TrndedIndex 1/1970

Large Cap DomesticEquities S&P5 s&P 500 Index 1/1970

Cash usm 90 Day US TreasuryBill 1/1970

Mid CapDomestic Equities RMID S&P Mid Cap 400 Index 1/1982

High Yield mYLD CSFBHigh Yield Bond Index 1/1986

ConvertibleSecurities CONY CSFB ConvertibleIndex 1/1982

CorporateBonds LBCORP LehmanBrothers Corporate Bond Index 1/1973

GovernmentBonds LBOOVf Lehman Brothers Government Bond Index 1/1973

MortgageBacked Securities LBMBS Lehman Brothers Mortgage Backed Securities Index 1/1986

Page41 of 4S The time seriesused in this study are monthly returns. With the exceptionof work relating to performance,all returnsare expressed as monthly returns.

For perfonnancemeasurement purposes, returns have been annualized using the following formulas.

Annualized Expected Return

Rp = (I+T,)12-1 (0.2) where,

R = annualized return, p r p = monthly return.

Annualized Varianceof Return

Vp =[vp +(1+ jlpt]12 -(I+MpY (0.3)

where,

annualizedvariance of return,

monthly variance of return,

expectedmonthly return.

M p = expectedannualized return.

Annualized GeometricReturn

The growth rate, g, for a holding period of n yearsis given by:

Page42 of 45 1+ g = (~tl" (0.4) Va

where,

v" = portfolio value at the end of the holding period 0,

v0 = portfolio value at the beginning of holding period.

Page43 of 45 References

Chopra,Vijay and Ziemba, William, "The Effect of Errors in Means,Variances, and Covarianceson Optimal Portfolio

Choice", WorldwideAsset and Liability Management,1998, Cambridge University Press,pp. 53-61

Con-enti,Salvatore, Sontin, StephenM. and Isaac,Daniel B., "Applying a DFA Model To Improve StrategicBusiness

Decisions," CasualtyActuarial SocietyForum, Summer1998, pp. 15-52.

Davison, AC. and Hinkley, D. V., BootstrapMethods and Their Application, 1997,Cambridge University Press,pp. 582.

Efron. GeraldF. and Tibshirani, Robert J.,An Introduction to the Bootstrap, 1993,Chapman & Hill, pp.436.

Elton, Edwin J. and Gruber,Martin J., Modem Portfolio Theoryand InvestmentAnalysis, 1995,John Wiley & Sons,Inc., pp.

715.

Kane, S.A., "What happened to the utility functions? Imprecise expectations of security prices," International Review of

Financial A na/ysis,8:2 (1999), pp. 165-175.

Kirschner, Gerald S., "A Cost/BenefitAnalysis of Alternative InvestmentStrategies Using Dynamic Financial Analysis

Tools", CasualtyACbJarial Society Forum, Summer2000, pp. 21-54.

Kirschner, Gerald S. and Scheel, William C., "The Mechanics of a Stochastic Financial Model," 1998 Proceedings of the

CasualtyActuarial Society,~ Nos. 162 and 163,pp. 404-408.

Laster,David S., "Measuring Gainsfrom InternationalEquity Diversification: The BootstrapApproach, " TheJournal of

Investing,Fall, 1998,pp. 52-60.

Page44 of 45 Levy, Haim and Gunthorpe,Deborah, "Optimal InvestmentProportions in SeniorSecurities and Equities Under Alternative

Holding Periods," TheJournal of Portfolio Management,Summer 1993, p. 31.

Malkiel, Burton G., A Random Walk Down Wall Street, 1996,W. W. Norton & Company,Inc., pp. 227-276.

Michaud, Richard 0., Efficient AssetManagement, 1998, Harvard BusinessSchool Press.

Michaud, Richard 0., "The Markowitz Optimization Enigma: is 'Optimized' Optimal", Financial AnalystsJournal, 45, Jan.-

Feb.,pp 32-42.

Mulvey, JohnM, Madsen,Chris K. and Morin, Fran~is, "Linking Strategicand Tactical Planning Systemsfor Dynamic

Financial Analysis," CasualtyActuarial SocietyForum, Summer1999, pp. 149-167.

Rowland, Vincent T. and Conde,Frank S., "Dynamic Financial Analysis Issuesin InvestmentPortfolio Management,"

CasualtyActuarial SocietyForum, Spring 1996,pp. 205-40.

ShaIpe,William F., "The SharpeRatio", TheJournal of Portfolio Management,Fall 1994pp. 49-58

Page45 of 45