Maximum Drawdown Is Generally Greater Than in IID Normal Case
Total Page:16
File Type:pdf, Size:1020Kb
THE HIGH COST OF SIMPLIFIED MATH: Overcoming the “IID Normal” Assumption in Performance Evaluation Marcos López de Prado Hess Energy Trading Company Lawrence Berkeley National Laboratory Electronic copy available at: http://ssrn.com/abstract=2254668 Key Points • Investment management firms routinely hire and fire employees based on the performance of their portfolios. • Such performance is evaluated through popular metrics that assume IID Normal returns, like Sharpe ratio, Sortino ratio, Treynor ratio, Information ratio, etc. • Investment returns are far from IID Normal. • If we accept first-order serial correlation: – Maximum Drawdown is generally greater than in IID Normal case. – Time Under Water is generally longer than in IID Normal case. – However, Penance is typically shorter than 3x (IID Normal case). • Conclusion: Firms evaluating performance through Sharpe ratio are firing larger numbers of skillful managers than originally targeted, at a substantial cost to investors. 2 Electronic copy available at: http://ssrn.com/abstract=2254668 SECTION I The Need for Performance Evaluation Electronic copy available at: http://ssrn.com/abstract=2254668 Why Performance Evaluation? • Hedge funds operate as banks lending money to Portfolio Managers (PMs): – This “bank” charges ~80% − 90% on the PM’s return (not the capital allocated). – Thus, it requires each PM to outperform the risk free rate with a sufficient confidence level: Sharpe ratio. – This “bank” pulls out the line of credit to underperforming PMs. Allocation 1 … Investors’ funds Allocation N 4 How much is Performance Evaluation worth? • A successful hedge fund serves its investors by: – building and retaining a diversified portfolio of truly skillful PMs, taking co-dependencies into account, allocating capital efficiently. – weeding out unskilled PMs to protect the invested principal. • Investors pay high fees for those services, typically: – 2% management fee. – 20% performance fee. • An accurate performance evaluation methodology is worth a lot of money!! 5 How are PMs Stopped-Out? • Drawdowns can be the result of – Poor investment skills: The PM should be weeded out. – Bad luck: The PM should be kept on platform. • Stopping-Out a PM is a decision under uncertainty. An accurate performance evaluation methodology is able to discriminate between both: • maximizing the probability of true negatives (retaining good PMs). • subject to a user-defined probability of false positives (“bad luck” stop-outs). 6 SECTION II Stop-Outs under the IID Normal Assumption The IID Normal Framework (1/2) • Suppose an investment strategy which yields a sequence of cash inflows ∆휋휏 as a result of a sequence of bets 휏 ∈ 1, … , ∞ , where ∆휋휏 = 휇 + 휎휀휏 such that the random shocks are IID distributed 휀휏~푁 0,1 . • Let us define a function 휋푡 that accumulates the outcomes ∆휋휏 over t bets. 푡 휋푡 = ∆휋휏 휏=1 where 푡 ∈ 0,1, … , ∞ and 휋0 = 0. 8 The IID Normal Framework (2/2) • Because 휋푡 is the aggregation of t IID random variables 2 2 ∆휋휏~푁 휇, 휎 , we know that 휋푡~푁 휇푡, 휎 푡 . 1 • For a significance level 훼 < , we define the quantile 2 function for 휋푡 푄훼,푡: = 휇푡 + 푍훼휎 푡 where 푍훼 is the critical value of the Standard Normal distribution associated with a probability 훼 of performing worse than 푄훼,푡, i.e. 훼: = 푃푟표푏 휋푡 ≤ 푄훼,푡 . Then, drawdown is defined as 퐷퐷훼,푡: = max 0, −푄훼,푡 9 Maximum Drawdown 2 • PROPOSITION 1: Assuming IID outcomes ∆휋휏~푁 휇, 휎 , and 휇 > 0, the maximum drawdown associated with a 1 significance level 훼 < is 2 2 푍훼휎 푀푎푥퐷퐷 = 훼 4휇 which occurs at the time (or bet) 2 푍훼휎 푡∗ = 훼 2휇 10 Maximum Time under Water 2 • PROPOSITION 2: Assuming IID outcomes ∆휋휏~푁 휇, 휎 , and 휇 > 0, the maximum time under water associated 1 with a significance level 훼 < is 2 2 푍훼휎 푀푎푥푇푢푊 = 훼 휇 • PROPOSITION 3: Given a realized performance 휋 푡 < 0 and assuming 휇 > 0, the implied maximum time under water is 2 휋 푡 휋 푡 푀푎푥푇푢푊 = − 2 + 푡 휋 푡 휇2푡 휇 • It does not only matter how much money a PM has lost, but critically, for how long. 11 The Triple Penance Rule (1/2) • THEOREM 1: Under IID Normal outcomes, a strategy’s maximum drawdown 푀푎푥퐷퐷훼 for a significance level 훼 ∗ occurs after 푡훼 observations. Then, the strategy is ∗ expected to remain under water for an additional 3푡훼 after the maximum drawdown has occurred, with a confidence 1 − 훼 . 푀푎푥푇푢푊훼 • If we define 푃푒푛푎푛푐푒: = ∗ − 1, then the “triple 푡훼 penance rule” tells us that, assuming independent ∆휋휏 identically distributed as Normal (which is the standard portfolio theory assumption), 푷풆풏풂풏풄풆 = ퟑ, regardless of the Sharpe ratio of the strategy. 12 The Triple Penance Rule (2/2) 5000000 It takes three time longer to recover from the maximum 4000000 drawdown (푀푎푥푇푢푊훼) than the time it took to produce it ∗ 1 (푡훼), for a given significance level 훼 < , regardless of the 3000000 2 PM’s Sharpe ratio. 2000000 푀푎푥푇푢푊훼 1000000 0 -0.1 0.1 0.3 0.5 0.7 0.9 1.1 1.3 1.5 Quantile (in US$) (in Quantile -1000000 푀푎푥퐷퐷훼 -2000000 -3000000 -4000000 ∗ ∗ 푡 3풕휶 -5000000 훼 Time Under the Water 13 Example 1 20000000 PM1 has an annual mean and standard deviation of 푀푎푥푇푢푊0.05[PM1] US$10m (SR=1), and PM2 15000000 has an annual mean of 푀푎푥푇푢푊0.05[PM2] US$15m and an annual standard deviation of 10000000 US$10m (SR=1.5). 5000000 For a 95% confidence level, PM1 reaches a maximum Quantile (in US$) (in Quantile drawdown at US$6,763,859 0 after 0.676 years, and 0 0.5 1 1.5 2 2.5 3 ] ] remains up to 2.706 years 푃푀2 푃푀1 [ [ under water. -5000000 05 05 . 0 0 PM2 reaches a maximum 푀푎푥퐷퐷 -10000000 푀푎푥퐷퐷 drawdown at US$4,509,239 Time Under the Water after 0.3 years, and remains PM1 PM2 1.202 years under water. 14 Example 2 2000000 PM1 has an annual mean and standard deviation of 1000000 US$10m (SR=1), and PM2 푀푎푥푇푢푊0.08 푃푀1 = 푀푎푥푇푢푊0.02 푃푀2 0 has an annual mean of 0 0.5 1 1.5 2 2.5 US$15m and an annual -1000000 standard deviation of 푀푎푥퐷퐷0.08[푃푀1] US$10m (SR=1.5). -2000000 -3000000 For a ~92% confidence level, PM1 reaches a maximum -4000000 Quantile (in US$) (in Quantile drawdown at US$5,000,000 after 0.5 years, and remains -5000000 up to 2 years under water. -6000000 푀푎푥퐷퐷0.02[푃푀2] For a ~98% confidence level, -7000000 ∗ ∗ PM2 reaches a maximum 푡훼 3풕휶 -8000000 drawdown at US$7,500,000 Time Under the Water after 0.5 years, and remains PM1 PM2 up to 2 years under water. 15 Implications of the Triple Penance Rule 1. It makes possible the translation of drawdowns in terms of time under water [Cf. Proposition 3]. 2. It sets expectations regarding how long it may take to earn performance fee (for a certain confidence level). – The remaining time under water may be so long that withdrawals are expected. This has implications for the firm’s cash management. 3. It shows that the penance period is independent of the Sharpe ratio (in the IID Normal case). – E.g., if a PM makes a fresh new bottom after being one year under water, it may take him 3 years to recover, under the confidence level associated with that loss. This holds true whether that PM has a Sharpe of 1 or a Sharpe of 10. 16 SECTION III The IID Normal Assumption The IID Normal Assumption (1/2) • In general, traditional performance statistics assume that returns are IID Normal: 0.014 – Returns are Independent. 0.012 • However, a test of “runs” shows that negative returns occur in sequences. 0.01 0.008 – Returns are Identically Distributed. 0.006 • However, squared returns exhibit positive autocorrelation (휎-clustering). 0.004 – The distribution is Gaussian (or 0.002 0 -0.1 -0.08 -0.06 -0.04 -0.02 0 0.02 0.04 0.06 0.08 0.1 Normal). pdf1 pdf2 pdf Mixture pdf Normal • However, hedge fund returns exhibit asymmetry and fat tails. • Unfortunately, the “IID Normal” assumption is not supported by the data. Then, why is it used? 18 The IID Normal Assumption (2/2) • It is a “Hail Mary pass”, a convenient leap of faith that simplifies the math involved (… at a substantial cost to firms and investors!) “Experience with real-world data, however, soon convinces one that both stationarity and Gaussianity are fairy tales invented for the amusement of undergraduates.” David J. Thomson, Bell Labs (1994) • A popular myth is that Central Limit Theorems (CLTs) justify the IID Normal assumption on a sufficiently large sample. This is false: − CLTs require either independence or weak dependence. − Normality is not recovered over time in the presence of dependence. 19 SECTION IV Stop-Outs under first-order auto-correlated outcomes First-order auto-correlation • It is well established that hedge fund strategies exhibit significant first-order auto-correlation. E.g., see Brooks and Kat [2002]. • There are various reasons why strategies’ returns exhibit first-order serial-correlation: – Unmonitored risk concentration (quite different from VaR). – Inconsistent profit taking and stop loss rules. – Serially correlated and cointegrated investments. • First-order auto-correlation introduces a serial dependence that explains by itself why returns are: – Non-Identically distributed. – Non-Normal. 21 Non IID Normal Perform. Eval. Framework (1/3) • Suppose an investment strategy which yields a sequence of cash inflows ∆휋휏 as a result of a sequence of bets 휏 ∈ 1, … , ∞ , where ∆휋휏 = 1 − 휑 휇 + 휑∆휋휏−1 + 휎휀휏 such that the random shocks are IID distributed 휀휏~푁 0,1 .