1 1 UNDERSTANDING INTERACTIONS THROUGH CROSS- CORRELATIONS S. Sinha TOPICS TO BE COVERED IN THIS CHAPTER: The market as a complex interacting systen open to exter- • nal influece Correlation between N infinite- and finite length random • walks Wigner matrices versus Wishart matrices • Bio-box on Wigner and Wishart • Eigenvalues and eigenvectors of random matrices • Semi-circle and Sengupta-Mitra distributions for eigen- • values Spacings of eigenvalues: Clue to the random nature of the • walks “Unfolding" the eigenvalues • Contributions of HE Stanley, JP Bouchaud et al • Connecting interactions to correlations by factor models • Building networks of interactions: Minimal spanning tree, • filtering the correlation matrix etc. Contributions of RN Mantegna, J Kertesz, JP Onnela, A Chakraborti, • K Kaski et al Econophysics. Sinha, Chatterjee, Chakraborti and Chakrabarti Copyright c 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim ISBN: 3-527-XXXXX-X 2 1 UNDERSTANDING INTERACTIONS THROUGH CROSS-CORRELATIONS Utility in Finance: Capital Asset Pricing Model and Markowitz • Portfolio optimization “Because nothing is completely certain but subject to fluctuations, it is dangerous for people to allocate their capital to a single or a small number of securities. [. ] No one has reason to expect that all securities . will cease to pay off at the same time, and the entire capital be lost.” – from the 1776 prospectus of an early mutual fund in the Netherlands [1] As the above quote makes it evident, those interested in the study of fi- nancial market movements have long been concerned with the correlation be- tween price movements of different commodities or stocks. Recently, with the emergence of the understanding that markets are examples of complex sys- tems with many interacting components, cross-correlations have been useful for inferring the existence of collective modes in the underlying dynamics of financial markets. While it may not be possible to directly observe the inter- actions among traders buying and selling various stocks, we can measure the resulting effect on the market by looking at how their actions introduce corre- lations into the price behavior of different stocks. Thus, the dynamics of the network of agents is being inferred from the observation of the interaction be- tween stock price movements. Note that, the interactions between stocks are indirect in the sense that they are mediated through the actions of the buyers and sellers of those stocks. This is analogous to using the process of Browinan motion to infer the motion of air molecules by observing pollen grains with which the air molecules are colliding. It is natural to expect that stocks which interact strongly between them- selves will have correlated price movements. Such interactions may arise be- cause the corresponding companies belong to the same business sector (so that they compete for the same set of customers and share similar market en- vironments), or they may belong to related sectors (e.g., automobile and en- ergy sector stocks would show similar response to a sudden rise in gasoline prices), or they may be owned by the same business house and therefore per- ceived by investors to be linked. In addition, all stocks may respond similarly to news breaks that affect the entire market (e.g., the outbreak of a war) and this induces market-wide correlations. On the other hand, information that is related only to a particular company will only affect the price movement of a single stock, effectively decreasing its correlation with that of other stocks. The study of empirical correlations between financial assets is extremely important from a practical point of view as it is vital for risk management of a stock portfolio. High correlation between the components of a given port- folio or option book increases the probability of large losses. For example, if a portfolio is dominated by stocks from a single sector, say petroleum, they 1.1 The Return Cross-Correlation Matrix 3 may all be hard hit by a crisis in oil production and supply and as a result suffer a steep decline in value. Thus, consideration of correlations had al- ways been important to theories for constructing optimal portfolios, such as that of Markowitz [2], which try to answer the following question: for a set of financial assets, for each of which the average return and risk (measured by the variance of fluctuations) is known, how shall we determine the optimal weight of each asset constituting the portfoilo that gives the highest return at a fixed level of risk, or alternatively, that has the lowest risk for a given average rate of return ? The effects governing the cross-correlation behavior of stock price fluctua- tions can be classified into that of (i) market (i.e., common to all stocks), (ii) sector (i.e., related to a particular business sector) and (iii) idiosyncratic (i.e., limited to an individual stock). The empirically obtained correlation struc- ture can then be analyzed to find out the relative importance of such effects in actual markets. Physicists investigating financial market structure have fo- cussed on the spectral properties of the correlation matrix. Pioneering studies in this area have investigated the deviation of these properties from those of a random matrix, which would have been obtained had the price movements been uncorrelated. It has been found that the bulk of the empirical eigenvalue distribution matches fairly well with those expected from a random matrix, as does the distribution of eigenvalue spacings [?, ?]. Among the few large eigen- values that deviates from the random matrix predictions, the largest represent the influence of the entire market common to all stocks, while the remain- ing eigenvalues correspond to different business sectors [?], as indicated by the composition of the corresponding eigenvectors [?]. However, although models in which the market is assumed to be composed of several correlated groups of stocks is found to reproduce many spectral features of the empirical correlation matrix [?], one needs to filter out the effects of the market-wide signal as well as noise in order to identify the group structure in an actual market. Recently, such filtered matrices have been used to reveal significant clustering among a large number of stocks from the NYSE [?]. 1.1 The Return Cross-Correlation Matrix To measure correlation between the price movements across different stocks, we first need to measure the price fluctuations such that the result is inde- pendent of the scale of measurement. For this, we calculate the logarithmic return of price. If Pi(t) is the stock price of the ith stock at time t, then the (logarithmic) price return is defined as R (t, ∆t) ln P (t + ∆t) ln P (t). (1.1) i ≡ i − i 4 1 UNDERSTANDING INTERACTIONS THROUGH CROSS-CORRELATIONS For daily return, ∆t = 1 day. By subtracting the average return and dividing the result with the standard deviation of the returns (which is a measure of 2 2 the volatility of the stock), σi = R Ri , we obtain the normalized qh i i−h i price return, Ri Ri ri(t, ∆t) −h i, (1.2) ≡ σi where ... represents time average. Once the return time series for N stocks h i over a period of T days are obtained, the cross-correlation matrix C is calcu- lated, whose element C = r r , represents the correlation between returns ij h i ji for stocks i and j. By construction, C is symmetric with Cii = 1 and Cij has a value in the domain [ 1,1]. Fig. 1.1 shows that, the correlation among stocks − in NSE is larger on the average compared to that among the stocks in NYSE. This supports the general belief that developing markets tend to be more cor- related than developed ones. To understand the reason behind this excess correlation, we perform an eigenvalue analysis of the correlation matrix. 6 NSE ) ij 4 NYSE P ( C 2 0 −0.2 0 0.2 0.4 0.6 0.8 C ij Table 1.1 The probability density function of the elements of the correlation matrix C for 201 stocks in the NSE of India and NYSE for the period Jan 1996-May 2006. The mean value of elements of C for NSE and NYSE, C , are 0.22 and 0.20 respectively. h iji 1.1.1 Eigenvalue spectrum of correlation matrix If the time series are uncorrelated, then the resulting random correlation ma- trix is known as a Wishart matrix, whose statistical properties are well known. In the limit N ∞, T ∞, such that Q T/N 1, the eigenvalues of this → → ≡ ≥ matrix are distributed according to the Sengupta-Mitra distribution [?]: Q (λ λ)(λ λ ) P(λ) = max − − min , (1.3) 2π p λ 1.1 The Return Cross-Correlation Matrix 5 for λ λ λ and, 0 otherwise. The bounds of the distribution are min ≤ ≤ max given by λ = [1 +(1/√Q)]2 and λ = [1 (1/√Q)]2. We now compare max min − this with the statistical properties of the empirical correlation matrix for the NSE. In the NSE data, there are N = 201 stocks each containing T = 2606 returns; as a result Q = 12.97. Therefore, it follows that, in the absence of any correlation among the stocks, the distribution should be bounded between λmin = 0.52 and λmax = 1.63. As observed in developed markets [?, ?, ?, ?], the bulk of the eigenvalue spectrum P(λ) for the empirical correlation ma- trix is in agreement with the properties of a random correlation matrix spec- trum Prm(λ), but a few of the largest eigenvalues deviate significantly from the RMT bound (Fig. 1.2). However, the number of these deviating eigenvalues are relatively few for NSE compared to NYSE. To verify that these outliers are not an artifact of the finite length of the observation period, we have randomly shuffled the return time series for each stock, and then re-calculated the result- ing correlation matrix.