<p>200 hPa versus SST in tropics:</p><p>140</p><p>120</p><p>100 y = 0.9188x - 1756.8 R2 = 0.4106 80</p><p>60</p><p>40</p><p> y = 1.6475x - 3196.7 20 R2 = 0.736</p><p>0 1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010</p><p>Red = 200 hPa 30S-40S, 240-260E, blue = SST tropics 20N-20S</p><p>We now calculate the ‘coefficient of determination’, also known as R2, for the cross correlation between the two curves. This number says how large a fraction of the variation in one curve matches or ‘causes’ – if you believe they are causually related. E.g. if R2 = 0.35, 35% of the variation matches that in the other curve. One can calculate R2 for different ‘lags’, i.e. move one series up a year [or down for a negative lag], then correlate again and note R2. Here is a plot of R2 for lags between -2 and +2 years:</p><p>0.5 R2</p><p>0.4</p><p>0.3</p><p>0.2</p><p>0.1</p><p>Lag 0 -3 -2 -1 0 1 2 3 The largest R2 occurs for lag = 0, but the coefficient is large for several lags. There is not much difference. The reason for this is obvious: There is a large ‘auto-correlation’ present: and both curves show an upwards trend, so the analysis so far just says that there is a similar upwards trend in both curves. Moving one curve over a year does not change the trend much. It is clear that the trend must first be removed if we want to compare the fine details. </p><p>Removing the trend, the curves look like this:</p><p>70 100</p><p>60 80 50</p><p>40 60 30</p><p>20 40</p><p>10 20 0 1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 -10 0 -20</p><p>-30 -20</p><p>And the plot of R2 as a function of lag looks like this:</p><p>0.06 R2</p><p>0.05</p><p>0.04</p><p>0.03</p><p>0.02</p><p>0.01</p><p>Lag 0 -3 -2 -1 0 1 2 3 Thus about 5% of the variation of one curve matches that of the other curve at lag zero years.</p><p>If we plot both set of coefficients as function of lag, we have:</p><p>0.5</p><p>0.4</p><p>0.3</p><p>0.2</p><p>0.1</p><p>0 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5</p><p>Just showing at a glance the effect of the high autocorrelation. The conclusion is that 95% of the variations are unrelated, and the 5% that match do that at zero lag.</p><p>Now, people are often very creative in ‘massaging’ the data. One standard trick is to plot the difference between successive values. This accentuates the variation:</p><p>40 80</p><p>30 60</p><p>20 40 10 20 0 1945 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 20100 -10 -20 -20 -40 -30</p><p>-40 -60</p><p>-50 -80 And indeed, the coefficient of determination more than triples:</p><p>0.18</p><p>0.16</p><p>0.14</p><p>0.12</p><p>0.1</p><p>0.08</p><p>0.06</p><p>0.04</p><p>0.02</p><p>0 -3 -2 -1 0 1 2 3</p><p>But still with the best match at zero lag. At this point it becomes clear to an old hand as I in data ‘massaging’ that further and more desperate measures will not alter the conclusion.</p><p>How good is this method? To answer that we take one of the time series and construct the other one by copying the data shifted one year plus adding some random noise:</p><p>160</p><p>140</p><p>120 y = 1.6475x - 3196.7 2 100 R = 0.736</p><p>80</p><p>60 y = 1.6613x - 3224.7 40 R2 = 0.6567 20</p><p>0 1940 1950 1960 1970 1980 1990 2000 2010 2020 Here it is clear there is a relationship with a lag of 1 year.</p><p>We now go through the same procedure as above, and plot the resulting R2s on one plot:</p><p>1</p><p>Raw simuated data 0.9 0.8 0.7</p><p>0.6 With Trends Removed 0.5</p><p>0.4</p><p>0.3 0.2</p><p>Difference curves 0.1</p><p>0 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5</p><p>You can clearly see the peak at lag one year.</p><p>Extensive trials with more and more random noise added confirm that if there is a signal, this method will find it.</p>