Quantifying the global parameter tensions between ACT, SPT and Planck Will Handley1, 2, 3, ∗ and Pablo Lemos4, y 1Astrophysics Group, Cavendish Laboratory, J.J.Thomson Avenue, Cambridge, CB3 0HE, UK 2Kavli Institute for Cosmology, Madingley Road, Cambridge, CB3 0HA, UK 3Gonville & Caius College, Trinity Street, Cambridge, CB2 1TA, UKz 4Department of Physics and Astronomy, University College London, Gower Street, London, WC1E 6BT, UK (Dated: Submitted 16th July 2020) The overall cosmological parameter tension between the Atacama Cosmology Telescope 2020 fourth data release (ACT) and Planck 2018 data within the concordance cosmological model is quantified using the Suspiciousness statistic to be 2:6σ. Between ACT and the South Pole Telescope (SPT) we find a tension of 2:4σ, and 2:8σ between ACT and Planck+SPT combined. While it is unclear whether the tension is caused by statistical fluctuations, systematic effects or new physics, caution should be exercised in combining these cosmic microwave background datasets in the context of the ΛCDM standard model of the universe. INTRODUCTION the primordial power spectrum ns are mildly discrepant, but it is always possible in a high dimensional parameter As cosmological datasets increase in quantity and qual- space that such discrepancies occur by chance and are ity, so does our capacity to use them to pin down the unremarkable. properties of our universe [1]. The error bars on the In this letter we discuss how this tension is quan- measurements of cosmological parameters have narrowed tified rigorously using the global Suspiciousness statis- over recent years and discrepancies between datasets (or tic [18, henceforth H19], and find that ACT is in mild- \tensions") have begun to emerge. Whilst this is most to-moderate tension with Planck and SPT, at a similar stark when examining differing observations of the Hub- or greater level to that found in weak lensing data. We ble parameter between early and late time cosmological place ACT's own global tension analysis in the context of probes [2{4], other more minor tensions arguably exist in the tensions literature, and extend it by considering SPT clustering parameters between weak lensing and the cos- data and further emphasise the perils of focussing too mic microwave background (CMB) [5, 6] and in cosmic closely on lower-dimensional views onto the cosmological curvature between the CMB and CMB lensing/Baryon constraints. Acoustic Oscillations [7{9]. When a substantial tension occurs, it may indicate METHODOLOGY either a systematic error in how either or both of the datasets have been gathered and analysed, or more ex- citingly may hint at evidence for new physics if extensions Quantifying tension between high dimensional poste- or modifications to our concordance model can bring the rior distributions is a non-trivial problem, even under the inferred parameters back into alignment. approximation of a Gaussian distribution. This has led In the case of the \Hubble tension" where a single ob- to a large number of papers describing methods to quan- vious cosmological parameter such as the present day ex- tify tension in high dimensional problems [for reviews, see 10, 11]. Working in a Bayesian framework, as most pansion rate H0 is discrepant by ∼ 5σ, there is little doubt that something is fundamentally wrong. The other cosmological analyses do, arguably the most natural way tensions are more subtle, in that they are only visible in to quantify tension is using the Bayes Ratio [19], defined complicated combinations of the parameters. As shown as the ratio of the probability that the two datasets are by Fig. 1, in modern cosmology, error bars on the param- described by a single set of parameters, to the probability that they are described by separate sets of parameters arXiv:2007.08496v4 [astro-ph.CO] 19 Apr 2021 eters of our universe are represented by high-dimensional Bayesian probability distributions. Visualising a \dis- P (A; B) ZAB tance" between these degrees of belief is challenging, and R = = ; (1) P (A)P (B) ZAZB in recent years a good deal of theory has been developed for defining a variety of metrics of discrepancy [10, 11]. where P represent a probability, we have omitted the de- The latest Atacama Cosmology Telescope (ACT) data pendence of both probabilities on an underlying model, release 4 [12{14] represents the most recently acquired such as ΛCDM, and Z is the Bayesian Evidence. Fur- CMB data, with two other measurements of the CMB thermore, we have assumed that both data sets are in- power spectrum across a wide range of multipoles being dependent, an assumption that we further comment on provided by the Planck satellite [15, 16], and the South later. High values of R correspond to concordance, and Pole Telescope (SPT) [17]. By eye it is clear that in the low values are indicative of discordance, with R often in- ACT data some parameters such as the spectral tilt of terpreted on a Jeffreys’ scale [20, 21]. The main issue of 2 this tension metric, in particular for the analysis of cos- and covariance µ and Σ, then the Suspiciousness is: mological data sets, is that it is easily proven that R is d χ2 proportional to the prior volume of shared parameters. log S = − ; (5) 2 2 Therefore, R cannot be used for analyses that use delib- 2 −1 erately flat and wide uninformative priors, such as the χ = (µA − µB)(ΣA + ΣB) (µA − µB): (6) analyses of Planck, Dark Energy Survey [DES, 21], Kilo This may be turned into a tension probability via the Degree Survey [KiDS, 6], ACT, SPT, etc. without the ar- survival function of the chi-squared distribution bitrary width of this prior affecting tension assessment. 1 A more detailed interpretation of this discussion can be Z xd=2−1e−x=2 p = dx; (7) found in H19. 2d=2Γ(d=2) 2 Motivated by this, H19 defined a new statistic, the χ Suspiciousness which keeps all the desired properties and calibrated using a σ-tension by analogy with the of Eq. (1), but corrects for this undesired dependence Gaussian case using the inverse of the complementary on the prior volume. To do so, we divide the Bayes Ra- error function: p tio in two components: Information and Suspiciousness. σ(p) = 2erfc−1(1 − p): (8) The Information is defined as: Note that, while several methods to quantify tension have been proposed in recent years, they are often built to log I = DA + DB − DAB; (2) recover Eq. (7) and Eq. (8) in the case of Gaussian poste- where D is the Kullback-Leibler divergence [22]. The rior distributions. Therefore, if this work were performed Information contains the dependence on the prior vol- using tension metrics such as Monte-Carlo Parameter ume, therefore by removing it, we obtain a statistic Shifts [26], Parameter Shifts in Update Form [27], or that does not depend on it, but is composed of well- EigenTension [28], we would expect to obtain very simi- defined Bayesian and information theoretic quantities lar, if not the same results, under the Gaussian approx- and is therefore covariantly insensitive to reparameterisa- imation used in this work. This is also equivalent to tion of the space. Therefore, we define the Suspiciousness the multivariate measure of tension used in the ACT pa- as: per [12]. It should be noted that alternative measures of tension R have also been defined and explored that are specialised S = : (3) I for the case when two datasets are correlated [26, 29]. In particular, [23] extended the formalism described in In the language of priors, the Suspiciousness may be in- this section to the case of correlated data sets. Applying terpreted as the most cautious Bayes Ratio R correspond- this to the case of CMB datasets such as Planck, ACT, ing to the narrowest possible priors that do not signifi- SPT and WMAP (which are correlated by virtue of their cantly alter the shape of the posteriors [23]. measuring the same sky) will form the subject of a future A significant innovation to the field which we highlight paper. here, first noted in the appendix F.3 of [24] and explored DATA in detail in [25] is that since log Z = hlog LiP − D, the suspiciousness can be computed from MCMC chains via In this work we analyse the three latest CMB data sets, Planck, SPT and ACT. As with all cosmological analyses, log S = hlog LABiPAB − hlog LAiPA − hlog LBiPB : (4) when considering combining or comparing them at the likelihood level we implicitly assume that the datasets are This observation means that so long as one has pos- independent, even though this may not strictly true. Ex- terior samples for each of the datasets run separately amining the effect of relaxing this assumption will form and in combination, one may compute the suspicious- the subject of future work. It should also be noted that ness without explicitly computing the Bayesian evidence. the prior treatment for τ is different across the three col- However, it should be noted that in non-CMB applica- laborations, one of the aims of future work will be to treat tions only a portion of the parameters are constrained, this in a consistent manner for all three cases. The ACT resulting in hypersurface-like posteriors which are ex- analysis uses a CMB-derived prior for τ, so there is cor- tremely challenging for traditional posterior samplers, relation between posteriors for the τ parameter. A more but present little challenge for nested samplers. complete analysis could adjust the tension in either direc- If the posteriors are such that we may approximate tion, since correlations in τ act to reduce the dimensional- them in the cosmological parameters by a Gaussian (an ity to less than d = 6, increasing the tension [18, 29, 30], approximation which is reasonably justified as shown by but since τ is a degeneracy-breaking parameter it can Fig.