Practical Statistics for Particle Physics

Center for Cosmology and Particle Physics Practical Statistics for Particle Physics

Kyle Cranmer, New York University

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 1 Center for Cosmology and Introduction Particle Physics Statistics plays a vital role in science, it is the way that we: ‣ quantify our knowledge and uncertainty ‣ communicate results of experiments Big questions: ‣ how do we make discoveries, measure or exclude theoretical parameters, ... ‣ how do we get the most out of our data ‣ how do we incorporate uncertainties ‣ how do we make decisions Statistics is a very big field, and it is not possible to cover everything in 4 hours. In these talks I will try to: ‣ explain some fundamental ideas & prove a few things ‣ enrich what you already know ‣ expose you to some new ideas I will try to go slowly, because if you are not following the logic, then it is not very interesting. ‣ Please feel free to ask questions and interrupt at any time

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 2 Center for Cosmology and Further Reading Particle Physics

By physicists, for physicists G. Cowan, Statistical Data Analysis, Clarendon Press, Oxford, 1998. R.J.Barlow, A Guide to the Use of Statistical Methods in the Physical Sciences, John Wiley, 1989; F. James, Statistical Methods in Experimental Physics, 2nd ed., World Scientific, 2006; ‣ W.T. Eadie et al., North-Holland, 1971 (1st ed., hard to find); S.Brandt, Statistical and Computational Methods in Data Analysis, Springer, New York, 1998. L.Lyons, Statistics for Nuclear and Particle Physics, CUP, 1986.

My favorite statistics book by a statistician:

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 3 Center for Cosmology and Other lectures Particle Physics Fred James’s lectures http://preprints.cern.ch/cgi-bin/setlink?base=AT&categ=Academic_Training&id=AT00000799 http://www.desy.de/~acatrain/ Glen Cowan’s lectures http://www.pp.rhul.ac.uk/~cowan/stat_cern.html Louis Lyons http://indico.cern.ch/conferenceDisplay.py?confId=a063350

Bob Cousins gave a CMS lecture, may give it more publicly Gary Feldman “Journeys of an Accidental Statistician” http://www.hepl.harvard.edu/~feldman/Journeys.pdf

The PhyStat conference series at PhyStat.org:

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 4 Practical Statistics for the LHC

Kyle Cranmer Center for Cosmology and Particle Physics, Physics Department, New York University, USA

Abstract This document is a pedagogical introduction to statistics for particle physics. Emphasis is placed on the terminology, concepts, and methods being used at the Large Hadron Collider. The document addresses both the statistical tests applied to a model of the data and the modeling itself . I expect to release updated versions of this document in the future.

Center for Cosmology and Lecture notes Particle Physics

Contents

Practical Statistics for the LHC 1 Introduction ...... 3 2 Conceptual building blocks for modeling ...... 3 Kyle Cranmer 2.1 Probability densities and the likelihood function ...... 3 Center for Cosmology and Particle Physics, Physics Department, New York University, USA 2.2 Auxiliary measurements ...... 5 2.3 Frequentist and Bayesian reasoning ...... 6 Abstract 2.4 Consistent Bayesian and Frequentist modeling of constraint terms ...... 7 This document is a pedagogical introduction to statistics for particle physics. 3 Physics questions formulated in statistical language ...... 8 Emphasis is placed on the terminology, concepts, and methods being used at 3.1 Measurement as parameter estimation ...... 8 the Large Hadron Collider. The document addresses both the statistical tests applied to a model of the data and the modeling itself . I expect to release 3.2 Discovery as hypothesis tests ...... 9 updated versions of this document in the future. 3.3 Excluded and allowed regions as conﬁdence intervals ...... 11 4 Modeling and the Scientiﬁc Narrative ...... 14 4.1 Simulation Narrative ...... 15 4.2 Data-Driven Narrative ...... 25 4.3 Effective Model Narrative ...... 27 4.4 The Matrix Element Method ...... 27 4.5 Event-by-event resolution, conditional modeling, and Punzi factors ...... 28 5 Frequentist Statistical Procedures ...... 28 5.1 The test statistics and estimators of µ and ✓ ...... 29 5.2 The distribution of the test statistic and p-values ...... 31 5.3 Expected sensitivity and bands ...... 32 5.4 Ensemble of pseudo-experiments generated with “Toy” Monte Carlo ...... 33 5.5 Asymptotic Formulas ...... 33 5.6 Importance Sampling ...... 36 5.7 Look-elsewhere effect, trials factor, Bonferoni ...... 37 5.8 One-sided intervals, CLs, power-constraints, and Negatively Biased Relevant Subsets . . 37 6 Bayesian Procedures ...... 38 6.1 Hybrid Bayesian-Frequentist methods ...... 39 6.2 Markov Chain Monte Carlo and the Metropolis-Hastings Algorithm ...... 40 6.3 Jeffreys’s and Reference Prior ...... 40 6.4 Likelihood Principle ...... 41 7 Unfolding ...... 42 8 Conclusions ...... 42

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 5

2 Center for Cosmology and Particle Physics

Lecture 1

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 6 Center for Cosmology and Outline Particle Physics Lecture 1: Building a probability model ‣ preliminaries, the marked Poisson process ‣ incorporating systematics via nuisance parameters ‣ constraint terms ‣ examples of different “narratives” / search strategies Lecture 2: Hypothesis testing ‣ simple models, Neyman-Pearson lemma, and likelihood ratio ‣ composite models and the profile likelihood ratio ‣ review of ingredients for a hypothesis test Lecture 3: Limits & Confidence Intervals ‣ the meaning of confidence intervals as inverted hypothesis tests ‣ asymptotic properties of likelihood ratios ‣ Bayesian approach

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 7 Center for Cosmology and Particle Physics

Preliminaries

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 8 Center for Cosmology and Probability Density Functions Particle Physics When dealing with continuous random variables, need to introduce the notion of a Probability Density Function (PDF... not parton distribution function) P (x [x, x + dx]) = f(x)dx Note, f ( x ) is NOT a probability

f(x) 0.4 PDFs are always normalized to0.35 unity: 0.3

0.25 ⇥ 0.2 f(x)dx =1 0.15 0.1 ⇥ 0.05 0 -3 -2 -1 0 1 2 3 x

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 9 Center for Cosmology and The Likelihood Function Particle Physics A Poisson distribution describes a discrete event count n for a real- valued mean µ. e µ Likelihood-RatioP ois(n µ)=µn Interval example | n! The likelihood of µ given n is the same equation evaluated as a function of µ 68% C.L. likelihood-ratio interval ‣ Now it’s afor continuous Poisson process function with n=3 ‣ But it is notobserved: a pdf!

3 LL ((µµ) )== µ Pexp(- oisµ()/3!n µ) Maximum at µ = 3. | Common to ∆plot2lnL the= 1 -22 for ln approximateL ±1 Gaussian standard deviation ‣ helps avoidyields thinking interval of [1.58, it as 5.08] a PDF ‣ connection to χ2 distribution Figure from R. Cousins, Am. J. Phys. 63 398 (1995)

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 10

Bob Cousins, CMS, 2008 35 Center for Cosmology and Particle Physics Change of variable x, change of parameter θ

P( y(x1)< y < y(x2) ) = P(x1 < x < x2 ), i.e., that Probabilities are invariant under change of variable x. – Mode of probability density is not invariant (so, e.g., criterion of maximum probability density is ill-defined). – Likelihood ratio is invariant under change of variable x. (Jacobian in denominator cancels that in numerator). • For likelihood L(θ) and reparametrization from θ to u(θ): L(θ) = L(u(θ)) (!). – Likelihood L (θ) is invariant under reparametrization of parameter θ (reinforcing fact that L is not a pdf in θ).

Bob Cousins, CMS, 2008 15 Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 11 Center for Cosmology and Change of variables Particle Physics If f(x) is the pdf for x and y(x) is a change of variables, then the pdf g(y) must satisfy xb y(xb) f(x)dx = g(y)dy Zxa Zy(xa) We can rewrite the integral on the right y(xb) xb dy g(y)dy = g(y(x)) dx dx Zy(xa) Zxa therefore, the two pdfs are related by a Jacobian factor dy f(x)=g(y) dx Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 12 Center for Cosmology and An example Particle Physics y(x) = cos(x) dy f(x)=g(y) dx 1 f(x)= 1 1 1 1 2⇡ g(y)= = 2⇡ sin(x) 2⇡ sin(arccos(y)) | | | |

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 13 Center for Cosmology and Particle Physics Probability Integral Transform

“…seems likely to be one of the most fruitful conceptions introduced into statistical theory in the last few years” − Egon Pearson (1938) Given continuous x ∈ (a,b), and its pdf p(x), let x y(x) = !a p(x′) dx′ . Then y ∈ (0,1) and p(y) = 1 (uniform) for all y. (!) So there always exists a metric in which the pdf is uniform. Many issues become more clear (or trivial) after this transformation*. (If x is discrete, some complications.) The specification of a Bayesian prior pdf p(µ) for parameter µ is equivalent to the choice of the metric f(µ) in which the pdf is uniform. This is a deep issue, not always recognized as such by users of flat prior pdf’s in HEP!

*And the inverse transformation provides for efficient M.C. generation of p(x) starting from RAN(). Bob Cousins, CMS, 2008 16 Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 14 Center for Cosmology and Axioms of Probability Particle Physics

These Axioms are a mathematical starting point for probability and statistics 1. probability for every element, E, is non- negative 2. probability for the entire space of possibilities is 1

3. if elements Ei are disjoint, probability is additive

Kolmogorov Consequences: axioms (1933)

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 15 Center for Cosmology and Different definitions of Probability Particle Physics Frequentist ‣ defined as limit of long term frequency ‣ probability of rolling a 3 := limit of (# rolls with 3 / # trials)

● you don’t need an infinite sample for definition to be useful

● sometimes ensemble doesn’t exist • eg. P(Higgs mass = 120 GeV), P(it will snow tomorrow) ‣ Intuitive if you are familiar with Monte Carlo methods ‣ compatible with orthodox interpretation of probability in Quantum Mechanics. Probability to measure spin projected on x-axis if spin of beam is polarized along +z 1 2 = Subjective Bayesian |⇤ | ⇥⌅| 2 ‣ Probability is a degree of belief (personal, subjective)

● can be made quantitative based on betting odds

● most people’s subjective probabilities are not coherent and do not obey laws of probability http://plato.stanford.edu/archives/sum2003/entries/probability-interpret/#3.1 Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 16 Center for Cosmology and Bayes’ Theorem Particle Physics Bayes’ theorem relates the conditional and marginal probabilities of events A & B P (B A)P (A) P (A B)= | | P (B)

▪P(A) is the prior probability or marginal probability of A. It is "prior" in the sense that it does not take into account any information about B. ▪P(A|B ) is the conditional probability of A, given B. It is also called the posterior probability because it is derived from or depends upon the speciﬁed value of B. ▪P(B |A) is the conditional probability of B given A. ▪P(B ) is the prior or marginal probability of B, and acts as a normalizing constant.

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 17 Center for Cosmology and ... in pictures (from Bob Cousins) Particle Physics P, Conditional P, and Derivation of Bayes’ Theorem in Pictures

P(A) = P(B) = Whole space A B P(A|B) = P(B|A) =

P(A ∩ B) =

P(A) × P(B|A) = × = = P(A ∩ B) Don’t forget about “Whole space” . I will drop it from the notation typically, but occasionally it is important. P(B) × P(A|B) = × = = P(A ∩ B)

Bob Cousins, CMS, 2008 ! P(B|A) = P(A|B) × P(B) / P(A) 7 Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 18 Center for Cosmology and Louis’s Example Particle Physics P (Data;Theory) ! P (Theory;Data)

Theory = male or female Data = pregnant or not pregnant

P (pregnant ; female) ~ 3% but P (female ; pregnant) >>>3%

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 19 Center for Cosmology and Bayes’ Theorem Particle Physics Bayes’ theorem relates the conditional and marginal probabilities of events A & B P (B A)P (A) P (A B)= | | P (B)

f(x ✓)⇡(✓) ⇡(✓ x)= | L(✓)⇡(✓) | / N

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 20 Center for Cosmology and Particle Physics

Modeling: The Scientific Narrative

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 21 Center for Cosmology and Building a model of the data Particle Physics Before one can discuss statistical tests, one must have a “model” for the data. ‣ by “model”, I mean the full structure of P(data | parameters)

● holding parameters fixed gives a PDF for data

● ability to evaluate generate pseudo-data (Toy Monte Carlo)

● holding data fixed gives a likelihood function for parameters • note, likelihood function is not as general as the full model because it doesn’t allow you to generate pseudo-data

Both Bayesian and Frequentist methods start with the model ‣ it’s the objective part that everyone can agree on ‣ it’s the place where our physics knowledge, understanding, and intuiting comes in ‣ building a better model is the best way to improve your statistical procedure

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 22 Center for Cosmology and RooFit: A data modeling toolkit Particle Physics

RooFit is a major tool developed at BaBar for data modeling. RooStats providesPar ahigher-levelmeters of comp ostatisticalsite PDF obj etoolscts based on these PDFs.

RooAddPdf sum

RooGaussian RooRealVar RooGaussian RooRealVar RooArgusBG gauss1 g1frac gauss2 g2frac argus

RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar Building realistic modmean1els sigma x mean2 argpar cutoff Building realistic models • Complex PDFs be can be trivially composed using operator classes – Composition (‘plug & play’) – Addition RooArgSet *paramList = sum.getParameters(data) ; paramList->Print("v") ; RooArgSet::parameters: 1) RooRealVar::argpar : -1.00000 C 2) RooRealVar::cutoff : 9.0000 C + 3) Ro=oRealVar::g1frac : 0.50000 C The parameters of sum = 4) RooRealVar::g2frac : 0.10000 C are the combined 5) RooRealVar::mean1 : 2.0000 C parameters 6) RooRealVar::mean2 : 3.0000 C mof(y its;a components,a ) g(x;m,s) 7) RooRealVar::sigma : 1.0000 C 0 1 g(x,y;a0,a1,s) Wouter Verkerke, UCSB Possible in any PDF No explicit support in PDF code needed – Multiplication – Convolution = * ⊗ =

Wouter Verkerke, UCSB Wouter Verkerke, UCSB Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 23 Center for Cosmology and The Scientific Narrative Particle Physics The model can be seen as a quantitative summary of the analysis ‣ If you were asked to justify your modeling, you would tell a story about why you know what you know

● based on previous results and studies performed along the way ‣ the quality of the result is largely tied to how convincing this story is and how tightly it is connected to model I will describe a few “narrative styles” ‣ The “Monte Carlo Simulation” narrative ‣ The “Data Driven” narrative ‣ The “Effective Modeling” narrative ‣ The “Parametrized Response” narrative

Real-life analyses often use a mixture of these

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 24 Center for Cosmology and The Monte Carlo Simulation narrative Particle Physics Calculation Let’sPredict startion v iawithMo nt“thee Ca rMontelo Simula Carlotion simulation narrative”, which is probably the most familiar The enormousFordet eachect eventors a, rcalculatee still b differentialeing const cross-section:ructed, but we have detailed simulations of the detectors response.

+ W µ L(x H ) =Phase-space H |P0 = Transfer Integral Functions! W Matrixµ− Element

Only partial information available The advancements inFixtheo measuredretical quantitiespredictions, detector simulation, tracking, calorimetry, triggeringIntegrate, and co overmput unmeasureding set the partonbar higquantitiesh for equivalent advances in our statistical treatment of the data. November 8, 2006 Daniel Whiteson/Penn Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 25

September 13, 2005 Statistical Challenges of the LHC (page 6) Kyle Cranmer PhyStat2005, Oxford Brookhaven National Laboratory Center for Cosmology and The simulation narrative Particle Physics

The language of the Standard Model is Quantum Field Theory 1) Phase space Ω defines initial measure, sampled via Monte Carlo f i 2 P = | | ⇥| f f i i | ⇥ | ⇥ P L ! d 2d⌦ 1 1 1 L = W · Wµν − B Bµν − Ga Gµν |M| SM 4 µν 4 µν 4 µν a ! "# $ kinetic energies and self-interactions of the gauge bosons

1 1 1 + L¯γµ(i∂ − gτ · W − g!YB )L + R¯γµ(i∂ − g!YB )R µ 2 µ 2 µ µ 2 µ ! "# $ kinetic energies and electroweak interactions of fermions

1 % 1 1 ! %2 + %(i∂µ − gτ · Wµ − g YBµ) φ % − V (φ) W, Z H 2 2 2 ! "# $ W ±,Z,γ,and Higgs masses and couplings

!! µ a ¯ ¯ + g (¯qγ Taq) Gµ +(G1LφR + G2RLφcRL + h.c.) ! "# $ ! "# $ interactions between quarks and gluons fermion masses and couplings to Higgs

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 26 Center for Cosmology and Particle Physics

A General Purpose Statistical Model

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 27 Center for Cosmology and Visualizing probability models Particle Physics I will represent PDFs graphically as below (directed acyclic graph) ‣ eg. a Gaussian G ( x µ, ⇥ ) is parametrized by ( µ, ⇥ ) | ‣ every node is a real-valued function of the nodes below

x µ σ

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 28 Center for Cosmology and Marked Poisson Process Particle Physics Channel: a subset of the data defined by some selection requirements. ‣ eg. all events with 4 electrons with energy > 10 GeV ‣ n: number of events observed in the channel ‣ ν: number of events expected in the channel Discriminating variable: a property of those events that can be measured and which helps discriminate the signal from background ‣ eg. the invariant mass of two particles ‣ f(x): the p.d.f. of the discriminating variable x = x ,...,x D { 1 n} Marked Poisson Process: n f( ⌫) = Pois(n ⌫) f(x ) D| | e e=1 Y Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 29 10 ATLAS collaboration: Search for the Standard Model Higgs Boson

Table 5. Numbers of events estimated as background, observed in data and expected from signal in the H ZZ !!qq search → → for low mass (mH < 360 GeV) and high mass (mH 360 GeV) selections. The signal, quoted at two mass points, includes small ≥ Center for contributions from !!!! and !!νν decays. Electron and muon channels are combined. The uncertainties shown are the statistical Cosmology and Mixture model Particle Physics and systematic uncertainties, respectively. Sample: a sample of simulated events corresponding to particular Source low mass selection high mass selection Z+jets 214 4 27type interaction 9.1 0 .9that1.4 populates the channel. W +jets 0.33 ±0.16± 0.17 ± ± tt¯ 0.94 ± 0.09 ± 0.25 0.08 0−.02 0.03 Multi-jet 3.81 ± 0.65 ± 1.91‣ statisticians 0.11 ± 0 .11call± 0. 06this a mixture model ZZ 3.80 ± 0.10 ± 0.73 0.30 ± 0.03 ± 0.06 WZ 2.83 ± 0.05 ± 0.88 0.29 ± 0.02 ± 0.10 ± ± 1 ± ± Total background 226 4 28 9.9 0.9 1.5 ± ± f(x)= ± ± ⌫ f (x) , ⌫tot = ⌫s H ZZ !!qq 0.60 0.01 0.12 (mH =200GeV) 0.24 (< 0.001) 0.05 (mH =400GeV)s s → → ± ± ± ± Observed 216 ⌫tot 11 s samples s samples 2 2 X h2mu2nu_200_model_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit X

6 data. The multi-jet background in the electron channel is 10 RooRealSumPdf derived from a sample where the electron identiﬁcation ATLAS 105 data requirements are relaxed. In the muon channel, the multi- Z+jets 4 H → ee νν (m =400 GeV) top jet background is estimated from a simulated sample of 10 H -1 Diboson semi-leptonically decaying b-andc-quarks and found to be 3 L dt = 35 pb 10 W+jets negligible after the application of the m!! selection. This ∫ 2 s = 7 TeV Multijet Events / 5 GeV L_x_MultiJet_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_Signal_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_WW_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit was veriﬁed in data using leptons with identical charges. 10 Signal (m =400 GeV) L_x_WZ_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_ZZ_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_W_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_Z_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit H L_x_tt_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit 10

6.3.2 Results for the H ZZ !!νν search 1 binWidth_obs_h2mu2nu_200_7_zz2l2nu binWidth_obs_h2mu2nu_200_6_zz2l2nu binWidth_obs_h2mu2nu_200_5_zz2l2nu binWidth_obs_h2mu2nu_200_4_zz2l2nu binWidth_obs_h2mu2nu_200_3_zz2l2nu binWidth_obs_h2mu2nu_200_2_zz2l2nu binWidth_obs_h2mu2nu_200_1_zz2l2nu binWidth_obs_h2mu2nu_200_0_zz2l2nu RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar → → RooProduct RooProduct RooProduct RooProduct RooProduct RooProduct RooProduct RooProduct 10-1 The H ZZ !!νν analysis is performed for Higgs -2 boson masses→ between→ 200 GeV and 600 GeV in steps of 10 20 GeV. Table 6 summarises the numbers of events ob- -3 10 0 50 100 150 200 250 served in the data, the estimated numbers of background miss events and the expected numbers of signal events for two ET [GeV] selected m values. For the low mass selections, five events H Fig. 7. Distribution of missing transverse energy in the H are observed in data compared to an expected number of Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 30 ZZ !!νν search in the electron channel before vetoing events→ events from background sources only of 5.8 0.5 1.3. The with→ low Emiss.TheexpectedyieldforaHiggsbosonwith corresponding results for the high mass selections± ± are five T mH =400GeVisalsoshown. events observed in data compared to an expected yield of 3.5 0.4 0.8 events from background sources only. In addition± to± the H ZZ !!νν decays, several other Higgs boson channels→ give a→ non-negligible contribution to the is more transparently defined and it has reduced overcov- ∗ total expected signal yield. In particular, H WW( ) erage resulting in a more precise meaning of the quoted !ν!ν decays can lead to final states that are→ very similar→ confidence level. The resulting PCL median limits have to H ZZ !!νν decays. They are found to contribute been found to be around 20% tighter than those obtained → → significantly to the signal yield at low mH values. The with the CLs method in several Higgs searches. The ap- ∗ expected number of events from H WW( ) !ν!ν de- plication of the PCL method to each of the individual cays relative to that from H ZZ→ !!νν decays→ is 76% Higgs boson search channels is described in Refs. [7–11]. → → for mH =200GeVand9%formH = 300 GeV. The kine- A similar procedure is used here. The individual analyses matic selections prevent individual candidates from being are combined by maximising the product of the likelihood miss accepted by both searches. The ET distribution before functions for each channel and computing a likelihood ra- miss vetoing events with low ET is shown in Fig. 7. tio. A single signal normalisation parameter µ is used for all analyses, where µ is the ratio of the hypothesised cross section to the expected Standard Model cross section. 7Combinationmethod Each channel has sources of systematic uncertainty, some of which are common with other channels. Table 7 The limit-setting procedure uses the power-constrained lists the common sources of systematic uncertainties, which profile likelihood method known as the Power Constrained are taken to be 100% correlated with other channels. Let Limit, PCL [13, 14, 64]. This method is preferred to the the search channels be labelled by l (l = H γγ, H → → more familiar CLs [15] technique because the constraint WW, . . . ), the background contribution, j,tochannell Center for Cosmology and Parametrizing the model ↵ =(µ, ✓) Particle Physics Parameters of interest (µ): parameters of the theory that modify the rates and shapes of the distributions, eg. ‣ the mass of a hypothesized particle ‣ the “signal strength” μ=0 no signal, μ=1 predicted signal rate

Nuisance parameters (θ or αp): associated to uncertainty in: ‣ response of the detector (calibration) ‣ phenomenological model of interaction in non-perturbative regime

Lead to a parametrized model: ⌫ ⌫(↵),f(x) f(x ↵) ! ! | n f( ↵) = Pois(n ⌫(↵)) f(x ↵) D| | e| e=1 Y Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 31 10 ATLAS collaboration: Search for the Standard Model Higgs Boson

Source low mass selection high mass selection Z+jets 214 4 27 9.1 0.9 1.4 ± ± ± ± Center for W +jets 0.33 0.16 0.17 Cosmology and Incorporating Systematic Effects Particle Physics tt¯ 0.94 ± 0.09 ± 0.25 0.08 0−.02 0.03 Multi-jet 3.81 ± 0.65 ± 1.91 0.11 ± 0.11 ± 0.06 ZZ 3.80 ± 0.10 ± 0.73Tabulate effect 0.30 ± 0 .of03 ± individual0.06 variations of sources of systematic uncertainty WZ 2.83 ± 0.05 ± 0.88 0.29 ± 0.02 ± 0.10 ± ± ± ± Total background 226 4 28‣ typically 9 .one9 0.9 at1 .a5 time evaluated at nominal and “± 1 σ” ± ± ± ± H ZZ !!qq 0.60 0.01 0.12 (mH =200GeV) 0.24 (< 0.001) 0.05 (mH =400GeV) → → ± ± ± ± Observed 216 ‣ use some form11 of interpolation to parametrize pth variation in terms of nuisance parameter αp data. The multi-jet background in the electron channel is 6 10 Z+jets top Diboson ... derived from a sample where the electron identification ATLAS 105 data 0.25 requirements are relaxed. In the muon channel, the multi- Z+jets 4 H → ee νν (m =400 GeV) top syst 1 jet background is estimated from a simulated sample of 10 H -1 Diboson 0.2 semi-leptonically decaying b-andc-quarks and found to be 3 L dt = 35 pb 10 W+jets negligible after the application of the m!! selection. This ∫ syst 2 2 s = 7 TeV Multijet Events / 5 GeV was verified in data using leptons with identical charges. 10 Signal (m =400 GeV) 0.15f(x) H 10 ... 0.1 6.3.2 Results for the H ZZ !!νν search 1 alpha_XS_BKG_ZZ_4l_zz4l → → 10-1 0.05 The H ZZ !!νν analysis is performed for Higgs → → 10-2 boson masses between 200 GeV and 600 GeV in steps of 0 20 GeV. Table 6 summarises the numbers of events ob- 10-3 170 180 190 200 210 220 230 0 50 100 150 200 250 obs_H_4e_channel_200 served in the data, the estimated numbers of background x miss events and the expected numbers of signal events for two ET [GeV] selected m values. For the low mass selections, five events n H Fig. 7. Distribution of missing transverse energy in the H are observed in data compared to an expected number of ZZ !!νν search in the electron channel before vetoing events→ events from background sources only of 5.8 0.5 1.3. The → miss with low ET .TheexpectedyieldforaHiggsbosonwithf( ↵) = Pois(n ⌫(↵)) f(x ↵) corresponding results for the high mass selections± ± are five e mH =400GeVisalsoshown. events observed in data compared to an expected yield of D| | | 3.5 0.4 0.8 events from background sources only. In ad- e=1 ± ± dition to the H ZZ !!νν decays, several other Higgs Kyle Cranmer (NYU) CLASHEP, Peru, March 2013Y 32 boson channels→ give a→ non-negligible contribution to the is more transparently defined and it has reduced overcov- ∗ total expected signal yield. In particular, H WW( ) erage resulting in a more precise meaning of the quoted !ν!ν decays can lead to final states that are→ very similar→ confidence level. The resulting PCL median limits have to H ZZ !!νν decays. They are found to contribute been found to be around 20% tighter than those obtained → → significantly to the signal yield at low mH values. The with the CLs method in several Higgs searches. The ap- ∗ expected number of events from H WW( ) !ν!ν de- plication of the PCL method to each of the individual cays relative to that from H ZZ→ !!νν decays→ is 76% Higgs boson search channels is described in Refs. [7–11]. → → for mH =200GeVand9%formH = 300 GeV. The kine- A similar procedure is used here. The individual analyses matic selections prevent individual candidates from being are combined by maximising the product of the likelihood miss accepted by both searches. The ET distribution before functions for each channel and computing a likelihood ra- miss vetoing events with low ET is shown in Fig. 7. tio. A single signal normalisation parameter µ is used for all analyses, where µ is the ratio of the hypothesised cross section to the expected Standard Model cross section. 7Combinationmethod Each channel has sources of systematic uncertainty, some of which are common with other channels. Table 7 The limit-setting procedure uses the power-constrained lists the common sources of systematic uncertainties, which profile likelihood method known as the Power Constrained are taken to be 100% correlated with other channels. Let Limit, PCL [13, 14, 64]. This method is preferred to the the search channels be labelled by l (l = H γγ, H → → more familiar CLs [15] technique because the constraint WW, . . . ), the background contribution, j,tochannell 10 ATLAS collaboration: Search for the Standard Model Higgs Boson

Source low mass selection high mass selection Z+jets 214 4 27 9.1 0.9 1.4 W +jets 0.33 ±0.16± 0.17 ± ± tt¯ 0.94 ± 0.09 ± 0.25 0.08 0−.02 0.03 Multi-jet 3.81 ± 0.65 ± 1.91 0.11 ± 0.11 ± 0.06 ZZ 3.80 ± 0.10 ± 0.73 0.30 ± 0.03 ± 0.06 WZ 2.83 ± 0.05 ± 0.88 0.29 ± 0.02 ± 0.10 ± ± ± ± Total background 226 4 28 9.9 0.9 1.5 ± ± ± ± H ZZ !!qq 0.60 0.01 0.12 (mH =200GeV) 0.24 (< 0.001) 0.05 (mH =400GeV) → → ± ± ± ± Observed 216 11 Center for Cosmology and Visualizing the model for one channel Particle Physics data. The multi-jet background in the electron channel is 106 derived from a sample where the electron identiﬁcation ATLAS 105 data requirements are relaxed. In the muon channel, the multi- Z+jets 4 H → ee νν (m =400 GeV) top jet background is estimated from a simulated sample of 10 H h2mu2nu_200_model_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit semi-leptonically decaying b-andc-quarks and found to be -1 Diboson f

3 L dt = 35 pb RooRealSumPdf 10 W+jets negligible after the application of the m!! selection. This ∫ 2 s = 7 TeV Multijet was veriﬁed in data using leptons with identical charges. Events / 5 GeV 10 Signal (m =400 GeV) H 10 L_x_MultiJet_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_Signal_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_WW_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_WZ_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_ZZ_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_W_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_Z_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_tt_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit 6.3.2 Results for the search 1

H ZZ !!νν binWidth_obs_h2mu2nu_200_7_zz2l2nu binWidth_obs_h2mu2nu_200_6_zz2l2nu binWidth_obs_h2mu2nu_200_5_zz2l2nu binWidth_obs_h2mu2nu_200_4_zz2l2nu binWidth_obs_h2mu2nu_200_3_zz2l2nu binWidth_obs_h2mu2nu_200_2_zz2l2nu binWidth_obs_h2mu2nu_200_1_zz2l2nu binWidth_obs_h2mu2nu_200_0_zz2l2nu → → RooGaussian RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooProduct RooProduct RooProduct RooProduct RooProduct RooProduct RooProduct RooProduct 10-1 JES_gaus The H ZZ !!νν analysis is performed for Higgs -2 boson masses→ between→ 200 GeV and 600 GeV in steps of 10 20 GeV. Table 6 summarises the numbers of events ob- -3 10 0 50 100 150 200 250 ⌫i ⌫i(↵), served in the data, the estimated numbers of background miss fi(x) fi(x ↵) !Signal_h2mu2nu_200_overallNorm_x_sigma_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit events and the expected numbers of signal events for two ET [GeV] ! |

selected mH values. For the low mass selections, ﬁve events PiecewiseInterpolation RooRealVar RooRealVar RooProduct JES_sigma Fig. 7. Distribution of missing transverse energy in the H JES_mean are observed in data compared to an expected number of ZZ !!νν search in the electron channel before vetoing events→ events from background sources only of 5.8 0.5 1.3. The with→ low Emiss.TheexpectedyieldforaHiggsbosonwith corresponding results for the high mass selections± ± are ﬁve 0.25 T mH =400GeVisalsoshown. events observed in data compared to an expected yield of Signal_h2mu2nu_200_Hist_alphanominal_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_8high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_7high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_6high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_5high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_4high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_3high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_2high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_1high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_0high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_8low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_7low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_6low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_5low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_4low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_3low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_2low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_1low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_0low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit 3.5 0.4 0.8 events from background sources only. In ad- 0.2 Signal_h2mu2nu_200_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ± ± RooStats::HistFactory::LinInterpVar

dition to the H ZZ !!νν decays, several other Higgs f(x) RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc M_RES_MS RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar M_RES_ID M_SCALE E_SCALE E_RES B_EFF JER JES boson channels→ give a→ non-negligible contribution to the is more0.15 transparently defined and it has reduced overcov- mu ∗ total expected signal yield. In particular, H WW( ) erage resulting in a more precise meaning of the quoted !ν!ν decays can lead to final states that are→ very similar→ confidence0.1 level. The resulting PCL median limits have ↵p alpha_XS_BKG_ZZ_4l_zz4l alpha_ShapeError_zz2l2nu alpha_StatsError_zz2l2nu to H ZZ !!νν decays. They are found to contribute been found to be around 20% tighter than those obtained obs_h2mu2nu_200 RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar MJET2MU XS_TOP MJET2E XS_WW XS_GG XS_WZ M_EFF E_EFF XS_ZZ XS_W → → 0.05 XS_Z significantly to the signal yield at low mH values. The with the CLs method in several Higgs searches. The ap- (∗) expected number of events from H WW !ν!ν de- plication of the PCL method to each of the individual x ↵p cays relative to that from H ZZ→ !!νν decays→ is 76% Higgs0 boson search channels is described in Refs. [7–11]. → → 170 180 190 200 210 220 230 for mH =200GeVand9%formH = 300 GeV. The kine- A similar procedure is used here. Theobs_H_4e_channel_200 individual analyses matic selections prevent individual candidates from being Kyleare combinedCranmer (NYU) by maximising the product ofx theCLASHEP, likelihood Peru, March 2013 33 miss accepted by both searches. The ET distribution before functions for each channel and computing a likelihood ra- miss vetoing events with low ET is shown in Fig. 7. tio. A single signal normalisation parameter µ is used for all analyses, where µ is the ratio of the hypothesised cross section to the expected Standard Model cross section. 7Combinationmethod Each channel has sources of systematic uncertainty, some of which are common with other channels. Table 7 The limit-setting procedure uses the power-constrained lists the common sources of systematic uncertainties, which profile likelihood method known as the Power Constrained are taken to be 100% correlated with other channels. Let Limit, PCL [13, 14, 64]. This method is preferred to the the search channels be labelled by l (l = H γγ, H → → more familiar CLs [15] technique because the constraint WW, . . . ), the background contribution, j,tochannell 10 ATLAS collaboration: Search for the Standard Model Higgs Boson

data. The multi-jet background in the electron channel is 106 derived from a sample where the electron identification ATLAS After parametrizing each 105 data requirements are relaxed. In the muon channel, the multi- Z+jets 4 H → ee νν (m =400 GeV) top jetcomponent background is estimated of the from mixture a simulated sample of 10 H -1 Diboson semi-leptonically decaying b-andc-quarks and found to be 3 L dt = 35 pb 10 W+jets negligiblemodel, after the the application pdf for ofa thesinglem!! selection. This ∫ 2 s = 7 TeV Multijet was verified in data using leptons with identical charges. Events / 5 GeV 10 Signal (m =400 GeV) channel might look like this H 10 6.3.2 Results for the H ZZ !!νν search 1 → → 10-1 The H ZZ !!νν analysis is performed for Higgs -2 boson masses→ between→ 200 GeV and 600 GeV in steps of 10 model_h2mu2nu_200_zz2l2nu_edit 20 GeV. TableRooProdPdf 6 summarises the numbers of events ob- 10-3 h2mu2nu_200_model_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit 0 50 100 150 200 250 M_RES_MS_gaus RooRealSumPdf M_RES_ID_gaus MJET2MU_gaus M_SCALE_gaus E_SCALE_gaus RooGaussian XS_TOP_gaus MJET2E_gaus XS_WW_gaus RooGaussian XS_GG_gaus XS_WZ_gaus RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian RooGaussian E_RES_gaus M_EFF_gaus E_EFF_gaus B_EFF_gaus XS_ZZ_gaus XS_W_gaus XS_Z_gaus LUMI_gaus JER_gaus servedJES_gaus in the data, the estimated numbers of background miss L_x_MultiJet_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_Signal_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_WW_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_WZ_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_ZZ_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_W_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit L_x_Z_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit events and the expected numbers of signal events for two L_x_tt_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ET [GeV] alpha_ShapeErrorConstraint_zz2l2nu alpha_StatsErrorConstraint_zz2l2nu binWidth_obs_h2mu2nu_200_7_zz2l2nu binWidth_obs_h2mu2nu_200_6_zz2l2nu binWidth_obs_h2mu2nu_200_5_zz2l2nu binWidth_obs_h2mu2nu_200_4_zz2l2nu binWidth_obs_h2mu2nu_200_3_zz2l2nu binWidth_obs_h2mu2nu_200_2_zz2l2nu binWidth_obs_h2mu2nu_200_1_zz2l2nu binWidth_obs_h2mu2nu_200_0_zz2l2nu M_RES_ID_sigma M_RES_ID_mean MJET2MU_sigma M_SCALE_sigma MJET2MU_mean M_SCALE_mean XS_TOP_sigma MJET2E_sigma XS_WW_sigma XS_TOP_mean MJET2E_mean XS_WW_mean XS_GG_sigma XS_WZ_sigma XS_GG_mean E_RES_sigma XS_WZ_mean M_EFF_sigma E_RES_mean E_EFF_sigma B_EFF_sigma M_EFF_mean XS_ZZ_sigma E_EFF_mean B_EFF_mean XS_ZZ_mean RooGaussian XS_W_sigma RooGaussian XS_W_mean XS_Z_sigma LUMI_sigma XS_Z_mean LUMI_mean RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar JER_sigma JES_sigma JER_mean JES_mean M_RES_MS_sigma M_RES_MS_mean E_SCALE_sigma E_SCALE_mean RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar selected m values. For the low mass selections, five eventsRooProduct RooProduct RooProduct RooProduct RooProduct RooProduct RooProduct RooProduct H Fig. 7. Distribution of missing transverse energy in the H are observed in data compared to an expected number of → Signal_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ !!νν search in the electron channel before vetoing events nom_alpha_ShapeError_zz2l2nu nom_alpha_StatsError_zz2l2nu RooConstVar RooAddition RooRealVar RooRealVar RooProduct lumi events from background sources only of 5.8 0.5 1.3. The1 with→ low Emiss.TheexpectedyieldforaHiggsbosonwith corresponding results for the high mass selections± ± are five T Signal_h2mu2nu_200_overallNorm_x_sigma_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit MultiJet_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_overallSyst_x_HistSyst_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit mH =400GeVisalsoshown. lumi_sigmaTimesDelta lumi_mean_val RooRealVar RooProduct RooProduct RooProduct RooProduct RooProduct RooProduct events observed in data compared to an expectedRooProduct yield ofRooProduct RooProduct 3.5 0.4 0.8 events from background sources only. In ad- MultiJet_h2mu2nu_200_Hist_alpha_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit MultiJet_h2mu2nu_200_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ± ± tt_h2mu2nu_200_epsilon_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit RooStats::HistFactory::LinInterpVar RooStats::HistFactory::LinInterpVar RooStats::HistFactory::LinInterpVar RooStats::HistFactory::LinInterpVar RooStats::HistFactory::LinInterpVar RooStats::HistFactory::LinInterpVar RooStats::HistFactory::LinInterpVar RooStats::HistFactory::LinInterpVar PiecewiseInterpolation PiecewiseInterpolation PiecewiseInterpolation PiecewiseInterpolation PiecewiseInterpolation dition to the H ZZ !!νν decays, several otherPiecewiseInterpolation Higgs PiecewiseInterpolation PiecewiseInterpolation lumi_sigma_val RooRealVar RooRealVar RooRealVar LUMI mu boson channels→ give a→ non-negligible contribution to the is more transparently defined and it has reduced overcov- MultiJet_h2mu2nu_200_Hist_alphanominal_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit MultiJet_h2mu2nu_200_Hist_alpha_0high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alphanominal_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit MultiJet_h2mu2nu_200_Hist_alpha_0low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_8high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_7high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_6high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_5high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_4high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_3high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_2high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_1high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_0high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_8low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_7low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_6low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_5low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_4low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_3low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_2low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_1low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Signal_h2mu2nu_200_Hist_alpha_0low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alphanominal_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alphanominal_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_8high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_7high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_6high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_5high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_4high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_3high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_2high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_1high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_0high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alphanominal_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_8high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_7high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_6high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_5high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_4high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_3high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_2high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_1high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_0high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_8low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_7low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_6low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_5low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_4low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_3low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_2low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_1low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WW_h2mu2nu_200_Hist_alpha_0low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alphanominal_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_7high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_6high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_5high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_4high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_3high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_2high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_1high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_0high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_8high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_8low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_7low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_6low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_5low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_4low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_3low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_2low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_1low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit WZ_h2mu2nu_200_Hist_alpha_0low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alphanominal_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alphanominal_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_9high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_8high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_7high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_6high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_5high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_4high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_3high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_2high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_1high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_0high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_8low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_7low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_6low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_5low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_4low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_3low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_2low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_1low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit ZZ_h2mu2nu_200_Hist_alpha_0low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_9high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_8high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_7high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_6high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_5high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_4high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_3high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_2high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_1high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_0high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_8high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_7high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_6high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_5high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_4high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_3high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_2high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_1high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_0high_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_9low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_8low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_7low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_6low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_5low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_4low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_3low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_2low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_1low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit W_h2mu2nu_200_Hist_alpha_0low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_9low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_8low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_7low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_6low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_5low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_4low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_3low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_2low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_1low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit Z_h2mu2nu_200_Hist_alpha_0low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit (∗) tt_h2mu2nu_200_Hist_alpha_8low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_7low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_6low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_5low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_4low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_3low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_2low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_1low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit tt_h2mu2nu_200_Hist_alpha_0low_zz2l2nu_model_h2mu2nu_200_zz2l2nu_edit alpha_ShapeError_zz2l2nu total expected signal yield. Inalpha_StatsError_zz2l2nu particular, H WW RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc RooHistFunc M_RES_MS RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar RooRealVar M_RES_ID MJET2MU M_SCALE E_SCALE XS_TOP MJET2E XS_WW erage resulting in a more precise meaning of the quoted XS_GG XS_WZ E_RES M_EFF XS_ZZ E_EFF B_EFF XS_W XS_Z JER JES !ν!ν decays can lead to final states that are→ very similar→ confidence level. The resulting PCL median limits have obs_h2mu2nu_200 to H ZZ !!νν decays. They are found to contribute been found toRooRealVar be around 20% tighter than those obtained → → significantly to the signal yield at low mH values. The with the CLs method in several Higgs searches. The ap- ∗ expected number of events from H WW( ) !ν!ν de- plication of the PCL method to each of the individual cays relative to that from H ZZ→ !!νν decays→ is 76% Higgs boson search channels is described in Refs. [7–11]. → → Kylefor mCranmerH =200GeVand9%for (NYU) mH = 300 GeV.CLASHEP, The kine- Peru,A March similar 2013 procedure is used here. The individual analyses34 matic selections prevent individual candidates from being are combined by maximising the product of the likelihood miss accepted by both searches. The ET distribution before functions for each channel and computing a likelihood ra- miss vetoing events with low ET is shown in Fig. 7. tio. A single signal normalisation parameter µ is used for all analyses, where µ is the ratio of the hypothesised cross section to the expected Standard Model cross section. 7Combinationmethod Each channel has sources of systematic uncertainty, some of which are common with other channels. Table 7 The limit-setting procedure uses the power-constrained lists the common sources of systematic uncertainties, which profile likelihood method known as the Power Constrained are taken to be 100% correlated with other channels. Let Limit, PCL [13, 14, 64]. This method is preferred to the the search channels be labelled by l (l = H γγ, H → → more familiar CLs [15] technique because the constraint WW, . . . ), the background contribution, j,tochannell Center for Cosmology and Simultaneous multi-channel model Particle Physics Simultaneous Multi-Channel Model: Several disjoint regions of the data are modeled simultaneously. Identification of common parameters across many channels requires coordination between groups such that meaning of the parameters are really the same. nc f ( ↵)= Pois(n ⌫ (↵)) f (x ↵) sim Dsim| c| c c ce| c channels " e=1 # 2 Y Y where = ,..., Dsim {D1 Dcmax } Control Regions: Some channels are not populated by signal processes, but are used to constrain the nuisance parameters ‣ attempt to describe systematics in a statistical language

‣ Prototypical Example: “on/off” problem with unknown ⌫b f(n, m µ, ⌫ ) = Pois(n µ + ⌫ ) Pois(m ⌧⌫ ) | b | b · | b signal region control region Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 35 | {z } | {z } Center for Cosmology and Constraint terms Particle Physics Often detailed statistical model for auxiliary measurements that measure certain nuisance parameters are not available.

‣ one typically has MLE for αp, denoted ap and standard error Constraint Terms: are idealized pdfs for the MLE.

fp(ap ↵p) for p S | 2 ‣ common choices are Gaussian, Poisson, and log-normal ‣ New: careful to write constraint term a frequentist way ‣ Previously: ⇡ ( ↵ a )= f ( a ↵ ) ⌘ ( ↵ ) with uniform η p| p p p| p p Simultaneous Multi-Channel Model with constraints:

nc f ( , ↵)= Pois(n ⌫ (↵)) f (x ↵) f (a ↵ ) tot Dsim G| c| c c ce| · p p| p c channels " e=1 # p 2 Y Y Y2S where sim = 1,..., c = a for p S D {D D max }, G { p} 2 Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 36 Probability models can be constructed to simultaneously describe several channels, that is several disjoint regions of the data deﬁned by the associated selection criteria. I will use e as the index over th events and c as the index over channels. Thus, the number of events in the c channel is nc and the th th value of the e event in the c channel is xce. In this context, the data is a collection of smaller datasets: sim = 1,..., c = xc=1,e=1 ...xc=1,e=n ,... xc=c ,e=1 ...xc=c ,e=n . In RooFit D {D D max } {{ c } { max max cmax }} the index c is referred to as a RooCategory and it is used to inside the dataset to differentiate events associated to different channels or categories. The class RooSimultaneous associates the dataset c with D the corresponding marked Poisson model. The key point here is that there are now multiple Poisson terms. Thus we can write the combined (or simultaneous) model

nc f ( ↵)= Pois(n ⌫(↵)) f(x ↵) , (2) sim Dsim| c| ce| c channels " e=1 # 2 Y Y

Center for remembering that the symbol product over channels has implications for the structureCosmology of the and dataset. Particle Physics

A Ensemble

B Experiment

Channel Constraint Term Legend: c ∈ channels f (a | α ) A "has many" Bs. p p p B "has a" C. f (x | α) Dashed is optional. c p ∈ parameters with constraints

Event Sample global observable e ∈ events s ∈ samples a {1…n } c

Observable(s) Distribution Expected Number of Events x f (x | α) ν ec sc s

Shape Variation Parameter f (x | α = X ) scp p α, θ, μ

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 37 Fig. 1: A schematic diagram of the logical structure of a typical particle physics probability model and dataset structures.

2.2 Auxiliary measurements Auxiliary measurements or control regions can be used to estimate or reduce the effect of systematic uncertainties. The signal region and control region are not fundamentally different. In the language that we are using here, they are just two different channels. A common example is a simple counting experiment with an uncertain background. In the frequentist way of thinking, the true, unknown background in the signal region is a nuisance parameter, 5 which I will denote ⌫B. If we call the true, unknown signal rate ⌫S and the number of events in the signal region n then we can write the model Pois(n ⌫ + ⌫ ). As long as ⌫ is a free parameter, SR SR| S B B 5Note, you can think of a counting experiment in the context of Eq. 1 with f(x)=1, thus it reduces to just the Poisson term.

5 Center for Cosmology and Combined ATLAS Higgs Search Particle Physics State of the art: At the time of the discovery, the combined Higgs search included 100 disjoint channels and >500 nuisance parameters April 18, 2012 – 13 : 01 DRAFT 10 ‣ Models for individual channels come from about 11 sub-groups performing dedicated searches for specific Higgs decay modes Table‣ In 3: addition Summary oflow-level the individual performance channels contributing groups to providethe combination. tools for The evaluating central number in the three-partsystematic mass ranges effects indicates and the corresponding transition from low- constraintmH to high-m termsH optimised event selections.

Subsequent m Higgs Decay Additional Sub-Channels H L[fb 1] Decay Range − H γγ –9sub-channels(p η conversion) 110-150 4.9 → Tt ⊗ γ ⊗ !!! ! 4e,2e2µ,2µ2e,4µ 110-600 4.8 $ $ { } H ZZ !!νν ee,µµ low pile-up, high pile-up 200-280-600 4.7 → { } ⊗ { } !!qq b-tagged, untagged 200-300-600 4.7 { } !ν!ν ee,eµ,µµ 0-jet, 1-jet, VBF 110-300-600 4.7 H WW { } ⊗ { } → !νqq e, µ 0-jet, 1-jet 300-600 4.7 $ { } ⊗ { } !!4ν eµ 0-jet 1-jet, VBF, VH 110-150 4.7 { } ⊗ { } ⊕ { } e,µ 0-jet Emiss ≷ 20 GeV H τ+τ !τ 3ν { } ⊗ { } ⊗ { T } 110-150 4.7 → − had e,µ 1-jet, VBF ⊕ { } ⊗ { } τ τ 2ν 1-jet 110-150 4.7 had had { } Z νν Emiss 120 160,160 200, 200 GeV 110-130 4.6 → T ∈ { − − ≥ } VH bb W !ν pW < 50,50 100,100 200, 200 GeV 110-130 4.7 → → T ∈ { − − ≥ } Z !! pZ < 50,50 100,100 200, 200 GeV 110-130 4.7 → T ∈ { − − ≥ } Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 38 ( ) + + 203 • H ZZ ! ! ! ! :Thisanalysisisunchangedwithrespecttothepreviouscombined → ∗ → − − 204 search [?]. The search is performed for mH hypotheses in the full 110 GeV to 600 GeV mass 1 205 range using data corresponding to an integrated luminosity of 4.8 fb− [?]. The main irreducible ( ) 206 ZZ ∗ background is estimated using Monte Carlo simulation. The reducible Z+jets background, 207 which has an impact mostly for low four-lepton invariant masses, is estimated from control re- 208 gions in the data. The top-quark (tt )backgroundnormalisationisvalidatedusingadedicated 209 control sample. The events are categorised according to the lepton flavour combinations. The 210 mass resolutions are approximately 1.5% in the four-muon channel and 2% in the four-electron 211 channel for m 120 GeV. The four-lepton invariant mass is used as a discriminating variable. H∼ + 212 • H ZZ ! !−νν update: The analysis described in [?, ?]wasbasedonanintegratedluminos- → → 1 213 ity of 2.05 fb− and was optimised for two search regions with mH hypotheses above and below 214 280 GeV and two lepton flavour categories. To achieve the best sensitivity, the present search, 1 1 215 which uses an integrated luminosity of 4.7 fb− [?], is additionally split between the first 2.3 fb− 216 of “low pile-up” collision data, where the average number of interactions per bunch crossing was 1 217 about 6, and the latter 2.4 fb− of “high pile-up” collisions, where the average number of interac- + 218 tions per bunch crossing was about 12. The selection is unaltered between the periods. The ! !− 219 pair invariant mass is required to be within 15 GeV of the Z-boson mass. The reverse requirement ( ) + 220 is applied to same-flavour leptons in the H WW ! ν! ν channel to avoid overlaps. The → ∗ → − 221 transverse mass of the dilepton and missing transverse energy system is used as a discriminating 222 variable.

+ 223 • H ZZ ! ! qq update: This analysis is updated with respect to the previouscombined → → − 224 search [?]. The previous analysis used a dataset corresponding to an integrated luminosity of 1 1 225 2.05 fb− [?], while the current analysis is based on an integrated luminosity of 4.7 fb− [?]. It 226 takes advantage of an improved b-tagging algorithm [?]andofthelargersampleofdatatobetter 227 constrain systematic uncertainties on the background yield. The analysis is separated into search 228 regions above and below mH =300 GeV, where the event selections are independently optimised. 229 The dominant background arises from Z+jets production, which is normalised from data using Center for Cosmology and Visualizing the combined model Particle Physics State of the art: At the time of the discovery, the combined Higgs search included 100 disjoint channels and >500 nuisance parameters

RooFit / RooStats: is the modeling language (C++) which provides technologies for collaborative modeling ‣ provides technology to publish likelihood functions digitally ‣ and more, it’s the full model so we can also generate pseudo-data

nc f ( , ↵)= Pois(n ⌫ (↵)) f (x ↵) f (a ↵ ) tot Dsim G| c| c c ce| · p p| p c channels " e=1 # p 2 Y Y Y2S

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 39 5

4 Center for Cosmology and Evolution of Model Complexity Particle Physics

14 4000 180 800 ATLAS Data 2011 ATLAS Data 2011 12 Data 2011 ATLAS 250 ATLAS (*) ATLAS H τ→ τ + 0j ATLAS H τ→ τ + 1j 12 (*) 3500 lep lep lep lep 160 H + 2j Data 2011 700 H γγ→ H→ZZ →4l Total background H→ZZ →4l τ→ lepτlep Total background Total background 10 Data 2011 200 140 10 3000 Data 2011 Total background Events / GeV 600 m =125 GeV, 1 x SM m =125 GeV, 1 x SM mH=125 GeV, 1 x SM H H 120 Events / 5 GeV 8 Total background Events / 10 GeV 2500 Total background mH=125 GeV, 10 x SM 8 -1 Events / 15 GeV Events / 20 GeV 150 Events / 40 GeV 500 -1 s = 7 TeV, Ldt = 4.8 fb 100 -1 s = 7 TeV, Ldt = 4.8 fb ∫ m =125 GeV, 10 x SM s = 7 TeV, Ldt = 4.9 fb 6 2000 H m =125 GeV, 10 x SM ∫ 6 ∫ H 80 400 s = 7 TeV, Ldt = 4.7 fb-1 1500 -1 100 ∫ 4 s = 7 TeV, Ldt = 4.7 fb -1 60 300 4 ∫ s = 7 TeV, ∫ Ldt = 4.7 fb 1000 40 200 2 2 50 500 20 100 0 0 100 110 120 130 140 150 160 0 100 200 300 400 500 600 100 120 140 160 180 200 220 240 0 0 0 40 60 80 100 120 140 160 180 200 50 100 150 200 250 300 50 100 150 200 250 m γγ [GeV] mllll [GeV] mllll [GeV] meff [GeV] m ττ [GeV] m ττ [GeV] (a) (b) (c) (a) (b) (c)

8 200 40 120 90 ATLAS Data 2011 ATLAS Data 2011 -1 ATLAS 180 H + 0/1j -1 s = 7 TeV, Ldt = 4.7 fb 7 1800 ATLAS τ→ lepτhad ATLAS H + 2j ATLAS H τ→ τ s = 7 TeV, Ldt = 4.7 fb H→ZZ→ll νν ∫ H→ZZ→llqq 35 τ→ lepτhad had had 80 H→ZZ→llbb Total background 160 Total background 100 ∫ 6 1600 70 Data 2011 140 Data 2011 30 Data 2011 Data 2011 mH=350 GeV, 1 x SM m =350 GeV, 1 x SM 1400 5 H 80 60 m =125 GeV, 10xSM m =125 GeV, 10 x SM Total background

Events / 50 GeV Events / 18 GeV Events / 20 GeV 120 Total background 1200 H 25 H

-1 Events / 10 GeV Events / 20 GeV Events / 12 GeV s = 7 TeV, Ldt = 4.7 fb -1 50 4 100 s = 7 TeV, Ldt = 4.7 fb m =125 GeV, 10 x SM ∫ 1000 Total backgrou nd 20 Total background 60 H mH=350 GeV, 1 x SM ∫ 40 80 3 800 -1 -1 15 30 60 s = 7 TeV, Ldt = 4.7 fb s = 7 TeV, Ldt = 4.7 fb 40 2 600 ∫ ∫ 20 10 40 400 20 10 1 20 200 5 0 0 200 300 400 500 600 700 800 900 200 300 400 500 600 700 800 200 300 400 500 600 700 800 0 0 0 0 50 100 150 200 250 300 350 400 0 50 100 150 200 250 300 350 400 60 80 100 120 140 160 180 mT [GeV] mllbb [GeV] mlljj [GeV] mMMC [GeV] mMMC [GeV] m ττ [GeV] (d) date Categories Nodes(e) Parameters (f) (d) (e) (f) Jan-09 3 50 10 100 30 5 Jun-10 6 374 37 50 180 40 90 ATLAS H→WW→lνlν+0j ATLAS Data 2011 4.5 ATLAS Data 2011 Jun-11 24 7000 82 45 ATLAS ZH→llbb Data 2011 ATLAS WH→lνbb Data 2011 ZH νν→ bb Data 2011 25 H→WW→lνlν+1j H WW l l +2j 160 35 ATLAS 80 m =125 GeV, 1xSM 4 → → ν ν Data 2011 Nov-11 48 10000 H 164 mH=125 GeV, 1 x SM -1 -1 Not final selection 40 s = 7 TeV, Ldt = 4.7 fb Total background s = 7 TeV, Ldt = 4.7 fb Total background -1 Total background 140 s = 7 TeV, Ldt = 4.6 fb 70 Mar-12 70 13700 Total background500 3.5 ∫ ∫ 30 m =125 GeV, 1 x SM 20 Tot al background 35 ∫ H m =125 GeV, 5 x SM 120 m =125 GeV, 5 x SM m =125 GeV, 5 x SM Jul-12 100 19000 580 H H H Events / 10 GeV 60 Events / 10 GeV -1 Events / 10 GeV 3 25

Events / 10 GeV 30 Events / 10 GeV Events / 10 GeV Total background s = 7 TeV, Ldt = 4.7 fb -1 50 15 ∫ 2.5 s = 7 TeV, Ldt = 4.7 fb 100 ∫ 25 20 40 -1 2 80 s = 7 TeV, Ldt = 4.7 fb 20 ∫ 10 15 30 1.5 60 15 20 1 10 5 10 40 10 0.5 5 20 5 0 0 0 60 80 100 120 140 160 180 200 220 240 60 80 100 120 140 160 180 200 220 240 50 100 150 200 250 300 350 400 450 0 0 0 80-150 80-150 80-150 80-150 80-150 80-150 80-150 80-150 80-150 80-150 80-150 mT [GeV] mT [GeV] mT [GeV] m [GeV] m [GeV] m [GeV] bb bb bb (g) (h) (i) (g) (h) (i)

120 35 Number100 of DatasetsATLAS H→ CombinedWW→lνqq+1j Number of Model Components Number of Parameters in Likelihood ATLAS H→WW→lνqq+0j ATLAS H→WW→lνqq+2j 30 FIG. 2. Invariant or transverse mass distributions for the selected candidate events, the total background and the signal expected 100 Data 2011 Data 2011 80 25 Data 2011 in the following channels: (a) H τlepτlep+0-jets, (b) H τlepτlep 1-jet, (c) H τlepτlep+2-jets, (d) H τlepτhad+0-jets and 100 Total background 20000 600 − 80 Total background → → → + ¯ → ¯ Total background 1-jet, (e) H τlepτhad+2-jets, (f) H τhadτhad.Thebb invariant mass for (g) the ZH " " bb,(h)theWH "νbb and (i) Events / 10 GeV Events / 10 GeV Events / 10 GeV 60 m =350 GeV, 1 x SM 20 m =350 GeV, 1 x SM H → ¯ → → → H the ZH ννbb channels. The vertical dashed lines illustrate the separation between the mass spectra of the subcategories in 60 mH=350 GeV, 1 x SM -1 15 Z W → miss -1 s = 7 TeV, Ldt = 4.7 fb p p E s = 7 TeV, Ldt = 4.7 fb 40 ∫ -1 T, T ,and T ,respectively.Thesignaldistributionsarelightlyshadedwheretheyhavebeenscaledbyafactorofﬁveor 40 ∫ s = 7 TeV, Ldt = 4.7 fb 10 ∫ ten for illustration purposes. 20 20 75 5 15000 450

0 0 0 300 400 500 600 700 800 900 1000 300 400 500 600 700 800 900 1000 300 400 500 600 700 800 900 1000

mlνjj [GeV] mlνjj [GeV] mlνjj [GeV] (j) 50 (k) 10000 (l) 300 FIG. 1. Invariant or transverse mass distributions for the selected candidate events, the total background and the signal expected ∗ − − ∗ − − in the following channels: (a) H γγ,(b)H ZZ( ) "+" "+" in the entire mass range, (c) H ZZ( ) "+" "+" in − − the low mass range, (d) H ZZ→ "+" νν,(e)→ b-tagged→selection and (f) untagged selection for H →ZZ "+→" qq,(g)H ∗ − → → ∗ − ∗ − → → # → WW( ) "+ν" ν+0-jets, (h)25H WW( ) "+ν" ν+1-jet, (i) H WW( ) "+ν" ν+2-jets,5000 (j) H WW "νqq +0- 150 → # → → # → → ∗ − → → jets, (k) H WW "νqq +1-jet and (l) H WW "νqq +2-jets. The H WW( ) "+ν" ν+2-jets distribution is shown before→ the ﬁnal→ selection requirements are→ applied.→ → → 0 0 0 Jan-09 Jan-10 Jan-11 Jan-12 Jan-13 Jan-09 Jan-10 Jan-11 Jan-12 Jan-13 Jan-09 Jan-10 Jan-11 Jan-12 Jan-13 Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 40 Center for Cosmology and Particle Physics

Constraint Terms Auxiliary Measurements and Priors on Nuisance Parameters

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 41 ATLAS Statistics Forum DRAFT 7 May, 2010

Comments and Recommendations for Statistical Techniques

1 Preliminaries

A simple ‘prototype problem’ has been considered as useful simplification of a common HEP situation and its coverage properties have been studied in Ref. [1] and generalized by Ref. [2]. The problem consists of a number counting analysis, where one observes non events and expects s + b events, b is uncertain, and one either wishes to perform a significance test against the null hypothesis s = 0 or create a confidence interval on s. Here s is considered the parameter of interest and b is referred to as a nuisance parameter (and should be generalized accordingly in what follows). In the setup, the background rate b is uncertain, but can be constrained by an auxiliary or sideband measurement where one expects ⇥b events and measures no events. This simple situation (often referred to as the ‘on/o⇥’ problem) can be expressed by the following probability density function: Center for Cosmology and What do we meanP (n ,n bys, uncertainty? b) = Pois(n s + b) Pois(n ⇥b). Particle Physics (1) on o | on| o | Let’s consider a simplified problem that has been studied quite a bit to Notegain that some in this insight situation into the our sideband more measurementrealistic and is difficult also modeled problems as a Poisson process and the expected number of counts due to background events can be related to the main measurement‣ number by a counting perfectly knownwith background ratio ⇥. In many uncertainty cases a more accurate relation between the sideband● in our measurement main measurementno and thewe observe unknown non background with s+b expected rate b may be a Gaussian with either an absolute or relative uncertainty b. These cases were also considered in Refs. [1, 2] and are referred to as the ‘GaussianPois( meann problem’.on s + b) While the prototype problem is a simplification,| it has been an instructive example. The first, and‣ and perhaps, the background most important has lesson some is uncertainty that the uncertainty on the background rate b has been● castbut aswhat a well-defined is “background statistical uncertainty”? uncertainty Where instead did it come of a vaguely-defined from? systematic uncertainty. To make this point more clearly, consider that it is common practice in HEP to ● maybe we would say background is known to 10% or that it has some pdf ⇡(b) describe the problem as • then we often do a smearing of the background: P (n s)= db Pois(n s + b) (b), (2) on| on| where (●b)Where is a distributiondoes ⇡ ( b ) come (usually from? Gaussian) for the uncertain parameter b,whichis then marginalized• did you (ie. realize ‘smeared’, that this ‘randomized’, is a Bayesian orprocedure ‘integrated that out’depends when on creating some prior pseudo- experiments).assumption But what isabout the naturewhat b ofis?(b)? The important fact which often evades serious consideration is that (b) is a Bayesian prior, which may or may-not be well-justified. It often is justified by some previous measurements either based on Monte Carlo, sidebands, or 42 controlKyle Cranmer samples. (NYU) However, even in thoseCLASHEP, cases Peru, one March does 2013 not escape an underlying Bayesian prior for b. The point here is not about the use of Bayesian inference, but about the clear accounting of our knowledge and facilitating the ability to perform alternative statistical tests.

1 4 3 Control of background rates from data

In the case of the eµ final state it is worth to note that the optimization was performed against tt and WW background only. As a consequence the results obtained are suboptimal since the background contribution from W+jets is not small and it affects the final cut requirements. In the NN analysis, additional variables have been used. They are: the separation angle between the isolated leptons in • the transverse mass of each lepton-Emiss pair, which help reduce non-W background; • T the of both leptons, as leptons from signal events are more central than the ones • | | from background events; miss the angle in the transverse plane between the ET and the closest lepton. This • miss variable discriminates against events with no real ET the di-lepton final states: ee, µµ or eµ, the background level and composition is quite • different depending on the type. The mass of the di-lepton system and the the angle between the isolated leptons in the transverse plane are shown in Figures 2, 3 and 4 for the Higgs boson signal (mH = 160 GeV) and for the main backgrounds. In these distributions, only events that satisfy the lepton identification, pre-selection cuts and the centralCenter jet for veto criteria are considered. Cosmology and The Data-driven narrative Particle Physics

4 10 CMS Preliminary CMS Preliminary - Regions in the data with negligible signal e+e Channel Signal, m =160 GeV Signal, m =160 GeV H 3 H expected are used as control samples W+Jets, tW 10 W+Jets, tW di-boson di-boson 103 tt tt ‣ simulated events are used to estimate events / bin Drell-Yan events / bin Drell-Yan 2 extrapolation coefficients e+e- Channel 10 102 ‣ extrapolation coefficients may have αWW 10 theoretical and experimental uncertainties 10

1 1 C.R.(WW ) S.R. C.R. S.R. S.R. N WW C.R. WW 10-1 10-1 = N WW 0 20 40 60 80 100 120 140 160 180 200 0 20 40 60 80 100 120 140 160 180 α WW Top 2 ∆Φ [dg.] H → WW mll [GeV/c ] ll W+jets Figure 2: Invariant mass of the di-lepton system (left) and azimuthal angular separation be- WW S.R. C.R.(WW ) N N Top Top C.R.(Top) β = αTop = C.R. ToptweenC.R.(Top) the two leptonsNotation (right) for for next the slides:e±e channel after the High Level Trigger, lepton identiﬁ- N Top N Top Top Top cation, pre-selection# in cutsS.R. and → the non central jet veto for a SM Higgs with mH = 160 GeV. C.R.(WW) S.R. N N W +jets # in C.R. → noff W +jets C.R.(W +jets) β = αW +jets = C.R. W +jets C.R.(W +jets) NW +jets N W +jets W+jets W+jets Figure 5 showsα the WW neural → τ network outputs for the mass hypotheses of mH = 130 GeV and mH = 170 GeV. The distributions are representative of other mass regions. There is a clear shape difference between signal and background events for both mass scenarios, although

Figure 10: Flow chart describing the four data samples used in the H WW ( ) there!ν!ν analysis. is no regionS.R completely free of background. Vertical lines indicate the cut values used. → ∗ → and C.R. stand for signal and control regions, respectively. Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 43 Figure 10 summarises the flow chart of the extraction of the main backgrounds.3Shown Controlare the four of background rates from data data samples and how the five scale factors are applied to get the background estimates in a graphical form. The top control region and the W +jets control regions are considered to be pure3.1samplestoft topandandWW normalization from data W+jets respectively. The normalisation for the WW background is taken from the WW control region after subtracting the contaminations from top and W+jets in the WW control region. To get the numbers of top and W+jets events in the signal region and in the WW control region there are four scale factors, αtop and αW+ jets to get the number of events in the signal region and βtop and βW+ jets to get the number of events in the WW control region. Finally there is a fifth scale factor, αWW to get the number of WW background events in the signal region from the number of background subtracted events in the WW control region. Table 12 shows the number of WW , top backgrounds and W+jets events in each of the four regions. Other smaller backgrounds are ignored for the purpose of estimating the scale factors. The assumption that the three control regions are dominated by these three sources of backgrounds is true to a level of 97% or higher. No uncertainty is assigned due to ignoring additional small backgrounds. The central values for the five scale factors are obtained from ratios of the event counts in Table 12, and are shown in Table 13. Table 14 shows the impact of systematic uncertainties on these scale factors for the H + 0 j, H + 1 j and H + 2 j analyses, respectively. The following is a list of the systematic uncertainties considered in the analyses together with a short description of how they are estimated: WW and Top Monte Carlo Q2 Scale: The uncertainty from higher order effects on the scale • factors for WW and top quark backgrounds is estimated from varying the Q2 scale of the WW and tt¯ Monte Carlo samples. Several WW and tt¯ Monte Carlo samples have been generated with different Q2 scales. Both renormalisation and factorisation scales are multiplied by factors of 8 and 1/8 (4 and 1/4) for the WW (tt¯) process. The uncertainties on the relevant scale factors (αWW , αtop and βtop) are taken to be the maximum deviation from the central value for these scale factors and the values for these scale factors in any of the Q2 scale altered samples [19].

Jet Energy Scale and Jet Energy Resolution: The Jet Energy Scale (JES) uncertainty is taken • to be 7% for jets with η < 3.2 and 15% for jets with η > 3.2. To estimate the effect of the | | | | 23 ATLAS Statistics Forum DRAFT 7 May, 2010 ATLAS Statistics Forum DRAFT Comments and Recommendations for Statistical25 May, Techniques 2010

We review a collection of statistical tests used for a prototype problem, characterize their generalizations, and provide comments on these generalizations. Where possible, concrete Commentsrecommendations and are made Recommendations to aid in future comparisons for and combinations Statistical with Techniques ATLAS and CMS results. We review a collection of statistical tests used for a prototype problem, characterize their generalizations, and provide comments on these generalizations. Where possible, concrete recommendations1 Preliminaries are made to aid in future comparisons and combinations with ATLAS and CMS results. These comments are quite general, and each experiment is expected to have well-developedA simple ‘prototype techniques problem’ that has are been (hopefully) considered consistent as useful simplification with what is of presented a common HEP here. situation and its coverage properties have been studied in Ref. [1] and generalized by Ref. [2]. The problem consists of a number counting analysis, where one observes non events and expects s + b events, b is uncertain, and one either wishes to perform a significance test 1against Preliminaries the null hypothesis s = 0 or create a confidence interval on s. Here s is considered the parameter of interest and b is referred to as a nuisance parameter (and should be generalized A simpleaccordingly ‘prototype in what problem’ follows). has In been the setup, considered the background as useful ratesimplificationb is uncertain, of a butcommon can HEP situationbe constrained and its coverage by an auxiliary properties or sideband have been measurement studied in where Ref. one [1] and expects generalized⇥b events by and Ref. [2]. measures n events. This simple situation (often referred to as the ‘on/o⇥’ problem) can be The problemo consists of a number counting analysis, where one observes non events and expressed by the following probability density function: expects s + b events, b is uncertain, and one either wishes to performCenter a for significance test Cosmology and against theThe null “on/off” hypothesis problems = 0 or create a confidence interval on s. HereParticles is Physics considered the P (non,no s, b) = Pois(non s + b) Pois(no ⇥b). (1) parameterNow of interest let’s sayand thatb isthe referred background| to as awasnuisance| estimated parameter from| some(and shouldcontrol be generalized accordinglyNote thatregion in in what thisor sideband situation follows). themeasurement. sidebandIn the setup, measurement the background is also modeled rate asb ais Poisson uncertain, process but can and the expected number of counts due to background events can be related to the main be constrained‣ by an auxiliary or sideband measurement where one expects ⇥b events and measurementWe bycan a perfectlytreat these known two ratio measurements⇥. In many cases simultaneously: a more accurate relation between measures no events. This simple situation (often referred to as the ‘on/o⇥’ problem) can be the sideband● main measurement measurement:n andobserve the unknownnon with s+b background expected rate b may be a Gaussian with expressed by the following probabilityo density function: either an absolute● sideband or relativemeasurement: uncertainty observeb .noff These with ⌧ cases b expected were also considered in Refs. [1, 2] and are referred to as the ‘Gaussian mean problem’. P (non,no s, b) = Pois(non s + b) Pois(no ⇥b) . (1) While the prototype problem| is a simplification,| it has been an instructive| example. The joint model main measurement sideband first, and perhaps, most important lesson is that the uncertainty on the background rate b ● In this approach “background uncertainty” is a statistical error Notehas that been in cast this as situation a well-defined⌅ the⇤⇥ statistical sideband⇧ uncertainty⌅ measurement⇤⇥ instead⇧ is⌅ alsoof a⇤⇥ vaguely-defined modeled⇧ as a systematic Poisson process anduncertainty. the expected● justification To make number this and ofpoint accounting counts more due clearly, of background to consider background uncertainty that it events is commonis much can more practice be related clear in HEP to the to main measurementdescribeHow the bydoes problem a perfectly this as relate known to the ratio smearing⇥. In manyapproach? cases a more accurate relation between the sideband measurement n and the unknown background rate b may be a Gaussian with Po(n s)= db Pois(n s + b) (b), (2) on| on| Ifeither we were an absolute actually or in relative a case described uncertainty by theb. These ‘on/o cases’ problem, were also then considered it would in be Refs. better [1, 2] to thinkand ofare⇤ referred(b) as the to posterior as the ‘Gaussian resulting mean from problem’. the sideband measurement where ‣(bwhile) is a distribution ( b ) is based (usually on data, Gaussian) it still depends for the uncertain on some parameter original priorb,whichis ⌘(b) Herethen marginalized we rely heavily (ie. ‘smeared’, on the correspondence ‘randomized’, or ‘integrated between hypothesis out’ when creating tests and pseudo- confidence experiments). But what is the nature of (b)? TheP important(no b)⇥ fact(b) which often evades serious intervals [3], and mainly frame⇤(b)= theP discussion(b no )= in terms| of confidence. intervals. (3) consideration is that (b) is a Bayesian| prior, whichdbP may(no orb) may-not⇥(b) be well-justified. It Whileoften is the justified prototype by some problem previous is measurements a simplification, either it based has| beenon Monte an instructive Carlo, sidebands, example. or The Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 44 Byfirst, doingcontrol and this perhaps, samples. it is clear However, most that important even the in term those lessonP ( casesno is thatb one) is does the an objectivenot uncertainty escape probabilityan on underlying the background density Bayesian that rate canb | behas usedprior been in for cast a frequentistb. Theas a point well-defined here context is not statistical and about that the uncertainty⇥ use(b) of is Bayesian the original instead inference, ofBayesian a but vaguely-defined about prior the assigned clear systematic ac- to b. uncertainty.counting of To our make knowledge this point and facilitating more clearly, the ability consider to perform that it alternative is common statistical practice tests. in HEP to Recommendation:describe the problemWhere as possible, one should express uncertainty on a parameter as statistical (eg. random) process (ie. Pois(n 1 ⌅b) in Eq. 1). o | P (n s)= db Pois(n s + b) (b), (2) on| on| Recommendation: When using Bayesian techniques, one should explicitly express and separatewhere the(b) prior is a fromdistribution the objective (usually part Gaussian) of the probability for the uncertain density function parameter (asb, in which Eq. 3). is thenNow marginalized let us consider (ie. some ‘smeared’, specific ‘randomized’, methods for or addressing ‘integrated the out’ on/o when problem creating and pseudo- their generalizations.experiments). But what is the nature of (b)? The important fact which often evades serious consideration is that (b) is a Bayesian prior, which may or may-not be well-justified. It

2 The frequentist solution: ZBi1

The goal for a frequentist solution to this problem is based on the notion of coverage (or Type I error). One considers there to be some unknown true values for the parameters s, b and attempts to construct a statistical test that will not incorrectly reject the true values above some speciﬁed rate .

A frequentist solution to the on/o problem, referred to as ZBi in Refs. [1, 2], is based on re-writing Eq. 1 into a dierent form and using the standard frequentist binomial parameter test, which dates back to the first construction of confidence intervals for a binomial parameter by Clopper and Pearson in 1934 [3]. This does not lead to an obvious generalization for more complex problems. The general solution to this problem, which provides coverage “by construction” is the Neyman Construction. However, the Neyman Construction is not uniquely determined; one must also specify: the test statistic T (n ,n ; s, b), which depends on data and parameters • on o a well-defined ensemble that defines the sampling distribution of T • the limits of integration for the sampling distribution of T • parameter points to scan (including the values of any nuisance parameters) • how the final confidence intervals in the parameter of interest are established • The Feldman-Cousins technique is a well-specified Neyman Construction when there are no nuisance parameters [6]: the test statistic is the likelihood ratio T (non; s)=L(s)/L(sbest), the limits of integration are one-sided, there is no special conditioning done to the ensemble, and there are no nuisance parameters to complicate the scanning of the parameter points or the construction of the final intervals. The original Feldman-Cousins paper did not specify a technique for dealing with nuisance parameters, but several generalization have been proposed. The bulk of the variations come from the choice of the test statistic to use.

2 Center for Cosmology and Common Constraints Terms Particle Physics Many uncertainties have no clear statistical description or it is impractical to provide Traditionally, we use Gaussians, but for large uncertainties it is clearly a bad choice ‣ quickly falling tail, bad behavior near physical boundary, optimistic p-values, ... For systematics constrained from control samples and dominated by statistical uncertainty, a Gamma distribution is a more natural choice [PDF is Poisson for the control sample] ‣ longer tail, good behavior near boundary, natural choice if auxiliary is based on counting For “factor of 2” notions of uncertainty log-normal is a good choice ‣ can have a very long tail for large uncertainties None of them are as good as an actual model for the auxiliary measurement, if available

0.1 To consistently switch between frequentist, Bayesian, and hybrid procedures, need to 0.08 be clear about prior vs. likelihood function Truncated Gaussian Gamma Projection of gprior 0.06 Log-normal PDF(y| β) Prior(β) Posterior(β|y) 0.04 Gaussian uniform Gaussian Poisson uniform Gamma 0.02 Log-normal 1/β Log-Normal 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 beta Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 45 Center for Cosmology and Classification of Systematic Uncertainties Particle Physics Taken from Pekka Sinervo’s PhyStat 2003 contribution

Type I - “The Good” ‣ can be constrained by other sideband/auxiliary/ ancillary measurements and can be treated as statistical uncertainties

● scale with luminosity Type II - “The Bad” ‣ arise from model assumptions in the measurement or from poorly understood features in data or analysis technique

● don’t necessarily scale with luminosity

● eg: “shape” systematics Type III - “The Ugly” ‣ arise from uncertainties in underlying theoretical paradigm used to make inference using the data

● a somewhat philosophical issue Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 46 Center for Cosmology and Particle Physics

Modeling: The Scientific Narrative (continued)

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 47 5 ) ) 2 2 CDF data (4.3 fb-1) 150 700 Bkg Sub Data (4.3 fb-1) WW+WZ 4.5% W+Jets 80.2% Center for 600 Cosmology and Choice: Data drivenTop 6.5% vs. Simulation100 WW+WZ (all bkg syst.)Particle Physics 500 Z+jets 2.8% (b) In the case of the CDFQCD bump, 5.3% the Z+jets control sample provides a data- Events/(8 GeV/c Events/(8 GeV/c driven400 estimate, but limited statistics. 50Using the simulation narrative over (a) 4.1 Signal and Background Definition 55 4.1 Signal and Background Definitionthe300 data-driven8.2 Background is a Modelingchoice Studies. If55 you trust that narrative, it’s a good choice.115 200 0 ν ν We compare the M distribution of data Z+jets events to ALPGEN MC. Fig. 8.17 ¯ 100 ¯ jj q¯ q¯ ¯ q ¯ q p¯ p¯ p p + + + shows the two distributions for+ muons and-50 electrons respectively. AlsoZ in this case, + Z 0 W + W 100 200 100 200 q within statistics, we do not observeq significant disagreement. q q p p 2 p 2 p q g g Mjj [GeV/c ] q Mjjg [GeV/c ] q g q⇥ ¯ ⇥ ¯ q¯ q¯ q q ) ) 2 2 -1 -1 180 -1 CDF dataMuon (4.3 fbData) (4.3 fb ) BkgElectron Sub Data Data (4.3 fb(4.3-1) fb ) Figure 4.2: Example of one of the numerous700 diagrams0.14 for the production of W +jets 0.16 Figure 4.2: Example of one of the numerous diagrams for the production of W +jets

Entries Gaussian 2.5% 160 Entries MC Z+jet MC Z+jet (left) and Z+jets (right). WW+WZ 4.8% 140 0.14 (left) and Z+jets (right).Gaussian 600 0.12 W+Jets 78.0% WW+WZ (all bkg syst.) Top 6.3% 120 0.12 500 Z+jets 2.8% 0.10 100 (d) QCD 5.1% 0.10 Events/(8 GeV/c Events/(8 GeV/c 80 ¯ 400 0.08 b¯ b 0.08 p¯ ¯ p¯ ¯ (c) 60 q t¯ q t¯ (a) (b) g W 300 g 0.06 W 40 0.06 q W + 0.04 W + 20 p t p q200 t 0.04 0 b b 100 0.02 -20 0.02

0.00 -40 0.00 0 0 50 100 150 200 250 300 0 50 100 150 200 250 300 100 200 2 100 Figure200 4.3: Feynman2 diagram for tt¯ production. Figure 4.3: Feynman diagram for tt¯ production. Mj1,j22 [GeV/c ] Mj1,j2 [GeV/c2 ] Mjj [GeV/c ] Mjj [GeV/c ] Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 48 FIG. 1: The dijet invariant mass distribution. The sum of electron and muon events is plotted. In the left plots we show the b¯ fits for knownFigure processes 8.17:only (a)M andjj within theZ+jets additionb¯ data of a hypothetical and MC Gaussian in the component muon sample (c). On the (a) right and plots in we in show, the W g by subtraction,W only the resonantg contribution to Mjj including WW and WZ productionp¯ (b) andq¯ the hypothesized narrow p¯ p¯ q¯ electronp¯ sample (b). ⇥ W + t b b ⇥ W + t Gaussian contribution (d). In plot (b)b and (d) datab points dier because the normalization of the background changes between t the two fits. The band in the subtractedt plots represents the sum of all background shape systematic uncertainties describedb + b + 2 2 W in the text. The distributions areW shown with a 8 GeV/c binning while the actual fitp is performedq using a 4 GeV/c bin size. p W q W p p b¯ q b¯ q ⇥ q In addition,⇥ we compareq several kinematic variables between Z+jets data and resonance with definite mass. The width of the Gaus- muon samples, while the mean is constrained to be the sian is fixedALPGEN to the expectedMC (see dijet Fig. mass8.18 resolution) and find by thatsame. the The agreement result of this isalternative good. fit is shown in Figs. 1 Figure 4.4: Feynman diagrams for single top production. Figure 4.4: Feynmanscaling diagrams the width for of single the W toppeak production. in the same spectrum: (c) and (d). The inclusion of this additional component Mjj 2 brings the fit into good agreement with the data. The resolution = W = 14.3 GeV/c ,whereW and MW fit ⇥2/ndf is 56.7/81 and the Kolmogorov-Smirnov test MW are the resolution and the average dijet invariant returns a probability of 0.05, accounting only for statis- mass for the hadronic8.2.3 WinR thejj WWModelingsimulations respec- tical uncertainties. The W +jets normalization returned q¯ tively, and Mjj is the dijet mass where¯ the Gaussian tem- q by the fit including the additional Gaussianp¯ componentq¯ is p¯ plate is¯ centered. g q In Fig. 8.4, we observed disagreementcompatible between with data the and preliminary our background estimation from model the E in In the combinedg fit, the normalization of the Gaus- T g g fit. The ⇥2/ndf in the region 120-160 GeV/c2 is 10.9/20. sian is free tothe varyR independentlydistribution for the of electron the electron and sample. q q jj p p q The main di⇥erenceq between muons and electrons is the method used to model the QCD contribution: high isolation candidates for muons and antielectrons for Figure 4.5: Feynman diagrams for QCD multijet production. Figure 4.5: Feynman diagramselectrons. for QCD multijet However, production. if we compare the R distribution of antieletrons and high isolation electrons, Fig. 8.19, we observe a significant di⇥erence and, in particular, high isolation electrons seems to behave such that they may cover the disagreement we see in R. Unfortunately, we cannot use high isolation electrons as a default

because they don’t model well other distribution such as the ET and quantities re-

lated to the ET . However, as already discussed in Sec. 8.2.1, high isolation electrons will be used to assess systematics due to the QCD multijet component.

To further prove that ALPGEN is reproducing the Rjj distribution, we have shown in Fig. 8.18 that there is a good agreement between the Z+jets data and ALPGEN SEARCH FOR NEW PARTICLES DECAYING INTO DIJETS ... PHYSICAL REVIEW D 79, 112002 (2009) renormalization and factorization scales to 2 instead of 104 0 -1 3 CDF Run II Data (1.13 fb ) )] 10 0 reduces the cross section prediction by 5%–10%, and 2 102 Fit setting Rsep 2 increases the cross section by & 10%. The ¼ Excited quark ( f = f’ = f = 1) PDF uncertainties estimated from 40 CTEQ6.1 error PDFs 10 s 1 and the ratio of the predictions using MRST2004 [37] and 2 10-1 300 GeV/c

CTEQ6.1 are shown in Fig. 1(b). The PDF uncertainty is [pb/(GeV/c

jj -2 2 the dominant theoretical uncertainty for most of the m 10 500 GeV/c jj -3 10 700 GeV/c2 range. The NLO pQCD predictions for jets clustered from / dm -4 σ 10 900 GeV/c2 partons need to be corrected for nonperturbative under- d 10-5 (a) lying event and hadronization effects. The multiplicative 2 10-6 1100 GeV/c parton-to-hadron-level correction (Cp h) is determined on ! 0.04 a bin-by-bin basis from a ratio of two dijet mass spectra. 0.8 0.02 The numerator is the nominal hadron-level dijet mass 0.6 0 -0.02 spectrum from the PYTHIA Tune A samples, and the de- 0.4 -0.04 200 300 400 500 600 700 nominator is the dijet mass spectrum obtained from jets 0.2 formed from partons before hadronization in a sample -0 simulated with an underlying event turned off. We assign (Data - Fit) / Fit -0.2 (b) the difference between the corrections obtained using -0.4 HERWIG and PYTHIA Tune A as the uncertainty on the 200 400 600 800 1000 1200 1400 2 Cp h correction. The Cp h correction is 1:16 0:08 at mjj [GeV/c ] ! ! Æ low m and 1:02 0:02 at high m . Figure 1 shows the jj Æ jj ratio of the measured spectrum to the NLO pQCD predic- FIG. 2 (color online). (a) The measured dijet mass spectrum Center for Cosmology and (points) fitted to Eq. (2) (dashed curve). The bin-by-bin unfold- The Effectivetions corrected Model Narrative for the nonperturbative effects. TheParticle data Physics ing corrections is not applied. Also shown are the predictions It is commonand to theoreticaldescribe a distribution predictions with are some found parametric to be in good function agree- 2 from the excited quark, qÃ, simulations for masses of 300, 500, ment. To quantify the agreement, we performed a test 2 ‣ “fit backgroundwhich is to the a polynomial”, same as the exponential, one used in the... inclusive jet cross 700, 900, and 1100 GeV=c , respectively (solid curves). (b) The fractional difference between the measured dijet mass distribu- ‣ While thissection is convenient measurements and the [ 15fit ,may17]. be The good, test the treats narrative the system- is weak 3 SEARCH FOR NEW PARTICLES DECAYING INTO DIJETS ... PHYSICAL REVIEW D 79, 112002 (2009) tion and the fit (points) compared to the predictions for qÃ signals 4 atic uncertainties from different sources and uncertainties situ renormalization and factorization scales to 2 instead of 10 dividedinformation. by the fit The to the systematic measured uncertainty dijet on mass the de- spectrum (curves). 0 -1 3 CDF Run II Data (1.13 fb ) )] 10 4 0 reduces the cross section prediction by 5%–10%, and 2 on C as independent but10 fully correlated over all m termination of the background was taken from the uncer- 102 p h Fit Data jj The inset shows the expanded view in which the vertical scale is setting Rsep 2 increases the cross section by & 10%. The ! Events ATLAS tainty on the parameters resulting from the fit of Eqn. 1 ¼ Excited quark2 ( f = f’ = f = 1) Fit PDF uncertainties estimated from 40 CTEQ6.1 error PDFs 10 s bins and yields =no: d:o:f: 3 21=21. q*(500) restrictedto the data to sample.0:04. The uncertainty on σ due to 1 10 s = 7 TeV and the ratio of the predictions using MRST2004 [37] and 2 ·A -1 300 GeV/c ¼ q*(800) -1 integrated luminosityÆ was estimated to be 11% [35]. 10 L dt = 315 nb CTEQ6.1 are shown in Fig. 1(b). The PDF uncertainty is [pb/(GeV/c (1200) ∫ ±

jj q* 10-2 500 GeV/c2 The JER uncertainty was treated as uniform in pT and the dominant theoretical uncertainty for most of the m 2 jj -3 10 jet 10 VI.700 SEARCH GeV/c2 FOR DIJET MASS RESONANCES η with a value of 14% on the fractional pT resolu-

/ dm decays to qg, we set its couplings to the SM SU 2 , U 1 , range. The NLO pQCD predictions for jets clustered from -4 ± σ 10 900 GeV/c2 tion of each jet [36]. The effects of JES, background partons need to be corrected for nonperturbative under- d -5 ð Þ ð Þ 10 (a) 10 and SU 3 gauge groups to be f f f 1 [1], re- 2 fit, integrated luminosity, and JER were incorporated0 s lying event and hadronization effects. The multiplicative We search for1100 GeV/c narrow mass resonances in the measured 10-6 ð Þ ¼ ¼ ¼ parton-to-hadron-level correction (Cp h) is determined on spectively,as nuisance and parameters the compositeness into the likelihood scale function to in the mass of q . ! 0.04dijet mass spectrum by fitting the measured spectrum to a Ã a bin-by-bin basis from a ratio of two dijet mass spectra. 0.8 0.02 1 Eqn. 2 and then marginalized by numerically integrating The numerator is the nominal hadron-level dijet mass 0.6 smooth0 functional form and by looking for data points that Forthe the product RS graviton of this modifiedGÃ that likelihood, decays the prior into in qs,q or gg, we use spectrum from the PYTHIA Tune A samples, and the de- -0.02 0.4 -0.04 and the priors corresponding to the nuisance parameters 200 300 400 500 600 700 B 500 1000 1500 the model parameter k=MPl 0:1 which determines the nominator is the dijet mass spectrum obtained from jets 0.2 show significant excess from the2 fit. We fit the measured to arrive at a modified posterior probability distribution. formed from partons before hadronization in a sample ¼ -0 0 couplings of the graviton to the SM particles. The produc- simulated with an underlying event turned off. We assign dijet mass spectrum before the bin-by-bin unfolding cor- In the course of applying this convolution technique, the (Data - Fit) / Fit -0.2 (b) JER was found to make a negligible contribution to the the difference between the corrections obtained using -0.4 -2 rection is applied. We use the(D - B) / following functional form: tion cross section increases with increasing k=MPl; how- HERWIG and PYTHIA Tune A as the uncertainty on the 200 400 600 800 1000 1200 1400 500 1000 1500 overall systematic uncertainty. 2 jj ever, values of k=M 0:1 are disfavored theoretically Cp h correction. The Cp h correction is 1:16 0:08 at mjj [GeV/c ] Reconstructed m [GeV] Figure 2 depicts the resultingPl 95% CL upper limits on ! ! Æ ∗ low mjj and 1:02 0:02 at high mjj. Figure 1 shows the d σ as a function of the q resonance) mass after incorpo- Æ p1 p2 p3 ln x [38]. For W0 and Z0, which decay to qq0 and qq respec- ratio of the measured spectrum to the NLO pQCD predic- FIG. 2 (color online). (a) The measuredp0 dijet1 massx spectrum=x þ Á ð Þ;xmjj=ps; (2) ration·A of systematic uncertainties. Linear interpolations (points) fitted to Eq.dm (2) (dashed curve). The bin-by-bin unfold- tions corrected for the nonperturbative effects. The data jj ¼ ð À Þ FIG. 1. The data (D) dijet¼ mass distribution (filled points)tively,between we test use masses the were SM used couplings. to determine The where leading-order the pro- and theoretical predictions are found to be in good agree- ing corrections is not applied. Also shown are the predictionsfitted using a binned background (B) distribution described Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 ∗ffiffiffi 49 experimental bound intersected with a theoretical pre- ment. To quantify the agreement, we performed a 2 test from the excited quark, qÃ, simulations for masses of 300, 500,by Eqn. 1 (histogram). The predicted q signals [2, 3] forduction cross sections of the RS graviton, W0, and Z0 are 700, 900, and 1100 GeV=c2, respectively (solid curves). (b) The which is the same as the one used in the inclusive jet cross where p0, p1, p2, and p3excited-quarkare free parameters. masses of 500, 800, This and form 1200 GeV fits are over- diction to yield a lower limit on allowed mass. The cor- fractional difference between the measured dijet mass distribu- multiplied by a factor of 1.3 to account for higher-order section measurements [15,17]. The test treats the system- laid, and the bin-by-bin significance of the data-background responding observed 95% CL excited-quark mass exclu- tion and the fit (points)well compared the dijet to the predictions mass for spectraqÃ signals from PYTHIA, HERWIG, and NLO ∗ atic uncertainties from different sources and uncertainties divided by the fit to the measured dijet mass spectrum (curves).difference is shown. effectssion region in the was strong found to couplingbe 0.30 proton-proton collisions produced by the LHC and The jet calibration relied on the MC∼ simulation of the recorded by the ATLAS detector. No evidence of a res- response of the ATLAS detector; its uncertainty was con- onance structure was found and upper limits at the 95% strained by varying the ATLAS simulation and from in CL were set on the products of cross section and signal HIGGS –SEARCH FOR THE STANDARD MODEL H ZZ 4l ⇥ ⇥

Zbb¯ ZZ H Zbb¯ ZZ H Zbb¯ ZZ H 4e 4µ 2e2µ Scale +0.5% (+1%) +1.5 +0.1 +0.9 +2.4 +0.4 +1.3 +1.9 +0.1 +0.9 Scale -0.5% (-1%) -1.1 -0.2 -0.5 -2.3 -0.3 -2.5 -1.7 -0.2 -1.4 Resolution -0.5 -0.1 -0.4 +0.1 -0.1 -2.6 -0.2 -0.1 -0.5 Rec. efﬁciency -1.0 -0.7 -0.5 -3.8 -4.0 -3.8 -2.0 -2.1 -1.7 Luminosity 3 3 3 Total 3.6 3.1 3.2 5.4 5.0 6.0 4.1 3.7 3.8

Table 13: Impact, in %, of the systematic uncertainties on the overall selection efﬁciency, as obtained for a mH = 130 GeVin the 4e,4µ, and 2e2µ ﬁnal states.

HIGGS –SEARCH FOR THE STANDARD MODEL H ZZ 4l the results obtained⇥ after varying⇥ the background shape, the fit range and even including background- only fits without the signal hypothesis, are the main subject of this section. An alternative approach with a global two-dimensional (2D) fit on the (mZ⇥ ,m4⇥) plane is also briefly discussed.

The first plateau, in the region where onlyThe one baseline of the method two Z usedbosons as input is toon the shell combination, is modelled of all ATLAS by the Standard first Model Higgs boson searches, is based on a fit of the 4-lepton invariant mass distribution over the full range from 110 to term, and its suppression, needed for a700 correct GeV. Thedescription method, whose at higher details are masses, described is incontrolled [21], is summarized by the below.p8 and p9 parameters. The second term in the aboveThe 4-lepton formula reconstructed accounts invariant for the mass shape after of the the full broad event selectionpeak and (except the the final cut on the tail at high masses. This function can describe4-lepton reconstructed with a negligible mass) is bias used asthe a discriminatingZZ background variable shape to construct with good a likelihood function. The accuracy over the full mass range. ThelikelihoodZbb¯ contribution is calculated is on relevant the basis ofto parametric the background forms of signal shape and only background when probability density functions (pdf) determined from the MC. For a given set of data, the likelihood is a function of the searching for very light Higgs boson (inpdf this parameters study,⇤p onlyand of at anm additionalH = 120 parameter GeV). Inµ thisdefined case, as the an ratio additional of the signal cross-section to term is added to the ZZ continuum, withthe a Standard functional Model form expectation similar (i.e. toµ the= 0 second means no part signal, of and equationµ = 1 corresponds 2. For to the signal rate expected for the Standard Model). To test a hypothesized value of µ the following likelihood ratio is the signal modelling a simple gaussian shape has been used for mH 300 GeV, while a relativistic Breit- constructed: ⇤ Wigner formula is needed at higher values of the Higgs boson mass. In Figs. 34 andˆ 35 two examples ofCenter for L(µ,⇤p) Cosmology and The Effective Model Narrative(µ)= Particle Physics (1) pseudo-experiments with the resulting fit functions for signal and background areL(µ shown.ˆ ,⇤pˆ) where ⇤pˆ is the set of pdf parameters that maximize the likelihood L for the analysed dataset and for a fixed value of µ (conditional Maximum Likelihood Estimators), and (µˆ ,⇤pˆ) are the values of µ and 30 50 ⇤p that maximise the likelihood function for the same dataset (Maximum Likelihood Estimators). The ATLAS H→ ZZ → 4l 25 profile likelihood ratio is used to reject the background only hypothesisATLAS (µ = 0) in the case of discovery, 40 -1 and L = the 30 signal+background fb hypothesis in the case of exclusion. The testsimulation statistic used is qµ = 2ln(µ), ∫ -1 Events / ( 4.2 ) 20 and the median discoveryEvents / ( 4.2 ) significance and limits are approximated using L = expected 30 fb signal and background 30 ∫ distributions, for different mH , luminosities and signal strength µ. The~2007 MC distributions, with the content 15 and error of each bin reweighted to a given luminosity (in the following referred to as “Asimov data”, see [21]), are fitted to derive20 the pdf parameters: in the fit, m is fixed to its true value, while ⇤ is allowed 10 H H to float in a 20% range around the value obtained from the signal MC distributions. All parameters ± 10 5 describing the background shape are floating within sensible ranges. The irreducible background has been modelled using a combination of Fermi functions which are suitable to describe both the plateau 0 0 100 150 200 250 300 350 400in the450 low mass500 region and100 the broad150 peak200 corresponding250 300 to350 the second400 Z450coming500on shell. The chosen modelm4l is [GeV] described by the following function: m4l [GeV] p0 p1 f (mZZ)= p6 m m p8 + p2 m p4 m (2) ZZ ZZ ZZ ZZ Figure 34: A pseudo-experiment corresponding to Figure 35:(1 + Ae pseudo-experimentp7 )(1 + e p9 ) (1 + correspondinge p3 )(1 + e p5 to) 1 1 30 fb of data for a Higgs boson mass of 130 GeV. 30 fb of data for a Higgs boson mass of 180 GeV. Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 50 22 The functions fitting the signal and the background The functions fitting the1264 signal and the background are shown. are shown. 68 The results presented in the following approximate the significance from the test statistics as 2ln⇥(µ). In order for the results of the method to be valid, the test statistic q = 2ln⇥(µ) should be distributed µ as a 2 with one degree of freedom. The results obtained with the strategy described above must thus be validated using toy MC. Such validation tests show a good agreement of the test statistic with the expected 2 distribution, as discussed in detail in [21]. This allows to approximate the significance from the test statistic as 2ln⇥(µ). The significances obtained as the square root of the median profile likelihood ratios for discovery, 2ln⇥(µ = 0) are shown in Table 14 for all m values considered in this H paper, and for various luminosities. In Fig. 36, the significance obtained from the profile likelihood ratio, after the fit of signal+background is shown. The significance is compared to the Poisson significance shown in Section 4. The slightly reduced discovery potential is due to the fact that several background shape and normalization parameters are derived from the data-like sample. Concerning exclusion, the median profile likelihood ratios are calculated under the background only hypothesis, and the integrated luminosity needed to exclude the signal at 95% C.L. is the one corresponding to ⌅ 2ln⇥=1.64. The integrated luminosity needed for exclusion is shown in Fig. 37. The median significance estimation with Asimov data can be validated using toy MC pseudo-experiments. For each mass point, 3000 background-only pseudo-experiments are generated. For each experiment, the profile likelihood ratio method is used to find which µ value can be excluded at 95% CL. The resulting distributions are then analysed to find the median and 1⌅ and 2⌅ intervals. The ± ± outcome of this test is summarized in Fig. 38, where the 95% CL exclusion µ obtained from single fits on the full MC datasets is plotted as well. As shown, the agreement is good over the full mass range. Fitting the 4-lepton mass distribution with all parameters left free in function 2 allows the fit to absorb

23 1265 69 680 where the ai are the parameters used to parameterize the fake-tau background and ν represents all nui- Center for 681 sance parameters of the model: ,m , ,r ,a ,a ,a . When using the alternateCosmology parameterization and The Effective ModelσH ZNarrativeσZ QCD 1 2 3 Particle Physics 682 of the signal, the exact form of Equation 14 is modified to coincide with parameters of that model. 683 FigureSometimes 14 shows the the fitef toective the signal model candidates comes for a frommH = 120a convincing GeV Higg with narrative (a,c) and without 684 (b,d) the‣ signalconvolution contribution. of detector It can be seen resolution that the background with known shapes distribution and normalizations are trying to 685 accommodate the excess near mττ = 120 GeV, but the control samples are constraining the variation. ● Ex: MissingET resolution propagated through Mττ in collinear approximation 686 Table 13 shows the significance calculated from the profile likelihood ratio for the ll-channel, the lh- ● Ex: lepton resolution convoluted with triangular Mll distribution 687 channel, and the combined fit for various Higgs boson masses. !"#"$%&'()*+'*%,#-.)$"*' % % $ $0 14 14 $ " $0 $ ! 1 ATLAS # ATLAS! 2 ! " 12 VBF H(120)→ ττ →lh 12 "VBF H(120)! → ττ →!lh s = 14 TeV, 30 fb-1 s = 14 TeV, 30 fb-1 10 ! /%**%#,'"#"$,+0'*.-12"'3$'4)-*%'*.-12"'5(610 2 / ndf 40.11 / 45 -1 ! Prob 0.679 Events / ( 5 GeV ) 7%,8'9".'3$'2":.3#';)2.%:2%&%.+0'7"-<+'319"&.'="&-+%#,'%#'="."&.3$Events / ( 5 GeV ) 8 8 50 Endpoint 99.66 ± 1.399 Norm. -0.3882 ± 0.02563 ! >%#";-.%&'*.$)&.)$"'?$3;'&-*&-="'="&-+*0'3#2+'%?'*:"&.$);'%*'40 Smearing 2.273 ± 1.339 6 @&3;:-&.A0'B""=*'23#,'&8-%#*'.3'C3$D'C"220';-#+'EF'%#<32<"=6 30

! 4 /3$"';3="2'=":"#="#.'G"H-;:2"*I4 20

Entries/4 GeV/ Entries/4 1 fb ATLAS " 53#,'2%<"='319"&.*10 2 2 ! (2":.3#*'3$',-),%#3*'%#'!/(E0

0 ! !2)%#3*'%#'*:2%.'(J(K0-10 60 80 100 120 140 160 180 60020406080100120140160180200 80 100 120 140 160 180 M (GeV) M (GeV) ττ Helmholtzm(ll)ττ [GeV]Ian Hinchliffe 12/1/07 28

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 51 (a) (b)

14 14 ATLAS ATLAS 12 VBF H(120)→ ττ →ll 12 VBF H(120)→ ττ →ll s = 14 TeV, 30 fb-1 s = 14 TeV, 30 fb-1 10 10

Events / ( 5 GeV ) 8 Events / ( 5 GeV ) 8

6 6

4 4

2 2

0 0 60 80 100 120 140 160 180 60 80 100 120 140 160 180 M ττ (GeV) M ττ (GeV)

Figure 14: Example ﬁts to a data sample with the signal-plus-background (a,c) and background only 1 (b,d) models for the lh- and ll-channels at mH = 120 GeV with 30 fb− of data. Not shown are the control samples that were ﬁt simultaneously to constrain the background shape. These samples do not include pileup.

27 Center for Cosmology and Tools for building effective models Particle Physics • RooFit’s convolution PDFs can aid in building more effective models with a more convincing narrative

// Construct landau (x) gauss (10000 samplings 2nd order interpolation) t.setBins(10000,”cache”) ; RooFFTConvPdf lxg("lxg","landau (X) gauss",t,landau,gauss,2) ;

–

Wouter Verkerke, NIKHEF Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 52 Center for Cosmology and The parametrized response narrative Particle Physics Calculation The Matrix-ElementPrediction via Mo techniquente Carlo Simula is tconceptuallyion similar to the simulation narrative, but the detector response is parametrized. For each event, calculate differential cross-section: ‣ Doesn’tThe eno requirermous detbuildingectors aparametrizedre still being co PDFnstruct byed, interpolatingbut we have detbetweenailed non- parametricsimulations templates.of the detect ors response.

+ W µ L(x H ) =Phase-space H | 0 Transfer Integral Functions! W Matrixµ− Element Data 20 example events…

Measurement! Only partial information available Fix measured quantities L = 350 pb-1 The advancements in theoretical predictions, detector simulation, tracking, Phys. Rev. Lett 96, 152002 (2006) Integrate over unmeasured parton quantities Phys. Rev. D Accepted (2006) calorimetry, triggering, and computing set the bar high for equivalent Thesis, A. Kovalev Penn (2005) L= 1000 pb-1 Thesis, B. Jayatilaka Michigan, 2006 advances in our statistical treatment of the data. Phys. Rev. Lett, In preparation November 8, 2006 Daniel Whiteson/Penn

September 13, 2005 Statistical Challenges of the LHC (page 6) Kyle Cranmer PhyStat2005, Oxford Brookhaven National Laboratory

M = 164.5 ± 3.9 ± 3.9 GeV/c2 Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 t stat syst 53 November 8, 2006 Daniel Whiteson/Penn

November 8, 2006 Daniel Whiteson/Penn Center for Cosmology and Homework / Discussion Problem Particle Physics Imagine a dataset with with observable m with range [0,10]

‣ ~100 events uniformly distributed in m fb(m)=1/10 ‣ ~30 events Gaussian distributed with µ= 5, σ=1f (m)=G(m µ, ) s |

Recall our “marked Poisson”, where s,b are model parameters n sf (m )+bf (m ) P (m s) = Pois(n s + b) s j b j | | s + b j Y ‣ What will be the “best fit”, maximum likelihood estimates of s,b? ‣ What is the role of the Poisson term here? Now imagine that we have a theory that predicts s/b=0.5. How do things change? Now imagine that we have a theory where µ is a free parameter, but the signal rate is given by s(µ) = 10*µ. How do things change?

Kyle Cranmer (NYU) CLASHEP, Peru, March 2013 54