Dissertation submitted to the Combined Faculties for the Natural Sciences and Mathematics of the Ruperto-Carola University of Heidelberg, Germany for the degree of Doctor of Natural Sciences Put forward by Alexander Helmboldt Born in Mannheim, Germany Oral examination: 19.07.2018 New insights into modified scalar sectors and exotic Higgs decays Referees: Prof. Dr. Dr. h.c. Manfred Lindner Prof. Dr. Tilman Plehn New insights into modified scalar sectors and exotic Higgs decays In the context of models with modified Higgs sectors, we examine different possible causes for a systematic absence of new physics from current searches at the Large Hadron Collider (LHC). First, having in mind the lack of clear signs for supersymmetry, we consider theo- ries based on classical scale invariance as an alternative approach towards addressing the gauge hierarchy problem. Within the aforementioned framework, we present a systematic analysis of simple Standard Model (SM) extensions. As a result, we identify the minimal model, which allows for radiative electroweak symmetry breaking, is consistent with all current experimental constraints and does not reintroduce any fine-tuning issues. Next, given that very light scalar particles as predicted by several well-motivated beyond-the- SM theories have not yet been detected either, we propose a novel search strategy based on exotic Higgs decays into three of these light scalars. We show that said processes can be abundant enough to be observable at the LHC or future lepton colliders in exciting new signatures containing six SM fermions in the final state. In particular, we demon- strate the three-body channel's capabilities in probing scenarios which are inaccessible to existing searches exclusively relying on the related two-body Higgs decay. Our last study is motivated by the fact that new physics may manifest itself only through effects which are out of the LHC's reach due to the large amounts of hadronic background. As an example, we consider quark-flavor-violating Higgs decays involving bottom quarks and evaluate the expected sensitivity of direct searches at the proposed International Linear Collider (ILC). In doing so, we identify hadronic SM Higgs decays with mistagged jets as the dominant background processes and find that a dedicated analysis is anticipated to explore branching ratios down to the per-mill level. Neue Erkenntnisse uber¨ modifizierte Higgs-Sektoren und exotische Higgs Zerf¨alle Im Rahmen von Modellen mit modifizierten Higgs-Sektoren untersuchen wir verschiedene m¨ogliche Ursachen fur¨ eine systematische Abwesenheit neuer Physik von entsprechenden Suchen am Large Hadron Collider (LHC). Angesichts des Fehlens klarer Hinweise auf Su- persymmetrie betrachten wir als Erstes klassisch skaleninvariante Theorien, die einen al- ternativen Zugang zur L¨osung des Hierarchieproblems darstellen. Innerhalb solcher Theo- rien fuhren¨ wir eine systematische Studie einfacher Standardmodell (SM) Erweiterungen durch. Somit k¨onnen wir das minimale Modell identifizieren, welches radiative elektro- schwache Symmetriebrechung erlaubt und vereinbar mit allen experimentellen Schranken ist, ohne dabei ein Feinabstimmungsproblem wiedereinzufuhren.¨ Auch leichte skalare Teil- chen, deren Existenz von etlichen gut motivierten Modellen jenseits des SM vorhergesagt wird, konnten bisher nicht gefunden werden. Daher schlagen wir als N¨achstes eine neue Suchstrategie vor, die auf exotischen Higgszerf¨allen in drei dieser leichten Skalare ba- siert. Wir finden, dass besagte Prozesse h¨aufig genug vorkommen k¨onnen, um am LHC oder zukunftigen¨ Leptonbeschleunigern in neuen, spektakul¨aren Signaturen mit sechs SM Fermionen im Endzustand beobachtet zu werden. Insbesondere zeigen wir, dass der Drei- K¨orper-Zerfall Szenarien testen kann, die fur¨ bestehende Suchen, welche ausschließlich auf dem verwandten Zwei-K¨orper-Zerfall beruhen, unzug¨anglich sind. Unsere letzte Stu- die ist dadurch motiviert, dass neue Physik sich m¨oglicherweise nur durch solche Signa- turen manifestiert, welche wegen des vielen hadronischen Untergrundes am LHC nicht nachweisbar sind. Als Beispiel betrachten wir Quark-Flavor-verletzende Higgszerf¨alle mit Bottom-Quarks und bestimmen die erwartete Sensitivit¨at direkter Suchen am geplanten International Linear Collider (ILC). Dabei erkennen wir hadronische SM Higgszerf¨alle mit falsch identifizierten Jets als die dominanten Untergrundprozesse und finden, dass eine ge- eignete Analyse es erlauben wird, Verzweigungsverh¨altnisse im Promillebereich zu prufen.¨ v Contents 1 Introduction and motivation 1 2 The Standard Model and beyond 7 2.1 The Standard Model in a nutshell ............... 8 2.2 Electroweak symmetry breaking . 13 2.3 The gauge hierarchy problem . 17 3 Minimal conformal extensions of the Higgs sector 21 3.1 Basics of scale-invariant model building . 24 3.2 The Gildener-Weinberg formalism . 27 3.3 Finding the minimal conformal model . 30 3.3.1 SM + one scalar representation . 30 A Real multiplet with vanishing vev . 31 B Real multiplet with finite vev . 36 C Complex multiplet with vanishing vev . 42 D Complex multiplet with finite vev . 46 3.3.2 SM + one scalar and one fermionic representation . 49 3.3.3 SM + two scalar representations . 49 A Two real multiplets with vanishing vev . 50 B The minimal conformal model . 53 C Non-minimal conformal models . 59 3.4 Summary and conclusion .................... 61 4 Three-body Higgs boson decays into extra light scalars 63 4.1 Higgs decays into scalars { a model-independent analysis . 66 4.1.1 Model-independent constraints . 68 A Experimental constraints . 68 B Theoretical constraints . 70 4.1.2 Model-independent results . 72 4.2 Scalar singlet-extension of the Standard Model . 75 4.2.1 Constraints on the parameter space . 79 A Experimental constraints . 79 B Theoretical constraints . 80 4.2.2 Decay properties of the light scalar . 81 vii viii Contents 4.3 Scalar Higgs decays at colliders . 84 4.3.1 Low-mass regime ..................... 84 4.3.2 Intermediate-mass regime . 90 4.3.3 High-mass regime ..................... 91 4.4 Summary and conclusion .................... 95 5 Quark-flavor-violating Higgs boson decays at the ILC 99 5.1 The ILC and Higgs physics . 101 5.2 Theoretical framework and indirect constraints . 105 5.3 Analysis and results . 109 5.3.1 General aspects of the analyses . 109 5.3.2 Hadronic channel at 250 GeV . 112 5.3.3 Leptonic channels . 116 A Estimated performance . 116 B Charged lepton channel at 250 GeV . 119 C Neutrino channel at 250 GeV and 500 GeV . 122 5.4 Summary and conclusion . 128 6 Final conclusion and outlook 131 Appendix A Calculating the Gildener-Weinberg scale 137 B One-loop renormalization group equations 141 B.1 SM + real scalar representation(s) . 142 B.2 SM + one complex scalar representation . 143 C Phase space threshold functions 145 D Tree-level perturbative unitarity 147 E Statistical methods 151 E.1 Setting upper limits . 151 E.2 Discovering signals . 154 List of Abbreviations 157 List of Figures 159 List of Tables 161 Bibliography 163 Chapter 1 Introduction and motivation July 4th 2012 will go down in history as one of the most memorable days for particle physicists during the last decades. The seminal discovery of the Higgs boson at the Large Hadron Collider (LHC) [1, 2], which was announced on that Wednesday, marked the long-awaited experimental ob- servation of the last missing piece of the Standard Model (SM) of particle physics, and thus the culmination of years of extensive effort to track down a particle which was already predicted in the 1960s, and which is of immense importance for high-energy physics. Whereas, for instance, a third genera- tion of quarks and leptons seems expendable from a purely formal point of view, the existence of the Higgs field is absolutely crucial for writing down a consistent quantum field theory describing the fundamental interactions of Nature, barring gravity [3{5]. Although the minimal SM has thus been completed from a theoretical perspective, it still leaves many fundamental questions of modern physics unanswered. For example, while the Standard Model as it stands accurately captures phenomena related to the electromagnetic, weak and strong forces, a con- sistent quantum theory of gravity could not be incorporated so far. It is furthermore still unclear if and how dark matter and dark energy, which after all make up more than 95 % of the universe's energy density [6, 7], may be accounted for within the framework of particle physics. But also the origins of phenomena whose ties to elementary particles are more obvious remain unresolved. Thus, for instance, the actual mechanisms behind mas- sive neutrinos or the baryon asymmetry of the universe are yet unknown. Driven by this large variety of open problems, the field of beyond-the-SM (BSM) model building has long pursued, and still pursues, a plethora of different and ever new directions with the aim of including one or several of the aforementioned phenomena in a consistent theory of particle physics. Interestingly, many of the existing approaches involve postulating further scalar bosons, or otherwise modifying the Higgs sector of the minimal SM 1 2 Chapter 1. Introduction and motivation (see e.g. [8] for a recent comprehensive review).a The question of whether or not the particle found by the ATLAS and CMS experiments is indeed precisely that predicted by the Standard Model is therefore one of the most pressing questions of contemporary high-energy physics. One additional reason to believe that the SM scalar sector is incomplete and requires modification stems from the very nature of said sector itself. Specifically, the SM Higgs being an elementary spinless boson implies the emergence of the infamous gauge hierarchy problem [9{14] as soon as the Standard Model is embedded in a more fundamental quantum field theory involving additional heavy particles. Although as a question about natu- ralness the hierarchy problem poses more of an aesthetic issue rather than a theoretical inconsistency, the quest for its solution has crucially shaped model building and the search for new physics over the last decades.