Eur. Phys. J. C (2018) 78:306 https://doi.org/10.1140/epjc/s10052-018-5777-1

Regular Article - Theoretical Physics

B- anomalies and Higgs physics in flavored U(1) model

Ligong Bian1,2,a, Hyun Min Lee2,b , Chan Beom Park3,4,c 1 Department of Physics, Chongqing University, Chongqing 401331, China 2 Department of Physics, Chung-Ang University, Seoul 06974, Korea 3 Center for Theoretical Physics of the Universe, Institute for Basic Science (IBS), Daejeon 34051, Korea 4 School of Physics, Korea Institute for Advanced Study, Seoul 02455, Korea

Received: 29 January 2018 / Accepted: 3 April 2018 / Published online: 18 April 2018 © The Author(s) 2018

Abstract We consider a simple extension of the Standard Recently, there have been interesting reports on the Model with flavor-dependent U(1), that has been proposed anomalies in rare semileptonic B-meson decays at LHCb ∗  to explain some of B-meson anomalies recently reported at such as RK [1], RK [2Ð4], P5 [5,6]. The reported value of  + − + − LHCb. The U(1) charge is chosen as a linear combination RK = B(B → K μ μ )/B(B → Ke e ) is of anomaly-free B3 − L3 and Lμ − Lτ . In this model, the fla- = . +0.097, 2 < 2 < 2, vor structure in the SM is restricted due to flavor-dependent RK 0 745−0.082 1GeV q 6GeV (1.1) U(1) charges, in particular, mixings are induced by a small vacuum expectation value of the extra Higgs dou- which deviates from the SM prediction by 2.6σ . On the other ∗ + − blet. As a result, it is natural to get sizable flavor-violating hand for vector B-, RK ∗ = B(B → K μ μ )/B ∗ + − Yukawa couplings of heavy Higgs involving the bot- (B → K e e ) is tom quark. In this article, we focus on the phenomenology + . ∗ = . 0 11( ) ± . ( ), of the Higgs sector of the model including extra Higgs dou- RK 0 66−0.07 stat 0 03 syst blet and singlet scalars. We impose various bounds on the 0.045 GeV2 < q2 < 1.1GeV2, extended Higgs sector from Higgs and electroweak preci- + . R ∗ = . 0 11( ) ± . ( ), sion data, B-meson mixings and decays as well as unitarity K 0 69−0.07 stat 0 05 syst and stability bounds, then discuss the productions and decays 1.1GeV2 < q2 < 6.0GeV2, (1.2) of heavy Higgs bosons at the LHC. which again differs from the SM prediction by 2.1Ð2.3σ and 2.4Ð2.5σ , depending on the energy bins. Explaining the B- 1 Introduction meson anomalies would require new physics violating the flavor universality at a few 100 GeV up to a few The observed masses and mixing angles are well 10 TeV, depending on the coupling strength of new parti- parametrized by the Higgs Yukawa couplings in the Stan- cles to the SM. We also note that there have been inter- (∗) dard Model (SM). However, the masses and mixing esting anomalies in B → D τν decays, the so called = B( → (∗)τν)/B( → (∗)ν)  = angles call for the addition of right-handed (RH) or RD(∗) B D B D with e, physics beyond the SM and, moreover, the flavor structures μ, whose experimental values are deviated from the SM val- of and are not understood yet. As there is no ues by more than 2σ [7Ð11]. flavor changing neutral current at tree level in the SM due Motivated by the B-anomalies RK (∗) , some of the authors to the GIM mechanism, the observation of flavor violation recently proposed a simple extension of the SM with extra  is an important probe of new physics up to very high energy U(1) gauge symmetry with flavor-dependent couplings [12].  scales and it can be complementary to direct searches at the The U(1) symmetry is taken as a linear combination of ( ) ( ) LHC. In particular, the violation of lepton flavor universality U 1 Lμ−Lτ and U 1 B3−L3 , which might be a good symme- would be a strong hint at new physics. try at low energy and originated from enhanced gauge sym- metries such as in the U(1) clockwork framework [13]. In a e-mail: [email protected] this model, the quark mixings and neutrino masses/mixings b e-mail: [email protected] require an extended Higgs sector, which has one extra Higgs c e-mail: [email protected] doublet and multiple singlet scalars beyond the SM. As 123 306 Page 2 of 25 Eur. Phys. J. C (2018) 78 :306

Table 1 U(1) charges of and scalars

q3L u3R d3R 2L e2R ν2R 3L e3R ν3R

 1 1 1 − − − − − − Q 3 x 3 x 3 xyy y x y x y x y

SH1 H2 1 2 3

 1 − 1 − + Q 3 x 0 3 x yxyx a result, nonzero off-diagonal components of quark mass for real parameters x and y [12].1 Introducing two Higgs matrices are obtained from the vacuum expectation value doublets H1,2 is necessary to have right quark masses and (VEV) of the extra Higgs doublet and correct electroweak mixings. We add one complex singlet scalar S for a correct symmetry breaking is ensured by the VEV of one of the sin- vacuum to break electroweak symmetry and U(1). More- glet scalars. over, in order to cancel the anomalies, the fermion sector is In this paper, we study the phenomenology of the heavy required to include at least two RH neutrinos νiR (i = 2, 3).   Higgs bosons in the flavored U(1) model mentioned above. One more RH neutrino ν1R with zero U(1) charge as well as  We first show that the correct flavor structure of the SM is extra singlet scalars, a (a = 1, 2, 3), with U(1) charges well reproduced in the presence of the VEV of the extra of −y, x + y, x, respectively, are also necessary for neutrino Higgs doublet. In particular, in the case with a small VEV masses and mixings. As Lμ − Lτ is extended to RH neutri- of the extra Higgs doublet or small tan β, we find that the nos, Lμ − Lτ and L2 − L3 can be used interchangeably in our heavy Higgs bosons have sizable flavor-violating couplings model. The U(1) charge assignments are given in Table 1. to the and reduced flavor-conserving Yukawa The Lagrangian of the model is given as couplings to the such that LHC searches for heavy Higgs bosons can be affected by extra or modified production 1  μν 1  μν L =− Zμν Z − sin ξ Zμν B + LS + LY (2.1) and decay channels. We also briefly mention the implication 4 2 of our extended Higgs sector for R (∗) anomalies. We discuss D with various constraints on the extended Higgs sector from Higgs and electroweak precision data, flavor data such as the B- 3 meson mixings and decays, as well as unitarity and stability 2 2 2 LS =|Dμ H1| +|Dμ H2| +|Dμ S| + |Dμa|−V (φi ), bounds. For certain benchmark points that can evade such a=1 bounds, we study the productions and decays of the heavy (2.2) Higgs bosons at the LHC and show distinct features of the     model with flavor-violating interactions in the Higgs sector. where Zμν = ∂μ Zν − ∂ν Zμ is the field strength of the U(1) This paper is organized as follows. First, we begin with a gauge , sin ξ is the gauge kinetic mixing between U(1) summary of the U(1) model with the extended Higgs sector   and SM hypercharge, and Dμφi = (∂μ − igZ  Qφ Zμ)φi are and new interactions. The Higgs spectrum and Yukawa cou-   i covariant derivatives. Here Qφ is the U(1) charge of φi , plings for heavy Higgs bosons are presented in Sect. 3.We i gZ  is the extra gauge coupling. The scalar potential V (φi ) then discuss various theoretical and phenomenological con- is given by V = V1 + V2 with straints on the Higgs sector are studied in Sect. 4, and collider signatures of the heavy Higgs bosons at the LHC are stud- = μ2| |2 + μ2| |2 − (μ † + . .) V1 1 H1 2 H2 SH1 H2 h c ied in Sect. 5. Finally, conclusions are drawn. There are four + λ |H |4 + λ |H |4 + λ |H |2|H |2 appendices dealing with the extended Higgs sector, unitarity 1 1 2 2 2 3 1 2 ( ) + λ ( † )( † ) bounds, quark Yukawa couplings, and the U 1 interactions. 2 4 H1 H2 H2 H1 + | |2(κ | |2 + κ | |2) + 2 | |2 + λ | |4, 2 S 1 H1 2 H2 mS S S S 2FlavoredU(1) model (2.3) 3 2 2 4  V = (μ | | + λ | | ) We consider a simple extension of the SM with U(1) , where 2 a i a a a new Z  couples specifically to heavy fla- a=1 ( ) vors. It is taken as a linear combination of U 1 Lμ−Lτ and ( ) 1 We note that we can take two independent parameters for the Z  U 1 B3−L3 with couplings to be either (xgZ  , ygZ  ) or (x/y, gZ  ) by absorbing y into  g  . Our following discussion does not depend on the choice of the Z  Z Q ≡ y(Lμ − Lτ ) + x(B3 − L3) couplings. 123 Eur. Phys. J. C (2018) 78 :306 Page 3 of 25 306

+ (λ 3† + μ   † + . .) S3 S 3 4 1 2 3 h c Since the mass matrix for charged leptons is already diago- 3 nal, the lepton mixings come from the mass matrix of RH 2 2 2 2 + 2 |a| (βa1|H1| + βa2|H2| + βa3|S| ) neutrinos. There are four other categories of neutrino mixing = matrices [14,15], that are compatible with neutrino data. In a 1 2 2 all the cases, we need at least three complex scalar fields with + 2 λab|a| |b| . (2.4) U( ) a

˜ ≡ σ ∗ with with H1,2 i 2 H1,2. After electroweak symmetry and  U(1)√are broken by the VEVs of scalar fields, H1,2=√ 2 2 2 2 2 2 1 2 2 2 2 2 2 2 2 v1,2/ 2 with v +√v = v = (246 GeV) , S=vs/ 2 m  = g  x v + y ω + (x + y) ω + x ω . 1 2 Z Z 9 s 1 2 3 and a=ωa/ 2, the quark and lepton mass terms are given as (2.14)

Here sϕ ≡ sin ϕ, cϕ ≡ cos ϕ, and tϕ ≡ tan ϕ. The modified ¯ ¯ ¯ c LY =−¯uMuu − dMd d − M − MDνR − (νR) MRνR + h.c. Z boson mass can receive constraints from electroweak pre- (2.6) cision data, which is studied in Sect. 4. We note that for a small mass mixing, the Z -like mass is approximately given 2 ≈ 2   with the following flavor structure: by m Z m Z  and we can treat m Z and gZ to be inde- 2 ω ⎛ ⎞ pendent parameters due to the presence of nonzero i ’s. The u  ˜  u  ˜  ( ) y11 H1 y12 H1 0 U 1 interactions are collected in Appendix D. ⎜ u  ˜  u  ˜  ⎟ Mu = ⎝y21 H1 y22 H1 0 ⎠ , (2.7) u ˜ u ˜ u ˜ h H2 h H2 y H1 31 32 33 3 Higgs spectrum and Yukawa couplings ⎛ ⎞ d   d   d   y11 H1 y12 H1 h13 H2 ⎜ d d d ⎟ We here specify the Higgs spectrum of our model and Md = ⎝y H1 y H1 h H2⎠ , (2.8) 21 22 23 identify the quark and lepton Yukawa couplings of neutral d   00y33 H1 and charged Higgs bosons for studies in next sections. The ⎛ ⎞    expressions are based on results in Appendices A and C . y11 H1 00 ⎜  ⎟ M = ⎝ 0 y H1 0 ⎠ , (2.9) 22 3.1 The Higgs spectrum 00y H  ⎛ 33 1 ⎞ yν H˜  00 The Higgs sector of our model has two Higgs doublets, which ⎜ 11 1 ⎟ = ⎜ ν  ˜  ⎟ , are expressed in components as MD ⎝ 0 y22 H1 0 ⎠ (2.10) ν ˜ + 00y H1 φ ⎛ 33 ⎞ = j √ ( = , ), (1) (2) H j j 1 2 (3.1) M z   z   (v j + ρ j + iη j )/ 2 ⎜ 11 12 1 13 2 ⎟ = ⎜ (1) (3) ⎟ . MR ⎝z 1 0 z 3⎠ (2.11) 21 23 and the complex√ singlet scalar decomposed into S = (2)  (3)  (v + S + iS ) / 2. z31 2 z32 3 0 s R I 123 306 Page 4 of 25 Eur. Phys. J. C (2018) 78 :306

In the limit of negligible mixing with the CP-even sin- Eqs. (3.4), (3.6) and (A.5). In this limit, the mixing angles glet scalar, the mass eigenstates of CP-even neutral Higgs between the SM-like Higgs and extra scalars can be negligi- scalars, h and H,aregivenby bly small and the resulting Higgs spectrum is consistent with Higgs data and electroweak precision tests (EWPT) as will h =−sin αρ1 + cos αρ2, be discussed in Subsec 4.2.But,asμvs is constrained by per- turbativity and unitarity bounds on the quartic couplings with H = cos αρ1 + sin αρ2. (3.2) Eqs. (A.7)or(A.9), as will be discussed in Sect. 4,theextra The general case where the CP-even part of the singlet scalars in our model remain non-decoupled. Since it is suffi- scalar S mixes with the Higgs counterpart is considered cient to take almost degenerate masses for two of m A, m H , in Appendix A. The mass eigenvalues of CP-even neutral and m H + for EWPT, we henceforth consider more general < < scalar masses but with small mixings between the SM-like Higgs scalars are denoted as mh1,2,3 with mh1 mh2 mh3 , ≡ ≡ ≡ Higgs and the extra neutral scalars. alternatively, mh mh1 , m H mh2 and ms mh3 , and there are three mixing angles, α1,2,3: α1 = α in the limit of a decoupled CP-even singlet scalar, while α2 and α3 3.2 Quark mass matrices are mixing angles√ between ρ1,2 and SR, respectively.√ For 2κ1v1vs ≈ μv2/ 2 and 2κ2v2vs ≈ μv1/ 2, the mixing We now consider the quark mass matrices and their diagonal- between ρ1,2 and SR can be neglected. For a later discussion, ization. After two Higgs doublets develop VEVs, we obtain we focus mainly on this case. the quark mass matrices from Eqs. (2.7) and (2.8)as The CP-odd parts of the singlet scalars, S and a, can mix  ⎛ ⎞ with the Higgs counterpart due to a nonzero U(1) charge of yu yu 0 ⎜ 11 12 ⎟ the second Higgs H2, but for a small x and small VEV of ( ) = √1 v β ⎜ u u ⎟ Mu ij cos ⎝y21 y22 0 ⎠ H2, the mixing effect is negligible. In this case, the neutral 2 G0 and the CP-odd Higgs scalar A0 are 00yu ⎛ ⎞33 turned out to be 000 ⎜ ⎟ + √1 v β ⎜ ⎟ , G0 = cos βη + sin βη , sin ⎝ 000⎠ (3.7) 1 2 2 0 = βη − βη hu hu 0 A sin 1 cos 2 (3.3) ⎛ 31 32 ⎞ d d y11 y12 0 with tan β ≡ v /v . The massless combination of η and η ⎜ ⎟ 2 1 1 2 ( ) = 1 v β ⎜ d d ⎟ Md ij √ cos ⎝y y 0 ⎠ is eaten by the Z boson, while a linear combination of SI and 2 21 22    d other pseudoscalars of a is eaten by the Z boson if the Z 00y33 mass is determined dominantly by the VEV of S. The other ⎛ ⎞ 00hd combination of the CP-odd scalars from two Higgs doublets ⎜ 13⎟ 1 ⎜ ⎟ has the mass of + √ v sin β ⎝00hd ⎠ . (3.8) 2 23 00 0 μ sin β cos β v2 m2 = √ v2 + s . (3.4) A v sin2 β cos2 β 2 s The quark mass matrices can be diagonalized by + ⎛ ⎞ On the other hand, the charged Goldstone bosons G and mu 00 charged Higgs scalar H + identified as † = D = ⎝ ⎠ , UL MuUR Mu 0 mc 0 00mt + = βφ+ + βφ+, ⎛ ⎞ G cos 1 sin 2 + + + md 00 H = sin βφ − cos βφ (3.5) † = D = ⎝ ⎠ , 1 2 DL Md DR Md 0 ms 0 (3.9) 00mb with nonzero mass eigenvalue given by † thus the CKM matrix is given as VCKM = U DL .We μ β β L 2 = 2 − sin√ cos + λ v2. note that the Yukawa couplings of the second Higgs doublet m H + m A 4 (3.6) 2vs are sources of flavor violation, which could be important in meson decays/mixings and collider searches for flavor- 2 We remark that in the limit of μvs v , the heavy scalars violating top decays and/or heavy Higgs bosons [16Ð19]. 2 ≈ in the Higgs doublets√ become almost degenerate as m A The detailed derivation of flavor-violating Higgs couplings 2 ≈ 2 ≈ μv /( β β) 2 ≈ λ v2 m H m H + s 2sin cos and ms 2 S s from is presented in the next section. 123 Eur. Phys. J. C (2018) 78 :306 Page 5 of 25 306

u u h/H/A cos(α − β) ∗ ∗ Since h31 and h32 correspond to rotations of right-handed −L = √ ¯ ( ˜d + ˜d ) Y bR h13 dL h23 sL h up-type quarks, we can take UL = 1, so VCKM = DL .In 2 cos β this case, we have an approximate relation for the down- λh λh + √b ¯ + √t ¯ type quark mass matrix, M ≈ V M D,uptom , /m bRbL h tRtL h d CKM d d s b 2 2 corrections. Then the Yukawa couplings between the third (α − β) sin ¯ ˜d∗ ˜d∗ and first two generations are given as follows. + √ bR(h dL + h sL )H 2 cos β 13 23 √ √ λH λH 2m 2m b ¯ t ¯ d b d b + √ bRbL H + √ tRtL H h = Vub, h = Vcb. (3.10) 13 v sin β 23 v sin β 2 2 i − √ ¯ ( ˜d∗ + ˜d∗ ) bR h13 dL h23 sL A . . d d 2 cos β For Vub 0 004 Vcb 0 04, we have h13 h23.The λA λA down-type Yukawa couplings are determined as i b ¯ i t ¯ + √ bRbL A − √ tRtL A + h.c. (3.19) 2 2 √ √ d 2md d 2ms where y = Vud, y = Vus, 11 v cos β 12 v cos β √ √ √ √ α ˜d (α − β) 2m 2ms 2m h 2mb sin h33 cos yd = d V , yd = V , yd = b V . λ =− + , (3.20) 21 v β cd 22 v β cs 33 v β tb b v cos β cos β cos cos cos √ (3.11) 2m sin α h˜u cos(α − β) λh =− t + 33 , (3.21) t v cos β cos β On the other hand, taking UL = 1 as above, we find √ 2m cos α h˜d sin(α − β) another approximate relation for the up-type quark mass λH = b + 33 , (3.22) = D † b v cos β cos β matrix: Mu Mu UR. Then the rotation mass matrix for √ −1 ˜u † = ( D) 2mt cos α h sin(α − β) right-handed down-type quarks becomes UR Mu Mu, λH = + 33 , (3.23) t v cos β cos β which is given as √ β h˜d λA = 2mb tan − 33 , ⎛ v v ⎞ (3.24) cos β yu cos β yu 0 b v cos β mu 11 mu 12 √ ⎜ ⎟ ˜u † 1 ⎜ v u v u ⎟ β h U = √ cos β y cos β y 0 . λA = 2mt tan − 33 . R ⎝ mc 21 mc 22 ⎠ t (3.25) 2 v v v v cos β sin β hu sin β hu cos β yu mt 31 mt 32 mt 33 (3.12) ˜d ≡ † d ˜u ≡ † u We note that h DL h DR and h UL h UR.Thus,by = ˜u = u ˜d = † d taking UL 1wegeth h UR and h VCKMh .In From the unitarity condition of UR we further find the fol- this case, as compared to two-Higgs-doublet model type I, lowing constraints on the up-type quark Yukawa couplings: extra Yukawa couplings are given by √ 2 v2 2 β 2m ˜u 2mt cos u 2 |yu |2 +|yu |2 = u , (3.13) h = 1 − |y | , (3.26) 11 12 v2 2 β 33 v sin β 2 33 cos 2mt 2 2m ˜d −2 mb |yu |2 +|yu |2 = c , (3.14) h = 1.80 × 10 , (3.27) 21 22 v2 2 β 13 v β cos sin 2 2m ˜d −2 mb |yu |2 + tan2 β(|hu |2 +|hu |2) = t , (3.15) h = 5.77 × 10 , (3.28) 33 31 32 v2 2 β 23 v β cos sin ∗ ∗ u ( u ) + u ( u ) = , − mb y11 y21 y12 y22 0 (3.16) ˜d = . × 3 . h33 2 41 10 (3.29) u ( u )∗ + u ( u )∗ = , v sin β y21 h31 y22 h32 0 (3.17) u ( u )∗ + u ( u )∗ = . y11 h31 y12 h32 0 (3.18) We find that the flavor-violating couplings for light up-type quarks vanish, while the top quark Yukawa can have a siz- ˜u 3.3 Quark Yukawa couplings able modification due to nonzero h33. On the other hand, the flavor-violating couplings for down-type quarks can be large Using the results in Appendix C, we get the Yukawa interac- if tan β is small, even though the couplings have the sup- tions for the SM-like h and heavy neutral Higgs pression factors of CKM mixing and smallness of bottom bosons H, A as quark mass. The couplings can be constrained by bounds 123 306 Page 6 of 25 Eur. Phys. J. C (2018) 78 :306 from B-meson mixings and decays as is discussed in the 4 Constraints on the Higgs sector next section. We note that the flavor-violating interactions of the SM-like Higgs boson are turned off in the alignment limit In this section we consider various phenomenological con- where α = β − π/2. straints on the model coming from B-meson mixings and The Yukawa terms of the charged Higgs boson are given decays as well as Higgs and electroweak precision data on as top of unitarity and stability bounds on the Higgs sector. − − − We also show how to explain the deficits in RK and RK ∗ − LH = b¯(λH P + λH P )tH− Y tL L tR R in the B-meson decays at LHCb in our model, and discuss ¯ H − H − − + b(λ P + λ P )cH the predictions for RD and RD∗ through the charged Higgs cL L cR R − exchange. + λH bP¯ uH− + . ., uL L h c (3.30) where 4.1 Unitarity and stability bounds √ ˜d ∗ − 2m tan β ∗ (VCKMh ) Before considering the phenomenological constraints, we λH = b V − 33 , (3.31) tL v tb cos β consider unitarity and stability bounds for the Higgs sector. √ ˜u As derived in Appendix B, the conditions for perturbativity − 2m tan β h ∗ λH =− t − 33 V , (3.32) and unitarity are tR v cos β tb √ ˜d ∗ |λ |≤ π, |κ |≤ π, − 2m tan β ∗ (VCKMh ) 1,2,3,S 4 1,2 4 λH = b V − 23 , (3.33) cL v cb cos β |λ ± λ |≤4π, |λ + 2λ |≤4π, √  3 4 3 4 − 2mc tan β ∗ λ (λ + 2λ ) ≤ 4π, λH =− V , (3.34) 3 3 4 cR v cb √ 2 2 ˜d ∗ |λ + λ ± (λ − λ ) + 4λ |≤8π − β (V h ) 1 2 1 2 4 λH = 2mb tan ∗ − CKM 13 u Vub (3.35) L v cos β a1,2,3 ≤ 8π, (4.1) with where a1,2,3 are the solutions to Eq. (B.7). The vacuum sta- ⎛ ⎞ bility conditions of the scalar potential can be obtained by ˜d ˜d ˜d 00Vudh + Vush + Vubh considering the potential to be bounded from below along ⎜ 13 23 33⎟ ˜d ⎜ ˜d ˜d ˜d ⎟ the directions of large Higgs doublet and singlet scalar fields. VCKMh = ⎝00Vcdh + Vcsh + Vcbh ⎠ . (3.36) 13 23 33 Following Refs. [20Ð22], we obtain the stability conditions ˜d + ˜d + ˜d 00 Vtdh13 Vtsh23 Vtbh33 as follows: √ u SM If y = y = 2m /v, the Higgs coupling to top quark λ1,2,S > 0 33 t t  becomes λ λ + λ + λ > 0,  1 2 3 4 λ λ + λ > , λH = SM (α − β), 1 2 3 0 t yt cos (3.37)  λ1λS + κ1 > 0, −  and λA = λH = 0. t tR λ λ + κ > 0, 2 S 2 (κ2 − λ λ )(κ2 − λ λ ) + λ λ >κ κ , 3.4 Lepton Yukawa couplings 1 S 2 S 3 S 1 2 1 2 2 2 (κ − λ1λS)(κ − λ2λS) + (λ3 + λ4)λS >κ1κ2. (4.2) As seen in (2.9), the mass matrix for charged leptons e j is 1 2 already diagonal due to the U(1) symmetry. Thus, the lepton The stability conditions along the other scalar fields  can Yukawa couplings are in a flavor-diagonal form given by a be obtained in the similar way, but they are not relevant for our α α study because a’s do not couple directly to Higgs doublets  me j sin me j cos −L =− e¯ j e j h + e¯ j e j H as long as the extra quartic couplings for  are positive and Y v cos β v cos β a large enough. ime tan β + j e¯ γ 5e A0 The unitarity and stability bounds are depicted in Figs. 1 v j j √ and 2 for the parameter space in terms of mh and tan β,orvs β   2 2me j tan + and μ, with assuming the alignment limit, cos(α−β) = 0.05, + ν¯ j PR e j H + h.c. (3.38) v and zero mixing between heavy CP-even scalars. In each 123 Eur. Phys. J. C (2018) 78 :306 Page 7 of 25 306

2=0.1, 3=0, cos( − )=0.05 2=0.1, 3=0, cos( − )=0.05 10 10

1 1

0.1 0.1 t t

10−2 10−2

10−3 10−3

10−4 10−4 100 200 300 400 500 600 700 100 200 300 400 500 600 700

mh =m ± [GeV] mh2=mA[GeV] 2 H

β = ± = Fig. 1 Parameter space in terms of mh2 and tan . The gray regions and mh2 m H and m A 140 GeV in the right panel. The mixing v = = are excluded by unitarity and stability bounds. s 2mh3 1 TeV and between heavy CP-even scalars is taken to be zero (α − β) = . = ± = cos 0 05 with mh2 m A and m H 500 GeV in the left,

= = = ( − )= = = = ( − )= ] ] [ [

[ ] [ ]

v μ = ± = = . (α −β) = . Fig. 2 Parameter space in terms of s and for mh3 m H mh2 0 5 TeV and cos 0 05. The gray regions are excluded by unitarity and stability bounds. tan β = 1 (0.5) in the left (right) panel. The mixing between heavy CP-even scalars is taken to be zero

figure, the gray region corresponds to the parameter space insensitive to the mixing angle of heavy CP-even scalars, excluded by the unitarity and stability conditions. In Fig. 1, in constraining the mass parameters. The allowed parame- = β we have taken the different choices of Higgs masses: mh2 ter space for mass parameters becomes narrower as tan is ± = = ± m A and m H 500 GeV in the left, while mh2 m H smaller. and m A = 140 GeV in the right panel. On the other hand, the parameter space in terms of vs and μ has been shown in 4.2 Higgs and electroweak precision data = ± = = . Fig. 2, with setting mh3 m H mh2 0 5 TeV, but tak- ing different values of tan β. We note that the unitarity and Provided that the Higgs mixings with the singlet scalar are stability bounds are sensitive to the choice of tan β, while small, the mixing angle α between CP-even Higgs scalars

123 306 Page 8 of 25 Eur. Phys. J. C (2018) 78 :306

u SM =0 2,3=0, y33 =yt 2,3 10 10

1 1 t t

0.1 0.1

10−2 10−2 −1.0 −0.8 −0.6 −0.4 −0.2 0.0 −1.0 −0.8 −0.6 −0.4 −0.2 0.0 s s

α β u = SM/ β = ± = Fig. 3 Parameter space for sin and tan allowed by Higgs data and y33 yt cos in the right panel. mh2 m H 450 GeV, within 1σ (green), 2σ (yellow), and 3σ (dark gray). The gray regions m A = 140 GeV and vs = 1TeVhasbeentakeninallpanels u = SM corresponds to the unitarity and stability bounds. y33 yt in the left

are constrained by Higgs precision data [23Ð32]. The param- From the Z-boson like mass given in Eq. (D.6) and the ZÐZ  eter space for sin α and tan β allowed by the Higgs data is mixing angle in Eq. (D.7), we find the correction to the ρ shown in Fig. 3. We take the (33) component of the up-type parameter as u = SM Higgs Yukawa coupling to be y33 yt in the left, and u SM 2 y = y / cos β in the right panel. For illustration, we mW 2 33 ρ = (cos ζ + sin θW tan ξ sin ζ) − 1 have also imposed unitarity and stability bounds discussed m2 cos2 θ Z1 W = ± =   in the previous subsection for mh2 m H 450 GeV, 2 2   sin θ m  g  2 m = 140 GeV and v = 1 TeV. As a result, we find W Z 2Q Z sin4 β − sin2 ξ , (4.5) A s 2 2 H2 cos ξ  g a wide parameter space close to the line of the alignment, m Z Y α = β − π/2, that is consistent with both the Higgs data where we assumed that tan 2ζ 2m2 /m2 1. Taking and unitarity/stability bounds for tan β  0.1. Thus, hence- 12 Z2 ξ = forth, for the phenomenology of the extra Higgs scalars, the limit of zero gauge kinetic mixing, i.e. sin 0, we have we focus on the parameter space near the alignment limit, m2 cos(α − β) ∼ 0. ρ = W − 1 2 2 θ To see bounds from electroweak precision data, we obtain m Z cos W   effective Lagrangian after integrating out 2 2 −4 x 2 4 β 400 GeV , 10 gZ  sin (4.6) as follows [33,34]: 0.05 m Z   μ μ which is consistent with the result in Ref. [12]. Therefore, 4G F 2  Leff =−√ sec θW J + J −,μ + ρ J JZ,μ β  . 2 2 W W Z for tan 1, gZ 1, and x 0 05, Z with the mass 2g sec θW  m   μ μ Z 400 GeV is consistent with electroweak precision +  +  +··· , 2aJZ JZ ,μ bJZ  JZ ,μ (4.3) data. The mass splittings between extra Higgs scalars can also be constrained by the electroweak precision data, but it μ μ μ can be easily satisfied if we take mh = m H ± or mh = m A, where J = J − sin2 θ∗ J with θ∗ being the modified 2 2 Z 3 EM and a small mixing between CP-even scalars. Weinberg angle. Here the non-oblique terms, a and b,are determined at tree level as  4.3 B-meson anomalies from Z

ρ ζ ξ 2 Before considering constraints from B-meson mixings and = sin sec , = a . a b (4.4) decays, we show how to explain the B-meson anomalies in cos ζ + sin θW tan ξ sin ζ ρ 123 Eur. Phys. J. C (2018) 78 :306 Page 9 of 25 306 our model and identify the relevant parameter space for that. Various phenomenological constraints on the Z  interac- This section is based on the detailed results on U(1) interac- tions coming from dimuon resonance searches, other meson tions presented in Appendix D and phenomenological find- decays and mixing, lepton decays and neutrino scattering ings in Ref. [12]. have been studied in Ref. [12], leading to the conclusion that  From the relevant Z interactions for B-meson anomalies the region of xgZ   0.05 for ygZ  1 and m Z   1TeVis and the Z  mass term, consistent with the parameter space for which the B-meson anomalies can be explained.   1 ∗ μ μ L  = g  Zμ xV Vtb s¯γ PL b + h.c. + yμγ¯ μ Z Z 3 ts 4.4 Bounds from B-meson mixings and decays 1 2 2 + m  Zμ , (4.7) 2 Z  We now consider the bounds from B-meson mixings and we get the classical equation of motion for Z as decays. After integrating out the heavy Higgs bosons, the + − effective Lagrangian for Bs(d) → μ μ from the flavor-  gZ  1 ∗ violating Yukawa interactions in (3.19)is Z =− xV V s¯γμ P b + h.c. + yμγ¯ μμ . √ μ 2 ts tb L (α − β) α m  3 2mμ sin cos Z L , →μ+μ− =− eff Bs(d) 2 v β (4.8) 2m H cos × (( ˜d )∗ ¯ + ( ˜d )∗ ¯ + . .)(μμ)¯ h23 bRsL h13 bRdL h c Then, by integrating out the Z  gauge boson, we obtain the √ β effective four-fermion interaction for b¯ →¯sμ+μ− as fol- − 2mμ tan 2 v β lows. 2m A cos × (( ˜d )∗ ¯ + ( ˜d )∗ ¯ + . .)(μγ¯ 5μ). h23 bRsL h13 bRdL h c (4.13) 2 xygZ  ∗ μ L ¯ + − =− (¯γ )(μγ¯ μμ) + . . eff,b→¯sμ μ 2 VtsVtb s PL b h c 3m  The extra contributions to the effective Hamiltonian for Z + − (4.9) Bs → μ μ are thus 2 2 G F mW Consequently, as compared to the effective Hamiltonian with H , →μ+μ− =− eff Bs π the SM normalization, ×[ BSM( ¯ )(μμ)¯ + BSM( ¯ )(μγ¯ 5μ)] CS bPL s CP bPL s (4.14) G α μ, μ H =−4√ F ∗ em NPO , ¯→¯μ+μ− VtsVtb C (4.10) with eff b s 2 4π 9 9 √ π (α − β) α μ μ BSM 2mμ sin cos ˜d ∗ with O ≡ (s¯γ PL b)(μγ¯ μμ) and αem being the electro- C =− · (h ) , 9 S G2 m2 2m2 v cos2 β 23 magnetic coupling, we obtain new physics contribution to F W √ H π β the Wilson coefficient, BSM =− 2mμ tan · ( ˜d )∗. CP h23 (4.15) G2 m2 2m2 v cos β 2 2 F W A μ, 8xyπ α  v C NP =− Z (4.11) 9 α  3 em m Z In the alignment limit with α = β − π/2 and m A m H , the Wilson coefficients become identical and suppressed for α  ≡ 2 /( π) β with Z gZ  4 , and vanishing contributions to other a small tan . The effective Hamiltonian in the above leads μ,NP = μ,NP = μ,NP = B → μ+μ− operators, C10 C9 C10 0. We note that to the corrections of the branching ratio for s as > μ xy 0 is chosen for a negative sign of C9 , being consis- follows [42]: tent with B-meson anomalies. Requiring the best-fit value, μ, NP / C =−1.10 [35Ð41], (while taking [−1.27, −0.92] and 4 4 2 1 2 9 + − G F mW 4mμ 2 2 [− . , − . ] σ σ B(B → μ μ ) = 1 − m f mμ τ 1 43 0 74 within 1 and 2 errors), to explain the B- s π 5 2 Bs Bs Bs 8 m anomalies yields ⎡ s   2 ( −  ) 2 m B CP CP   1/2 × ⎣ s − (C − C ) αZ   ( + ) A A  m  = . × xy . 2 mb ms mμ Z 1 2TeV α (4.12) em   ⎤  2 ( −  ) 2 2  m B CS CS  4mμ  α  α +  s  − ⎦ , Therefore, m Z 1TeVfor xy 1 and Z em.For 1 2  2(mb + ms)mμ  m values of xy less than unity or αZ   αem, Z can be even Bs lighter. (4.16) 123 306 Page 10 of 25 Eur. Phys. J. C (2018) 78 :306

τ s (μ) where m Bs , fBs , and Bs are mass, decay constant, and life- where B is a combination of bag-parameters [43] and    123 ( ), ( ), ( ) CSM 4.95 [44]. The SM prediction and the experimental time of Bs-meson, respectively. C A CS CP are Wilson VLL ()  ( )SM = ( . ± . ) −1 O =[¯γ ] values of Ms are given by MBs 17 4 2 6 ps coefficients of the effective operators, b μ PL(R)s −  A  ( )exp = ( . ± . ) 1 μ ( ) ¯ ( ) ¯ [44] and MBs 17 757 0 021 ps [45], respec- [¯μγ γ 5μ] O =[bP ( )s][μμ ¯ ] O =[bP ( )s] , S L R and P L R tively. Then, taking into account the SM uncertainties, we [¯μγ μγ 5μ] , respectively. We note that there is no contribu- obtain the bounds on M as 16 (13) ps−1 <M <  → μ+μ− Bs Bs tion from Z interactions to Bs since the ( ) −1 ( )BSM < . ( . ) −1 σ σ  21 23 ps or MBs 3 0 5 6 ps at 1 (2 ) couplings to Z are vector-like. On the other hand, in the level for new physics. We also note that the most recent lat- → μ+μ− alignment limit the bounds obtained from Bs,d in tice calculations show considerably large values for the bag Ref. [42] can be translated to our case as ( )SM = ( . ± . ) −1 parameters, leading to MBs 20 01 1 25 ps     [46]. It needs an independent confirmation, but if it is true,   cos β m , 2  ˜d  < . × −2 H A , h23 3 4 10 the new physics contributions coming from the heavy Higgs tan β 500 GeV      bosons and Z would be constrained more tightly. β 2  ˜d  −2 cos m H,A ( )SM = ( . ± h  < 1.7 × 10 . (4.17) Taking the SM prediction as MBs 17 4 13 β − tan 500 GeV 2.6) ps 1 [44], from Eq. (4.22) with Eqs. (4.20) and (4.21), we get the bound on the flavor-violating Yukawa coupling in From Eqs. (3.27) and (3.28), we find that flavor constraints the alignment limit of heavy Higgs bosons as are satisfied as far as    ˜d  2 1/2 2 |h | m 500 GeV − 500 GeV 23  H − 1 < 4.6(6.4) × 10 3 sin β< 1 − 0.033 . (4.18)  2  cos β m m H m H,A  A

  2 xgZ  2 300 GeV This leads to tan β<5.4form H,A = 500 GeV. × 1 − 0.1(0.06) . (4.23) .  The flavor-violating Yukawa couplings of heavy Higgs 0 05 m Z  ¯ bosons as well as Z interactions [12] can modify the BsÐBs   .   mixing. The additional effective Hamiltonian relevant for the Here, since we need to choose xgZ 0 05 for m Z 1TeV mixing is given by to satisfy the B-meson anomalies and the LHC dimuon  bounds at the same time as discussed in the previous sec- H ¯ = C (s¯α P bα)(s¯β P bβ )  eff,Bs −Bs 2 R R tion, we can safely ignore the contribution of Z interactions 2 2 to the B ÐB¯ mixing on the right-hand side of Eq. (4.23). G F mW ∗ 2 NP s s + (V Vtb) C  ¯ 16π 2 ts VLL Furthermore, with the Z contribution ignored, the Bd ÐBd μ ×(s¯αγ PL bα)(s¯β γμ PL bβ ), (4.19) mixing leads to a similar bound [43]: with      2 −1/2 ˜d  ˜d  −3 m H  m H  h h  < 0.91(1.3) × 10 cos β  − 1 . C = 23 13 m2 500 GeV 2 4 cos2 β m2 A H (4.24) m2 m2 cos2(α − β) × H − 2(α − β) − H , 2 sin 2 m A mh + − Comparing to the bounds from Bs → μ μ in (4.17), the (4.20) BÐB¯ mixings could lead to tighter constraints on the flavor- 16π 2 (xg  )2v4 violating Yukawa couplings for down-type quarks unless m CNP = Z H VLL 2 2 and m are almost degenerate. The upper frames of Fig. 4 9 m Z  mW A   show that a wide range of heavy Higgs masses up to 600Ð  2 2 = . xgZ 300 GeV . = β = O( ) 0 27 (4.21) 700 GeV are allowed for mh2 m A and tan 1 . .  0 05 m Z On the other hand, for tan β = 0.5, the neutral Higgs boson can be as heavy as 400 GeV, but the charged Higgs mass is The mass difference in the Bs system becomes constrained as 240 GeV  m H ±  650 GeV.For illustration, 2  = 2 s (μ) the case with mh = m H ± has also been shown in the lower MBs m Bs fB B123 2  3 s  frames of Fig. 4, where the narrower region is allowed as   G2 m2 = F W ∗ 2 SM NP  compared with the case with mh2 m A. × (V Vtb) C + C +|C | , 16π 2 ts VLL VLL 2 Another important bound comes from the inclusive radia- → γ (4.22) tive decay, B Xs . The effective Hamiltonian relevant for the b → sγ transition is 123 Eur. Phys. J. C (2018) 78 :306 Page 11 of 25 306

= = = ( − )= = = = ( − )= ] ] [ [ ± ±

= [ ] = [ ]

= = = ( − )= 2=0.1, 3=0, t =0.5, cos( − )=0.05 800

700

600 excl. S excl. + 500 U s ] ] X B GeV 400 [ [ A

m 300 Bd−Bd excexccl. 200

BS−BS exexclc . 100

100 200 300 400 500 600 700

= ± [ ] m =m ± [GeV] h2 H

± Fig. 4 Parameter space in terms of mh2 and m H (upper frames), and zero. The gray regions are excluded by unitarity and stability bounds. m A (lower frames). tan β = 1 in the left and 0.5 in the right panels. We The magenta regions are excluded by B → Xs γ , and cyan region is v = = (α − β) = . u = SM → μ+μ− have chosen s 2mh3 1TeV,cos 0 05, and y33 yt excluded by Bs . The yellow and orange regions are excluded in all frames. The mixing between heavy CP-even scalars is taken to be by Bs and Bd mixings, respectively

− − 4G F ∗ 2 (λH )∗λH H =− √ ( O + O ) v t t ( ) eff,b→sγ VtbVts C7 7 C8 8 (4.25) CBSM = R R C 1 (x ) 2 7 2 ∗ 7 t 2mt VtbVts − − v2 (λH )∗λH with + tL tR (2)( ), ∗ C7 xt 2mt mb VtbVts e μν O = m s¯σ P bFμν, − ∗ − 7 π 2 b R v2 (λH ) λH 16 BSM tR tR (1) C = ∗ C (xt ) gs μν a a 8 2 V V 8 O = mb s¯σ PR T bGμν. (4.26) 2mt tb ts 8 16π 2 − − v2 (λH )∗λH + tL tR (2)( ) The charged Higgs contributions to the Wilson coefficients ∗ C8 xt (4.27) 2mt mb VtbV are given by [19,47] ts 123 306 Page 12 of 25 Eur. Phys. J. C (2018) 78 :306

= ( − )= = ( − )=

± [ ] ± [ ]

± β → γ σ u = SM = = Fig. 5 Parameter space for m H and tan excluded by B Xs within 2 (red) and unitarity bounds (gray) with y33 yt for m A mh2 = = 160 GeV (left panel) and m A mh2 350 GeV (right panel)

2 with xt ≡ (mt /m H ± ) , and mass as shown in Fig. 5, where unitarity and stability bounds u =   are displayed as well. We also find that the case with y33 ( ) x −8x3 + 3x2 + 12x − 7 + (18x2 − 12) ln x SM/ β → γ C 1 (x) = , yt cos has been excluded by B Xs , hence the case 7 72 (x − 1)4 with yu = ySM is considered in Figs. 4 and 5 and collider   33 t studies in the next section. ( ) x −5x2 + 8x − 3 + (6x − 4) ln x C 2 (x) = , 7 12 (x − 1)3   4.5 Predictions for RD and RD∗ ( ) x −x3 + 6x2 − 3x − 2 − 6x ln x C 1 (x) = , 8 24 (x − 1)4   We briefly discuss the implications of flavor-violating cou- 2 R R ∗ (2) x −x + 4x − 3 − 2lnx plings with charged Higgs on D and D . The effective C (x) = . (4.28) → (∗)τν 8 4 (x − 1)3 Hamiltonian relevant for B D in our model is given as follows: − Here λH are given by Eqs. (3.31) and (3.32). The Wil- H = cb (¯ γ )(τ¯ γ μν ) + cb(¯ )(τ¯ ν ) tL,R eff CSM cL μbL L L CR cL bR R L SM = cb son coefficients in the SM at one loop are given by C7 + C (c¯ b )(τ¯ ν ), (4.31) ( ) ( ) L R L R L 3C 1 (m2/m2 ) and CSM = 3C 1 (m2/m2 ). CBSM mixes 7 t W 8 8 t W 8 cb 2 BSM where the Wilson coefficient in the SM is C = 2Vcb/v , into the C at the scale of μb = mb through the renormal- SM 7 and the new Wilson coefficients generated by charged Higgs ization group equations and contribute to B(B → Xsγ)[48]. The next-to-next-leading order SM prediction for B(B → exchanges are Xsγ)is [49,50] √ 2mτ tan β − ∗ Ccb =− (λH ) , −4 R 2 cL B(B → X γ)= (3.36 ± 0.23) × 10 , (4.29) v m ± s √ H 2mτ tan β − ∗ B(B → X γ) Ccb =− (λH ) . (4.32) whereas the experimentally measured value of s L 2 cR v m ± from HFAG is [45] H

− B( → γ)= ( . ± . ± . ) × −4. λH B Xs 3 43 0 21 0 07 10 (4.30) See Eqs. (3.33) and (3.34)for cL,R . The ratios of the branching ratios for B → D(∗)τν to (∗) As a result, the SM prediction for B → Xsγ is consistent B → D ν with  = e, μ are defined by with experiments, so we obtain the bounds on the modified − . < BSM(μ )< . σ (∗) Wilson coefficients as 0 032 C7 b 0 027 at 2 B(B → D τν) β R (∗) = . (4.33) level [51]. This constrains tan in terms of charged Higgs D B(B → D(∗)ν) 123 Eur. Phys. J. C (2018) 78 :306 Page 13 of 25 306

1.2 5.1 Heavy neutral Higgs boson RD/RD,SM,t =1

RD/RD,SM,t =0.5 1.1 The main channels for neutral Higgs productions are the R ∗/R ∗ ,t =1 D D ,SM fusion gg → H, bottom-quark fusion bb¯ → H, and R ∗/R ∗ ,t =0.5 D D ,SM additional productions through the flavor-violating interac- 1.0 ¯ ¯ tions for the bottom quark, bdi → H and di b → H, where di denotes light down-type quarks, di = d, s. There are bottom quark associated productions, bg → bH and d g → bH,as 0.9 i well. The leading-order cross section for the gluon fusion pro- cess at parton level is 0.8 1 5 10 50 100 500 α2m2 m [GeV] σ(ˆ gg → H) = s H H± πv2  576    2 / ∗ / ∗ 3 cos α v sin(α − β)  Fig. 6 The ratios of RD RD,SM and RD RD ,SM as the functions of ×  + √ h˜q AH (τ ) charged Higgs mass for given tan β  β β 33 1/2 q  4 q cos 2mq cos ×δ(ˆ − 2 ), s m H (5.1) = . ± . ∗ = The SM expectations are RD 0 300 0 008 and RD 2 2 H where τq = m /(4m ). The loop function A / (τ) is given 0.252 ± 0.003 [52], but the experimental results for R (∗) H q 1 2 D in Ref. [54]. sˆ is the partonic center-of-mass energy. Here are deviated from the SM values by more than 2σ [7Ð11]. the contributions of only top and bottom quarks have been Including the additional contributions from charged Higgs taken into account. Note that the top quark contribution is exchanges, we find the simplified forms for R and R ∗ as D D vanishing if one takes yu = ySM and the alignment limit as follows [47,53]: 33 t can be seen in Eq. (3.37). The parton-level cross section for ⎡   ⎤ bottom-quark fusion bb¯ → H is cb + cb  cb + cb 2 ⎣ CR CL CR CL  ⎦ R = R , 1 + 1.5Re +   , π 2 α v (α − β) 2 D D SM cb  cb  ¯ mb cos sin ˜d CSM CSM σ(ˆ bb → H) = + √ h  18v2 cos β 2m cos β 33 cb cb b C − C / ∗ = ∗ + . R L 1 2 RD RD ,SM 1 0 12 Re 4m2 Ccb × − b δ(ˆ − 2 ).  SM 1 2 s m H (5.2)   m Ccb − Ccb 2 H + 0.05 R L  . (4.34) Ccb There are other single Higgs production channels through the SM ¯ ¯ flavor-violating interactions, bdi → H and di b → H.The As can be seen in Fig. 6, a light charged Higgs is neces- corresponding cross section is given by sary to have large deviations of R and R ∗ . However, it is D D π| ˜d |2 2(α − β) excluded by B → X γ . [See Fig. 5]. Therefore, our model ¯ hi3 sin 2 s σ(ˆ di b → H) = δ(sˆ − m ), (5.3) 72 cos2 β H cannot explain the experimental results for RD(∗) simultane- ously with the other bounds. ¯ ¯ and σ(ˆ bdi → H) =ˆσ(di b → H) at parton level. The bottom quark associated production of the Higgs boson can occur by initial states with a bottom quark, that 5 Productions and decays of heavy Higgs bosons at the is, bg → bH, through the flavor-conserving interactions or LHC initial states with a light down-type quark, di g → bH,via the flavor-violating interactions. The former is nonvanishing We investigate the main production channels for heavy Higgs even if all the components of h˜d are zero. The diagrams of bosons at the LHC, including the contributions from flavor- the bottom quark associated production are shown in Fig. 7. violating interactions of quarks. The decay modes of the The differential cross section for bg → bH at parton level is heavy Higgs bosons for some benchmark points are also stud-  H 2 2 ied, and we discuss smoking gun signals for heavy Higgs dσˆ αs(λ ) 2F1 − F − 2G1G2 (bg → bH) = b 2 searches at the LHC. In this section, mixings with singlet dtˆ 96(sˆ − m2)2 (sˆ − m2)(tˆ − m2) scalar have been neglected and the heavy neutral Higgs b  b b boson H denotes h . h ≡ h is the SM-like Higgs with G1 G2 2 1 +2m2 + , (5.4) = b (ˆ − 2)2 (ˆ − 2)2 mh 125 GeV. s mb t mb 123 306 Page 14 of 25 Eur. Phys. J. C (2018) 78 :306

d, s, b H d, s, b H

g b g b

Fig. 7 Diagrams of the bottom quark associated productions of neutral Higgs bosons

Fig. 8 Production cross sections of the heavy neutral Higgs H at 14 TeV -proton collisions. We have chosen tan β = 1 (left panel) and β = . (α − β) = . u = SM tan 0 5 (right panel) with cos 0 05 and y33 yt where proton collisions of 14 TeV and employ the NNPDF2.3 par- ton distribution function (PDF) set [55] via the LHAPDF F =ˆstˆ − m4, F =ˆs + tˆ − 2m2, 1 b 2 b 6 library [56]. The renormalization and factorization scales G = m2 − m2 −ˆs, G = m2 − m2 − tˆ, 1 H b 2 H b are set to m H , and mb = 4.7 GeV. The resulting production λH → cross sections as a function of m H are shown in Fig. 8.In and b is given in (3.22). For the di g bH process, it is all frames we set cos(α − β) = 0.05, close to the alignment u = SM σˆ α | ˜d |2 2(α − β) limit, and y33 yt . A constant K -factor of 2.5 has been d s hi3 sin (di g → bH) = multiplied to the gluon fusion production cross section, while dtˆ 96sˆ2(tˆ − m2) cos2 β  b  the leading-order expressions have been used for the other 2F − F2 − 2G G 2m2G production channels. × 1 2 1 2 + b 2 sˆ tˆ − m2 In the alignment limit, the neutral Higgs coupling to the b λH (5.5) top quarks t is vanishing as can be seen in Eq. (3.37). In this case, the single Higgs production through the gluon with fusion process is suppressed compared to the SM case, though nonvanishing due to the bottom quarks in the loop. =ˆˆ, =ˆ+ ˆ− 2, = 2 − 2 −ˆ, = 2 − ˆ. Still, the gluon fusion production convoluted with PDF is F1 st F2 s t mb G1 m H mb s G2 m H t the most dominant channel for the single Higgs production ¯ ¯ ¯ → β  O( . ) And again, σ(ˆ di g → bH) =ˆσ(di g → bH) at parton level. and bb H is the subdominant one for tan 0 1 . We perform the integration by using the Monte Carlo On the other hand, for smaller tan β, the flavor-violating method to obtain the production cross sections at proton- Higgs couplings to light quarks become larger and contribu- 123 Eur. Phys. J. C (2018) 78 :306 Page 15 of 25 306

3 2 tions from the initial states with the light down-type quarks δV m cos (α − β) (H → VV) = H d b¯ → H is subdominant, and become even the most dom- πv2 i 32 β = O( . ) 1/2 inant channel in the case of very small tan 0 01 . 4m2 4m2 12m4 However, since we find that such scenarios with very small × 1 − V 1 − V + V , (5.9) m2 m2 m4 tan β have been excluded by bounds from the experimen- H H H tal results on B-meson mixings and decays, particularly by where δW = 2 and δZ = 1. These partial widths are vanishing → γ B Xs as seen in the previous section, we have cho- in the alignment limit. If m H > 2m Z  , the decay mode of sen tan β = 1 and 0.5 as benchmarks for this study. For H → Z  Z  opens. Ignoring the small mixing with the Z m H = 200 GeV and tan β = 1 (0.5), σpp→H 225.2 boson, the decay width is (σ + σ )/σ = . (110.5) fb, and bd¯ →H d b¯→H gg→H 0 62% / i i 4 4 3 v2 2 β 2 α 2 1 2 (1.6%), while (σ ¯ → +σ ¯→ )/σ ¯→ 1.6% (10.9%)   g  x m H sin sin 4m  bdi H di b H bb H (H → Z Z ) = Z 1 − Z at the LHC. As the neutral Higgs gets heavier, the produc- π 4 2 2592 m Z m H = tion cross sections rapidly decreases. For m H 400 GeV 2 4 4m  12m  and tan β = 1 (0.5), σpp→H 38.4 (31.7) fb. × − Z + Z . 1 2 4 (5.10) As can be seen in Fig. 8, the production cross section m H m H β of the bottom quark associated process increases as tan However, we find that this decay mode is almost negligible is smaller since the effect of the flavor-violating couplings for small g  x O(0.05) and m   400 GeV,which would  Z Z become larger. In particular, if m H 200 GeV the produc- be necessary to evade constraints from the Z  searches at the O( ) tion cross section is 10 fb, so it can be served as a good LHC.  search channel at the LHC. Meanwhile, if m H 2mt ,the The neutral Higgs boson can also decay into γγ and gg  O( ) cross section decreases down to 1 fb. through fermion or gauge boson loops. At leading order, the We now turn to the decay widths of the neutral Higgs decay widths are given as bosons and obtain their branching ratios. Ignoring the mixing  among the SM-like Higgs and singlet scalar, the partial decay α2 3   λH v m H  2 q H widths to quarks are (H → γγ)=  3Q √ A / (τq ) 256π 3v2 q 1 2 q=t, b 2mq  α 2 ( → ¯ ) = ( → ¯) cos H H  H bdi H di b + A (ττ ) + cos(α − β)A (τ ) , β 1/2 1 W  2 cos 3|h˜d |2 sin2(α − β) m2   = i3 − b ,  2 m H 1 α2 3   λH v  32π cos2 β m2 s m 3 q H (H → gg) = H  √ AH (τ ) , 3 2  1/2 q  3/2 72π v 4 2m  (λH )2 2 q=t, b q 3 q 4mq (H → qq¯) = m 1 − , (5.6) π H 2 (5.11) 16 m H

where Qq is the of the quark and τi ≡ 2 /( 2) H H = λH λH m H 4mi . The loop functions A1/2 and A1 can be found where q t, b, c. b and t are given in (3.22) and (3.23), and in Ref. [54]. If m H > 2mh, the heavy neutral Higgs can decay into a √ pair of SM-like Higgs bosons.2 The triple interaction comes 2mc cos α from the scalar potential in (2.3), λH = . (5.7) c v cos β g v V ⊃ Hhh Hhh, (5.12) 1 2 On the other hand, the Higgs interactions to the charged lep- tons are flavor-conserving and the corresponding decay width where is given as gHhh = 3(λ1 sin α cos β + λ2 cos α sin β)sin(2α)

3/2 + (λ3 + λ4) [3 cos(α + β)cos(2α) − cos(α − β)] . 2 2 2 + − mτ cos α 4mτ (H → τ τ ) = m 1 − . (5.8) (5.13) πv2 2 β H 2 8 cos m H 2 If the singlet scalar h3 = S is light enough, additional decay modes H → Sh = such as can occur and become important channels (See, for The partial widths to electroweak gauge bosons V W, example, [57,58]). Here we assume that S is heavy, mS  0.5Ð1 TeV, Z are given as and the mixings with doublet Higgs bosons are negligible. 123 306 Page 16 of 25 Eur. Phys. J. C (2018) 78 :306

Fig. 9 Branching ratios of the heavy neutral Higgs H.tanβ = 1andμ = 200 GeV (left panel), and tan β = 0.5andμ = 50 GeV (right panel) have been taken. vs = 1 TeV and cos(α − β) = 0.05 for both panels The decay width for the H → hh process is given as conjunction with the dijet channel with one b jet, is important to search the heavy neutral Higgs boson at the LHC. Thus,

1/2 < g2 v2 4m2 the neutral Higgs boson with m H 250 GeV can receive (H → hh) = Hhh 1 − h . (5.14) constraints from dijet searches [59Ð61]. Although the dijet 32πm m2 H H channel has typically been used to seek for heavy resonances in a few TeV scales, it can probe lower scales if it is associ- The quartic couplings in the Higgs potential can be eval- ated with a hard or jet from initial state radiations. The μv β α uated by choosing values of s,tan ,sin , and m H if ATLAS collaboration has searched light resonance with dijet α α mixing with the singlet scalar is negligible, 2 3 0. down to 200 GeV in the final states of dijet in See Appendix A. association with a photon [62,63]. In our case, gluon fusion By combining all the decay widths, we obtain the branch- production is the most dominant channel and it is not associ- ing ratio of each decay mode. Fig. 9 shows the branching ated with a hard photon. It can have a hard jet from the (α − β) = . ratios of the neutral Higgs boson H for cos 0 05 in the initial states, but the mass region below 250 GeV has v = μ and s 1 TeV, but with different values of to sat- not been searched yet in the final states of dijet in association isfy the unitary and stability bounds studied in Sect. 4.1. with a hard jet. For m H > 250 GeV, bounds from di-Higgs → ¯ ¯ We observe that H bdi /di b is the predominant decay searches can be imposed, but we find that they do not have < → mode if m H 2mh, whereas the di-Higgs mode H hh enough sensitivities for heavy neutral Higgs bosons in our becomes the most important if the mode is kinematically model yet [64Ð67]. allowed, irrespective of tan β. In practice, the branching ratio B( → ) of di-Higgs mode H hh depends on the choice of 5.2 Heavy charged Higgs boson μvs value. If we take a smaller μvs value, for instance, μ = v = β = 200 GeV and s 500 GeV with tan 1, we One of the conventional search channels for the heavy find that H → bd¯ /d b¯ is always the most dominant decay i i charged Higgs with m ± > m at colliders is the = H t mode. The dip near m H 580 GeV in the left panel of top quark associated production, bg → tH−, by the similar Fig. 9 is due to the accidental cancellation in the Higgs triple diagrams as bg → bH. Since the charged Higgs boson can coupling (5.13). The position of dip also depends on the have enhanced couplings with the light up-type quarks due μv β (α − β) value of s for given tan and cos . On the other to nonzero components of h˜d , we can also have a sizable pro- ¯ / hand, the bb mode and diboson modes such as WW ZZ are duction cross section of the bottom quark associated process subdominant. + from the initial states with light up-type quarks, ui g → bH From these observations, we expect that the search strate- 3 where ui = u, c. gies would be different depending on the mass of the heavy Higgs boson. For m < 2m , pp → H → bd¯ /d b¯, i.e., H h i i 3 We note that there have been collider studies on the production of dijet final states containing one b jet is the most important, heavy Higgs bosons due to flavor-violating interactions for up-type but for m H > 2mh, the di-Higgs channel, and possibly in quarks. See, for instance, Refs. [17,18]. 123 Eur. Phys. J. C (2018) 78 :306 Page 17 of 25 306

Fig. 10 Production cross sections of the heavy charged Higgs H ± at 14 TeV proton-proton collisions. We have chosen tan β = 1 (left panel) and β = . u = SM tan 0 5 (right panel) with y33 yt The differential cross section for bg → tH− at parton The leading-order cross sections evaluated by convoluting level is the partonic cross section with the PDFs at proton-proton    collisions of 14 TeV are shown in Fig. 10. In each figure, dσˆ α − − = s |λH |2 +|λH |2 σ(pp → H ±q) = σ(pp → H +q) + σ(pp → H −q).The ˆ (ˆ − 2)2 tL tR dt 48 s mb production cross sections are quite sensitive to tan β.For ± 2F − F2 − 2G G 2m2G 2 tan β = 1, the top quark associated production, pp → H t, × 1 2 1 2 + b 1 + 2mt G2 (ˆ − 2)(ˆ − 2) (ˆ − 2)2 (ˆ − 2)2 is the dominant channel, while the bottom quark associated s m t mt s m t mt b b production, pp → H ±b, which is the characteristic channel   2 − − − − 4m m m ± + λH (λH )∗ + λH (λH )∗ b t H of our model, can also be served as a good channel to search tL tR tR tL (sˆ − m2)(tˆ − m2)  b t the charged Higgs boson at the LHC. On the other hand, forsmallertanβ, the bottom quark associated production × − F1 F2 , 1 2 2 2 (5.15) becomes the dominant channel due to the enhanced charged- m ± (sˆ − m )(tˆ − m ) H b t Higgs couplings with light up-type quarks. The suppression of top quark associated production is also due to the partial where − λH cancellation of two terms in tL . ˆ 2 2 ˆ 2 2 Concerning the decays of charged Higgs, the most impor- F1 =ˆst − m m , F2 =ˆs + t − m − m , b t b t tant fermionic decay mode is H + → tb¯. The decay width = 2 − 2 −ˆ, = 2 − 2 − ˆ. G1 m H ± mt s G2 m H ± mb t (5.16) is

Since the diagrams contributing to bottom quark associ- + − (H → tb¯) = (H → bt¯) ated processes has the same Lorentz structure as those for −   → 1/2 bg tH , we can obtain their parton-level cross sections 2 2 H − H − 3 (mt + mb) (mt − mb) by replacing λ with λ , mb with mu 0, and mt with = ± − − tL,R uiL,R i m H 1 2 1 2 16π m ± m ± mb. They are given as H H − −  σˆ α (|λH |2 +|λH |2)   2 + 2 d + s uiL uiR H − 2 H − 2 mt mb (ui g → bH ) = × |λ | +|λ | 1 − ˆ 2 ˆ 2 tL tR 2 dt 48sˆ (t − m ) m ±  b H 2F − F2 − 2G G 2m2G  × 1 2 1 2 + b 2   (5.17) − − ∗ − − ∗ m m sˆ tˆ − m2 −2 λH (λH ) + λH (λH ) t b . (5.19) b tL tR tR tL 2 m H ± with

ˆ ˆ 2 2 2 H − H − F =ˆst, F =ˆs + t − m , G = m ± − m −ˆs, m m m λ λ 1 2 b 1 H b By replacing t with c or u and tL,R with cL,R or 2 H − + ¯ G = m ± − tˆ. λ H → cb 2 H (5.18) uL,R , one can obtain the decay widths of and 123 306 Page 18 of 25 Eur. Phys. J. C (2018) 78 :306

Fig. 11 Branching ratios of the heavy charged Higgs H ±.tanβ = 1 (left panel) and tan β = 0.5 (right panel) have been taken. cos(α − β) = 0.05 u = SM and y33 yt for both panels

H + → ub¯. The other fermionic decay modes are H + → cs¯ next-to-dominant and this is not constrained by the current ¯ 2 2 and cd, whose decay widths are proportional to tan β|Vcs| LHC data [71,72], because the production cross section for 2 2 and tan β|Vcd| , respectively. The decay widths of leptonic the heavy charged Higgs in our model is less than 10 fb in decay modes are given as most of the parameter space. (H + → +ν) = (H − → −ν)¯

2 2 2 2 m tan β m = m ± 1 − . (5.20) 6 Conclusions 8πv2 H 2 m H ± Meanwhile, if H + → W + A and W + H are kinematically We have considered an extra local U(1) with flavor- + − forbidden, the only non-fermionic decay mode is H → dependent couplings as a linear combination of B3 L3 W +h. The decay width is and Lμ − Lτ , that has been recently proposed to explain the B-meson anomalies. In our model, we have reproduced the (H + → W +h) = (H − → W −h) correct flavor structure of the quark sector due to the VEV 2 2 3 of the second Higgs doublet, at the expense of new flavor g cos (α − β)m ± = H violating couplings for quarks and the violation of lepton π 2 64 mW universality. ⎡ ⎤ / 2 3 2 The extra gauge boson leads to flavor violating interac- m2 m2 4m2 m2 × ⎣ − W − h − W h ⎦ . tions for down-type quarks appropriate for explaining B- 1 2 2 4 (5.21) m ± m ± m ± H H H meson anomalies in RK (∗) whereas heavy Higgs bosons ren- der up-type quarks have modified flavor-conserving Yukawa By combining all the decay modes in the above we obtain couplings and down-type quarks receive flavor-violating the branching ratios of the heavy charged Higgs, which are Yukawa couplings. We also found that the B-meson anoma- shown in Fig. 11. Interestingly, the dominant decay mode of lies in RD(∗) cannot be explained by the charged Higgs boson the charged Higgs boson is H + → W +h if it is kinemati- in our model, due to small flavor-violating couplings. cally allowed, although we have taken the alignment limit. We showed how the extended Higgs sector can be con- H + → tb¯ is subdominant. Together with the production, strained by unitarity and stability, Higgs and electroweak we expect that pp → H ±b → W ±h + b can be served as precision data, B-meson decays/mixings. Taking the align- the important process to probe the charged Higgs boson at ment limit of heavy Higgs bosons from Higgs precision data, the LHC and future hadron colliders. Most LHC searches for we also investigated the production of heavy Higgs bosons W +h have been dedicated to heavy resonances [68Ð70] that at the LHC. We found that there are reductions in the cross decay directly into W +h, so our model is not constrained sections of the usual production channels in 2HDM, such as by W +h at the moment. On the other hand, the tb¯ mode is pp → H and pp → H ±t at the LHC. In addition, new 123 Eur. Phys. J. C (2018) 78 :306 Page 19 of 25 306

± production channels such as pp → Hb and pp → H b The mass matrix MS can be then diagonalized as become important for tan β  1. Decay products of heavy RM RT = diag(m2 , m2 , m2 ). (A.3) Higgs bosons lead to interesting collider signatures due to S h1 h2 h3 large branching fractions of bd +bs modes for neutral Higgs bosons and W ±h mode for charged Higgs boson if kinemat- We use a convention such that the mass eigenstates are < < ically allowed, thus requiring a more dedicated analysis for ordered as mh1 mh2 mh3 . Here, the orthogonal matrix α α the LHC. R is parametrized in terms of the mixing angles 1 to 3 as ⎛ ⎞ Acknowledgements The work is supported in part by Basic Science cα1 cα2 sα1 cα2 sα2 = ⎝−( + ) − ⎠ , Research Program through the National Research Foundation of Korea R cα1 sα2 sα3 sα1 cα3 cα1 cα3 sα1 sα2 sα3 cα2 sα3 (NRF) funded by the Ministry of Education, Science and Technology − + −( + ) cα1 sα2 cα3 sα1 sα3 cα1 sα3 sα1 sα2 cα3 cα2 cα3 (NRF-2016R1A2B4008759). The work of LGB is partially supported (A.4) by the National Natural Science Foundation of China (under Grant no. 11605016), Korea Research Fellowship Program through the National ≡ α ≡ α Research Foundation of Korea (NRF) funded by the Ministry of Science where sαi sin i and cαi cos i . Without loss of gener- and ICT (2017H1D3A1A01014046). The work of CBP is supported by ality the angles can be chosen in the range of IBS under the project code, IBS-R018-D1. π π − ≤ α , , < . Open Access This article is distributed under the terms of the Creative 2 1 2 3 2 Commons Attribution 4.0 International License (http://creativecomm ons.org/licenses/by/4.0/), which permits unrestricted use, distribution, In the text we focus mainly on the situation where mixings and reproduction in any medium, provided you give appropriate credit ρ to the original author(s) and the source, provide a link to the Creative between 1,2 and SR are small. Commons license, and indicate if changes were made. The mass eigenvalues of CP-even neutral scalars are Funded by SCOAP3. given by √ 2 = 1( + − ) ≡ 2, mh a b D mh Appendix A: The extended Higgs sector 1 2 1 √ m2 = (a + b + D) ≡ m2 , h2 2 H By using the minimization condition of the Higgs potential μv v 2 = λ v2 + √ 1 2 ≡ 2, given by mh 2 S s ms (A.5) 3 2v √ s 2μv v − 2λ v3 − 2λ v v2 − 2λ v v2 − 2κ v v2 μ2= 2 s 1 1 3 1 2 4 1 2 1 1 s , where 1 2v 1 μv v μv v √ 2 2 s 2 1 s 2 2 a≡2λ1v + √ , b≡2λ2v + √ , D≡(a −b) +4d 2μv v − 2λ v2v − 2λ v2v − 2λ v3 − 2κ v v2 1 v 2 v μ2= 1 s 3 1 2 4 1 2 2 2 2 2 s , 2 1 2 2 2 2v2 (A.6) √ √ 2 2 3 ≡ v v (λ +λ )−μv / 2μv1v2 − 2κ1v vs − 2κ2v vs − 2λSv with d 2 1 2 3 4 s 2. We can trade off quartic m2= 1 2 s , (A.1) s λ , , , κ , 2vs couplings, 1 2 3 4 and 1 2, for mixing angles and Higgs masses. the mass matrix for CP-even scalars can be written as √ 2 m2 R2 − 2μv tan β i hi i1 s ⎛ μv v μv ⎞ λ1 = , λ v2 + √ 2 s v v (λ + λ ) − μv√ s κ v v − √ 2 v2 2 β 2 1 1 v 2 1 2 3 4 2 1 1 s 4 cos ⎜ 2 1 2 2 ⎟ √ ⎜ μv v μv ⎟ 2 2 ⎜ μv√ s 2 √ 1 s √ 1 ⎟ 2 m R − 2μvs cot β M = 2v v (λ + λ ) − 2λ v + 2κ v vs − . i hi i2 S ⎜ 1 2 3 4 2 2 2 2v 2 2 2 ⎟ λ = , ⎝ 2 ⎠ 2 2 2 μv μv μv v 4v sin β κ v v − √ 2 κ v v − √ 1 λ v2 + √ 1 2 2 1 1 s 2 2 2 s 2 S s √ 2 2 2vs 2 2μvs + 2 m Ri1 Ri2 (A.2) λ + λ = i hi , 3 4 4v2 sin 2β √ 2v m2 R2 − 2μv2 sin β cos β We introduce a rotation matrix R to change the interaction s i hi i3 λS = , basis (ρ1,ρ2, SR) to the physical mass eigenstates, h1, h2 4v3 √ s and h3 as 2μv sin β + 2 m2 R R i hi i1 i3 ⎛ ⎞ ⎛ ⎞ κ1 = , 4vvs cos β h1 ρ1 √ ⎝ ⎠ = ⎝ρ ⎠ . 2μv cos β + 2 m2 R R h2 R 2 i hi i2 i3 κ2 = . (A.7) h3 SR 4vvs sin β 123 306 Page 20 of 25 Eur. Phys. J. C (2018) 78 :306

2 2 In the case when the Higgs mixings with the singlet scalar whose eigenvalues are 2λ1,2λ2, λ1+λ2± (λ1 − λ2) + 4λ , α α 4 are negligible, 2 3 0, the rotation matrix can be and 2λS. simplified as (φ+ φ+ φ+ φ+ ) In the basis of 1 SR, 2 SR, 1 SI , 2 SI , the submatrix ⎛ ⎞ is given by cos α sin α 0 ≈ ⎝− α α ⎠ , ⎛ ⎞ R sin cos 0 (A.8) 2κ1 000 001 ⎜ ⎟ ⎜ ⎟ ⎜ 02κ 00⎟ ⎜ 2 ⎟ where α = α . Then the Higgs quartic couplings are given M2 = ⎜ ⎟ (B.2) 1 ⎜ κ ⎟ by ⎝ 0021 0 ⎠ √ 2 2 2 2(m cos2 α + m sin α) − 2μv tan β 0002κ2 λ ≈ h H s , 1 4v2 cos2 β √ κ , 2 2 2 2 with eigenvalues being 2 1 2. 2(m sin α + m cos α) − 2μvs cot β λ ≈ h H , In the basis of (ρ1η1, ρ2η2, SR SI ), the matrix is 2 v2 2 β 4 sin√ ⎛ ⎞ (m2 − m2 ) sin 2α + 2μv 2λ1 00 λ + λ ≈ h H s . ⎜ ⎟ 3 4 (A.9) ⎜ ⎟ 4v2 sin 2β M = ⎜ λ ⎟ 3 ⎝ 022 0 ⎠ (B.3) Here h = h with m = 125 GeV and H = h . This relations 1 h 2 λ show that the values of quartic couplings can be evaluated 002S solely by m if one chooses a benchmark point in terms of H λ μv ,tanβ, and sin α. with eigenvalues being 2 1,2,s. s (φ+φ− φ+φ− ρ η ρ η η η ρ ρ ) In the basis of 1 2 , 2 1 , 1 2, 2 1, 1 2, 1 2 , we have Appendix B: Unitarity bounds

The initial scattering states can be classified by hyper- (φ+φ− charges and√ isospins√ [73Ð75]. In√ the basis√ of 1 √1 , φ+φ− η η / ρ ρ / η η / ρ ρ / / 2 2 √, 1 1 2, 1 1 2, 2 2 2, 2 2 2, SR SR 2, SI SI / 2), the scattering amplitude is

⎛ √ √ √ √ √ √ ⎞ 4λ 2(λ + λ ) 2λ 2λ 2λ 2λ 2κ 2κ ⎜ 1 3 4 √ 1 √ 1 √ 3 √ 3 √ 1 √ 1⎟ ⎜ (λ + λ ) λ λ λ λ λ κ κ ⎟ ⎜2 √3 4 √4 2 2 3 2 3 2 2 2 2 2 2 2 2⎟ ⎜ ⎟ ⎜ 2λ1 2λ3 3λ1 λ1 λ3 + λ4 λ3 + λ4 κ1 κ1 ⎟ ⎜ √ √ ⎟ ⎜ 2λ 2λ λ 3λ λ + λ λ + λ κ κ ⎟ M = ⎜ √ 1 √ 3 1 1 3 4 3 4 1 1 ⎟ , (B.1) 1 ⎜ λ λ λ + λ λ + λ λ λ κ κ ⎟ ⎜ 2 3 2 2 3 4 3 4 3 2 2 2 2 ⎟ ⎜ √ √ ⎟ ⎜ 2λ 2λ λ + λ λ + λ λ 3λ κ κ ⎟ ⎜ √ 3 √ 2 3 4 3 4 2 2 2 2 ⎟ ⎝ κ κ κ κ κ κ λ λ ⎠ √2 1 √2 2 1 1 2 2 3 S S 2κ1 2κ2 κ1 κ1 κ2 κ2 λS 3λS

⎛ ⎞ 02λ3 + 2λ4 iλ4 −iλ4 λ4 λ4 ⎜ ⎟ ⎜2λ3 + 2λ4 0 −iλ4 iλ4 λ4 λ4 ⎟ ⎜ ⎟ ⎜ iλ −iλ 2λ + 2λ 000⎟ M = ⎜ 4 4 3 4 ⎟ 4 ⎜ − λ λ λ + λ ⎟ (B.4) ⎜ i 4 i 4 023 2 4 00⎟ ⎝ ⎠ λ4 λ4 002λ3 + 2λ4 0 λ4 λ4 0002λ3 + 2λ4

123 Eur. Phys. J. C (2018) 78 :306 Page 21 of 25 306

λ (λ + λ ) (λ + λ ) with√ eigenvalues being 2 3,2 3 4 ,2 3 2 4 , and In the basis of mass eigenstates the quark Yukawa interactions ±2 λ3(λ3 + 2λ4). of the CP-even neutral scalars are + + + + + Finally, in the basis of (ρ1φ , ρ2φ , η1φ , η2φ , ρ1φ , 1 1 1 1 2     ρ φ+ η φ+ η φ+) −Lq = (u¯ Y u u + d¯ Y d d )H + h.c., (C.2) 2 2 , 1 2 , 2 2 , we obtain the matrix as Y L Hi R L Hi R i

⎛ ⎞ where primed fields are mass eigenstates, and 2λ1 00 0 0λ4 0 iλ4 ⎜ ⎟ 02λ3 00λ4 0 −iλ4 0 ⎜ ⎟ R11 R11 tan β − R12 ⎜ 002λ 00−iλ 0 λ ⎟ Y u =− M D + √ h˜u, ⎜ 1 4 4 ⎟ H1 v β u ⎜ λ λ λ ⎟ cos 2 M = ⎜ 00023 i 4 0 4 0 ⎟ , 5 ⎜ ⎟ R11 R11 tan β − R12 ⎜ 0 λ4 0 −iλ4 2λ3 000⎟ Y d =− M D + √ h˜d , ⎜ ⎟ H1 v cos β u ⎜ λ4 0 iλ4 002λ2 00⎟ 2 ⎝ λ λ λ ⎠ R R tan β − R 0 i 4 0 4 0023 0 Y u =− 21 M D + 21 √ 22 h˜u, H2 v β u −iλ4 0 λ4 000 02λ2 cos 2 R R β − R (B.5) d =− 21 D + 21 tan√ 22 ˜d , Y Md h H2 v cos β 2 with eigenvalues being 2λ1,2λ2,2λ3,2(λ3 ± λ4), and λ1 + R31 R31 tan β − R22 Y u =− M D + √ h˜u, λ ± (λ − λ )2 + λ2 H3 v cos β u 2 1 2 4 4. 2 R R β − R The eigenvalues obtained in the above are constrained by d =− 31 D + 31 tan√ 22 ˜d , Y Md h (C.3) unitarity as H3 v cos β 2

|2λ1,2,3,S|≤8π, |2κ1,2|≤8π, Assuming the singlet scalars are decoupled and using Eqs. (3.2)to(3.5), the above quark Yukawa interactions |2(λ3 ± λ4)|≤8π, |2(λ3 + 2λ4)|≤8π,  become |2 λ3(λ3 + 2λ4)|≤8π,  u  ¯ d   u  ¯ d  −L , = (u¯ Y u + d Y d )h + (u¯ Y u + d Y d )H |λ + λ ± (λ − λ )2 + λ2|≤ π, q Y L h R L h R L H R L H R 1 2 1 2 4 4 8 + i(u¯ Y uu + d¯ Y d d )A0 a1,2,3 ≤ 8π. (B.6) L A R L A R   + +¯u (Y2,H + PR + Y1,H + PL )d H + h.c., (C.4) Here a1,2,3 are three other solutions of the following equa- tion: where

3 2 sin α cos(α − β) x − 2x (3λ1 + 3λ2 + 2λS) u =− D + √ ˜u,  Yh Mu h v cos β 2 cos β − κ2 + κ2 − λ λ − λ λ 4x 2 1 2 2 9 1 2 6 1 S α (α − β)  d =− sin D + cos√ ˜d , Y Md h − λ λ + λ2 + λ λ + λ2 h v cos β 2 cos β 6 2 S 4 3 4 3 4 4  α (α − β) u = cos D + sin√ ˜u, + κ2λ − κ κ ( λ + λ ) + κ2λ Y Mu h 16 3 1 2 2 1 2 2 3 4 3 2 1 H v cos β β   2 cos α (α − β) +λ (2λ + λ )2 − 9λ λ = 0. (B.7) d = cos D + sin√ ˜d , S 3 4 1 2 Y Md h H v cos β 2 cos β β u =−tan D + √ 1 ˜u, YA Mu h Appendix C: The quark Yukawa couplings v 2 cos β β d = tan D − √ 1 ˜d , Y Md h The quark Yukawa couplings in the interaction basis are given A v 2 cos β √ by β 2tan D 1 ˜u † Y , + =− M − (h ) V , 1 H v u cos β CKM −Lq = √1 ¯ ((ρ − η ) u + (ρ − η ) u) √ Y uL 1 i 1 y 2 i 2 h u R β 2 2tan D 1 ˜d Y , + = V M − h 2 H CKM v d β (C.5) 1 ¯ u u cos + √ dL ((ρ1 + iη1)y + (ρ2 + iη2)h )dR 2 with − ¯ ( u(φ+)∗ + u(φ+)∗) dL y 1 h 2 u R +¯ ( d φ+ + d φ+) + . . ˜u ≡ † u , ˜d ≡ † d . uL y 1 h 2 dR h c (C.1) h UL h UR h DL h DR (C.6) 123 306 Page 22 of 25 Eur. Phys. J. C (2018) 78 :306

˜u u For UL = 1, we have h = h UR. As a result, After diagonalizing the terms simultaneously with ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ v β Bμ c −s −tξ 10 0 Aμ ˜u = √1 cos ( u ( u )∗ + u ( u )∗) = , W W h31 h31 y11 h32 y12 0 ⎝ 3⎠ = ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ mu Wμ sW cW 0 0 cζ sζ Z1μ 2  v β Zμ 001/cξ 0 −sζ cζ Z2μ ˜u = √1 cos ( u ( u )∗ + u ( u )∗) = , ⎛ ⎞ ⎛ ⎞ h32 h31 y21 h32 y22 0 2 mc cW −sW cζ + tξ sζ −sW sζ − tξ cζ Aμ ⎝ ⎠ ⎝ ⎠ v β = s c cζ c sζ Z μ , ˜u = √1 sin (| u |2 +| u |2) W W W 1 h33 h31 h32 − / / 2 mt 0 sζ cξ cζ cξ Z2μ √ 2m v2 cos2 β (D.3) = t 1 − |yu |2 , (C.7) v sin β 2m2 33 t where ζ is the mass mixing angle and sW ≡ sin θW , cW ≡ cos θW , etc, we obtain the mass eigenvalues for massive where use is made of Eqs. (3.15), (3.17), and (3.18). Other gauge bosons: components of h˜u are vanishing. Moreover, with h˜d =   V † hd and using Eq. (3.10)forhd and hd , we obtain 1 CKM 13 23 m2 = m2 + m2 ∓ (m2 − m2 )2 + 4m4 . (D.4) nonzero components of h˜u as Z1,2 2 Z 22 Z 22 12

√ 2 ≡ ( 2 + 2 )v2/ Here m Z g gY 4 and ˜d = ∗ d + ∗ d = 2mb ( ∗ + ∗ ) h13 Vudh13 Vcdh23 VudVub Vcd Vcb v sin β 2 2 2 2 2 2 −1  2 m ≡ m s t + m  /c − c eg  Q v tξ /cξ , 22 Z W ξ Z ξ W Z H2 w = . × −2 mb , 1 80 10 2 2 1 −1 −1  2 v sin β m ≡ m sW tξ − c s eg  Q v /cξ . (D.5) √ 12 Z 2 W W Z H2 2 ˜d ∗ d ∗ d 2mb ∗ ∗ h = V h + V h = (V Vub + V Vcb) 23 us 13 cs 23 v sin β us cs We can rewrite the Z-boson like mass in terms of the heavy  Z mass and the mixing angle ζ as = . × −2 mb , 5 77 10 v β sin 2 ζ + 2 ( − ζ) √ 2m Z sec 2 m Z 1 sec 2 2m m2 = 2 , (D.6) h˜d = V ∗ hd + V ∗ hd = b (V ∗ V + V ∗ V ) Z1 1 + sec 2ζ 33 ub 13 cb 23 v β ub ub cb cb sin − m and the mixing angle as = 2.41 × 10 3 b . (C.8) v sin β 2m2 (m2 − m2 ) tan 2ζ = 12 Z2 Z . (D.7) (m2 − m2 )2 − m4 Z2 Z 12 Appendix D: U(1) interactions We note that the modified Z-boson mass is constrained by The gauge kinetic terms and mass terms for U(1) and U(1) electroweak precision data, in particular, ρ or T parameter. Y  are The current interactions including Z are given by

L = μ + 3 μ +  μ = μ + 1 μν 1  μν 1  μν g Bμ JB Wμ J3 Zμ JZ  Aμ JEM Z1μ Lg.kin =− Bμν B − Zμν Z − sin ξ Zμν B   4 4 2 × μ + ( − ) μ − μ / tξ sζ cW JEM cζ tξ sζ sW JZ sζ JZ  cξ 1 T 2 μ   − Vμ M V , (D.1) 2 V + − μ +( − ) μ + μ / Z2μ tξ cζ cW JEM sζ tξ cζ sW JZ cζ JZ  cξ

3  T (D.8) where Vμ = (Bμ, Wμ, Zμ) , and

⎛ ⎞ 1 −  2 2 − 2 1  v2 ⎜ m s m cW sW c egZ Q ⎟ ⎜ Z W Z 2 W H2 2 ⎟ ⎜ ⎟ 2 = ⎜ 2 2 2 1 −1  2⎟ . M ⎜ −m cW sW m c − s egZ  Q v ⎟ (D.2) V ⎜ Z Z W 2 W H2 2⎟ ⎝ ⎠ 1 −1  2 1 −1  2 2 c egZ  Q v − s egZ  Q v m  2 W H2 2 2 W H2 2 Z

123 Eur. Phys. J. C (2018) 78 :306 Page 23 of 25 306   2 with hi 2mW 1 2 μ L = (cos β Ri + sin β Ri )hi + h WμW V v 1 2 2v i μ ¯ μ   J = e f γ Q f f, 2 EM m Z 1 2 μ + (cos β Ri1 + sin β Ri2)hi + h Zμ Z . μ = e ¯γ μ(σ 3 − 2 ) , v 2v i JZ f 2sW Q f f 2cW sW (D.13) μ μ  =  ¯γ . JZ  gZ f Q f f (D.9) For a negligible mixing with the singlet scalar, the above Q Q U( ) Here f is the electric charge and f is the 1 charge couplings become of fermion f . For a small gauge kinetic mixing and/or the  mass mixing ζ ,theZ -like gauge boson Z2μ couples to the  2 ε = / / 0 2m electromagnetic current with the overall coefficient of Lh H A = W − (α − β) + (α − β) V sin h cos H tξ cζ cW . v    Ignoring the ZÐZ mixing, the interaction terms for Z 1 2 2 0 2 μ + (h + H + (A ) ) WμW interactions is 2v  m2  1 ¯ μ 1 ¯ μ μ + Z − sin(α − β)h + cos(α − β)H LZ  = gZ  Zμ x tγ t + x bγ b + yμγ¯ μ v 3 3  μ μ + ν¯ γ ν − ( + ) τγ¯ τ 1 2 2 0 2 μ y μ PL μ x y + (h + H + (A ) ) Zμ Z . (D.14) μ μ 2v − (x + y) ν¯τ γ P ντ + y ν¯ γ P ν L 2R R 2R μ − (x + y) ν¯3Rγ PRν3R . (D.10) One can see that in the alignment limit with α = β − π/2 the gauge interactions of h are the same as for the SM Higgs while the triple couplings of heavy Higgs boson to gauge Now we change the basis into the one with mass eigenstates =  =  =  =  bosons vanish. by dR DRdR, u R URu R, dL DL dL and uL UL uL = † = = = such that VCKM UL DL . Taking DR UL 1 and DL  VCKM, the above Z interactions become References v2 2 β| u |2  1 ¯ μ  1 cos y33 ¯ μ  LZ  = gZ  Zμ x t γ PL t + x t γ PRt 3 3 2m2 1. R. Aaij et al. [LHCb Collaboration], Phys. Rev. Lett. 113, t 151601 (2014). https://doi.org/10.1103/PhysRevLett.113.151601. 1 ¯ μ dL  1 ¯ μ  arXiv:1406.6482 [hep-ex] + x d γ  PL d + x b γ PRb 3 i ij j 3 2. S. Bifani, Seminar at CERN (2017). https://indico.cern.ch/event/ μ μ μ 580620/. Accessed 18 Apr 2017 + yμγ¯ μ − (x + y) τγ¯ τ + y ν¯μγ P νμ L 3. S. Bifani [LHCb Collaboration]. arXiv:1705.02693 [hep-ex] μ − (x + y) ν¯τ γ PL ντ 4. R. Aaij et al. [LHCb Collaboration], JHEP 1708, 055 (2017). https://doi.org/10.1007/JHEP08(2017)055. arXiv:1705.05802 μ μ [hep-ex] + y ν¯2Rγ PRν2R − (x + y) ν¯3Rγ PRν3R , 5. R. Aaij et al. [LHCb Collaboration], Phys. Rev. Lett. 111, (D.11) 191801 (2013). https://doi.org/10.1103/PhysRevLett.111.191801. arXiv:1308.1707 [hep-ex] 6. R. Aaij et al. [LHCb Collaboration], JHEP 1602, 104 (2016). where https://doi.org/10.1007/JHEP02(2016)104. arXiv:1512.04442 ⎛ ⎞ [hep-ex] 000 7. J.P. Lees et al. [BaBar Collaboration], Phys. Rev. Lett. 109, dL ≡ † ⎝ ⎠ 101802 (2012). https://doi.org/10.1103/PhysRevLett.109.101802. VCKM 000 VCKM 001 arXiv:1205.5442 [hep-ex] ⎛ ⎞ 8. J.P. Lees et al. [BaBar Collaboration], Phys. Rev. D 88(7), 072012 | |2 ∗ ∗ (2013) Vtd VtdVts VtdVtb ⎜ ⎟ 9. M. Huschle et al. [Belle Collaboration], Phys. Rev. D 92(7), = ⎜ ∗ | |2 ∗ ⎟ . ⎝VtsVtd Vts VtsVtb⎠ (D.12) 072014 (2015). https://doi.org/10.1103/PhysRevD.92.072014. arXiv:1507.03233 [hep-ex] ∗ ∗ | |2 VtbVtd VtbVts Vtb 10. A. Abdesselam et al. [Belle Collaboration]. arXiv:1603.06711 [hep-ex] Considering the general mixing of CP-even scalars while 11. R. Aaij et al. [LHCb Collaboration], Phys. Rev. Lett. ignoring the ZÐZ  mixing, we obtain the interaction between 115(11), 111803 (2015) Erratum: [Phys. Rev. Lett. 115 = (2015) no.15, 159901] https://doi.org/10.1103/PhysRevLett. neutral massive electroweak gauge bosons (V W, Z) and 115.159901, https://doi.org/10.1103/PhysRevLett.115.111803.  Z as arXiv:1506.08614 [hep-ex] 123 306 Page 24 of 25 Eur. Phys. J. C (2018) 78 :306

12. L. Bian, S.M. Choi, Y.J. Kang, H.M. Lee, Phys. Rev. 36. W. Altmannshofer, P. Stangl, D.M. Straub, Phys. Rev. D D 96(7), 075038 (2017). https://doi.org/10.1103/PhysRevD.96. 96(5), 055008 (2017). https://doi.org/10.1103/PhysRevD.96. 075038. arXiv:1707.04811 [hep-ph] 055008. arXiv:1704.05435 [hep-ph] 13. H.M. Lee, Phys. Lett. B 778, 79 (2018). https://doi.org/10.1016/j. 37. L.S. Geng, B. Grinstein, S. Jger, J. Martin Camalich, X.L. Ren, physletb.2018.01.010. arXiv:1708.03564 [hep-ph] R.X. Shi, Phys. Rev. D 96(9), 093006 (2017). https://doi.org/10. 14. H. Fritzsch, Z z Xing, S. Zhou, JHEP 1109, 083 (2011). https:// 1103/PhysRevD.96.093006. arXiv:1704.05446 [hep-ph] doi.org/10.1007/JHEP09(2011)083. arXiv:1108.4534 [hep-ph] 38. M. Ciuchini, A.M. Coutinho, M. Fedele, E. Franco, A. Paul, L. Sil- 15. T. Araki, J. Heeck, J. Kubo, JHEP 1207, 083 (2012). https://doi. vestrini, M. Valli, Eur. Phys. J. C 77(10), 688 (2017). https://doi. org/10.1007/JHEP07(2012)083. arXiv:1203.4951 [hep-ph] org/10.1140/epjc/s10052-017-5270-2. arXiv:1704.05447 [hep- 16. A. Crivellin, G. D’Ambrosio, J. Heeck, Phys. Rev. D 91(7), ph] 075006 (2015). https://doi.org/10.1103/PhysRevD.91.075006. 39. D. Ghosh, Eur. Phys. J. C 77(10), 694 (2017). https://doi.org/10. arXiv:1503.03477 [hep-ph] 1140/epjc/s10052-017-5282-y. arXiv:1704.06240 [hep-ph] 17. M. Buschmann, J. Kopp, J. Liu, X.P.Wang, JHEP 1606, 149 (2016). 40. A.K. Alok, B. Bhattacharya, A. Datta, D. Kumar, J. Kumar, D. https://doi.org/10.1007/JHEP06(2016)149. arXiv:1601.02616 London, Phys. Rev. D 96(9), 095009 (2017). https://doi.org/10. [hep-ph] 1103/PhysRevD.96.095009. arXiv:1704.07397 [hep-ph] 18. W. Altmannshofer, J. Eby, S. Gori, M. Lotito, M. Martone, D. 41. D. Bardhan, P. Byakti, D. Ghosh, Phys. Lett. B 773, Tuckler, Phys. Rev. D 94(11), 115032 (2016). https://doi.org/10. 505 (2017). https://doi.org/10.1016/j.physletb.2017.08.062. 1103/PhysRevD.94.115032. arXiv:1610.02398 [hep-ph] arXiv:1705.09305 [hep-ph] 19. A. Crivellin, J. Heeck, D. Mueller. arXiv:1710.04663 [hep-ph] 42. A. Crivellin, A. Kokulu, C. Greub, Phys. Rev. D 87(9), 20. A.W. El Kaffas, W. Khater, O.M. Ogreid, P. Osland, Nucl. Phys. B 094031 (2013). https://doi.org/10.1103/PhysRevD.87.094031. 775, 45 (2007). https://doi.org/10.1016/j.nuclphysb.2007.03.041. arXiv:1303.5877 [hep-ph] arXiv:hep-ph/0605142 43. A. Faessler, T. Gutsche, S. Kovalenko, V.E. Lyubovitskij, I. 21. B. Grzadkowski, O.M. Ogreid, P. Osland, Phys. Rev. D Schmidt, Phys. Rev. D 82, 075012 (2010). https://doi.org/10.1103/ 80, 055013 (2009). https://doi.org/10.1103/PhysRevD.80.055013. PhysRevD.82.075012. arXiv:1007.2496 [hep-ph] arXiv:0904.2173 [hep-ph] 44. B. Bhattacharya, A. Datta, J.P. Guvin, D. London, R. 22. A. Drozd, B. Grzadkowski, J.F. Gunion, Y. Jiang, JHEP Watanabe, JHEP 1701, 015 (2017). https://doi.org/10.1007/ 1411, 105 (2014). https://doi.org/10.1007/JHEP11(2014)105. JHEP01(2017)015. arXiv:1609.09078 [hep-ph] arXiv:1408.2106 [hep-ph] 45. Y. Amhis et al. [Heavy Flavor Averaging Group (HFAG)]. 23. G. Aad et al. [ATLAS Collaboration], Phys. Rev. D 92(1), arXiv:1412.7515 [hep-ex] 012006 (2015). https://doi.org/10.1103/PhysRevD.92.012006. 46. L. Di Luzio, M. Kirk, A. Lenz. arXiv:1712.06572 [hep-ph] arXiv:1412.2641 [hep-ex] 47. P. Ko, Y. Omura, Y. Shigekami, C. Yu, Phys. Rev. D 24. S. Chatrchyan et al. [CMS Collaboration], JHEP 1401, 096 (2014). 95(11), 115040 (2017). https://doi.org/10.1103/PhysRevD.95. https://doi.org/10.1007/JHEP01(2014)096. arXiv:1312.1129 115040. arXiv:1702.08666 [hep-ph] [hep-ex] 48. F. Borzumati, C. Greub, Phys. Rev. D 58, 074004 (1998). https:// 25. G. Aad et al. [ATLAS Collaboration], Phys. Rev. D 91(1), doi.org/10.1103/PhysRevD.58.074004. arXiv:hep-ph/9802391 012006 (2015). https://doi.org/10.1103/PhysRevD.91.012006. 49. M. Misiak et al., Phys. Rev. Lett. 114(22), 221801 (2015). https:// arXiv:1408.5191 [hep-ex] doi.org/10.1103/PhysRevLett.114.221801. arXiv:1503.01789 26. S. Chatrchyan et al. [CMS Collaboration], Phys. Rev. D [hep-ph] 89(9), 092007 (2014). https://doi.org/10.1103/PhysRevD.89. 50. M. Czakon, P. Fiedler, T. Huber, M. Misiak, T. Schutzmeier, 092007. arXiv:1312.5353 [hep-ex] M. Steinhauser, JHEP 1504, 168 (2015). https://doi.org/10.1007/ 27. G. Aad et al. [ATLAS Collaboration], Phys. Rev. D 90(11), JHEP04(2015)168. arXiv:1503.01791 [hep-ph] 112015 (2014). https://doi.org/10.1103/PhysRevD.90.112015. 51. A. Paul, D.M. Straub, JHEP 1704, 027 (2017). https://doi.org/10. arXiv:1408.7084 [hep-ex] 1007/JHEP04(2017)027. arXiv:1608.02556 [hep-ph] 28. V. Khachatryan et al. [CMS Collaboration], Eur. Phys. J. C 74(10), 52. S. Fajfer, J.F. Kamenik, I. Nisandzic, Phys. Rev. D 85, 3076 (2014). https://doi.org/10.1140/epjc/s10052-014-3076-z. 094025 (2012). https://doi.org/10.1103/PhysRevD.85.094025. arXiv:1407.0558 [hep-ex] arXiv:1203.2654 [hep-ph] 29. G. Aad et al. [ATLAS Collaboration], JHEP 1504, 117 (2015). 53. A. Crivellin, C. Greub, A. Kokulu, Phys. Rev. D 86, https://doi.org/10.1007/JHEP04(2015)117. arXiv:1501.04943 054014 (2012). https://doi.org/10.1103/PhysRevD.86.054014. [hep-ex] arXiv:1206.2634 [hep-ph] 30. S. Chatrchyan et al. [CMS Collaboration], JHEP 1405, 104 (2014). 54. A. Djouadi, Phys. Rep. 457, 1 (2008). https://doi.org/10.1016/j. https://doi.org/10.1007/JHEP05(2014)104. arXiv:1401.5041 physrep.2007.10.004. arXiv:hep-ph/0503172 [hep-ex] 55. R.D. Ball et al., Nucl. Phys. B 867, 244 (2013). https://doi.org/10. 31. G. Aad et al. [ATLAS Collaboration], Eur. Phys. J. C 75(7), 1016/j.nuclphysb.2012.10.003. arXiv:1207.1303 [hep-ph] 349 (2015). https://doi.org/10.1140/epjc/s10052-015-3543-1. 56. A. Buckley, J. Ferrando, S. Lloyd, K. Nordstrm, B. Page, M. Rfe- arXiv:1503.05066 [hep-ex] nacht, M. Schnherr, G. Watt, Eur. Phys. J. C 75, 132 (2015). https:// 32. V. Khachatryan et al. [CMS Collaboration], Eur. Phys. J. C 75(6), doi.org/10.1140/epjc/s10052-015-3318-8. arXiv:1412.7420 [hep- 251 (2015). https://doi.org/10.1140/epjc/s10052-015-3454-1. ph] arXiv:1502.02485 [hep-ex] 57. S. von Buddenbrock et al., Eur. Phys. J. C 76(10), 580 33. S. Cassel, D.M. Ghilencea, G.G. Ross, Nucl. Phys. B (2016). https://doi.org/10.1140/epjc/s10052-016-4435-8. 827, 256 (2010). https://doi.org/10.1016/j.nuclphysb.2009.10.029. arXiv:1606.01674 [hep-ph] arXiv:0903.1118 [hep-ph] 58. S. von Buddenbrock, A.S. Cornell, A. Fadol, M. Kumar, B. Mel- 34. D.C. Kennedy, B.W. Lynn, Nucl. Phys. B 322, 1 (1989). https:// lado, X. Ruan. arXiv:1711.07874 [hep-ph] doi.org/10.1016/0550-3213(89)90483-5 59. V.Khachatryan et al. [CMS Collaboration], Phys. Rev. Lett. 117(3), 35. B. Capdevila, A. Crivellin, S. Descotes-Genon, J. Matias, J. Virto. 031802 (2016). https://doi.org/10.1103/PhysRevLett.117.031802. arXiv:1704.05340 [hep-ph] arXiv:1604.08907 [hep-ex] 60. ATLAS Collaboration, ATLAS-CONF-2016-030 123 Eur. Phys. J. C (2018) 78 :306 Page 25 of 25 306

61. CMS Collaboration, CMS-PAS-EXO-16-030 69. CMS Collaboration [CMS Collaboration], CMS-PAS-B2G-16-003 62. The ATLAS collaboration [ATLAS Collaboration], ATLAS- 70. M. Aaboud et al. [ATLAS Collaboration], Phys. Lett. B CONF-2016-029 765, 32 (2017). https://doi.org/10.1016/j.physletb.2016.11.045. 63. The ATLAS collaboration [ATLAS Collaboration], ATLAS- arXiv:1607.05621 [hep-ex] CONF-2016-070 71. V. Khachatryan et al. [CMS Collaboration], JHEP 1511, 64. V. Khachatryan et al. [CMS Collaboration], Phys. Lett. B 018 (2015). https://doi.org/10.1007/JHEP11(2015)018. 749, 560 (2015). https://doi.org/10.1016/j.physletb.2015.08.047. arXiv:1508.07774 [hep-ex] arXiv:1503.04114 [hep-ex] 72. G. Aad et al. [ATLAS Collaboration], JHEP 1603, 127 (2016). 65. V. Khachatryan et al. [CMS Collaboration], Eur. Phys. J. C 76(7), https://doi.org/10.1007/JHEP03(2016)127. arXiv:1512.03704 371 (2016). https://doi.org/10.1140/epjc/s10052-016-4206-6. [hep-ex] arXiv:1602.08762 [hep-ex] 73. A.G. Akeroyd, A. Arhrib, E.M. Naimi, Phys. Lett. B 490, 66. G. Aad et al. [ATLAS Collaboration], Eur. Phys. J. C 75(9), 119 (2000). https://doi.org/10.1016/S0370-2693(00)00962-X. 412 (2015). https://doi.org/10.1140/epjc/s10052-015-3628-x. arXiv:hep-ph/0006035 arXiv:1506.00285 [hep-ex] 74. I.F. Ginzburg, I.P. Ivanov, Phys. Rev. D 72, 115010 (2005). https:// 67. M. Aaboud et al. [ATLAS Collaboration], Phys. Rev. D doi.org/10.1103/PhysRevD.72.115010. arXiv:hep-ph/0508020 94(5), 052002 (2016). https://doi.org/10.1103/PhysRevD.94. 75. S. Kanemura, K. Yagyu, Phys. Lett. B 751, 289 (2015). https://doi. 052002. arXiv:1606.04782 [hep-ex] org/10.1016/j.physletb.2015.10.047. arXiv:1509.06060 [hep-ph] 68. V. Khachatryan et al. [CMS Collaboration], JHEP 1602, 145 (2016). https://doi.org/10.1007/JHEP02(2016)145. arXiv:1506.01443 [hep-ex]

123