Polarizations of Gravitational Waves in Horndeski Theory

Eur. Phys. J. C (2018) 78:378 https://doi.org/10.1140/epjc/s10052-018-5869-y

Regular Article - Theoretical Physics

Polarizations of gravitational waves in Horndeski theory

Shaoqi Houa , Yungui Gongb , Yunqi Liuc School of Physics, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China

Received: 27 January 2018 / Accepted: 6 May 2018 / Published online: 12 May 2018 © The Author(s) 2018

Abstract We analyze the polarization content of gravi- VIRGO, the future space-borne Laser Interferometer Space tational waves in Horndeski theory. Besides the familiar Antenna (LISA) [7] and TianQin [8], and pulsar timing arrays plus and cross polarizations in Einstein’s General Relativ- (e.g., the International Pulsar Timing Array and the European ity, there is one more polarization state which is the mix- Pulsar Timing Array [9,10]). In fact, in the recent GW170814 ture of the transverse breathing and longitudinal polariza- [4], the Advanced VIRGO detector joined the two aLIGO tions. The additional mode is excited by the massive scalar detectors, so they were able to test the polarization content field. In the massless limit, the longitudinal polarization dis- of gravitational waves for the first time. The result showed appears, while the breathing one persists. The upper bound that the pure tensor polarizations were favored against pure on the graviton mass severely constrains the amplitude of the vector and pure scalar polarizations [4,11]. Additionally, longitudinal polarization, which makes its detection highly GW170817 is the first observation of a binary neutron star unlikely by the ground-based or space-borne interferometers inspiral, and its electromagnetic counterpart, GRB 170817A, in the near future. However, pulsar timing arrays might be was later observed by the Fermi Gamma-ray Burst Moni- able to detect the polarization excited by the massive scalar tor and the International Gamma-Ray Astrophysics Labora- field. Since additional polarization states appear in alterna- tory [5,12,13]. The new era of multi-messenger astrophysics tive theories of gravity, the measurement of the polarizations comes. of gravitational waves can be used to probe the nature of In GR, the gravitational wave propagates at the speed of gravity. In addition to the plus and cross states, the detection light and it has two polarization states, the plus and cross of the breathing polarization means that gravitation is medi- modes. In alternative metric theories of gravity, there may ated by massless spin 2 and spin 0 fields, and the detection of exist up to six polarizations, so the detection of the polar- both the breathing and longitudinal states means that gravi- izations of gravitational waves can be used to distinguish tation is propagated by the massless spin 2 and massive spin different theories of gravity and probe the nature of gravity 0 fields. [14,15]. For null plane gravitational waves, the six polarizations are classified by the little group E(2) of the Lorentz group with the help of the six independent Newman-Penrose 1 Introduction (NP) variables Ψ2, Ψ3, Ψ4 and Φ22 [16–18]. In particular, the complex variable Ψ4 denotes the familiar plus and cross The detection of gravitational waves by the Laser Interferom- modes in GR, the variable Φ22 denotes the transverse breath- eter Gravitational-Wave Observatory (LIGO) Scientific Col- ing polarization, the complex variable Ψ3 corresponds to the laboration and VIRGO Collaboration further supports Ein- vector-x and vector-y modes, and the variable Ψ2 corresponds stein’s General Relativity (GR) and provides a new tool to to the longitudinal mode. Under the E(2) transformation, all study gravitational physics [1–6]. In order to confirm gravi- other modes can be generated from Ψ2,soifΨ2 = 0, then we tational waves predicted by GR, it is necessary to determine may see all six modes in some coordinates. The E(2) clas- the polarizations of gravitational waves. This can be done by sification tells us the general polarization states, but it fails the network of ground-based Advanced LIGO (aLIGO) and to tell us the correspondence between the polarizations and gravitational theories. In Brans-Dicke theory [19], in addi- a e-mail: [email protected] tion to the plus and cross modes Ψ4 of the massless gravitons, b e-mail: [email protected] there exists another breathing mode Φ22 due to the massless c e-mail: [email protected] Brans-Dicke scalar field [17]. 123 378 Page 2 of 15 Eur. Phys. J. C (2018) 78 :378

Brans-Dicke theory is a simple extension to GR. In Brans- effect of the longitudinal polarization on the geodesic devi- Dicke theory, gravitational interaction is mediated by both ation depends on the mass and it is much smaller than that the metric tensor and the Brans-Dicke scalar field, and the of the transverse polarization in the aLIGO and LISA fre- Brans-Dicke field plays the role of Newton’s gravitational quency bands. In the massless limit, the longitudinal mode constant. In more general scalar-tensor theories of gravity, the disappears, while the breathing mode persists. scalar field φ has self-interaction and it is usually massive. In The paper is organized as follows. Section 2 briefly 1974, Horndeski found the most general scalar-tensor theory reviews ELLWW framework for classifying the polarizations of gravity whose action has higher derivatives of gμν and of null gravitational waves. In Sect. 3, the linearized equa- φ, but the equations of motion are at most the second order tions of motion and the plane gravitational wave solution [20]. Even though there are higher derivative terms, there is are obtained. In Sect. 3.1, the polarization states of gravita- no Ostrogradsky instability [21], so there are three physical tional waves in Horndeski theory are discussed by examining degrees of freedom in Horndeski theory, and we expect that the geodesic deviation equations, and Sect. 3.2 discusses the there is an extra polarization state in addition to the plus and failure of ELLWW framework for the massive Horndeski cross modes. If the scalar field is massless, then the additional theory. Section 4 discusses the possible experimental tests of polarization state should be the breathing mode Φ22. the extra polarizations. In particular, Sect. 4.1 mainly calcu- When the interaction between the quantized matter fields lates the interferometer response functions for aLIGO, and and the classical gravitational field is considered, the quadratic Section 4.2 determines the cross-correlation functions for the μναβ terms Rμναβ R and R2 are needed as counterterms to longitudinal and transverse breathing polarizations. Finally, remove the singularities in the energy-momentum tensor this work is briefly summarized in Sect. 5. In this work, we [22]. Although the quadratic gravitational theory is renor- use the natural units and the speed of light in vacuum c = 1. malizable [23], the theory has ghost due to the presence of higher derivatives [23,24]. However, the general nonlinear f (R) gravity [25] is free of ghost and it is equivalent to 2 Review of ELLWW Framework a scalar-tensor theory of gravity [26,27]. The effective mass squared of the equivalent scalar field is f (0)/3 f (0), and the Eardley et. al. devised a model-independent framework [17, massive scalar field excites both the longitudinal and trans- 18] to classify the null gravitational waves in a generic metric verse breathing modes [28–30]. The polarizations of gravi- theory of gravity based on the Newman-Penrose formalism ( ) μ μ μ tational waves in f R gravity were previously discussed in [16]. The quasiorthonormal, null tetrad basis Ea = (k , l , [31–37]. The authors in [33,34] calculated the NP variables mμ, m¯ μ) is chosen to be and found that Ψ2, Ψ4 and Φ22 are nonvanishing. They then μ 1 claimed that there are at least four polarization states in f (R) k = √ (1, 0, 0, 1), (1) gravity. Recently, it was pointed out that the direct applica- 2 μ 1 tion of the framework of Eardley et. al. (ELLWW frame- l = √ (1, 0, 0, −1), (2) work) [17,18] derived for the null plane gravitational waves 2 μ 1 to the massive field is not correct, and the polarization state m = √ (0, 1, i, 0), (3) of the equivalent massive field in f (R) gravity is the mixture 2 of the longitudinal and the transverse breathing modes [37]. μ 1 m¯ = √ (0, 1, −i, 0), (4) Furthermore, the longitudinal polarization is independent of 2 the effective mass of the equivalent scalar field, so it cannot where bar means the complex conjugation. They satisfy the be used to understand how the polarization reduces to the μ μ relation −k lμ = m m¯ μ = 1 and all other inner prod- transverse breathing mode in the massless limit. ucts vanish. Since the null gravitational wave propagates in Since the polarizations of gravitational waves in alterna- the +z direction, the Riemann tensor is a function of the tive theories of gravity are not known in general, so we need retarded time u = t − z, which implies that Rabcd,p = 0, to study it [38–40]. In this paper, the focus is on the polariza- where (a, b, c, d) range over (k, l, m, m¯ ) and (p, q, r, ···) tions of gravitational waves in the most general scalar-tensor range over (k, m, m¯ ). The linearized Bianchi identity and the theory, Horndeski theory. It is assumed that matter minimally symmetry properties of Rabcd imply that couples to the metric, so that test particles follow geodesics. The gravitational wave solutions are obtained from the lin- 1 R [ ; ] = Rab[pq,l] = Rabpq,l = 0. (5) earized equations of motion around the flat spacetime back- ab pq l 3 ground, and the geodesic deviation equations are used to reveal the polarizations of the massive scalar field. The anal- So Rabpq is a trivial, nonwavelike constant, and one can set ysis shows that in Horndeski theory, the massive scalar field Rabpq = Rpqab = 0. One could also verify that Rkk = excites both breathing and longitudinal polarizations. The Rkm = Rkm¯ = Rmm = Rmm¯ = Rm¯ m¯ = 0. The nonvanishing 123 Eur. Phys. J. C (2018) 78 :378 Page 3 of 15 378 components of the Riemann tensor are Rplql. Since Rplql is Class O1 Ψ2 = Ψ3 = Ψ4 = 0,Φ22 = 0. The presence or symmetric in exchanging p and q, there are six independent absence of all modes is observer-independent. nonvanishing components and they can be expressed in terms Class O0 Ψ2 = Ψ3 = Ψ4 = Φ22 = 0. No wave is observed. of the following four NP variables, Apparently, Class II6 is the most general one. If Ψ2 = 0, then 1 we may see all six modes in some coordinates. By setting Ψ2 =− Rklkl, 6 successive variables {Ψ2,Ψ3,Ψ4,Φ22} to zero, one obtains 1 Ψ =− , less and less general classes, and eventually, Class O0 which 3 Rklml¯ (6) 2 is trivial and represents no wave. These NP variables can also Ψ4 =−Rml¯ ml¯ , be grouped according to their helicities. By setting ρ = 0in Ψ Φ Ψ Φ22 =−Rmlml¯ , Eq. (7), one sees that 2 and 22 have helicity 0, 3 has helicity 1 and Ψ4 has helicity 2. {Ψ ,Ψ ,Ψ ,Φ } and the remaining nonzero NP variables are Φ11 = 3Ψ2/2, In order to establish the relation between 2 3 4 22 ¯ ¯ 1 Φ12 = Φ21 = Ψ3 and Λ = Ψ2/2. Ψ2 and Φ22 are real and the polarizations of the gravitational wave, one needs while Ψ3 and Ψ4 are complex. So there are exactly six real to examine the geodesic deviation equation in the Cartesian independent variables, and they can be used to replace the coordinates [17], six components of the electric part of the Riemann tensor. 2 j These four NP variables {Ψ2,Ψ3,Ψ4,Φ22} can be clas- j d x k x¨ = =−Rtjtkx , (8) sified according to their transformation properties under the dt2 group E(2), the little group of the Lorentz group for massless particles. E(2) consists of local transformations preserving where x j represents the deviation vector between two nearby μ the null vector k , i.e., under E(2) transformation, geodesics and j, k = 1, 2, 3. The so-called electric part Rtjtk of the Riemann tensor is given by the following matrix, Ψ = Ψ , 2 2 ⎛ ⎞ − ϑ 1 1 Ψ = i (Ψ + ρΨ¯ ), − (Ψ4 + Φ22) Ψ4 −2Ψ3 3 e 3 3 2 2 2 (7) R = ⎝ 1 Ψ 1 (Ψ − Φ ) 2Ψ ⎠ , Ψ = −i2ϑ (Ψ + ρΨ¯ + ρ¯2Ψ ), tjtk 2 4 2 4 22 3 4 e 4 4 3 6 2 −2Ψ3 2Ψ3 −6Ψ2 Φ = Φ + ρΨ + ρ¯Ψ¯ + ρρΨ¯ , 22 22 2 3 2 3 6 2 (9) where ϑ ∈[0, 2π) represents a rotation around the z direc- where and stand for the real and imaginary parts. So tion and the complex number ρ denotes a translation in the these NP variables can be used to label the polarizations of Euclidean 2-plane. From these transformations, one finds null gravitational waves. More specifically, Ψ4 and Ψ4 out that only Ψ is invariant and the amplitudes of the four 2 represent the plus and the cross polarizations, respectively; NP variables are not observer-independent. However, the Φ22 represents the transverse breathing polarization, and Ψ2 absence (zero amplitude) of some of the four NP variables is represents the longitudinal polarization; finally, Ψ3 and observer-independent. Based on this, six classes are defined Ψ3 represent vector-x and vector-y polarizations, respec- as follows [17]. tively. Figure 1 in Ref. [17] displays how these polarizations deform a sphere of test particles, which will not be Class II Ψ = 0. All observers measure the same nonzero 6 2 reproduced here. In terms of Rtjtk, the plus mode is spec- amplitude of the Ψ2 mode, but the presence or ˆ ified by P+ =−Rtxtx + Rtyty, the cross mode is speci- absence of all other modes is observer-dependent. ˆ fied by P× = R , the transverse breathing mode is rep- Class III Ψ = 0, Ψ = 0. All observers measure the txty 5 2 3 ˆ = + Ψ resented by Pb Rtxtx Rtyty, the vector-x mode is rep- absence of the 2 mode and the presence of the ˆ Ψ Ψ resented by Pxz = Rtxtz, the vector-y mode is represented 3 mode, but the presence or absence of 4 and ˆ by Pyz = Rtytz, and the longitudinal mode is represented by Φ22 is observer-dependent. Pˆ = R . For null gravitational waves, the four NP vari- Class N3 Ψ2 = Ψ3 = 0,Ψ4 = 0 = Φ22. The presence or l tztz {Ψ ,Ψ ,Ψ ,Φ } absence of all modes is observer-independent. ables 2 3 4 22 with six independent components are related with the six electric components of Riemann Class N2 Ψ2 = Ψ3 = Φ22 = 0,Ψ4 = 0. The presence or absence of all modes is observer-independent. tensor, and they provide the six independent polarizations ˆ ˆ ˆ ˆ ˆ ˆ {P+, P×, Pb, Pxz, Pyz, Pl }. According to the E(2) classification, the longitudinal mode which corresponds to nonzero 1 Λ =−Ψ / et. al. Note that in Ref. [17], 2 2 because Eardley chose a Ψ different sign convention relating NP variables to the components of 2 belongs to the most general class 6. The presence of the Riemann tensor. the longitudinal mode means that all six polarizations are 123 378 Page 4 of 15 Eur. Phys. J. C (2018) 78 :378 present in some coordinate systems. Although we obtain the G4 = φ. And finally, to reproduce f (R) gravity, one can set six polarizations according to the E(2) classification for gen- G3 = G5 = 0, K = f (φ) − φ f (φ) and G4 = f (φ) with eral metric theory of gravity, the polarizations of gravitational f (φ) = df(φ)/dφ. waves in alternative theories of gravity are unknown in gen- The variational principle gives rise to the equations of eral, so we need to discuss them case by case. motion. The full set of equations can be found in Ref. [41], It is now ready to apply this framework to discuss some for instance. For the purpose of this work, one expands the specific metric theories of gravity. For example, GR predicts metric tensor field gμν and φ in the following way, the existence of the plus and the cross polarizations, and it gμν = ημν + hμν, (12) can be easily checked that only Ψ = 0. For Brans-Dicke 4 φ = φ + ϕ, theory [19], there is one more polarization, the transverse 0 (13) breathing polarization, excited by the massless scalar field, so where φ0 is a constant. The equations of motion are expanded Ψ = Φ in addition to 4 0, 22 is also nonzero [17]. In particular, up to the linear order in hμν and ϕ, Ψ2 = 0, so the longitudinal polarization is absent in Brans- 1 (1) Dicke theory. So for Brans-Dicke theory, − K (0) + G (0)Gμν − G ,φ(0)(ϕμν − ημνϕ) = 0, 2 4 4 ⎛ ⎞ − 1 (Ψ + Φ ) 1 Ψ (14) 2 4 22 2 4 0 BD = ⎝ 1 Ψ 1 (Ψ − Φ ) ⎠ . K,φ(0) +[K,X (0) − 2G3,φ(0)]ϕ + K,φφ(0)ϕ Rtjtk 2 4 2 4 22 0 (10) (1) 000 +G4,φ(0)R = 0, (15) μ where = ∂ ∂μ from now on, G (0) = G (φ , 0), K (0) = In the next section, the plane gravitational wave solution to 4 4 0 (φ , ) (1) (1) the linearized equation of motion will be obtained for Horn- K 0 0 , and Gμν and R are the linearized Einstein tensor ( ) deski theory, and the polarization content will be determined. and Ricci scalar, respectively. In addition, the symbol K,X 0 = ∂ /∂ φ = φ = = It will show that because of the massive scalar field, the elec- means the value of K,X K X at 0 and X X0 tric part of the Riemann tensor takes a different form from 0, and the remaining symbols can be understood similarly. Eq. (10). Its components are expressed in terms of a different In order to obtain the gravitational wave solutions around = η φ = φ set of NP variables. the flat background, one requires that gμν μν and 0 be the solution to Eqs. (14) and (15), which implies that

K (0) = 0, K,φ(0) = 0. (16) 3 Gravitational wave polarizations in Horndeski theory Substituting this result into Eqs. (14) and (15), and after some In this section, the polarization content of the plane gravi- algebraic manipulations, one gets, tational wave in Horndeski theory will be determined. The action of Horndeski theory is [20], ( − m2)ϕ = 0, (17) ( ) (1) G4,φ 0 √ Gμν − (∂μ∂νϕ − ημνϕ) = 0, (18) 4 G (0) S = d x −g(L2 + L3 + L4 + L5), (11) 4 where G4(0) = 0 and the mass squared of the scalar field is where K,φφ(0) m2 =− . (19) = (φ, ), =− (φ, )φ, K, ( ) − G ,φ( ) + G2 ( )/G ( ) L2 K X L3 G 3 X X 0 2 3 0 3 4,φ 0 4 0 2 μ ν L4 = G4(φ, X)R + G4,X (φ) − (∇μ∇νφ)(∇ ∇ φ) , For Brans-Dicke theory, K,φφ(0) = 0, so m = 0. For the 2 μ ν 1 3 f (R) theory, one gets m = f (0)/3 f (0), which agrees L = G (φ, X)Gμν∇ ∇ φ − G ,X [(φ) 5 5 6 5 with the previous work [37]. μ ν −3(φ)(∇μ∇νφ)(∇ ∇ φ) One can decouple Eq. (18)fromEq.(17) by reexpressing μ α β ˜ +2(∇ ∇αφ)(∇ ∇β φ)(∇ ∇μφ)]. them in terms of the auxiliary tensor field hμν defined by, μ μ Here, X =−∇μφ∇ φ/2, φ =∇μ∇ φ, the functions ˜ 1 αβ hμν = hμν − ημνη hαβ − ημνσ ϕ, (20) K , G3, G4 and G5 are arbitrary functions of φ and X, and 2 G , (φ, X) = ∂G (φ, X)/∂ X with j = 4, 5. Horndeski j X j σ = ( )/ ( ) theory reduces to several interesting theories as its subclasses with G4,φ 0 G4 0 . The original metric tensor pertur- by suitable choices of these functions. For example, one bation can be obtained by inverting the above relation, i.e., obtains GR with K = G = G = 0 and G = 1/(16πG). 3 5 4 ˜ 1 αβ ˜ G = G = K = ωX/φ hμν = hμν − ημνη hαβ − ημνσ ϕ. (21) For Brans-Dicke theory, 3 5 0, 2 and 2 123 Eur. Phys. J. C (2018) 78 :378 Page 5 of 15 378

Therefore, Eq. (18) becomes wave solution (27) and (28). Written as a 3 × 3 matrix, it is given by ˜ρ 1 ˜ 1 ˜ρσ ∂ρ∂(μhν) − hμν − ημν∂ρ∂σ h = 0, (22) 2 2 R = ⎛tjtk ⎞ 1 2 1 2 ˜ 1 2 ˜ The equations of motion remain invariant under the gauge − q σ ϕ + Ω hxx Ω hxy 0 ⎜ 2 t 2 2 ⎟ transformation, ⎜ ⎟ ⎜ 1 Ω2h˜ − 1 q2σ ϕ − 1 Ω2h˜ ⎟ . ⎝ 2 xy 2 t 2 xx 0 ⎠ ˜ ˜ ρ ϕ = ϕ, hμν = hμν − ∂μξν − ∂νξμ + ημν∂ρξ , (23) − 1 2σ ϕ 002 m (29) with xμ = xμ + ξ μ. Therefore, one can choose the trans- ∂ ˜μν = ημν ˜ = verse traceless gauge ν h 0, hμν 0byusing From this expression, one can easily recognize the familiar the freedom of coordinate transformation, and the linearized plus and cross polarizations by setting ϕ = 0, which leads Eqs. (17) and (18) become the wave equations to a symmetric, traceless matrix with nonvanishing compo- =− = Ω2 ˜ / = = ( − m2)ϕ = 0, (24) nents Rtxtx Rtyty hxx 2 and Rtxty Rtytx 2 ˜ ˜ ˜ Ω hxy/2. Indeed, hμν generates the plus and the cross polar- hμν = 0. (25) izations. ¯ ρσ By the analogue of hμν = hμν − ημνη hρσ in GR, it is To study the polarizations caused by the scalar field, one ˜ ˜ = = easy to see that the field hμν represents the familiar massless sets hμν 0. If the scalar field is massless (m 0) as in = graviton and it has two polarization states: the plus and cross Brans-Dicke theory, one finds out that Rtztz 0, and the polarizations. The scalar field ϕ is massive in general, and it scalar field excites only the transverse breathing polarization ˜ = decouples from the massless tensor field hμν. Suppose the with Rtxtx Rtyty. If the scalar field is massive, one can = massless and the massive modes both propagate in the +z perform a Lorentz boost such that qz 0, i.e., one works in direction with the wave vectors, the rest frame of the scalar field. In the rest frame, from Eq. 2 = 2 = = = (29), we find that qt m and Rtxtx Rtyty Rtztz 0. μ μ the geodesic deviation equations are, k = (Ω, 0, 0,Ω), q = (qt , 0, 0, qz), (26) 1 respectively, where the dispersion relation for the scalar field x¨ j = m2σ ϕx j , j = 1, 2, 3. (30) 2 − 2 = 2 2 is qt qz m . The propagation speed of the massive scalar v = 2 − 2/ field is qt m qt . The plane wave solutions to Eqs. Suppose the initial deviation vector between two geodesics j = ( , , ) (24) and (25) take the following form is x0 x0 y0 z0 , and one integrates the above equations twice to obtain the changes in the deviation vector, ˜ −ik·x hμν = eμνe , (27) − · ϕ = ϕ e iq x , (28) 1 0 δx j ≈− σ ϕx j . (31) 2 0 where ϕ0 and eμν are the amplitudes of the waves with ν μν k eνμ = 0 and η eμν = 0. From the above equation, one finds that if σ is independent of the mass m,soareδx j . However, for f (R) gravity, m2σ = 3.1 Polarizations 1/3, so δx j is proportional to 1/m2 [37]. Eq. (31)impliesthat the sphere of test particles will oscillate isotropically in all The polarizations of gravitational wave can be extracted by directions, so the massive scalar field excites the longitudinal studying the relative acceleration of two nearby test particles mode in addition to the breathing mode. Note that in the rest moving in the field of the gravitational wave. One assumes frame of a massive field (such as ϕ) one cannot take the that the matter fields minimally couple with the metric tensor massless limit, as there is no rest frame for a massless field gμν, while there are no direct interactions between the matter with the speed of light as its propagation speed. However, fields and the scalar field φ. Therefore, freely falling test the massless limit can be taken in Eq. (29) if the rest frame particles follow geodesics, and the relative acceleration is condition qt = m is not imposed a prior. In the massless given by the linearized geodesic deviation equations Eq. (8). limit, Rtztz = 0 and Rtxtx = Rtyty = 0. The polarizations of gravitational wave can be understood However, in the actual observation, it is highly unlikely by placing a sphere of test particles in the spacetime, and that the test particles, such the mirrors in aLIGO/VIRGO, are studying how this sphere deforms. With the solutions (27) at the rest frame of the (massive) scalar gravitational wave. and (28), one calculates the electric part Rtjtk for the plane So one should also study how the scalar gravitational wave 123 378 Page 6 of 15 Eur. Phys. J. C (2018) 78 :378 deforms the sphere when qz = 0. In this case, the deviation vector is given by 1 δx ≈− σ ϕx , (32) 2 0 1 δy ≈− σ ϕy , (33) 2 0 2 δ ≈−1 m σ ϕ . z 2 z0 (34) 2 qt When the gravitational wave passes by, this sphere deforms asshowninFigs.1 and 2 for the massive and massless scalars, respectively. In each figure, Panel (a) shows how the sphere (a) deforms in the 3 dimensional space in which the semitrans- Top view parent ellipsoids are the results of deforming the spheres, and y Panels (b) and (c) show the top view and the side view with the dashed ellipses representing the results of the deforma- tion of the circles, respectively. Because of the cylindrical symmetry around the z axis, the xOy cross section is not displayed in each figure. In both figures, one finds out that Panels (b) behave the same, as Eqs. (32) and (33) are inde- x pendent of the mass. So Panels (b) represent the transverse breathing polarization, as the circle (the intersection of the original sphere with the xOy plane) deforms to a circle of a different size. From Panel (c) in Fig. 1, one finds out that the z coordinates of the test particles change (except the test parti- (b) cles on xOyplane), and the change in z is smaller than that in z Side view x (or y). This panel represents the longitudinal polarization. In contrast, Panel (c) in Fig. 2 shows that the z coordinates of the test particles remain the same, so in the massless case, there does not exist the longitudinal polarization. Note that in the above discussion, we distinguish the breathing polarization [described by Eqs. (32) and (33)] from the longitudinal x one [described by Eq. (34)] in the massive case, but since they are excited by the same field ϕ, they represent a single degree of freedom. We thus call the polarization state excited by the scalar a mix polarization of the breathing and the longitudinal polarizations. (c) In summary, the polarizations of gravitational waves in Horndeski theory include the plus and cross polarizations Fig. 1 The transverse breathing and the longitudinal polarizations ˜ induced by the spin 2 field hμν. The scalar field excites both excited by the massive scalar field the transverse breathing and longitudinal polarizations if it is massive. However, if it is massless, the scalar field excites merely the transverse breathing polarization. In terms of the 3.2 Newman-Penrose variables basis introduced for the massless fields in [17], there are three ˆ ˆ polarizations: the plus state P+, the cross state P× and the Now we calculate the NP variables with the plane wave solu- ˆ ˆ mix state of Pb and Pl . In the massless limit, the mix state tion (27) and (28), we get ˆ ˆ reduces to the pure state Pb. Note that the vector modes Pxz ˆ and Pyz are absent in Horndeski theory, this seems to be in conflict with the E(2) classification because the longitudinal 1 ˆ Ψ2 = (Rtxtx + Rtyty − 2Rtztz mode Pl is present, so we need to discuss the application of 12 ( ) E 2 classification. 1 (35) + 2Rxyxy − Rxzxz − Ryzyz) + iRtzxy 2 = 0, 123 Eur. Phys. J. C (2018) 78 :378 Page 7 of 15 378

m2 Φ11 = Λ = Φ22. (39) ( + 2 − 2)2 2 qt qt m

Note that Ψ2 =−Rtztz/6 for null gravitational waves only, in general case we should use Eq. (35). The naive application of the ELLWW framework to the massive case gives the conclusion that Horndeski gravity has the plus, cross and transverse breathing polarizations, because Ψ2 = Ψ3 = 0. Moreover, Φ00 is absent and both Φ11 and Λ are proportional to Ψ2 in the ELLWW framework, which is in contradiction with Eqs. (38) and (39). These indicate the failure of this (a) framework for the massive Horndeski theory. In particular, Ψ y Top view the absence of 2 means that there would be no longitudinal polarization if the ELLWW framework were correct. However, the previous discussion in Sect. 3.1 clearly shows the existence of the longitudinal polarization. Therefore, the ELLWW framework cannot be applied to a theory which predicts the existence of massive gravitational waves. The x massive scalar ﬁeld excites both the breathing and longitudinal polarizations. In fact, Rtjtk can be rewritten in terms of NP variables as a matrix displayed below,

⎛ ⎞ ϒ − 1 Ψ 1 Ψ (b) 2 4 2 4 0 = ⎝ 1 Ψ ϒ + 1 Ψ ⎠ , z Side view Rtjtk 2 4 2 4 0 (40) 00−2(Λ + Φ11)

Φ +Φ ϒ =− Λ− 00 22 with 2 2 . It is rather different from Eq. (10). From the above expression, it is clear that some linear com- Φ Φ Φ Λ x binations of 00, 11, 22 and correspond to the transverse breathing and the longitudinal polarizations. In particular, ϒ =−2Λ − (Φ00 + Φ22)/2 represents the transverse breathing polarization, while −2(Λ + Φ11) represents the longitudinal polarization. The plus and cross polarizations are still represented by Ψ4 and Ψ4, respectively. In the (c) massless limit, Φ00 = Φ11 = Λ = 0 according to Eqs. (38) Fig. 2 The transverse breathing polarization excited by the massless and (39), so Eq. (40) takes the same form as Eq. (10) since scalar field the ELLWW framework applies in this case. Note that when the scalar field is massive, Eq. (40) actually takes the same form as Eq. (29) by setting qz = 0 and qt = m in Eqs. (36), (37), (38) and (39). and the nonvanishing ones are These discussion tells us that the detection of polarizations probes the nature of gravity. If only the plus and cross modes are detected, then gravitation is mediated by massless spin Ψ =−Ω2(h˜ − ih˜ ), (36) 2 field and GR is confirmed. The detection of the breath- 4 xx xy ing mode in addition to the plus and cross modes means (q + q2 − m2)2 t t that gravitation is mediated by massless spin 2 and spin 0 Φ22 = σϕ, (37) 4 fields. If the breathing, plus, cross and longitudinal modes ( − 2 − 2)2 are detected, then gravitation is mediated by massless spin 2 qt qt m Φ00 = Φ22, (38) and massive spin 0 fields. For the discussion on the detection ( + 2 − 2)2 qt qt m of polarizations, please see Refs. [42,43]. 123 378 Page 8 of 15 Eur. Phys. J. C (2018) 78 :378

4 Experimental Tests 1

4.1 Interferometers 10–5

In this subsection, the response functions of the interferom- |Yb| eters will be computed for the transverse breathing and lon- |Y | –10 l

)| 10 f gitudinal polarizations following the ideas in Refs. [28,44]. ( Y The detection of GW170104 placed an upper bound on the | mass m of the graviton [3], g 10–15

< = . × −23 / 2. mg mb 7 7 10 eV c (41) 10–20

10–7 10–5 0.001 0.100 10 1000 In obtaining this bound,the assumed dispersion relation E2 = f(Hz) 2 2 + 2 δΨ ( ) p c mg, leads to dephasing mg of the waves relative to the phase evolution in GR, which is given by the template Fig. 3 The absolute values of the longitudinal and transverse response | ( )| | ( )| = of the waveform predicted by GR. The constraint on δΨ (mg) functions Yl f and Yb f as functions of f for aLIGO (L 4km) to a scalar gravitational wave with the mass m = mb (dashed red curve). from the observations was used to derive the bound on mg. The solid curve denotes |Yb( f )| and the dashed curve denotes |Yl ( f )| Since there are only the plus and cross polarizations in GR, this bound might not be simply applied to the scalar mode. 2 Whether Eq. (41) can be applied to the scalar graviton is m i2π fL(v+1) Yl ( f ) = 4π fe beyond the scope of the present work. Here we assume that 16π 3 f 3 the upper bound mb is applicable to the scalar field and study 4iπ fL e i2π fL(v−1) how the upper bound affects the detector responses to the − e [2π fL(v − 1) (v − 1)3 L scalar mode. This bound potentially places severe constraint (42) on the effect of the longitudinal polarization on the geodesic + i2]+2π fL(v − 1) − i2 deviation in the massive case in the high frequency band, 1 π (v+ ) + 2π fL(v + 1) + ei2 fL 1 while the transverse breathing mode can be much stronger as (v + 1) 3 L long as the amplitude of scalar field ϕ is large enough. There- × π (v + ) + − , fore, although there exits the longitudinal mode in the mas- [2 fL 1 2i] i2 sive scalar-tensor theory, the detection of its effect is likely very difficult in the high frequency band. However, in f (R) where f = qt /(2π) is the frequency of gravitational waves gravity, the displacement in the longitudinal direction is inde- and the propagation speed v varies with the frequency. To pendent of the mass and the displacements in the transverse obtain the response function for the transverse mode, project directions become larger for smaller mass, so this mix mode the light in the x direction and place the mirror at x = L. can place strong constraints on f (R) gravity. The response function is, In the interferometer, photons emanate from the beam i2π fL[ π − ( π )] splitter, are bounced back by the mirror and received by the i2π fL e 2 fL sin 2 fL Yb( f ) = e + , (43) beam splitter again. The round-trip propagation time when 2π fL the gravitational wave is present is different from that when the gravitational wave is absent. To simplify the calculation which is independent of the mass m of the scalar field. of the response functions, the beam splitter is placed at the Figure 3 shows the absolute values of the longitudinal and origin of the coordinate system. Then the change in the round- transverse response functions for aLIGO (L = 4km)toa trip propagation time comes from two effects: the change in scalar gravitational wave with the mass m = mb. Comparing the relative distance between the beam splitter and the mir- the response functions shows that the transverse response is ror due to the geodesic deviation, and the distributed grav- much larger than that of the longitudinal mode in high fre- itational redshift suffered by the photon in the field of the quencies, so the detection of the longitudinal mode becomes gravitational wave [44]. very difficult in the high frequency band. First, consider the response function for the longitudinal One can also understand the difficulty to use the interfer- polarization. To this end, project the light in the z direction ometers to detect the longitudinal polarizations in the follow- = |¨j / j | and place the mirror at z L. The response function is thus ing way. Table 1 lists the magnitudes of x x0 normalized given by, with σ ϕ for the longitudinal and the transverse modes assum- 123 Eur. Phys. J. C (2018) 78 :378 Page 9 of 15 378

|¨j / j | Pulsar Table 1 The dependence of the magnitudes of x x0 for the longitudinal and transverse modes on the frequencies of gravitational waves 2 z in units of Hz σϕ assuming the scalar mass is mb −3 −7 q 100 Hz 10 Hz 10 Hz z n Longitudinal 2.81×10−15 2.81×10−15 2.81×10−15 Transverse 1.97 × 105 1.97×10−5 1.97×10−13

ing the graviton mass is also mb at three different frequencies: 100 Hz, 10−3 Hz, and 10−7 Hz. The first frequency lies in the frequency band of the ground-based detector like aLIGO Earth l x and VIRGO, the second frequency lies in the frequency band ˆ of space-borne observatory like LISA, and the last frequency Fig. 4 The gravitational wave is propagating in the direction of qz, and the photon is traveling in −ˆn direction at the leading order. lˆ is lies in the sensitive band of pulsar timing arrays [45]. perpendicular to qˆz and in the same plane determined by qˆz and nˆ.The Table 1 shows that the effect of the longitudinal mode angle between nˆ and lˆ is β on the geodesic deviation is smaller than that of the transverse mode by 19 to 9 orders of magnitude at higher frequen- −7 cies. But at the lower frequencies, i.e., at 10 Hz, the two works have been done, for example, Refs. [56–59] and ref- modes have similar amplitudes. Therefore, aLIGO/VIRGO erences therein. and LISA might find it difficult to detect the longitudinal In their treatments, it is assumed that all the polarization mode, but pulsar time arrays should be able to detect the mix modes have the same mass, either zero or not. If all polar- polarization state with both the longitudinal and transverse izations propagate in the +z direction at the speed of light, modes. The results are consistent with those for the massive there will be three linearly independent Killing vector fields, graviton in a specific bimetric theory [46]. χ μ = δμ,χμ = δμ χ μ = δμ + δμ 1 1 2 2 , and 3 0 3 . Using the con- χ μ = , , μ servation of pμ j ( j 1 2 3) for photon’s 4-velocity p ν μ 4.2 Pulsar timing arrays satisfying p ∇ν p = 0, one obtains the change in the locally observed frequency of the radiation and integrates to obtain A pulsar is a rotating neutron star or a white dwarf with a very the time residual R(t) [48,60,61]. One could also directly strong magnetic field. It emits a beam of the electromagnetic integrate the time component of the photon geodesic equa- radiation. When the beam points towards the Earth, the radi- tion to obtain the change in the frequency [54,62]. The later ation can be observed, which explains the pulsed appearance method can be applied to massive case. In the present work, of the radiation. Millisecond pulsars can be used as stable a different method will be used by simply calculating the clocks [47]. When there is no gravitational wave, one can 4-velocities of the photon and observers on the Earth and observe the pulses at a steady rate. The presence of the grav- the pulsar. Since different polarizations propagate at differ- itational wave will alter this rate, because it will affect the ent speeds, there are not enough linearly independent Killing propagation time of the radiation. This will lead to a change vector fields. The first method cannot be used any way. in the time-of-arrival (TOA), called time residual R(t).Time In order to calculate the time residual R(t) caused by the residuals caused by the gravitational wave will be corre- gravitational wave solution (27) and (28), one sets up a coor- lated between pulsars, and the cross-correlation function is dinate system shown in Fig. 4, so that when there is no gravi- C(θ) = Ra(t)Rb(t) , where θ is the angular separation of tational wave, the Earth is at the origin, and the distant pulsar pulsars a and b, and the brackets imply the ensemble aver- is assumed to be stationary in the coordinate system and one age over the stochastic background. This enables the detec- can always orient the coordinate system such that the pul- tion of gravitational waves and the probe of the polarizations. sar is located at x p = (L cos β,0, L sin β). qˆz is the unit The effects of the gravitational wave in GR on the time vector pointing to the direction of the gravitational wave, residuals were first considered in Refs. [48–50]. Hellings and nˆ is the unit vector connecting the Earth to the pulsar, and ˆ Downs [51] proposed a method to detect the effects by using l =ˆqz ∧ (nˆ ∧ˆqz)/ cos β =[ˆn −ˆqz(nˆ ·ˆqz)]/ cos β is the the cross-correlation of the time derivative of the time resid- unit vector parallel to the y axis. In the leading order, i.e., uals between pulsars, while Jenet et. al. [52] directly worked in the absence of gravitational waves, the 4-velocity of the μ with the time residuals instead of the time derivative. The photon is u = γ0(1, − cos β,0, − sin β) with γ0 = dt/dλ later work was generalized to massless gravitational waves a constant and λ an arbitrary affine parameter. The perturbed μ μ μ μ ν in alternative metric theories of gravity in Ref. [53], and fur- photon 4-velocity is u = u +v , and since gμνu u = 0, ther to massive gravitational waves in Refs. [54,55]. More one obtains 123 378 Page 10 of 15 Eur. Phys. J. C (2018) 78 :378 γ v0 = 0 h˜ cos2 β − v cos β − v sin β. (44) So the 4-velocity of the Earth is approximately 2 11 1 3 =− = μ 1 qz Note that one chooses the gauge such that e11 e22, e12 u = 1 + σ ϕ cos q t, 0, 0, − σ ϕ cos q t . (54) ˜ e 0 t 0 t e21 are the only nonvanishing amplitudes for hμν, which can 2 2qt always be made. The photon geodesic equation is Similarly, one can obtain the 4-velocity of the pulsar, which μ du μ ρ σ is 0 = + Γ ρσ u u dλ μ (45) 1 du μ ρ σ μ ≈γ + Γ . u = 1 + σ ϕ0 cos(qt t − qz L sin β),0, 0 ρσ u u p 2 dt (55) , − qz σ ϕ ( − β) , The calculation shows that 0 0 cos qt t qz L sin 2qt v0 = γ σ ϕ cos[(q + q sin β)t − q (L + t ) sin β] 0 0 t z z e μ up to linear order. The form of u p can be understood, realizing e11 μ μ − (1 − sin β)cos[Ω(1 + sin β)t that χ and χ are still the Killing vector fields. 2 1 2 Therefore, the frequency of the photon measured by an −Ω( + ) β] , L te sin (46) observer comoving with the pulsar is v1 = γ0{−σ ϕ0 cos β cos[(qt + qz sin β)t − qz(L + te) sin β] =− μ +e cos β cos[Ω(1 + sin β)t − Ω(L + t ) sin β]}, (47) fe uμu p 11 e q − q sin β v2 = γ0e12 cos β cos Ω[(1 + sin β)t − (L + te) sin β], (48) = γ + t z σ ϕ ( − β) 0 1 0 cos qt te qz L sin (56) 2qt v3 = γ0 − σ ϕ0 sin β cos[(qt + qz sin β)t − qz(L + te) sin β] − e11 ( − β) Ω( − β) , e 1 sin cos te L sin − 11 (1 − sin β)cos[Ω(1 + sin β)t 2 2 and the frequency measured by another observer on the Earth −Ω(L + te) sin β] , (49) is where te is the time when the photon is emitted from the =− μ pulsar. Eq. (46) is consistent with Eq. (44). fr uμue The 4-velocities of the Earth and the pulsar also change q − q sin β =γ 1 + t z σ ϕ cos q (t + L) due to the gravitational wave. Take the 4-velocity of the Earth 0 0 t e (57) 2qt for instance. Suppose when the gravitational wave is present, − e11 ( − β) Ω( + ) . μ 0 1 sin cos te L its 4-velocity is given by ue = u (1, ve). The normalization 2 μ e of ue implies that The frequency shift is thus given by 0 = + 1 + (v2). ue 1 h00 O e (50) 2 f − f q − q qˆ ·ˆn e r = t z z σ ϕ(t − L, Lnˆ) − ϕ(t, 0) The geodesic equation for the Earth is, fr 2qt ˆ j ˆk − n n ˜ ( − , ˆ) − ˜ ( , ) , 2 μ ρ ν h jk t L Ln h jk t 0 d x μ dx dx 2(1 +ˆqz ·ˆn) 0 = + Γ ρν τ 2 τ τ d d d (58) 2 μ 0 μ (51) ≈( 0)2 d x + Γ μ + 0 due dx , ue 00 ue dt2 dt dt where t = te + L is the time when the photon arrives at the Earth at the leading order. This expression (58) can be μ where τ is the proper time and x are the coordinates of the easily generalized to a coordinate system with an arbitrary Earth. One sets x = y = 0, which is consistent with Eq. (51). orientation and at rest relative to the original frame. It can be Then, one obtains the following solution checked that in the massless limit, the change in the ratio of frequencies agrees with Eq. (2) in Ref. [53]. The contribution v3 ≈−qz σ ϕ . e 0 cos qt t (52) 2qt of the massive scalar field also agrees with Eq. (2) in Ref. [55] which was derived provided all polarizations have the same In addition, with Eq. (50), mass. Eq. (58) gives the frequency shift caused by a monochro- 0 1 u = 1 + σ ϕ cos qt t. (53) e 2 0 matic gravitational wave. Now, consider the contribution of 123 Eur. Phys. J. C (2018) 78 :378 Page 11 of 15 378 a stochastic gravitational wave background which consists and xb = L2nˆ2, respectively. The angular separation is θ = of monochromatic gravitational waves, arccos(nˆ1 ·ˆn2). The cross-correlation function is thus given by h˜ (t, x) jk ∞ Ω d 2 ˆ ˜ P ˆ P ˆ = d k h (Ω, k) exp[iΩ(t − k · x)] , (θ) = ( ) ( ) −∞ 2π jk C Ra T Rb T P=+,× ∞ (q − q qˆ ·ˆn )(q − q qˆ ·ˆn ) (59) = dq d2qˆ t z z 1 t z z 2 t z π 2 5 ϕ(t, x) m 4 qt ∞ ×|ϕ |2( )P, dqt 2 c qt = d qˆz ϕ0(qt , qˆz) exp[i(qt t − qzqˆz · x)] , (60) −∞ 2π (66) where h˜ P (Ω, kˆ) is the amplitude of the gravitational wave propagating in the direction kˆ at the angular frequency Ω with where =+ × P the polarization P or , jk is the polarization tensor for the polarization P, and ϕ0(qt , qˆz) is the amplitude for P = 1 − cos Δ1 − cos Δ2 + cos(Δ1 − Δ2), (67) the scalar gravitational wave propagating in the direction qˆz ˆ, ˆ, ˆ at the angular frequency qt .Leti j k form a right-handed and Δ = (q + q qˆ ·ˆn )L with j = 1, 2. In obtaining this ˆ × ˆ = ˆ P j t z z j j triad frame so that i j k, then jk is given by result, one uses Eq. (63), and takes the real part. In addition, T drops out, since the ensemble average also implies the + = ˆ ˆ − ˆ ˆ ,× = ˆ ˆ + ˆ ˆ . jk i j ik j j jk jk i j jk j j ik (61) average over the time [53]. Since the gravitational wave background is assumed to be Assuming the gravitational wave background is isotropic, isotropic, one can calculate C(θ) by setting stationary and independently polarized, one defines the char- ˜ P ϕ acteristic strains hc and c in the following way, nˆ1 = (0, 0, 1), (68) ∗ ˆ = ( θ, , θ). h˜ P (Ω, kˆ)h˜ P (Ω , kˆ ) n2 sin 0 cos (69) ˜ P 2 π|h | (Ω) = δ(Ω − Ω )δ(kˆ − kˆ )δ PP c , (62) Also, let qˆz = (sin θg cos φg, sin θg sin φg, cos θg),so 4Ω |ϕ |2( ) Δ = ( + θ ) , ϕ∗( , ˆ )ϕ ( , ˆ ) =δ( − )δ( ˆ −ˆ ) c qt , 1 qt qz cos g L1 (70) 0 qt qz 0 qt qz qt qt qz qz qt Δ2 =[qt + qz(sin θg cos φg sin θ + cos θg cos θ)]L2, (71) (63) and where the star ∗ indicates the complex conjugation. Since the gravitational wave with the plus or cross polar- ∞ 2 ization behaves exactly the same way as in GR which can |ϕc| (qt ) C(θ) = dq dθ dφ sin θ (q − q cos θ ) t g g g π 2 5 t z g be found in Ref. [53], the focus will be on the contribution m 4 qt of the scalar field in the following discussion. The total time [qt − qz(sin θg cos φg sin θ + cos θg cos θ)]P. residual in TOA caused by the stochastic gravitational wave (72) background is ∞ T − In the observation, pulsars are far away enough, so that ( ) = dqt 2 ˆ fe fr , R T d qz dt (64) q L 1. This implies that one can approximate P ≈ 1 −∞ 2π 0 fr t j when θ = 0, as the phases in cosines in the definition (67) where the argument T is the total observation time. Substi- of P oscillate fast enough. The integration can be partially tuting Eq. (58) (ignoring the second line) in, one obtains done, yielding

∞ dq q − q qˆ ·ˆn ∞ t 2 t z z iqt T |ϕ |2( ) 2 R(T ) = d qˆ ϕ (q , qˆ )(e − 1) c qt qz π z 2 0 t z C(θ) ≈ dq 1 + cos θ . (73) −∞ 2 i2qt t 3 2 m πq 3q − ( + ˆ · ) t t ×[1 − e i qt qzqz n L ]. (65) But for θ = 0, one cannot simply set cosines in P to 1. In this case, one actually considers the auto-correlation function, so Therefore, one can now consider the correlation between two nˆ1 =ˆn2 and L1 = L2 = L. The auto-correlation function is pulsars a and b which are located at positions xa = L1nˆ1 thus given by 123 378 Page 12 of 15 Eur. Phys. J. C (2018) 78 :378 ∞ 2 2 |ϕc| (qt ) q 2 0.5 C(0) =2 dq 1 + z + cos q L cos q L t π 3 2 2 2 t z m qt 3qt qt L GR 0.4 Breathing 2 1 2 qz + − + − cos q L sin q L 2 3 2 t z 0.3 qt L qz L qt qz L qt L

2 ) 0.2

− sin q L sin q L . ( 2 t z qt qz L (74) 0.1

0.0

Note that the observation time T sets a natural cutoff for the –0.1 frequency, i.e., qt ≥ 2π/T . So the lower integration limits { , π/ } in Eqs. (73) and (74) should be replaced by Min m 2 T . 0 50 100 150 Usually, one assumes that ϕc(qt ) takes a form of ϕc(qt ) ∝ ( / c)α c α qt qt with qt some characteristic angular frequency. is called the power-law index, and usually, α = 0, −2/3or−1 0.50 [53,63]. One can numerically integrate Eqs. (73) and (74)to =0 obtain the so-called normalized correlation function ζ(θ) = 0.48 =–2/3 C(θ)/C(0). In the integration, suppose the observation time =–1 0.46 T = 5 years and the mass of the scalar ﬁeld is mb = 7.7 × −23 / 2 10 eV c [3]. The distance L takes a large enough value 0.44 so that q L 1 is satisﬁed. This gives rise to the right panel )

t ( in Fig. 5, where the power-law index α takes different values. 0.42 Together shown are the normalized correlation functions for the plus and cross polarizations (labeled by “GR”), and for the 0.40 transverse breathing polarization (labeled by “Breathing”) 0.38 in the left panel. The normalized correlation function for the transverse breathing polarization can be obtained simply 0.36 by setting the mass m = 0. It is clear that ζ(θ) behaves ˜ 0 50 100 150 very differently for hμν and ϕ, so it is possible to determine whether there are polarizations induced by the scalar field using pulsar timing arrays. Note that since L is large enough, Fig. 5 The normalized cross-correlation functions ζ(θ) = ζ(θ) C(θ)/C(0). The upper panel shows the cross-correlations when barely changes with L. the scalar field is massless, i.e., when there is no longitudinal polariza- One could also vary the mass of the scalar field. The result tion. The solid curve is for familiar GR polarizations (i.e., the plus or is shown in Fig. 6. This figure displays ζ(θ) for six different cross ones), and the dashed curve for the transverse breathing polarization. The lower panel shows the cross-correlations induced together multiples of mb, including m = 0 labeled by “Breathing”. by the transverse breathing and longitudinal polarizations when the The power-law index α is chosen to be 0. It shows that at −23 2 mass of the scalar field is taken to be mb = 7.7 × 10 eV/c .The large angles, ζ(θ) is very sensitive to smaller masses with calculation was done assuming T = 5yrs mb/5 m mb,butform mb or m mb/5, ζ(θ) almost remains the same. Therefore, the measurement of the correlation can be used to constrain the mass of the scalar 5 Conclusion field. In Ref. [55], Lee also analyzed the time residual of TOA This work analyzes the gravitational wave polarizations in caused by massive gravitational waves and calculated the the most general scalar-tensor theory of gravity, Horndeski cross-correlation functions. His results (the right two pan- theory. It reveals that there are three independent polariza- els in his Fig. 1) differ from those on the right panel in tion modes: the mixture state of the transverse breathing ˆ ˆ Fig. 5, because in his treatment, the longitudinal and the Pb = Rtxtx + Rtyty and longitudinal Pl = Rtztz polar- transverse polarizations were assumed to be independent. In izations for the massive scalar field, and the usual plus ˆ ˆ Horndeski theory, however, it is not allowed to calculate the P+ =−Rtxtx + Rtyty and cross P× = Rtxty polarizations cross-correlation function separately for the longitudinal and for the massless gravitons. These results are consistent with the transverse polarizations, as they are both excited by the the three propagating degrees of freedom in Horndeski the- same field ϕ and the polarization state is a single mode. ory. Since the propagation speed of the massive gravitational 123 Eur. Phys. J. C (2018) 78 :378 Page 13 of 15 378

=0 aLIGO/VIGO and LISA, too. It is interesting that the trans- 0.60 verse mode becomes stronger for smaller graviton mass in Breathing the f (R) gravity, so the detection of the mixture state can 0.55 mb/10 place strong constraint on f (R) gravity. mb/5 0.50 For null gravitational waves, the presence of the longitu- mb dinal mode means that all six polarizations can be detected 5mb 0.45 in some coordinate systems. For Horndeski theory, we find 10mb ) that the vector modes are absent even though the longitudinal ( 0.40 mode is present. Since the massive scalar field excites both the breathing and longitudinal polarizations, while the mass- 0.35 less scalar field excites the breathing mode only, the detection of polarizations can be used to understand the nature of 0.30 gravity. If only the plus and cross modes are detected, then

0.25 gravitation is mediated by massless spin 2 field and GR is confirmed. The detection of the breathing mode in addition 0 50 100 150 to the plus and cross modes means that gravitation is mediated by massless spin 2 and spin 0 fields. If the breathing, Fig. 6 ζ(θ) as a function of scalar mass m at α = 0. The calculations plus, cross and longitudinal modes are detected, then grav- were also done for a 5-year observation itation is mediated by massless spin 2 and massive spin 0 fields. wave depends on the frequency and is smaller than the speed Acknowledgements We would like to thank Ke Jia Lee for useful of light, the massive mode will arrive at the detector later discussions. This research was supported in part by the Major Program of the National Natural Science Foundation of China under Grant No. than the massless gravitons. In addition to the difference of 11690021 and the National Natural Science Foundation of China under the propagation speed, the presence of both the longitudinal Grant No. 11475065. and breathing states without the vector-x and vector-y states Open Access This article is distributed under the terms of the Creative are also the distinct signature of massive scalar degree of Commons Attribution 4.0 International License (http://creativecomm freedom for graviton. ons.org/licenses/by/4.0/), which permits unrestricted use, distribution, Using the NP variables, we find that Ψ2 = 0. For null and reproduction in any medium, provided you give appropriate credit gravitational waves, this means that the longitudinal mode to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. does not exist. However, our results show that the longitudi- Funded by SCOAP3. nal mode exists in the massive case even though Ψ2 = 0. We also find that the NP variable Φ00 = 0, and Φ00, Φ11 and Λ are all proportional to Φ22. These results are in conflict with those for massless gravitational waves in [17], so the results further support the conclusion that the classification of the References six polarizations for null gravitational waves derived from the little group E(2) of the Lorentz group is not applicable 1. B.P. Abbott et al., Observation of gravitational waves from a binary to the massive case. Although the longitudinal mode exists black hole merger. Phys. Rev. Lett. 116(6), 061102 (2016) 2. B.P. Abbott et al., GW151226: observation of gravitational waves for the massive scalar field, it is difficult to be detected in from a 22-solar-mass binary black hole coalescence. Phys. Rev. the high frequency band because of the suppression by the Lett. 116(24), 241103 (2016) extremely small graviton mass upper bound. Compared with 3. Benjamin P. Abbott et al., GW170104: observation of a 50-solar- aLIGO/VIRGO and LISA, pulsar timing arrays might be the mass binary black hole coalescence at redshift 0.2. Phys. Rev. Lett. 118(22), 221101 (2017) primary tool to detect both the transverse breathing and lon- 4. Benjamin P. Abbott et al., GW170814: a three-detector observation gitudinal modes due to the massive scalar field. In the mass- of gravitational waves from a binary black hole coalescence. Phys. less case, the longitudinal mode disappears and the mix state Rev. Lett. 119, 141101 (2017) reduces to the pure transverse breathing mode. Brans-Dicke 5. Benjamin P. Abbott et al., GW170817: observation of gravitational 119 ( ) waves from a binary neutron star inspiral. Phys. Rev. Lett. (16), theory and f R gravity are subclasses of Horndeski theory, 161101 (2017) so the general results obtained can be applied to those the- 6. B.P. Abbott et al., GW170608: observation of a 19-solar-mass ories. Despite the fact that in f (R) gravity, the longitudinal binary black hole coalescence. Astrophys. J. 851(2), L35 (2017) mode does not depend on the mass of graviton, its magnitude 7. P. Amaro-Seoane et al. Laser Interferometer Space Antenna. arXiv:1702.00786 is much smaller than the transverse one in the high frequency 8. Jun Luo et al., TianQin: a space-borne gravitational wave detector. band, which makes its detection unlikely by the network of Class. Quant. Grav. 33(3), 035010 (2016) 123 378 Page 14 of 15 Eur. Phys. J. C (2018) 78 :378

9. G. Hobbs et al., The international pulsar timing array project: using 33. M.E.S. Alves, O.D. Miranda, C.N. de Araujo, Probing the f(R) pulsars as a gravitational wave detector. Class. Quant. Grav. 27, formalism through gravitational wave polarizations. Phys. Lett. B 084013 (2010) 679, 401–406 (2009) 10. M. Kramer, D.J. Champion, The European pulsar timing array and 34. M.E.S. Alves, O.D. Miranda, J.C.N. de Araujo, Extra polarization the large European array for pulsars. Class. Quant. Grav. 30(22), states of cosmological gravitational waves in alternative theories 224009 (2013) of gravity. Class. Quant. Grav. 27, 145010 (2010) 11. Benjamin P.Abbott et al., First search for nontensorial gravitational 35. H.Rizwana Kausar, Lionel Philippoz, Philippe Jetzer, Gravitational waves from known pulsars. Phys. Rev. Lett. 120(3), 031104 (2018) wave polarization modes in f (R) theories. Phys. Rev. D 93(12), 12. A. Goldstein et al., An ordinary short gamma-ray burst with extraor- 124071 (2016) dinary implications: fermi-GBM fetection of GRB 170817A. 36. Yun Soo Myung, Propagating degrees of freedom in f (R) gravity. Astrophys. J. 848(2), L14 (2017) Adv. High Energy Phys. 2016, 3901734 (2016) 13. V. Savchenko et al., Integral detection of the frst prompt gamma- 37. Dicong Liang, Yungui Gong, Shaoqi Hou, Yunqi Liu, Polarizations ray signal coincident with the gravitational-wave event GW170817. of gravitational waves in f (R) gravity. Phys. Rev. D 95, 104034 Astrophys. J. 848(2), L15 (2017) (2017) 14. Maximiliano Isi, Alan J. Weinstein, Carver Mead, Matthew Pitkin, 38. Yungui Gong and Shaoqi Hou. Gravitational Wave Polarizations Detecting beyond-Einstein polarizations of continuous gravita- in f (R) Gravity and Scalar-Tensor Theory. In Proceedings, 13th tional waves. Phys. Rev. D 91(8), 082002 (2015) International Conference on Gravitation, Astrophysics and Cos- 15. Maximiliano Isi, Matthew Pitkin, Alan J. Weinstein, Probing mology and 15th Italian-Korean Symposium on Relativistic Astro- Dynamical Gravity with the Polarization of Continuous Gravita- physics (IK15): Seoul, Korea, July 3-7, 2017, volume 168, page tional Waves. Phys. Rev. D 96(4), 042001 (2017) 01003, 2018 16. Ezra Newman and Roger Penrose. An approach to gravitational 39. Yungui Gong, Shaoqi Hou, Dicong Liang, Eleftherios radiation by a method of spin coefficients. J. Math. Phys., 3(3):566– Papantonopoulos, Gravitational Waves in Einstein-Æther Theory 578, 1962. [Errata: J.Math.Phys.4,no.7,998(1963)] and Generalized TeVeS Theory after GW170817. Phys. Rev. D 17. D.M. Eardley, D.L. Lee, A.P. Lightman, Gravitational-wave obser- 97(8), 084040 (2018) vations as a tool for testing relativistic gravity. Phys. Rev. D 8, 40. Qing Gao, Yungui Gong, Dicong Liang, Chin. Sci. Bull. 63, 801– 3308–3321 (1973) 805 (2018) 18. D.M. Eardley, D.L. Lee, A.P. Lightman, R.V. Wagoner, C.M. Will, 41. Tsutomu Kobayashi, Masahide Yamaguchi, Ju’ichi Yokoyama, Gravitational-wave observations as a tool for testing relativistic Generalized G-inflation: Inflation with the most general second- gravity. Phys. Rev. Lett. 30, 884–886 (1973) order field equations. Prog. Theor. Phys. 126, 511–529 (2011) 19. C. Brans, R.H. Dicke, Mach’s principle and a relativistic theory of 42. Maximiliano Isi and Alan J. Weinstein. Probing gravitational gravitation. Phys. Rev. 124(3), 925–935 (1961) wave polarizations with signals from compact binary coalescences. 20. Gregory Walter Horndeski, Second-order scalar-tensor field equa- arXiv:1710.03794 tions in a four-dimensional space. Int. J. Theor. Phys. 10(6), 363– 43. I. Di Palma, M. Drago, Estimation of the gravitational wave polar- 384 (1974) izations from a non template search. Phys. Rev. D 97(2), 023011 21. M. Ostrogradsky, Mémoires sur les équations différentielles, rela- (2018) tives au problème des isopérimètres. Mem. Acad. St. Petersbourg 44. Malik Rakhmanov, Response of test masses to gravitational waves 6(4), 385–517 (1850) in the local Lorentz gauge. Phys. Rev. D 71, 084003 (2005) 22. R. Utiyama, S. Bryce, DeWitt. Renormalization of a classical gravi- 45. C.J. Moore, R.H. Cole, C.P.L.Berry, Gravitational-wave sensitivity tational field interacting with quantized matter fields. J. Math. Phys. curves. Class. Quant. Grav. 32(1), 015014 (2015) 3, 608–618 (1962) 46. Wayne L. S. de Paula, Oswaldo D. Miranda, and Rubens M. Mar- 23. K.S. Stelle, Renormalization of Higher Derivative Quantum Grav- inho. Polarization states of gravitational waves with a massive ity. Phys. Rev. D 16, 953–969 (1977) graviton. Class. Quant. Grav., 21:4595–4606, (2004) 24. K.S. Stelle, Classical gravity with higher derivatives. Gen. Rel. 47. J.P.W. Verbiest et al., Timing stability of millisecond pulsars and Grav. 9, 353–371 (1978) prospects for gravitational-wave detection. Mon. Not. Roy. Astron. 25. Hans A. Buchdahl, Non-linear lagrangians and cosmological the- Soc. 400, 951–968 (2009) ory. Mon. Not. Roy. Astron. Soc. 150, 1 (1970) 48. F.B. Estabrook, H.D. Wahlquist, Response of Doppler spacecraft 26. John O’Hanlon, Intermediate-range gravity: a generally covariant tracking to gravitational radiation. Gen. Rel. Grav. 6, 439–447 model. Phys. Rev. Lett. 29(2), 137–138 (1972) (1975) 27. P. Teyssandier, Ph Tourrenc, The cauchy problem for the R + R2 49. M.V. Sazhin, Opportunities for detecting ultralong gravitational theories of gravity without torsion. J. Math. Phys. 24(12), 2793– waves. Sov. Astron. 22, 36 (1978) 2799 (1983) 50. Steven L. Detweiler, Pulsar timing measurements and the search 28. Christian Corda, The production of matter from curvature in a par- for gravitational waves. Astrophys. J. 234, 1100–1104 (1979) ticular linearized high order theory of gravity and the longitudinal 51. R.W. Hellings, G.S. Downs, Upper limits on the isotropic gravi- response function of interferometers. JCAP 0704, 009 (2007) tational radiation background from pulsar timing analysis. Astro- 29. Christian Corda, Massive gravitational waves from the R**2 theory phys. J. 265, L39–L42 (1983) of gravity: production and response of interferometers. Int. J. Mod. 52. Fredrick A. Jenet, George B. Hobbs, K.J. Lee, Richard N. Manch- Phys. A 23, 1521–1535 (2008) ester, Detecting the stochastic gravitational wave background using 30. Salvatore Capozziello, Christian Corda, Maria Felicia De Lau- pulsar timing. Astrophys. J. 625, L123–L126 (2005) rentis, Massive gravitational waves from f(R) theories of gravity: 53. K.J. Lee, F.A. Jenet, R.H. Price, Pulsar timing as a probe of non- potential detection with LISA. Phys. Lett. B 669, 255–259 (2008) Einsteinian polarizations of gravitational waves. Astrophys. J. 685, 31. S. Capozziello, R. Cianci, M. De Laurentis, S. Vignolo, Testing 1304–1319 (2008) metric-affine f(R)-gravity by relic scalar gravitational waves. Eur. 54. Kejia Lee, Fredrick A. Jenet, Richard H. Price, Norbert Wex, Phys. J. C 70, 341–349 (2010) Michael Kramer, Detecting massive gravitons using pulsar timing 32. P. Prasia, V.C.Kuriakose, Detection of massive gravitational waves arrays. Astrophys. J. 722, 1589–1597 (2010) using spherical antenna. Int. J. Mod. Phys. D 23, 1450037 (2014) 55. K.J. Lee, Pulsar timing arrays and gravity tests in the radiative regime. Class. Quant. Grav. 30(22), 224016 (2013) 123 Eur. Phys. J. C (2018) 78 :378 Page 15 of 15 378

56. Sydney J. Chamberlin, Xavier Siemens, Stochastic backgrounds 60. William L. Burke, Large-scale random gravitational waves. Astro- in alternative theories of gravity: overlap reduction functions for phys. J. 196, 329–334 (1975) pulsar timing arrays. Phys. Rev. D 85, 082001 (2012) 61. Massimo Tinto and Márcio Eduardo da Silva Alves, LISA Sensi- 57. Nicolás Yunes,Xavier Siemens, Gravitational-wave tests of general tivities to Gravitational Waves from Relativistic Metric Theories relativity with ground-based detectors and pulsar timing-arrays. of Gravity. Phys. Rev. D 82, 122003 (2010) Living Rev. Rel. 16, 9 (2013) 62. Melissa Anholm, Stefan Ballmer, Jolien D.E. Creighton, Larry R. 58. Jonathan Gair, Joseph D. Romano, Stephen Taylor, M.F. Chiara, Price, Xavier Siemens, Optimal strategies for gravitational wave Mingarelli. Mapping gravitational-wave backgrounds using meth- stochastic background searches in pulsar timing data. Phys. Rev. ods from CMB analysis: Application to pulsar timing arrays. Phys. D 79, 084030 (2009) Rev. D 90(8), 082001 (2014) 63. Joseph D. Romano, Neil J. Cornish, Detection methods for stochas- 59. Jonathan R. Gair, Joseph D. Romano, Stephen R. Taylor, Mapping tic gravitational-wave backgrounds: a uniﬁed treatment. Living gravitational-wave backgrounds of arbitrary polarisation using pul- Rev. Rel. 20, 2 (2017) sar timing arrays. Phys. Rev. D 92(10), 102003 (2015)

123