Constraint on Brans-Dicke theory from Intermediate/Extreme Mass Ratio Inspirals with self-force method

Tong Jiang,1, ∗ Ning Dai,1, † Yungui Gong,1, ‡ Dicong Liang,1, § and Chao Zhang1, ¶ 1School of Physics, Huazhong University of Science and Technology, Wuhan, Hubei 430074, China Intermediate/Extreme mass ratio inspiral (IMRI/EMRI) system provides a good tool to test the nature of gravity in strong field. We construct the self-force and use the self-force method to generate accurate waveform templates for IMRIS/EMRIs on quasi-elliptical orbits in Brans-Dicke theory. The extra monopole and dipole emissions in Brans-Dicke theory accelerate the orbital decay, so the observations of gravitational waves may place stronger constraint on Brans-Dicke theory. With a two-year observations of gravitational waves emitted from IMRIs/EMRIs with LISA, we can get the 5 most stringent constraint on the Brans-Dicke coupling parameter ω0 > 10 .

Since the first direct detection of gravitational wave orbital dynamics and the GW waveform of a compact (GW) event GW150914 [1,2] by the Laser Interferom- binary system different from those in GR [46–52], so BD eter Gravitational-Wave Observatory (LIGO) Scientific theory can also be probed by the observations of GWs Collaboration [3,4] and Virgo Collaboration [5], there [53–63]. However, the orbital evolution and gravitational have been tens of confirmed GW detection [6,7]. It is radiation from binary black holes (BBHs) are identical well known that in general relativity (GR) GWs prop- in GR and BD theory, so GW observations of BBHs are agate with the speed of light and have only two tensor unable to distinguish BD theory from GR [38, 46, 61]. By polarizations. However, six possible polarization states in using the simple approximate waveform template with GWs are allowed in general metric theory of gravity [8,9] the dipolar correction in the phase and the Fisher matrix and the number of polarization states depends on the par- method, it was estimated that the observation of a 0.7M ticular theory of gravity [10–14]. The detected GWs are neutron star (NS) on a quasicircular inspiral into a 3M useful to understand the nature of gravity and test Ein- black hole (BH) with a signal-to-noise (SNR) of 10 by stein’s GR in strong-field and nonlinear regions [15–18]. LIGO/Virgo detectors could give the constraint ωBD & The observation of GW170817 and its electromagnetic 2000 [53]. Using the Bayesian inference method, the GW counterpart GRB170817A constrained the speed of GWs event GW190426 152155 of a possible 1.5M NS/5.7M −15 −16 as −3 × 10 < cgw/c − 1 ≤ 7 × 10 [19] and this BH binary gave the constraint ωBD & 10 [64]. measurement on the propagation speed of GWs was al- If the BHs are more massive, then the frequency of ready used to exclude some alternative theories of gravity emitted GWs is in the mHz band and the GWs should [20–31]. be measured by space-based GW detectors like the Laser One of the simple alternative theories of gravity is Interferometer Space Antenna (LISA) [65, 66], TianQin Brans-Dicke (BD) theory of gravity [32, 33]. In BD the- [67] and Taiji [68]. One particular interest in space-based ory, the BD scalar field ϕ not only takes the role of 1/G GW detectors is the stellar-mass BHs or NS captured into but also mediates gravity and excites the scalar breath- inspiral orbits around massive BHs (MBHs), the Inter- ing mode in GWs. The most stringent constraint on mediate/Extreme mass ratio inspiral (IMRI/EMRI). The BD theory comes from the Cassini measurements of the mass ratio between the MBHs and the compact stellar ob- Shapiro time delay in the solar system [34] and the re- ject is about 102-104 : 1 for IMRIs and 104 : 1 for EM- sult is ω > 40000 [35]. For a binary system, the or- & BD RIs. In EMRIs, the timescale on the modification of the bital period of the system will decrease due to the loss orbit due to the back-reaction from gravitational radia- of energy by the emission of GWs. In BD theory [36], tion is much larger than the orbital period, so it takes the arXiv:2107.02700v2 [gr-qc] 7 Jul 2021 the extra dipolar emission channel of GWs can further compact object (CO) the last few years to inspiral deep decrease the orbital period of a binary system [36, 37], inside the strong field region of the MBH with a speed of so the measurement of the change of the orbital period a significant fraction of the speed of light and there are of a binary can be used to constrain BD theory [36–44]. 105-106 GW cycles in the detector band [69]. The emit- By using the measurement of the orbital decay from the ted GWs from (IMRIs/EMRIs) encode rich information pulsar-white dwarf binary PSR J1738+0333, the BD pa- about the spacetime geometry around the MBH and they rameter ω was constrained to be ω > 25000 [45]. 0 0 can be used to confirm whether the MBH is a Kerr BH The extra energy loss in BD theory makes both the predicated by GR. Using the Fisher matrix method, it was shown that the observation of a 1.4M NS on a qua- 3 sicircular inspiral into a 10 M BH with a SNR of 10 by ∗ [email protected] LISA could give the constraint ωBD & 24000 [58], and a † [email protected] two-year observation of a 1.4M NS on a quasicircular in- ‡ 3 Corresponding author. [email protected] spiral into a 10 M BH by LISA could give the constraint § 5 [email protected] ωBD > 3×10 [59]. Including the spin-spin coulpling and ¶ chao [email protected] small eccentricity, the constraint from a 1.4M NS inspi- 2 √ 3 ralling into a 10 M BH with a SNR of 200 by LISA elliptical orbits in the framework of GR and BD theory. became ωBD & 6944 [60]. Because BD theory modifies We then discuss the constraint on ωBD with LISA. With gravity in the weak field and the deviation in the en- two-year observation of IMRI/EMRI, we derive the con- 5 ergy flux are largest at small velocities, the constraints straint ω0 > 10 , which is more stringent than current on ωBD from EMRIs on quasicircular orbits are worse solar system tests. The constraint obtained in this paper than those derived from comparable-mass binaries [61]. should be more reliable than those derived in [58–60] with For EMRIs, half the total energy is radiated earlier than the approximate analytic waveform in frequency domain 10 years before the final plunge [70] and monopole radi- and the Fisher matrix approximation. In this paper we ation appears when the eccentricity is nonzero [38, 57], adopt natural units c = G = 1. so the early inspiral of EMRIs on quasi-elliptical orbits For EMRI systems, at the zeroth order of approxima- may place stronger constraint on BD theory. Therefore, tion the small compact object moves along the geodesic it is interesting to consider EMRIs when the small CO of the central MBH. The bound geodesic is a Keplerian moves slowly in a quasi-elliptical orbit. orbit parameterized by χ as The long inspiral time of EMRIs makes the genera- tion of accurate template waveforms for matched filter- pm r(χ) = 1 , (1) ing with the numerical relativity method computation- 1 + e cos (χ − w) ally impossible. However, the problem can be approached based on the expansion in mass ratio. To the lowest or- der, the small CO can be treated as a point like test parti- where m1 is the mass of the central MBH, the parameter cle moving in the geodesics of the central MBH. To higher χ runs from 0 to 2π over one radial cycle, w is the value orders, the gravitational field, the internal structure of of χ at periapsis, e is the orbital eccentricity and p is the small CO and the back-reaction of gravitational ra- the semilatus rectum. The self-force including the back diation are treated as perturbations. The waveforms are reaction of radiative GW and the interaction between calculated by solving the Teukolsky equation [71] and the small compact object and its own gravitational per- summing all the multipole modes. The Teukolsky-based turbation, is considered as a perturbation acting on the waveforms are computationally expensive. In order to geodesic of the central MBH. With the self-force, the or- quickly derive the equation of motion (EoM) and the bital parameters p, e and w evolve with time and we can waveform template for EMRIs, the kludge models in- get more accurate template waveforms. Here we mainly cluding the analytical kludge (AK) model [70] and the discuss the method of Pound and Poisson which uses the numerical kludge (NK) model [72] have been proposed. PN theory to construct the SF up to 2.5PN [88]. The AK model assumes that the small CO moves on a Now we apply the above SF method to the BD theory Keplerian orbit with relativistic corrections such as peri- with the action [32, 36] apsis precession, Lense-Thirring precession and inspiral from radiation reaction given by analytic post-Newtonian Z −1  −1 ,α  √ 4 (PN) evolution equations. It is extremely quick to cal- S =(16π) ϕR − ϕ ω(ϕ)ϕ ϕ,α −gd x culate, but it dephases relative to true waveforms within (2) hours. The augmented AK model improved the accuracy + Sm(Ψ, gαβ), of waveform template [73, 74]. The NK model combines Kerr geodesic with PN orbital evolution which is caused where ϕ is the BD scalar field, the coupling parameter ω by radiation reaction of GWs, and numerically integrates is a function of ϕ and Sm is the matter action in which the Kerr geodesic equations along the inspiral trajectory. the matter field Ψ couples to the metric gαβ only but It is more accurate and computationally expensive than the mass of a self-gravitating body depends on the BD the AK model. scalar field. We denote ϕ0 as the vacuum expectation On the other hand, the interaction between the value of ϕ and ω0 = ω(ϕ0). To parametrize the sensi- small CO and its own gravitational perturbation can tivity of the body’s binding energy on the background be thought as an effective gravitational self-force driving scalar field, we introduce the the sensitivity of body A as the radiative evolution of the geodesic orbit of the cen- sA = d ln mA(ϕ)/d ln ϕ0. In the original BD theory [32], tral MBH. The perturbative force includes the radiation- ω(φ) = ωBD, the effective Newtonian gravitational cou- reaction of GWs and the gravitational effect caused by pling constant measured by Cavendish-like experiments the small CO [75, 76]. There were some developments is G = (4 + 2ωBD)/[φ0(3 + 2ωBD)], and the stationary, on the construction of self-force [77–87]. To integrate asymptotically flat BHs in vacuum in BD theory are the the equations of motion that govern accelerated motion BHs of GR. due to self-force in Schwarzschild spacetime, the method For a compact binary, the relative acceleration between of osculating orbits was proposed [88]. The self-force the two COs up to the 2.5PN in BD theory is [46] method can be applied in strong field regions and can generate waveform templates for EMRIs with a good accuracy [88–90]. In this paper, we use the self-force d2~x αm 2 = − 2 (ABDn − BBDv), (3) method to generate waveforms for EMRIs with quasi- dt rh 3 where the orbits in GR and BD theory are almost identical. Af- 8 αm ter 18 months of inspirals, the orbits in BD theory with ABD = 1 − A1PN − A2PN − η r˙h(A1.5PN + A2.5PN), ω0 = 50000 start to deviate from those in GR and the or- 5 rh bits in BD theory with ω = 2000 are different from those 8 αm 0 BBD = (B1PN + B2PN)r ˙h − η (B1.5PN + B2.5PN), in GR. The main difference in the orbital evolution can 5 rh be manifested by the orbital phase accumulation. In Fig. (4) 2, we show the evolutions of the orbital phase difference ∆φ = φ − φ between GR and BD theory with dif- r ≡ |~x| = |~x − ~x |, the subscript h means that we BD GR 1 2 ferent ω . As expected, the phase difference increases as use the harmonic gauge in the PN expansion, the har- 0 ω becomes smaller which is consistent with the results monic coordinate is related to Schwarzschild coordinate 0 of the orbital evolution. From Fig.2, we see that ∆ φ by the transformation r = r − m , n = ~x/r , m = h 1 h oscillates with time. After around one-year inspiral, the m + m , η = m m /m2, v = v − v ,r ˙ = dr /dt, 1 2 1 2 1 2 h h phase difference is larger than 40 rad for ω = 2000; the α = 1 − ζ + ζ(1 − 2s )(1 − 2s ), ζ = 1/(4 + 2ω ), the 0 1 2 0 phase difference oscillates between 6 rad and 12 rad for variables A and B through 2.5PN are given in the ap- ω = 10000; and the maximum phase difference reaches pendix. For the equations of motion through 2PN and 0 5 rad for ω = 50000. The accumulated phase difference 3PN, please refer [47, 48, 50]. From Eq. (3) we construct 0 will be manifested in the GW waveform which provides the SF and then we solve the osculating orbits for the us tool to constrain BD. orbital parameters to see whether the orbital evolutions in GR and BD theory are distinguishable. Since the or- 50 ω = bital evolution of BBHs cannot distinguish GR from BD 0 2000 ω0=10000 ω = theory, so we consider a binary with a NS and a BH. 40 0 50000 The mass and the sensitivity of the NS are m2 = 1.3M and s2 = 0.2 respectively, and the mass ratio between −4 30 the NS and the BH is q = m2/m1 = 10 . The initial rad conditions are as follows: at the initial time t0 = 0, the Δϕ / 20 dimensionless semilatus rectum p0 = 80, the eccentricity e0 = 0.8, the orbital parameter at periapsis w0 = 10, and the phase at periapsis φ0 = 0. The results for the orbital 10 evolution are shown in Fig.1.

0 0 200 400 600 800 1000 1200

Initial 18months, 0=50000 t/days 400 400 BD BD GR GR

200 200 FIG. 2. The orbital phase difference ∆φ = φBD−φGR between 1 1

m 0 m 0 / / GR and BD theory with different ω0. The solid, dashed and y y dot-dashed lines are the envelops of the oscillating ∆φ. The

-200 -200 insets show the oscillation behavior of ∆φ within 0.2 day.

-400 -400 -400 -200 0 200 400 -400 -200 0 200 400 Without loss of generality, we take the inclination an- x/m1 x/m1 18months, 0=10000 18months, 0=2000 gle ι = π/6 and longitude of pericenter ξ = 0 to gen- 400 400 BD BD GR GR erate the GW waveforms of EMRIs. Using the results

200 200 of the orbital evolution as shown in Fig.1, we get the GW waveforms in the time domain. In Fig.3 we show 1 1

m 0 m 0 / / the GW waveforms of the plus and cross polarizations y y in GR and BD theory. From Fig.3, it is obvious that -200 -200 the mismatch of the phases in GR and BD theory starts earlier for smaller ω0. As a result, the accumulation of -400 -400 -400 -200 0 200 400 -400 -200 0 200 400 phase difference leads to distinguishable waveforms, so /m /m x 1 x 1 it is possible to distinguish GR and BD theory from the observations of GWs from EMRIs. To detect GWs from FIG. 1. Comparisons of the orbital evolution in GR (blue solid EMRIs, the SNR should exceed the threshold value of 7. curves) and BD theory (red dashed curves) with different ω0. Take the luminosity distance of the EMRI as DL = The upper left panel displays the initial behaviors of the orbits 100 Mpc and use the GW waveform obtained in Fig. and the other panels show the behaviors of the orbits after 18 3, with two-year observation, we get ρ(H˜GR) ≈ 7.3 for months. All the plots cover the same range of time. GR and ρ(H˜BD) ≈ 7.3 for the BD theory with ω0 = 105. Thus, these GW signals are detectable for LISA-like From Fig.1, we see that during the initial inspirals, space-based detectors. To quantify the difference of the 4

ω0=2000 ω0=2000 0.10 0.10

0.05 0.05 / μ / μ L L BD BD D 0.00 D 0.00  + H H GR GR

-0.05 -0.05

-0.10 -0.10 14.9 15.0 15.1 15.2 15.3 15.4 14.9 15.0 15.1 15.2 15.3 15.4 t/day t/day

ω0=100000 ω0=100000 0.10 0.10

0.05 0.05 / μ / μ L L BD BD D 0.00 D 0.00  + H H GR GR

-0.05 -0.05

-0.10 -0.10 692.5 692.6 692.7 692.8 692.9 692.5 692.6 692.7 692.8 692.9 t/day t/day

FIG. 3. The plus and cross waveforms in the time domain in GR and BD theory with different ω0. The reduced mass µ = m1m2/m. Different time windows are chosen for different ω0 to display the mismatch of the phases in GR and BD theory.

GWs from different theories, following [91], we compute hard to distinguish the two GW waveforms. Otherwise, the SNR of the difference of these two signals ∆H(t) = the two GW waveforms are distinguishable. For the GW 5 HBD(t) − HGR(t) and we get ρ(∆H˜ ) ≈ 10.4, Therefore, waveforms in GR and BD theory with ω0 = 10 , with ˜ ˜ LISA can detect GWs from a binary consisting of a NS one-month observation we find that FF (HGR, HBD) = with the mass m2 = 1.3M and a BH with the mass 0.87, with three-month observation, the fitting factor 4 m1 = 10 m2 located at DL = 100 Mpc away, and a drops to 0.32, and with two-year observation we have ˜ ˜ −4 two-year observation of the binary can put the constraint FF (HGR, HBD) = 5 × 10 . The result is consistent 5 ω0 > 10 . with the analysis above by SNR. Our results show that The mismatch of two GW waveforms can also be quan- with a two-year observation of IMRIs or EMRIs, we can 5 tified by the fitting factor [92]: get the constraint ω0 > 10 which is stronger than the current constraint obtained by the solar system tests. ˜ ˜ ACKNOWLEDGMENTS ˜ ˜ (h1|h2) FF (h1, h2) ≡ q , (5) ˜ ˜ ˜ ˜ (h1|h1)(h2|h2) TJ and YG are grateful to Adam Pound for helpful discussion on self force method. The numerical com- where the inner product(a(f)|b(f)) is defined as putations were performed at the public computing ser- vice platform provided by Network and Computing Cen- ter of Huazhong University of Science and Technology Z ∞ a˜∗(f)˜b(f) +a ˜(f)˜b∗(f) (a(f)|b(f)) = 2 df , (6) (HUST). This research is supported in part by the Na- 0 Sn(f) tional Key Research and Development Program of China under Grant No. 2020YFC2201504, the National Natural and Sn(f) is the noise power spectral density [93]. For Science Foundation of China under Grant No. 11875136, two identical GW waveforms, the fitting factor is 1. If and the Major Program of the National Natural Science the fitting factor is very close to 1, then it means that it is Foundation of China under Grant No. 11690021.

[1] B. P. Abbott et al. (LIGO Scientific and Virgo Collabo- Quant. Grav. 27, 084006 (2010). rations), Phys. Rev. Lett. 116, 061102 (2016). [4] J. Aasi et al. (LIGO Scientific Collaboration), Class. [2] B. P. Abbott et al. (LIGO Scientific and Virgo Collabo- Quant. Grav. 32, 074001 (2015). rations), Phys. Rev. Lett. 116, 131103 (2016). [5] F. Acernese et al. (Virgo Collaboration), Class. Quant. [3] G. M. Harry (LIGO Scientific Collaboration), Class. Grav. 32, 024001 (2015). 5

[6] B. P. Abbott et al. (LIGO Scientific and Virgo Collabo- 042001 (1998). rations), Phys. Rev. X 9, 031040 (2019). [41] J. Alsing, E. Berti, C. M. Will, and H. Zaglauer, Phys. [7] R. Abbott et al. (LIGO Scientific and Virgo Collabora- Rev. D 85, 064041 (2012). tions), Phys. Rev. X 11, 021053 (2021). [42] J. Antoniadis et al., Science 340, 6131 (2013). [8] D. M. Eardley, D. L. Lee, A. P. Lightman, R. V. Wagoner, [43] X. Zhang, R. Niu, and W. Zhao, Phys. Rev. D 100, and C. M. Will, Phys. Rev. Lett. 30, 884 (1973). 024038 (2019). [9] D. M. Eardley, D. L. Lee, and A. P. Lightman, Phys. [44] B. C. Seymour and K. Yagi, Class. Quant. Grav. 37, Rev. D 8, 3308 (1973). 145008 (2020). [10] R. V. Wagoner, Phys. Rev. D 1, 3209 (1970). [45] P. C. C. Freire, N. Wex, G. Esposito-Farese, J. P. W. Ver- [11] D. Liang, Y. Gong, S. Hou, and Y. Liu, Phys. Rev. D 95, biest, M. Bailes, B. A. Jacoby, M. Kramer, I. H. Stairs, 104034 (2017). J. Antoniadis, and G. H. Janssen, Mon. Not. Roy. Astron. [12] S. Hou, Y. Gong, and Y. Liu, Eur. Phys. J. C 78, 378 Soc. 423, 3328 (2012). (2018). [46] S. Mirshekari and C. M. Will, Phys. Rev. D 87, 084070 [13] Y. Gong, S. Hou, E. Papantonopoulos, and D. Tzortzis, (2013). Phys. Rev. D 98, 104017 (2018). [47] R. N. Lang, Phys. Rev. D 89, 084014 (2014). [14] Y. Gong, S. Hou, D. Liang, and E. Papantonopoulos, [48] R. N. Lang, Phys. Rev. D 91, 084027 (2015). Phys. Rev. D 97, 084040 (2018). [49] N. Sennett, S. Marsat, and A. Buonanno, Phys. Rev. D [15] B. P. Abbott et al. (LIGO Scientific and Virgo Collabo- 94, 084003 (2016). rations), Phys. Rev. Lett. 116, 221101 (2016), [Erratum: [50] L. Bernard, Phys. Rev. D 98, 044004 (2018). Phys.Rev.Lett. 121, 129902 (2018)]. [51] L. Bernard, Phys. Rev. D 99, 044047 (2019). [16] B. P. Abbott et al. (LIGO Scientific and Virgo Collabo- [52] L. Bernard, Phys. Rev. D 101, 021501 (2020). rations), Phys. Rev. Lett. 123, 011102 (2019). [53] C. M. Will, Phys. Rev. D 50, 6058 (1994). [17] B. P. Abbott et al. (LIGO Scientific and Virgo Collabo- [54] M. Shibata, K.-i. Nakao, and T. Nakamura, Phys. Rev. rations), Phys. Rev. D 100, 104036 (2019). D 50, 7304 (1994). [18] R. Abbott et al. (LIGO Scientific and Virgo Collabora- [55] M. Saijo, H.-a. Shinkai, and K.-i. Maeda, Phys. Rev. D tions), Phys. Rev. D 103, 122002 (2021). 56, 785 (1997). [19] B. P. Abbott et al. (LIGO Scientific and Virgo Collab- [56] C. M. Will, Phys. Rev. D 57, 2061 (1998). orations, Fermi-GBM, INTEGRAL), Astrophys. J. Lett. [57] M. Brunetti, E. Coccia, V. Fafone, and F. Fucito, Phys. 848, L13 (2017). Rev. D 59, 044027 (1999). [20] S. Mirshekari, N. Yunes, and C. M. Will, Phys. Rev. D [58] P. D. Scharre and C. M. Will, Phys. Rev. D 65, 042002 85, 024041 (2012). (2002). [21] J. Beltran Jimenez, F. Piazza, and H. Velten, Phys. Rev. [59] C. M. Will and N. Yunes, Class. Quant. Grav. 21, 4367 Lett. 116, 061101 (2016). (2004). [22] P. M. Chesler and A. Loeb, Phys. Rev. Lett. 119, 031102 [60] K. Yagi and T. Tanaka, Phys. Rev. D 81, 064008 (2010), (2017). [Erratum: Phys.Rev.D 81, 109902 (2010)]. [23] T. Baker, E. Bellini, P. G. Ferreira, M. Lagos, J. Noller, [61] N. Yunes, P. Pani, and V. Cardoso, Phys. Rev. D 85, and I. Sawicki, Phys. Rev. Lett. 119, 251301 (2017). 102003 (2012). [24] P. Creminelli and F. Vernizzi, Phys. Rev. Lett. 119, [62] K. G. Arun, Class. Quant. Grav. 29, 075011 (2012). 251302 (2017). [63] E. Barausse, N. Yunes, and K. Chamberlain, Phys. Rev. [25] J. Sakstein and B. Jain, Phys. Rev. Lett. 119, 251303 Lett. 116, 241104 (2016). (2017). [64] R. Niu, X. Zhang, B. Wang, and W. Zhao. [26] J. M. Ezquiaga and M. Zumalac´arregui, Phys. Rev. Lett. [65] K. Danzmann, Class. Quant. Grav. 14, 1399 (1997). 119, 251304 (2017). [66] P. Amaro-Seoane et al. (LISA). [27] M. A. Green, J. W. Moffat, and V. T. Toth, Phys. Lett. [67] J. Luo et al. (TianQin), Class. Quant. Grav. 33, 035010 B 780, 300 (2018). (2016). [28] A. Nishizawa, Phys. Rev. D 97, 104037 (2018). [68] W.-R. Hu and Y.-L. Wu, Natl. Sci. Rev. 4, 685 (2017). [29] S. Arai and A. Nishizawa, Phys. Rev. D 97, 104038 [69] L. Barack and A. Pound, Rept. Prog. Phys. 82, 016904 (2018). (2019). [30] Y. Gong, E. Papantonopoulos, and Z. Yi, Eur. Phys. J. [70] L. Barack and C. Cutler, Phys. Rev. D 69, 082005 (2004). C 78, 738 (2018). [71] S. A. Teukolsky, Astrophys. J. 185, 635 (1973). [31] R. A. Battye, F. Pace, and D. Trinh, Phys. Rev. D 98, [72] S. Babak, H. Fang, J. R. Gair, K. Glampedakis, and 023504 (2018). S. A. Hughes, Phys. Rev. D 75, 024005 (2007), [Erratum: [32] C. Brans and R. H. Dicke, Phys. Rev. 124, 925 (1961). Phys.Rev.D 77, 04990 (2008)]. [33] R. H. Dicke, Phys. Rev. 125, 2163 (1962). [73] A. J. K. Chua and J. R. Gair, Class. Quant. Grav. 32, [34] B. Bertotti, L. Iess, and P. Tortora, Nature 425, 374 232002 (2015). (2003). [74] A. J. K. Chua, C. J. Moore, and J. R. Gair, Phys. Rev. [35] C. M. Will, Living Rev. Rel. 17, 4 (2014). D 96, 044005 (2017). [36] D. M. Eardley, Astrophys. J. 196, L59 (1975). [75] L. Barack, Class. Quant. Grav. 26, 213001 (2009). [37] C. M. Will, Astrophys. J. 214, 826 (1977). [76] E. Poisson, A. Pound, and I. Vega, Living Rev. Rel. 14, [38] C. M. Will and H. W. Zaglauer, Astrophys. J. 346, 366 7 (2011). (1989). [77] Y. Mino, M. Sasaki, and T. Tanaka, Phys. Rev. D 55, [39] T. Damour and G. Esposito-Farese, Class. Quant. Grav. 3457 (1997). 9, 2093 (1992). [78] M. J. Pfenning and E. Poisson, Phys. Rev. D 65, 084001 [40] T. Damour and G. Esposito-Farese, Phys. Rev. D 58, (2002). 6

[79] L. Barack, Phys. Rev. D 64, 084021 (2001). where the prime indicates the derivative with respect to [80] L. Barack and C. O. Lousto, Phys. Rev. D 66, 061502 χ. We can relate the proper time τ and the parameter χ (2002). using dτ/dχ = r0/r˙, so the geodesic Eqs. (A1)-(A3) can [81] T. S. Keidl, J. L. Friedman, and A. G. Wiseman, Phys. be parameterized with χ and they become Rev. D 75, 124009 (2007). [82] L. Barack and D. A. Golbourn, Phys. Rev. D 76, 044020 r 0 p (2007). φ (χ) = , (A5) p − 6 − 2e cos(χ − w) [83] A. C. Ottewill and B. Wardell, Phys. Rev. D 77, 104002 m p2 t0(χ) = 1 (2008). [e cos(χ − w) + 1]2[p − 2 − 2e cos(χ − w)] [84] S. L. Detweiler, Phys. Rev. D 77, 124026 (2008). s (A6) [85] L. Barack and N. Sago, Phys. Rev. D 81, 084021 (2010). (p − 2 − 2e)(p − 2 + 2e) × . [86] A. Pound, Phys. Rev. Lett. 109, 051101 (2012). p − 6 − 2e cos(χ − w) [87] A. Pound, B. Wardell, N. Warburton, and J. Miller, Phys. Rev. Lett. 124, 021101 (2020). In terms of the orbital parameters p and e, the energy [88] A. Pound and E. Poisson, Phys. Rev. D 77, 044013 (2008). and the angular momentum of the system are [89] N. Warburton, S. Akcay, L. Barack, J. R. Gair, and (p − 2 − 2e)(p − 2 + 2e) N. Sago, Phys. Rev. D 85, 061501 (2012). E2 = , [90] T. Osburn, N. Warburton, and C. R. Evans, Phys. Rev. p(p − 3 − e2) (A7) D 93, 064024 (2016). 2 2 2 p m1 [91] B. Kocsis, N. Yunes, and A. Loeb, Phys. Rev. D 84, L = . p − 3 − e2 024032 (2011). [92] T. A. Apostolatos, Phys. Rev. D 52, 605 (1995). With the SF, the small CO moves along the worldline [93] T. Robson, N. J. Cornish, and C. Liu, Class. Quant. zα(λ) parametrized by the affine parameter λ, Grav. 36, 105011 (2019). α α β γ α z¨ (λ) + Γβγ z˙ (λ)z ˙ (λ) = f , (A8) where the components of SF f α are [88] Appendix A: Overview of the SF method    2  ˙ r 2 dφ φ 2 dr dφ t ap F − r dt + ap r dt dt For EMRI systems, at the zeroth order of approxi- f r = , (A9)   2 mation the small CO moves along the geodesic of the F −1 F 2 − ( dr )2 − F r2 dφ central MBH. Because of the spherical symmetry of the dt dt Schwarzschild BH, the geodesics are in the equatorial plane with θ = π/2, h  i ˙ r dr dφ φ 2 dr
2 t ap dt dt + ap F − dt f φ = , (A10) t˙ = E/F, (A1)  2 2 dr
2 2 dφ F − dt − F r dt

α the subscript p in the acceleration means that ap in- r˙2 = E2 − U , (A2) 2 α 2 α eff volves only the perturbative terms in d z /dt , and ap α A can be constructed from PN theory. Take zG(I (λ), λ) as a geodesic with orbital parameters IA(λ) and using the the osculating condition [88] ˙ L φ = 2 , (A3) r dzα(λ) ∂zα (λ) zα(λ) = zα (IA(λ), λ), = G , (A11) G dλ ∂λ where the constants E and L correspond to the energy and angular momentum of the system, F = 1 − 2m1/r, we get the evolution equations for IA(λ) as [88] m1 is the mass of the central MBH, the effective potential U = F (1+L2/r2), and the overdot means a derivative ∂zα eff G I˙A = 0, with respect to the proper time τ. The Keplerian orbit A ∂I (A12) (1) parameterized by χ is the solution to the geodesic ∂z˙α G I˙A = f α. Eqs. (A1)-(A3). With the parameterization (1), the ra- ∂IA dial component of the velocity becomes Explicitly, the osculating orbits (A12) give the evolu- pm e sin (χ − w) tion equations for the orbital parameters p, e and w as r0(χ) = 1 , (A4) [88] [1 + e cos (χ − w)]2 7

2p7/2m2(p − 3 − e2)(p − 6 − 2e cos v)1/2(p − 3 − e2 cos2 v) p0 = 1 f φ (p − 6 + 2e)(p − 6 − 2e)(1 + e cos v)4 (A13) 2p3m e(p − 3 − e2) sin v − 1 f r, (p − 6 + 2e)(p − 6 − 2e)(1 + e cos v)2

(p − 6 − 2e2) [(p − 6 − 2e cos v)e cos v + 2(p − 3)] cos v + e(p2 − 10p + 12 + 4e2) e0 = (p − 6 + 2e)(p − 6 − 2e)(p − 6 − 2e cos v)1/2(1 + e cos v)4 (A14) p2m (p − 3 − e2)(p − 6 − 2e2) sin v × p5/2m2(p − 3 − e2)f φ + 1 f r, 1 (p − 6 + 2e)(p − 6 − 2e)(1 + e cos v)2

p5/2m2(p − 3 − e2) (p − 6) [(p − 6 − 2e cos v)e cos v + 2(p − 3)] − 4e3 cos v sin v w0 = 1 f φ e(p − 6 + 2e)(p − 6 − 2e)(p − 6 − 2e cos v)1/2(1 + e cos v)4 (A15) p2m (p − 3 − e2) [(p − 6) cos v + 2e] − 1 f r, e(p − 6 + 2e)(p − 6 − 2e)(1 + e cos v)2 where v = χ − w(χ). For a compact binary, the relative acceleration between the two COs up to the 2.5PN in BD theory is [46]

d2~x αm αm = − n + [n(A + A ) +r ˙v(B + B )] dt2 r2 r2 1PN 2PN 1PN 2PN h h (A16) 8 (αm)2 + η 3 [r ˙n(A1.5PN + A2.5PN) − v(B1.5PN + B2.5PN)], 5 rh where

2 3 2 ¯ ¯ αm A1PN = −(1 + 3η +γ ¯)v + ηr˙h + 2(2 + η +γ ¯ + β+ − ψβ−) , (A17) 2 rh

B1PN = 2(2 − η +γ ¯), (A18)

5 A = ζS2 , (A19) 1.5PN 2 −

5 B = ζS2 , (A20) 1.5PN 6 −

4 1 ¯ ¯ 2 αm A2PN = − η(3 − 4η +γ ¯)v + [η(13 − 4η + 4¯γ) − 4(1 − 4η)β+ + 4ψ(1 − 3η)β−]v 2 rh 15 3  1 − η(1 − 3η)r ˙4 + η(3 − 4η +γ ¯)v2r˙2 + 2 + 25η + 2η2 + 2(1 + 9η)¯γ + γ¯2 8 h 2 h 2   ¯ ¯ ¯ ¯ αm 2 87 1 2 (A21) − 4η(3β+ − ψβ−) + 2δ+ + 2ψδ− r˙h − 9 + η + (9 + 8η)¯γ + (9 − 2η)¯γ rh 4 4 ¯ ¯ ¯ ¯ + (8 + 15η + 4¯γ)β+ − ψ(8 + 7η + 4¯γ)β− + (1 − 2η)(δ+ − 2¯χ+) + ψ(δ− + 2¯χ−) β¯ β¯ αm2 − 24η 1 2 , γ¯ rh

1 3 B = η(15 + 4η + 8¯γ)v2 − η(3 + 2η + 2¯γ)r ˙2 2PN 2 2 h (A22) 1 2 2 ¯ ¯ ¯ ¯ αm − [4 + 41η + 8η + 4(1 + 7η)¯γ +γ ¯ − 8η(2β+ − ψβ−) + 4δ+ + 4ψδ−] , 2 rh 8

2 αm 2 A2.5PN = a1v + a2 + a3r˙h , rh 2 αm 2 B2.5PN = b1v + b2 + b3r˙h , (A23) rh

5 15 5 15 a = 3 − γ¯ + β¯ + ζS2 (9 + 4¯γ − 2η) + ζψS S , (A24a) 1 2 2 + 8 − 8 − + 17 35 95 5 S β¯ + S β¯  a = + γ¯ − β¯ − ζS2 135 + 56¯γ + 8η + 32β¯  + 30ζS − + + − 2 3 6 6 + 24 − + − γ¯ 5  32 S β¯ + S β¯  S β¯ + S β¯ 2 − ζψS S − S β¯ + 16 + + − − − 40ζ + + − − , (A24b) 8 − + 3 − − γ¯ γ¯ 25 a = 2¯γ − ζS2 (1 − 2η) − 4β¯ − ζψS S  , (A24c) 3 8 − + − + 5 5 5 5 b = 1 − γ¯ + β¯ − ζS2 (7 + 4¯γ − 2η) + ζψS S , (A24d) 1 6 2 + 24 − 8 − + 5 5 5 b = 3 + γ¯ − β¯ − ζS2 23 + 8¯γ − 8η + 8β¯  (A24e) 2 2 2 + 24 − + 10 S β¯ + S β¯  + ζS − + + − 3 − γ¯ 5  8 16 S β¯ + S β¯  − ζψS S − S β¯ + + + − − , (A24f) 8 − + 3 − − 3 γ¯ 5 b = 6¯γ + ζS2 (13 + 8¯γ + 2η) − 12β¯ − 3ζψS S  . (A24g) 3 8 − + − +

m − m ψ = 1 2 = p1 − 4η, m1 + m2 −1/2 (A25) S+ = −α (s1 − s2), −1/2 S− = −α (1 − s1 − s2).

The parameters in the above equations are defined in where TableI. Here the subscripts ”+” and ”-” on various pa- 8 αm rameters denote sums and differences, such as ABD = 1 − A1PN − A2PN − η r˙h(A1.5PN + A2.5PN), 5 rh 1 1 8 αm x = (x + x ), x = (x − x ). (A26) BBD = (B1PN + B2PN)r ˙h − η (B1.5PN + B2.5PN). + 2 1 2 − 2 1 2 5 rh (A28) From now on, the overdot means the derivative with re- To separate the perturbation from the Schwarzschild spective to t. In the GR limit ω → ∞, Eq. (A16) 0 background and derive the acceleration aα, we write reduces to that of GR. The first term in Eq. (A16) is p ˜ ˜ Newtonian gravity, and the presence of α shows the vi- ABD = As + A and BBD = Bs + B, where the coeffi- olation of the strong equivalence principle in BD theory. cients As and Bs coming from the geodesic equations of If the mass of an self-gravitating body is independent of the Schwarzschild BH through 2.5PN are the BD scalar field, then the sensitivity s = 0 and α = 1, αm1 2 αm1 2 αm1 drh 2 we recover the Newtonian gravity. Since the sensitivity of AS = 1 − 4 + v + 9( ) − 2 ( ) , (A29) rh rh rh dt BHs is s = 1/2, so for binary BHs, the equation of motion (A16) in BD theory is the same as that in GR through drh αm1 BS = − (4 − 2 ), (A30) 2.5PN if we rescale each mass by α. In the extreme mass dt rh ratio limit, it was found that there is no dipolar radiation ˜ ˜ to all orders in PN theory for binary BHs [61]. and A and B denote the contributions from perturba- For convenience, we rearrange Eq. (A16) as tions. Combining Eqs. (A27)-(A30), we get the per- turbed accelerations 2 d ~x αm r m1 ˜ ˜ drh 2 = − 2 (ABDn − BBDv), (A27) ap = − 2 (A + B ), (A31) dt rh rh dt 9

TABLE I. Parameters used in the equations of motion. We used similar notation as in [46]

Parameter Definition Parameter Definition Scalar-tensor parameters Equation of motion parameters −1 G φ0 (4 + 2ω0)/(3 + 2ω0) Newtonian ζ 1/(4 + 2ω0) α 1 − ζ + ζ(1 − 2s1)(1 − 2s2) 2 λ1 (dω/d(ϕ/ϕ0))0ζ /(1 − ζ) post-Newtonian 2 2 3 −1 λ2 (d ω/d(ϕ/ϕ0) )0ζ /(1 − ζ)γ ¯ −2α ζ(1 − 2s1)(1 − 2s2) ¯ −2 2 0 Sensitivities β1 α ζ(1 − 2s2) (λ1(1 − 2s1) + 2ζs1) ¯ −2 2 0 sA [d ln mA(ϕ)/d ln ϕ]0 β2 α ζ(1 − 2s1) (λ1(1 − 2s2) + 2ζs2) 0 2 2 sA [d ln mA(ϕ)/d ln ϕ ]0 2nd post-Newtonian 00 3 3 ¯ −2 2 sA [d ln mA(ϕ)/d ln ϕ ]0 δ1 α ζ(1 − ζ)(1 − 2s1) −2 2 δ¯2 α ζ(1 − ζ)(1 − 2s2) −3 3  2 0 2 00 χ¯1 α ζ(1 − 2s2) (λ2 − 4λ1 + ζλ1)(1 − 2s1) − 6ζλ1s1 + 2ζ s1 −3 3  2 0 2 00 χ¯2 α ζ(1 − 2s1) (λ2 − 4λ1 + ζλ1)(1 − 2s2) − 6ζλ1s2 + 2ζ s2

φ m1 ˜ dφ Appendix B: GW Waveform ap = − 2 B . (A32) rh dt The waveforms up to quadrupole radiation for GR and BD are Substituting Eqs. (A31) and (A32) into Eqs. (A9) and   (A10), we derive the SF in BD theory and the we solve ij 4ηm i j m i j hGR = v v − n n , (B1) the evolution Eqs. (A13)-(A15) for the orbital parame- DL rh ters.

To see the effect of the mass ratio on the result, we ij m1 ij 2ζA ij hBD =2ζ δ − δ also compute the phase difference between GR and BD DL DL theory with ω = 2000 for EMRIs with different mass   (B2) 0 4(1 − 2ζ)ηm i j m i j ratio q and the results are shown in Fig.4. + v v − n n , DL rh

where DL is luminosity distance between the source and 25 - the detector. The other parameters are q=10 4 - q=10 6 1 20 A = E + E˙jN − I¨jkN N , (B3) j 2 j k

15   rad m m m E = 2(1 + 2λ) 1 2 + ηm v2 + (1 + 4λ) , (B4) Δϕ / 10 rh rh

5   j j 2 m j E = −2(1 + 2λ)ηmSrh − η∆m v + (1 + 4λ) rh, 2rh 0 0 50 100 150 200 (B5) t/days  m  I¨jk = 2ηm vivj − ninj , (B6) rh FIG. 4. The orbital phase difference ∆φ = φBD−φGR between where ∆ = (m − m )/(m + m ) and S = (m − GR and BD theory with ω = 2000 for EMRIs with different 1 2 1 2 1 0 m )/(2r ). For GWs propagating along the z direction mass ratio q. The intitial conditions and curves are the same 2 h as described in Fig.2. with the unit vector eZ in the detector-adapted frame, the polarizations of GWs are expressed as 1 H = h (ei ej − ei ej ), (B7) From Fig.4, we see that the change of the phase dif- + 2 ij X X Y Y ference is small. The smaller the mass ratio, the smaller the phase difference due to the smaller perturbation of 1 H = h (ei ej + ej ei ), (B8) the small compact object. × 2 ij X Y X Y 10

1 H = h (ei ej + ei ej ), (B9) k (k = 0, 1, 2), b 2 ij X X Y Y 0 hij = hij(t − 2L/c), where the unit vectors eX and eY are perpendicular to 1 ˆ the unit vector eZ and they along with eZ form an or- hij = hij[t − (L/c)(1 + Ω · uˆ)], (B12) thonormal basis in the detector-adapted frame. In par- h2 = h (t), ticular, in the heliocentric coordinate system, ij ij where c is the speed of light, L is the arm length of the eX = [cos ξ, − sin ξ, 0], (B10) detector and Ωˆ is the propagating direction of GWs. The eY = [cos ι sin ξ, cos ι cos ξ, − sin ι], phase shift induced by the GWs is where the inclination angle ι measures the angle between Z t 0 0 the propagation direction of GWs and the normal vector ∆Φ(t, uˆ1) = 2π ∆ν(t , uˆ1)dt . (B13) 0 of the orbital plane, the longitude of pericenter ξ is the The strain recorded in the interferometric detector is angle between the pericenter and the line of nodes as c measured in the orbital plane. Substituting the results H(t) = (∆Φ(t, uˆ1) − ∆Φ(t, uˆ2)), (B14) of the orbital evolution as shown in Fig.1 to Eqs. (B7)- 4πν0L (B9), we get the GW waveforms in the time domain as whereu ˆ andu ˆ are the unit vectors along the two arms shown in Figs.5 and6. The GW waveforms of plus 1 2 of the detector. and cross poloarizations in GR and BD theory are shown In this paper, we take LISA as an example to calculate in Fig.5. The GW waveforms of the breathing mode the detector response. The result can be easily extended present in BD theory are shown in Fig.6. to other space-based detectors like Tianqin or Taiji. In From Fig.6, we see that the amplitude of the breath- the heliocentric coordinate system, the unit vectors of ing polarization is several orders of magnitude smaller two detector arms, i.e.u ˆ andu ˆ of LISA are than those of the two tensor polarizaitons, making it hard 1 2 to be directly detected. uˆ1x = − sin(ωst) cos(ωst) + cos(ωst) sin(ωst)/2, For the binary system with non-negligible orbital ec- uˆ = cos(ω t) cos(ω t) + sin(ω t) sin(ω t)/2, centricity, it radiates GWs in multiple harmonics in the 1y s s s s inspiral stage. In this case, the GWs contain multi- uˆ1z = sin(π/3) sin(ωst), frequency contribution at any moment, which is very dif- uˆ2x = − sin(ωst) cos(ωst − π/3) + cos(ωst) sin(ωst − π/3)/2, ferent from the GWs emitted by circular binary whose uˆ2y = cos(ωst) cos(ωst − π/3) + sin(ωst) sin(ωst − π/3)/2, frequency is twice the orbital frequency. When the or- bital eccentricity is high, the high-frequency harmonics uˆ2z = sin(π/3) sin(ωst − π/3), become dominant, the detector response to such multi- where the rotation frequency ω = 2π/(365 days) and we band GWs is complicated. s set the initial phase to be zero. When we calculate the Consider a photon emitted at spacetime event 0, trav- response of the detector in the heliocentric coordinate, eling in the directionu ˆ . It arrives at the end-mirror at 1 we have to consider the phase modulation induced by spacetime event 1 and then returns at spacetime event the translatory motion of the detector which introduces 2. The frequency shift induced by GWs for this single an extra time delay round trip is ˆ ˆ 2 1 1 0 ! Ω · dLISA(t) ∆ν(t, uˆ1) 1 i j hij − hij hij − hij td(t) = , (B15) = uˆ1uˆ1 + . (B11) c ν0 2 1 + Ωˆ · uˆ1 1 − Ωˆ · uˆ1 ˆ where we adopt dLISA(t) = (cos ωst, sin ωst, 0) × 1AU for k where hij is the metric perturbation at spacetime event simplicity. Then the frequency shift in one arm becomes

ˆ ∆ν(t, uˆ1) 1 i j hij(t − td) − hij(t − td − (L/c)(1 + Ω · uˆ1)) = uˆ1uˆ1 ν0 2 1 + Ωˆ · uˆ1 ! (B16) h (t − t − (L/c)(1 + Ωˆ · uˆ )) − h (t − t − 2L/c) + ij d 1 ij d . 1 − Ωˆ · uˆ1

The frequency shift in the other arm can be obtained by Substitute Eqs. (B1) and (B2) into Eqs. (B13), (B14) replacingu ˆ1 withu ˆ2. and (B16) separately, we can obtain the strain HGR(t) 11

0.10 0.10

0.05 0.05 / μ / μ L L D 0.00 D 0.00

 BD + H H GR

-0.05 -0.05

-0.10 -0.10 0.0 0.1 0.2 0.3 0.4 0.5 0.0 0.1 0.2 0.3 0.4 0.5 t/day t/day

ω0=2000 ω0=2000 0.10 0.10

0.05 0.05 / μ / μ L L BD D 0.00 D 0.00  + H H GR

-0.05 -0.05

-0.10 -0.10 15.0 15.1 15.2 15.3 15.4 15.0 15.1 15.2 15.3 15.4 t/day t/day ω0=10000 ω0=10000 0.10 0.10

0.05 0.05 / μ / μ L L BD D 0.00 D 0.00  + H H GR

-0.05 -0.05

-0.10 -0.10 79.3 79.4 79.5 79.6 79.7 79.3 79.4 79.5 79.6 79.7 t/day t/day ω0=50000 ω0=50000 0.10 0.10

0.05 0.05 / μ / μ L L BD D 0.00 D 0.00  + H H GR

-0.05 -0.05

-0.10 -0.10 368.9 369.0 369.1 369.2 369.3 368.9 369.0 369.1 369.2 369.3 t/day t/day ω0=100000 ω0=100000 0.10 0.10

0.05 0.05 / μ / μ L L BD D 0.00 D 0.00  + H H GR

-0.05 -0.05

-0.10 -0.10 692.5 692.6 692.7 692.8 692.9 692.5 692.6 692.7 692.8 692.9 t/day t/day

FIG. 5. The plus and cross waveforms in the time domain in GR and BD theory with different ω0. Different time windows are chosen for different ω0 to display the mismatch of the phases in GR and BD theory. 12

FIG. 6. The GW waveform of the breathing polarization in BD theory with different ω0.

for GR and HBD(t) for BD theory. the noise power spectral density Sn(f) of LISA is [93]

With a signal H(t), the SNR in LISA is S 2S 1 + cos2(2πfL/c)
S (f) = x + a n L2 (2πf)4L2 ! (B18) 4 × 10−4Hz2 Z ∞ 1 × 1 + , ρ2 = 4 df H˜ (f)H˜ ∗(f), (B17) f 0 Sn(f) √ −15 −2 1/2 the acceleration noise is Sa = 3 × 10 m s /Hz , √ 1/2 the displacement noise is Sx = 15 pm/Hz and the where H˜ (f) is the Fourier transform of the signal H(t), arm length is L = 2.5 × 106 km [66].