<<

Chiral Tricritical Point: A New Universality Class in Dirac Systems

Shuai Yin,1 Shao-Kai Jian,1, ∗ and Hong Yao1, 2, 3, † 1Institute for Advanced Study, Tsinghua University, Beijing 100084, China 2State Key Laboratory of Low Dimensional Quantum Physics, Tsinghua University, Beijing 100084, China 3Collaborative Innovation Center of Quantum Matter, Beijing 100084, China (Dated: May 28, 2018) Tricriticality, as a sister of criticality, is a fundamental and absorbing issue in condensed-matter physics. It has been verified that the bosonic Wilson-Fisher universality class can be changed by gapless fermionic modes at criticality. However, the counterpart phenomena at tricriticality have rarely been explored. In this Letter, we study a model in which a tricritical is coupled to massless Dirac fermions. We find that the massless Dirac fermions result in the emergence of a new tricritical point, which we refer to as the chiral tricritical point (CTP), at the phase boundary between the Dirac semimetal and the charge-density wave insulator. From functional analysis of the effective action, we obtain the critical behaviors of the CTP, which are qualitatively distinct from both the tricritical Ising universality and the chiral Ising universality. We further extend the calculations of the chiral tricritical behaviors of Ising spins to the case of Heisenberg spins. The experimental relevance of the CTP in two-dimensional Dirac semimetals is also discussed.

Introduction.— Discovering and understanding uni- H⊥

versality classes is among the central issues in modern 0.6 ◼ condensed-matter physics as the fundamental concept 1st order of universality class is crucial to classifying the infrared 0.4 ◼ Dirac Semimetal CTP 2 physics in many-body systems [1, 2]. The universality nd 0.2 ◼ CDW order class is dictated by the renormalization group (RG) flow

near the corresponding fixed point [3, 4]. According to ◼ ◼ ◼ ◼ ◼ ◼ ◼ 0 0.2 0.4 0.6 0.8 1.21.0 the number of relevant directions of the RG flow, the H singular behavior near the fixed point can be classified into criticality, tricriticality, and so on [5]. In contrast FIG. 1. The quantum phase diagram as a function of Zee- to the criticality controlled by only one relevant direc- man fields H⊥ and Hk with h = 0.2, t = 1,J1 = 2J2 = 1/6. The blue dashed and blue solid lines refer to first- and tion, tricriticality often shows more intriguing properties, second-order phase transitions between the charge-density since it has two relevant directions [5, 6]. Thus, tricrit- wave (CDW) insulator and the Dirac semimetal. The red icality has aroused enormous theoretical as well as ex- point at (Hk,H⊥) = (0.7, 0.375) denotes the chiral tricritial perimental studies in condensed-matter systems [7–17]. point (CTP). The gray dashed and gray solid lines refer to For instance, the tricritical Ising fixed point in 1 + 1 first- and second-order phase transitions of the Ising field dimensions enjoys superconformal symmetry [8]. The without coupling to fermions (h = 0). quantum tricritical point (TP) connecting the deconfined quantum critical point [9] and first-order transition in two-dimensional spin systems exhibits O(4) symmetry different universality class, usually called the chiral Ising and self-duality [10]. Furthermore, even supersymmetric universality class (or the Gross-Neveu-Yukawa universal- quantum electrodynamics can emerge at the pair-density- ity class) if the Ising field couples to the Dirac fermions wave quantum TP on the surface of a three-dimensional as a chiral mass [18–28]. Recently, it was proposed in topological insulator [11]. Among them, some theoretical Ref. [29] by two of us and collaborators that the pres- predictions [12, 13] have been verified in experiments [14– ence of gapless fermionic modes can even change the type 17]. of from first-order to continuous, realiz- ing the so-called fermion-induced quantum critical points arXiv:1711.10473v3 [cond-mat.str-el] 25 May 2018 On the other hand, quantum phase transitions in sys- (also see Refs. [30–32]). However, for the more colorful tems of interacting Dirac fermions have been investigated tricriticality, the effects induced by the massless Dirac intensively in recent years [3, 4]. Besides their fundamen- fermions have rarely been studied. Some questions then tal relevance in relativistic quantum field theories, such arise: To what extent is the tricriticality affected by the fermion systems also emerge in the low-energy sector of Dirac fermion? Will a new fixed point emerge? If so, various condensed-matter systems [1, 2]. It was realized how are we to characterize the critical properties in this that gapless fermionic excitations at quantum criticality new universality class? play an essential role in determining the critical behav- To answer these questions, in this Letter we report iors. For instance, coupling massless Dirac fermions to a first theoretical study of the chiral tricritical point the Ising quantum critical point results in a qualitatively (CTP). As suggested by its name, the CTP refers to a tri- 2 critical point coupled to massless Dirac fermions. More plicity, because m decouples from the MF Hamiltonian specifically, we first construct a microscopic model on the for J− = 0. Such a simplification will not qualitatively honeycomb lattice, where the Ising field couples to the change the physics discussed here, because the global Z2 Q x itinerant fermions as a staggered on-site potential. Us- symmetry—namely, the symmetry generated by i σi — ing mean-field (MF) calculations, we find that this mi- is already broken by the external longitudinal field H⊥. croscopic model features a zero-temperature transition As a function of H⊥ and Hk, the Hamiltonian Hb ex- between a charge-density wave insulator (CDW) and a hibits a phase transition from hφi = 0 to hφi= 6 0 (break- Dirac semimetal as shown in Fig. 1, where the transition ing the lattice inversion or Z2 symmetry that exchanges can be either discontinuous or continuous depending on two sublattices), which is continuous or discontinuous two tuning parameters. The CTP emerges at the location depending on the external fields (see the Supplemental where first- and second-order transitions meet. Material for details) and a quantum tricritical point sep- To understand the universal behaviors of the CTP, arating these two types of transitions, as shown by the we carry out functional renormalization group (FRG) gray line in Fig. 1. The Landau free energy up to sixth calculations of the effective theory near the transition order near the transitions is given by (see the Supple- point. By solving the RG equations, we find four nontriv- P3 00 2n mental Material for details) fb = n=0 λ2nφ , where ial fixed points, corresponding to the Ising, chiral Ising, 00 2 2 3/2 2 00 2 2 00 λ2 ∝ (H⊥ + Hk ) − Hk J+, λ4 ∝ Hk − 4H⊥, and λ6 tricritical Ising, and chiral tricritical Ising universality is a positive constant. The tricritical point is located classes, distinguished by the relevant directions. The at λ2 = λ4 = 0, which can be accessed by tuning two former three universal classes have been extensively ex- parameters, H⊥ and Hk. Since the critical behaviors of plored [5, 6, 19, 21, 33], while the last one— i.e., the this quantum tricritical point are governed by a φ6 the- chiral tricritical Ising universality class—is new and is ory whose upper is 3, the MF theory our main focus in the present Letter. Various critical should be reliable. Various critical exponents of this tri- exponents characterizing this novel universality class are critical Ising universality class are listed in Table I. obtained. We further study tricritical points of Heisen- To investigate a CTP, we consider massless Dirac berg transitions in Dirac semimetals [19, 21, 34–37] by fermions on the honeycomb lattice which couple to the extending the Ising fields to Heisenberg spins. The rel- Ising variables as follows: evance of these universality classes to real materials like X † X z † graphene or graphene-like systems will also be discussed. Hf = −t (ciscjs + H.c.) + h σi ciscis, (3) Lattice model and phase diagram.— It is known that hiji,s i,s the (2+1)-dimensional Ising model under both transver- † sal and longitudinal Zeeman fields can realize a tricritical where cis labels the operator creating a fermion on site i point at the phase boundary between first- and second- with spin polarization s, t is the hopping amplitude, and order transitions [7]. The Hamiltonian on the honeycomb h refers to the coupling strength between fermions and lattice is given by Ising variables. Note that in Eq. (3) there is an implicit P † chemical potential term −µ c cis. The condensa- X z z X z z X z X x i,s is Hb =J1 σi σj −J2 σi σj −H⊥ σi −Hk σi , (1) tion of the Ising field φ gives rise to staggered on-site po- hiji hhijii i i tentials for the fermions on the honeycomb lattice which where σ denotes the Ising spins defined at each site, and can gap out the Dirac fermions, as shown by the dis- h i and hh ii refer to nearest neighbors (NN) and next- persions of the fermions in the Ising symmetry-breaking ~ 2 2 2 nearest neighbors (NNN), respectively. J and J refer to phase: ε(k) = ±{h φ +t [3+2(cos k1 +cos k2 +cos(k1 − 1 2 1 k ))]} 2 , where the lattice constant is set to be 1 for sim- the NN antiferromagnetic and NNN ferromagnetic cou- 2 √ 1 3 ~ plings, respectively. H⊥ and Hk refer to the longitudinal plicity, ande ˆ1 = (1, 0),e ˆ2 = ( 2 , 2 ), and ki =e ˆi · k. and transversal Zeeman fields, respectively. The quantum phase diagram of the Hamiltonian H = We assume the vacuum expectation value (VEV) of Hb + Hf is shown in Fig. 1, where the blue line refers to z Ising variables is given by hσA/Bi = m ± φ, where A and the phase boundary, and the CTP is indicated by the red B refer to two sublattices of the honeycomb lattice, and point (see the Supplemental Material for details). The m (φ) denotes the uniform (staggered) VEV of the Ising evolutions of the free energy as a function of the expec- variables. Note that we refer to microscopic degrees of tation value of the Ising field across the transition point freedom σ as the Ising variable while referring to φ as the are shown in Fig. 2, where one can clearly see that the Ising field. By employing the MF decoupling, we obtain phase transition changes from first to second order as the HMF tuning parameters cross the CTP. b = (J m − J φ − H )σz − H σx Near the phase boundary, the free energy can be ex- M − + ⊥ A k A z x 2 2 panded as (see the Supplemental Material for details) + (J−m + J+φ − H⊥)σB − HkσB − J−m + J+φ , (2) P3 0 2n 3 0 f = n=0 λ2nφ + λ3|φ| , where λ2n are modified cou- where J± ≡ 3J1 ± 6J2, and M is the number of unit pling constants owing to the presence of Dirac fermions. cells. Below, we focus on the case where J− = 0 for sim- These modifications only renormalize the location of the 3

f[ϕ] f[ϕ] To solve Eq. (5), we employ the modified local poten- tial approximation (LPA’). According to LPA’, one can obtain the β functions for the expansion coefficients from 1 ∂n ∂ λ˜ = ∂ U˜(˜ρ)| . (9) τ n n! ∂ρ˜n t ρ˜=0 ϕ ϕ 2 (a) (b) Similarly, one can obtain the β function for h˜ according to FIG. 2. The evolution of the free energy as a function of −→ ←− −i δ δ δ the expectation value of the Ising field across the transition ˜ ∂τ h = Tr[γ3 ¯ ∂tΓk ]|ρ˜=0. (10) point. (a) The evolution of free energy shows a first-order Nf dγ δφ(0) δψ(0) δψ(0) transition at Hk = 0.6 as one tunes H⊥. (b) The evolution of free energy shows a second-order transition at H = 0.8 Using the Litim cutoff regulators [42], i.e., Rk = k 2 2 2 2 as one tunes H⊥. These provide the evidence that there is a Zφ,k(k − q )Θ(k − q ) for the boson cutoff and Rk = 2 2 quantum tricritical point between 0.6 < Hk < 0.8. Zψ,kiγµqµ(k/q−1)Θ(k −q ) for the fermion cutoff where Θ is the step function, the flow equations read [40] ˜ phase boundary. However, a new nonanalytical term ˜ ˜ ηb,k λ4 24vD ∂τ λ2 = (ηb,k − 2)λ2 − (1 − ) 3 ˜ 2 λ3|φ| appears due to the IR singularity of the gapless D + 2 (1 + λ2) D 4h3 fermionic modes, where λ3 = 2 , and changes the η 8v 9πt + (1 − f,k )h2d N D , (11) universal critical behaviors in the IR. Since the gapless D + 1 γ f D modes involve both fermions and bosons, it is important ˜2 ˜ ˜ ηb,k 12vD 12λ4 to keep the massless fermionic modes at the critical point. ∂τ λ4 = (D − 4 + 2ηb,k)λ4 + D + 2 D ˜ 3 Thus, to explore the critical behaviors, we use the effec- (1 + λ2) ˜ tive Lagrangian given as follows (see the Supplemental ηb,k 12vD 5λ6 16vD 4 ηf − − dγ Nf h (1− ), (12) Material for details): ˜ 2 D + 2 D (1 + λ2) D D + 1 1 ˜3 L = ψγ¯ µ∂ ψ + (∂ φ)2 + hφψψ¯ + U(ρ), (4) ˜ ˜ ηb 144vD 6λ4 µ µ ∂τ λ6 = (2D − 6 + 3ηb,k)λ6 − 2 ˜ 4 D + 2 D (1 + λ2) where ψ refers to the Dirac fermions, while φ denotes the ˜ ˜ ¯ † 0 0 z 1 y z ηb,k 144vD 5λ4λ6 32vD ˜6 ηf,k Z2 order parameter. ψ = ψ γ , γ = τ , γ = τ µ , + − dγ Nf h (1− ),(13) 2 x i i D + 2 D ˜ 3 D D + 1 γ = τ , and ∂µ = (∂τ , ∇~ ). τ and µ are Pauli matri- (1 + λ2) ces referring to sublattice and valley degrees of freedom, 16vD 1 ηf,k ∂ h˜2 = (D − 4 + η + 2η )h˜2 − (1 − ) τ b,k f,k ˜ respectively. We set both fermion and boson velocity to D 1 + λ2 D + 1 1 due to the emergent Lorentz symmetry [38, 39]. U(ρ) 16vD 1 ηb,k 1 2 − (1 − ) , (14) is an analytical function of ρ ≡ φ /2, representing the D 1 + λ˜ D + 2 (1 + λ˜ )2 potential of the Ising field. Note that the potential is 2 2 −1 D+1 D/2 an even function of φ owing to the Z2 symmetry of the where vD = 2 π Γ(D/2), D is the spacetime di- model. The expansion of potential can be expressed as mension, and the running anomalous dimensions for bo- P i U(ρ) = i λ2iρ . In the following calculations, we set son and fermion fields, defined as ηb,k ≡ −∂τ Zφ,k/Zφ,k the number of the four-component Dirac fermions to be and ηf,k ≡ −∂τ Zψ,k/Zψ,k, respectively, can be solved Nf = 2, which corresponds to spin-1/2 fermions on the explicitly as honeycomb lattice. 4vD 2 4 − 3D + 2ηf,k FRG analysis.— The RG flow of the effective action ηb,k = 2Nf dγ h , (15) satisfies the Wetterich equation [40, 41] D 8 − 4D 8vD 2 ηb,k 1 (2) −1 ηf,k = h 2 . (16) ∂τ Γk = STr[(Γk + Rk) ∂τ Rk], (5) D 1 + D (1 + λ2) where τ = lnk,Γk is the effective action, Rk is the regula- For k → 0, the equations above give anomalous dimen- (2) tor function, STr denotes the supertrace, and Γk reads sions of the boson field and fermion field, respectively. −→ ←− We then solve the flow equations in Eqs. (11)-(14) for (2) δ δ the case of Nf = 2 (namely spin-1/2 fermions). To iden- Γk (i, j) ≡ Γk , (6) δΦi δΦj tify the CTP fixed point, we resort to the stability ma- ¯ trix at each fixed point. The stability matrix is defined with Φi ≡ (φ, ψ, ψ). We then define the dimensionless ˜ ˜2 variables as as Si,j = −∂Bi/∂αj, in which B ≡ (∂tλ2i, ∂th ) and α ≡ (λ˜ , h˜2). As we mentioned above, Eqs. (11)-(14) ad- 2−D ˜ p 1−D 2i ρ˜ ≡ Zφ,kk ρ, ψ ≡ Zψ,kk ψ, (7) mit four nontrivial fixed points. Among them, two show ˜2 D−4 −1 −2 2 ˜ −D h = k Zφ,kZψ,kh , U(˜ρ) ≡ k U(ρ). (8) double positive eigenvalues of S (see the Supplemental 4

TABLE I. Critical exponents of the chiral tricritical Ising and chiral tricritical Heisenberg universality classes for Nf = 2. The critical exponents of tricritical Ising, chiral Ising, and chiral Heisenberg universality are also listed for comparison. Note that the mean field calculation is exact for tricritical Ising universality in 2 + 1 dimensions.

Universality class α β γ δ ν ηb ηf CTP Ising 0.694 0.378 0.550 2.456 0.435 0.736 0.036 Tricritical Ising 1/2 1/4 1 5 1/2 0 Chiral Ising [22] −1.018 0.894 1.230 2.376 1.006 0.777 0.028 CTP Heisenberg 0.791 0.404 0.401 1.994 0.403 1.004 0.102 Chiral Heisenberg [36] −1.773 1.278 1.217 1.953 1.257 1.032 0.071

Material for details). One corresponds to the Ising crit- cording to Eqs. (15) and (16) (see the Supplemental Ma- ical point without the coupling to the massless fermion terial for details), ν is given by the inverse of the eigen- ∗ (h = 0). Note that at the Ising critical point, h is a value of the stability matrix along the λ2 direction, and relevant direction. The other is the CTP fixed point, others are obtained via the scaling law. We show the re- at which h∗ 6= 0. The remaining two fixed points cor- sults in Table I. For comparison, the critical exponents respond to the chiral Ising critical point and tricritical for the tricritical Ising and the chiral Ising universality Ising fixed point with one and three relevant directions, class are also displayed therein. Apparently, the chiral respectively. From the eigenvalues of S (See the Supple- tricritical universality class is a new universality class mental Material), we find that the coupling to the mass- distinct from both tricritical Ising and chiral Ising uni- less fermions changes the critical properties dramatically, versality classes. From Table I, one can find that ν in i.e., the eigenvalues corresponding to the CTP are very the chiral trcritical Ising universal class is smaller than distinct from those of the tricritcal Ising point. that in the chiral Ising universality class. The reason can We are ready to obtain the relevant directions in the be traced back to the fluctuation effect, which is weaker CTP by solving the relevant eigenvectors of the pos- in the chiral tricritical universality class because of the itive eigenvalues. Two eigenvectors are (in the ba- absence of the divergent length scale in the first-order sis of α) v1 = (0.996, 0.086, 0.005, −0.015) and v2 = phase transition side of the CTP. (−0.016, 0.349, 0.022, 0.937). For v1, the relevant direc- Chiral Heisenberg tricritical point.— We extend the tion is dominated by λ2, while for v2, the relevant direc- discussion of the chiral tricritical Ising universality to the 2 tion is mainly lying in the λ4-h plane, and the weight in chiral tricritcal point involving vector bosons, namely, h2 direction is larger. Therefore, in contrast to the usual the chiral tricritical Heisenberg point. The Lagrangian is tricritical point whose relevant directions are λ2 and λ4, given by the coupling to the massless Dirac fermions plays signif- ¯ µ 1 ~ 2 ~ ¯ icant roles in the CTP. L = ψγ ∂µψ + (∂µφ) + hφ · (ψ~sψ) + U(ρ), (17) This relevant direction can be understood heuristically 2 P3 2i as follows: Consider the free energy f = i=1 λ2iφ where ψ labels the fermionic operator the same as be- ~ with λ4 < 0 and λ6 > 0. Without coupling to fermions, fore, φ refers to a three-component vector boson, and ~s i.e., h = 0, the phase transition is discontinuous; two lo- is the Pauli matrix in the spin space. U(ρ) is an analyt- √ q 2 −(4+ 12)λ2 ical function of ρ ≡ (φ~) /2, representing the potential of cal potential minima appear at ±φmin ≡ ± 3λ4 2 vector bosons. λ4 at the transition point which locates at λ2c = where 4λ6 Similar to the process in the previous model, we solve the three local minima are degenerate. The fourth- the RG flow equations and study the properties of the order term dominates near φ = φmin. Now, turning stability matrix near the fixed point (see the Supplemen- on the boson-fermion coupling will result in a nonan- tal Material). We find that the CTP also exists in the 3 3 2 alytical term λ3|φ| , where λ3 ∝ h /t , with t being chiral Heisenberg model in which the two largest eigen- the energy scale capturing the kinetic energy of fermions values are positive, in contrast to the chiral Heisenberg (the hopping amplitude in our lattice model). If λ3 is fixed point, which has only one positive eigenvalue. We 3 4 sufficiently large, especially λ3|φmin|  λ4φmin (equiv- also calculate critical exponents and show the results in 2/3 1/2 −1/6 alently, h  t |λ4| λ6 ), the transition will be Table I. driven to continuous. As a result, between h = 0 and Discussion.— The CTP studied in this Letter is a new 2/3 1/2 −1/6 h  t |λ4| λ6 , there must exist a tricritical point fixed point located in an unexploited region of the pa- separating first- and second-order transitions. This also rameter space. Given the theoretic prediction in this explains why besides λ2, the other relevant direction Letter, experimental and numerical studies are urgently 2 mainly lies in the λ4-h plane. called for. In particular, the critical exponents obtained We further calculate other critical exponents along the above provide perceptible information for experiments. λ2 direction. Among them, ηb and ηf are obtained ac- As hosted in the lattice model we construct, the CTP 5 can manifest itself in various systems, such as graphene [9] T. Senthil, A. Vishwanath, L. Balents, S. Sachdev, and and gaphene-like materials [43–51]. This provides broad M. P. A. Fisher, Science 303, 1490 (2004). candidates to detect the CTP. For instance, recent exper- [10] C.-M. Jian, A. Rasmussen, Y.-Z. You, and C. Xu, iments with a controlled epitaxial graphene have visual- arXiv:1708.03050. [11] S.-K. Jian, C.-H. Lin, J. Maciejko, and H. Yao, Phys. ized a chiral symmetry breaking phase [52]. The CTP Rev. Lett. 118, 166802 (2017). could possibly be realized therein by tuning the interac- [12] D. Belitz, T. R. Kirkpatrick, and T. Vojta, Phys. Rev. tion between the boson modes. Moreover, the control- Lett. 82, 4707 (1999). lable magnetism, which has been realized in ultracold [13] A. V. Chubukov, C. P´epin,and J. Rech, Phys. Rev. Lett. atom systems [53–55], particularly in the honeycomb lat- 92, 147003 (2004). tice [54], also provides promising platforms to achieve the [14] C. Pfleiderer, G. MeMullan, and G. Lonzarich, Phys. B CTP. 199, 634 (1994). [15] C. Pfleiderer, G. J. McMullan, S. R. Julian, and G. G. Conclusion.— In this Letter, we have constructed a mi- Lonzarich, Phys. Rev. B 55, 8330 (1997). croscopic model on honeycomb lattice with both fermions [16] M. Uhlarz, C. Pfleiderer, and S. M. Hayden, Phys. Rev. and Ising variables featuring a new CTP. From the pro- Lett. 93, 256404 (2004). posed microscopic model, we find a phase transition be- [17] S. Friedemann, W. J. Duncan, M. Hirschberger, T. W. Bauer, R. K¨uchler, A. Neubauer, M. Brando, C. Pflei- tween a Dirac semimetal and a CDW insulator that derer, and F. M. Grosche, Nat. Phys. 14, 62 (2017). breaks sublattice symmetry as a function of external [18] D. J. Gross and A. Neveu, Phys. Rev. D 10, 3235 (1974). fields, and show the CTP residing between the first-order [19] B. Rosenstein, H.-L. Yu, and A. Kovner, Phys. Lett. B phase transition and the continuous phase transition. Af- 314, 381 (1993). ter extracting the effective action, we have employed the [20] L. N. Mihaila, N. Zerf, B. Ihrig, I. F. Herbut, and M. M. FRG method to explore the critical properties. We have Scherer, Phys. Rev. B 96, 165133 (2017). found that this CTP is dictated by a new fixed point of [21] N. Zerf, L. N. Mihaila, P. Marquard, I. F. Herbut, and M. M. Scherer, Phys. Rev. D 96, 096010 (2017). the RG flow. The eigenvalues of the stability matrix at [22] B. Knorr, Phys. Rev. B 94, 245102 (2016). this fixed point indicate new critical phenomena which [23] S. Raghu, X.-L. Qi, C. Honerkamp, and S.-C. Zhang, are qualitatively different from both the tricitical Ising Phys. Rev. Lett. 100,156401 (2008). universality class and the chiral Ising universality class. [24] Z.-X. Li, Y.-F. Jiang, and H. Yao, Phys. Rev. B 91, The experimental feasibility of realizing the CTP is also 241117(R) (2015). discussed. [25] Z.-X. Li, Y.-F. Jiang, and H. Yao, New J. Phys. 17, 085003 (2015). Acknowledgement. — We thank M. M. Scherer for [26] L. Wang, P. Corboz, and M. Troyer, New J. Phys. 16, helpful discussions. This work is supported in part by 103008 (2014). the NSFC under Grant No. 11474175 (S. Y., S.-K. J. [27] S. Chandrasekharan and A. Li, Phys. Rev. D 88, 021701 and H. Y.) and by the MOST of China under Grant No. (2013). 2016YFA0301001 (H. Y.). S. Y. is supported in part [28] Y.-F. Jiang, Z.-X. Li, S. A. Kivelson, and H. Yao, Phys. Rev. B 95, 241103(R) (2017). by China Postdoctoral Science Foundation (Grant No. [29] Z.-X. Li, Y.-F. Jiang, S.-K. Jian, and H. Yao, Nat. Com- 2017M620035). mun. 8, 314 (2017). [30] S.-K. Jian and H. Yao, Phys. Rev. B 96, 155112 (2017). [31] S.-K. Jian and H. Yao, Phys. Rev. B 96, 195162 (2017). [32] L. Classen, I. F. Herbut, and M. M. Scherer, Phys. Rev. B 96, 115132 (2017). ∗ [email protected] [33] S. El-Showk, M. F. Paulos, D. Poland, S. Rychkov, D. † [email protected] Simmons-Duffin, and A. Vichi, Phys. Rev. D 86, 025022 [1] S. Sachdev, Quantum Phase Transitions, (John Wiley (2012). and Sons, Ltd. 2007). [34] I. F. Herbut, Phys. Rev. Lett. 97, 146401 (2006). [2] S. L. Sondhi, S. M. Girvin, J. P. Carini, and D. Shahar, [35] L. Janssen and I. F. Herbut, Phys. Rev. B 89, 205403 Rev. Mod. Phys. 69, 315 (1997). (2014). [3] E. Fradkin, Field Theories of Condensed Matter Physics [36] B. Knorr, Phys. Rev. B 97, 075129 (2018). (Cambridge Univ. Press, 2013), 2nd ed. [37] F. F. Assaad and I. F. Herbut, Phys. Rev. X 3, 031010 [4] X.-G. Wen, Quantum Field Theory of Many-Body Sys- (2013). tems, (Oxford University Press, New York, 2004). [38] S.-K. Jian, Y.-F. Jiang, and H. Yao, Phys. Rev. Lett. [5] J. Cardy, Scaling and Renormalization in Statistical 114, 237001 (2015). Physics, (Cambridge University Press, Cambridge, Eng- [39] B. Roy, V. Juricic, and I. F. Herbut, J. High Eenergy land, 1996). Phys. 04 (2016) 018 . [6] I. D. Lawrie and S. Sarbach, Phase Transition and Crit- [40] J. Berges, N. Tetradis, and C. Wetterich, Phys. Rep. 363, ical Phenomena, edited by C. Domb and J. L. Lebowitz, 223 (2002). Vol. 9 (Academic Press, London, 1984). [41] W. Metzner, M. Salmhofer, C. Honerkamp, V. Meden, [7] Y. Kato and T. Misawa, Phys. Rev. B 92, 174419 (2015). and K. Sch¨onhammer,Rev. Mod. Phys. 84, 299 (2012). [8] D. Friedan, Z. Qiu, and S. Shenker. Phys. Lett. 151B, [42] D. F. Litim, Phys. Rev. D 64, 105007 (2001). 37 (1985). [43] K. S. A. Novoselov, A. K. Geim, S. V. Morozov, D. Jiang, 6

M. I. Katsnelson, I. V. Grigorieva, S. V. Dubonos, and [49] Y. Otsuka, S. Yunoki and S. Sorella, Phys. Rev. X 6, A. A. Firsov, Nature (London) 438, 197 (2005). 011029 (2016). [44] A. H. Castro Neto, F. Guinea, N. M. R. Peres, K. S. [50] H.-K. Tang, E. Laksono, J. N. B. Rodrigues, P. Sengupta, Novoselov, and A. K. Geim, Rev. Mod. Phys. 81, 109 F. F. Assaad, and S. Adam, Phys. Rev. Lett. 115, 186602 (2009). (2015). [45] C. Honerkamp, Phys. Rev. Lett. 100, 146404 (2008). [51] D. S. de la Pe˜na,J. Lichtenstein, C. Honerkamp, and M. [46] I. F. Herbut, V. Juriˇci´c,and B. Roy, Phys. Rev. B 79, M. Scherer, Phys. Rev. B 96, 205155 (2017). 085116 (2009). [52] C. Gutierrez, et. al., Nat. Phys. 12, 950 (2016). [47] F. Parisen Toldin, M. Hohenadler, F. F. Assaad, I. F. [53] D. Greif, T. Uehlinger, G. Jotzu, L. Tarruell, and T. Herbut, Phys. Rev. B 91, 165108 (2015). Esslinger, Science 340, 1307 (2013). [48] M. V. Ulybyshev, P. V. Buividovich, M. I. Katsnelson, [54] D. Greif, G. Jotzu, M. Messer, R. Desbuquois, and T. and M. I. Polikarpov, Phys. Rev. Lett. 111, 056801 Esslinger, Phys. Rev. Lett. 115, 260401 (2015). (2013). [55] A. Mazurenko, et al., Nature (London) 545, 462 (2017).

SUPPLEMENTAL MATERIAL

A. Mean field theory of tricritical Ising point without fermions

The Ising Hamiltonian is given by X z z X z z X z X x Hb = J1 σi σj − J2 σi σj − H⊥ σi − Hk σi . (S1) ij i i In order to formulate the mean field theory, let’s assume the vacuum expectation value (VEV) of Ising variable is z given by hσA/Bi = m ± φ, where A/B refer to two sublattices in honeycomb lattice and m (φ) refer to the uniform (staggered) VEV of Ising variable. Using the mean field decoupling, z z z z z z z z σi σj = hσi iσj + σi hσj i − hσi ihσj i, (S2) we have HMF b = (J m − J φ − H )σz − H σx + (J m + J φ − H )σz − H σx − J m2 + J φ2, (S3) M − + ⊥ A k A − + ⊥ B k B − + where J± = 3J1 ± 6J2 and M refers to the number of unit cells. Thus the MF energy per unit cell is given by EMF q q b = ± (J m − J φ − H )2 + H2 ± (J m + J φ − H )2 + H2 − J m2 + J φ2 (S4) M − + ⊥ k − + ⊥ k − +

For simplicity, we consider the case J− = 0 or equivalently J1 = 2J2, then the energy is independent of m (note that the general features of the phase diagram does not depend on J− since we consider the condensation of φ field), EMF q q b = ± (J φ + H )2 + H2 ± (J φ − H )2 + H2 + J φ2. (S5) M + ⊥ k + ⊥ k + MF 1 −βHb The free energy per unit cell is fb = − βN log Tr(e ). At T = 0, according to the free energy, the MF phase diagram is shown in Fig. S1. Near the phase boundary, assuming the free energy can be expanded as function of P3 00 2n small φ, we have fb = n=0 λ2nφ where 2 2 2 2 4 4 2 6 2 4 6  H J+  (H − 4H⊥)H J+ (12H H⊥ − H − 8H H⊥)J+ λ00 = −2∆, λ00 = 1 − k J , λ00 = k k , λ00 = k k k , (S6) 0 2 ∆3 + 4 4∆7 6 8∆11 q 2 2 where ∆ ≡ H⊥ + Hk .

B. Mean field theory of chiral tricritical point

Using Fourier transformation, the fermion Hamiltonian in momentum space reads

 z ik2 i(k2−k1)    X † † hσA − µ −t(1 + e + e ) cA,k X † h zi Hf = (cA,k, cB,k) −ik −i(k −k ) z = c˜k hm − µ + ε(k)τ c˜k, −t(1 + e 2 + e 2 1 ) hσ − µ cB,k k B k (S7) 7

z T where in the last step we approximate σ as its VEV andc ˜ = (˜cA, c˜B) is the annihilation operator in eigenwave basis and p 2 2 2 ε(k) = h φ + t [3 + 2(cos k1 + cos k2 + cos(k1 − k2))], (S8) √ 1 3 here the lattice constant is set to be unity, ande ˆ1 = (1, 0),e ˆ2 = ( 2 , 2 ) and ki =e ˆi · k for i = 1, 2. To perform P † (2n+1)π the functional integral, we use the Lagrangian, Lf = k c˜k[−iωn + E(k, τ)]˜ck with ωn ≡ β the Matsubara frequency. The free energy per site is Z 1 −β(HMF+H ) 1 X −βH [σ] † −S[˜c,c˜†] f = − log Tr(e b f ) = − log (e MF Dc˜ Dce˜ ) (S9) βM βN σ

1 X h −βHMF[σ] X 1 i 1 X h −βHMF[σ] X −βE(k,τ) i = − log e b exp log (−iω + E(k, τ)) = − log e b exp log(1 + e ) , βM β n βM σ n,k,τ σ k,τ (S10) where E(k, τ) = hm − µ + ε(k)τ z. At zero temperature, we have q q 1 X f ≈ − (J φ + H )2 + H2 − (J φ − H )2 + H2 + J φ2 + E(k, τ)θ(−E), (S11) + ⊥ k + ⊥ k + M k,τ 1 ∂ where θ is step function. In the disorder phase, we consider the neutral point ρ = βM ∂µ log Z = 0. Thus the chemical potential is fixed to be µ = hm at zero temperature, and we have 1 X Z d2k E(k, τ)θ(−E) = − ph2φ2 + t2[3 + 2(cos k + cos k + cos(k − k ))]. (S12) M (2π)2 1 2 1 2 k,τ From the free energy, we get the phase diagram shown in the main text. Assuming small φ near the phase boundary, we can expand the free energy as a function φ. A subtlety arises here due to the coupling between Ising field and 4π 0 4π gapless Dirac fermions. Near K = ( 3 , 0) and K = (− 3 , 0) points, we make the approximation, 3 + 2(cos k1 + 3 2 2 2 2 2 cos k2 + cos(k1 − k2)) ≈ 4 (kx + ky) for kx + ky ≤ Λ where Λ is a cutoff, and Z d2k Z d2k r 3 − ph2φ2 + t2[3 + 2(cos k + cos k + cos(k − k ))] ≈ −2 h2φ2 + t2k2 (S13) (2π)2 1 2 1 2 (2π)2 4 2(2π) Z r 3 tΛ3 h2Λ 4h3 h4 2h6 = − dk h2φ2 + t2k2 ≈ − √ − √ φ2 + |φ|3 − √ φ4 + √ φ6. (S14) 2 2 3 5 3 (2π) Λ 4 2 3π 3πt 9πt 3 3πt Λ 27 3πt Λ P3 0 2n 3 As a result the free energy near the phase transition is given by f = n=0 λ2nφ + λ3|φ| , where H2J ! 2 (H2 − 4H2 )H2J 4 4 0 0 k + h Λ 0 k ⊥ k + h λ = −2∆, λ = 1 − J+ − √ , λ = − √ , (S15) 0 2 ∆3 3πt 4 4∆7 3 3πt3Λ (12H4H2 − H6 − 8H2H4 )J 6 6 3 0 k ⊥ k k ⊥ + 2h 4h λ = + √ , λ3 = . (S16) 6 8∆11 27 3πt5Λ3 9πt2

H⊥ 0.8 ◼ Uniform Phase 0.6 ◼ 1st order 0.4 ◼ TP

2 nd ◼ 0.2 Stagered Phase order

◼ ◼ ◼ ◼ ◼ ◼ 0.2 0.4 0.6 0.8 1.00 H

FIG. S1. Mean field phase diagram of Hb at zero temperature as a function of Zeeman fields H⊥ and Hk with J+ = 1,J− = 0. The dashed and solid lines refer to first- and second-order phase transition between staggered phase and uniform phase. The red point denotes the tricritial point (TP). 8

Despite renormalization of the original coupling constants, coupling to gapless fermions gives rise to a nonanalyitical terms |φ|3.

C. The effective Lagrangian near chiral tricritical point

As mentioned in SM B, two Dirac cones locate at K and K0. We now expand the free part of fermion Hamiltonian near the Dirac cones in the disordered phase, the low-energy Lagrangian of the Dirac fermion is given by

† x z y Lf = ψ [−iω + vF (kxτ µ − kyτ )]ψ, (S17)

T where ψ = (ψ , ψ , ψ 0 , ψ 0 ) is fermion operator. ω is frequency while momentum ~k are measured from K K,A K,B K ,A K ,B √ 0 3 or K respectively. τ (µ) refers to the Pauli matrix in sublattice (valley) space and vF = 2 ta, where a is the lattice constant. The coupling between the Ising field φ and fermions are described by

† z Lbf = hφψ τ ψ, (S18) where h captures the coupling strength. The Lagrangian involving the Ising field is given by 1 L = [(∂ φ)2 + v2 (∇~ φ)2] + U(ρ), (S19) f 2 τ B 2 where vB is boson velocity and U(ρ) is an analytical function of ρ ≡ φ . Note that odd power of potential in the Ising P i field vanishes due to Ising symmetry. The expansion of potential near criticality is denoted U(ρ) = i λ2iρ . It is well known that Lorentz symmetry will emerge at criticality, in the other word, the boson velocity and fermion velocity will flow to the same value in the IR. Thus, we set vF = vB = 1 for simplicity. To further simplify the ¯ † 0 z 1 y z 2 x notation, we define ψ = ψ γ0 and γ = τ , γ = τ µ , γ = τ , and ∂µ = (∂τ , ∇~ ), 1 L = ψγ¯ µ∂ ψ + (∂ φ)2 + hφψψ¯ + U(ρ), (S20) µ 2 µ which gives the Lagrangian in the main text.

TABLE S1. The largest three eigenvalues of the stability matrix for different universality classes with Nf = 2 in 2+1 dimensions. The eigenvalues corresponding to the conventional tricritical point are exact because its upper critical dimension is three.

Universality class θ1 θ2 θ3 Chiral tricritical Ising 2.2972 0.9816 −0.8739 Tricritical Ising 2 1 1 Chiral Ising 0.9819 −0.8723 −1.0897 Chiral tricritical Heisenberg 2.4818 0.7829 −0.9364 Chiral Heisenberg 0.7716 −0.9238 −1.5235

D. Eigenvalues of the stability matrices at different fixed points

To explore critical properties near different fixed points, we distill the stability matrix at each fixed point. We list 1 the largest three eigenvalues of S in Table S1. The ν is obtained by using ν ≡ θ , where θ is the eigenvalue of the stability matrix in the λ2 direction.

E. Method to determine the critical exponents

Here we show the method to determine anomalous dimensions used in the main text. We use the extrapolation method to obtain the critical exponents at N → ∞, where N denotes the truncation order of the boson potential. At first, we fit out the dependence of the exponents as a polynomial function of 1/N. The intercept of this function is just the exponents at N → ∞. Then we extrapolate the fixed point value of λi at N → ∞ in the similar way and substitute the result to the function of ηb/f . Comparing the results obtained by these two different approaches, we can check the reliability of our method. 9

0 . 8 1 b

؈ D a t a 0 . 7 2 0 . 8 1 F i t ג 3 6 . 0 ،

0 . 5 4 0 . 7 8 D a t a F i t 0 . 4 5 0 . 1 4 0 . 2 1 0 . 2 8 0 . 3 5 0 . 4 2 0 . 4 9 0 . 1 4 0 . 2 1 0 . 2 8 0 . 3 5 0 . 4 2 0 . 4 9 1 / N 1 / N (a) (b)

FIG. S2. Third-order polynomial fit of the curve of ηb and λ2 versus 1/N, where N denotes the truncation order of the boson potential. The resulting intercept gives ηb = 0.7359 and λ2 = 0.2763 at N → ∞.

The Nf = 2 case in the chiral Ising model will be take as an example. In Fig. S2 we show the fitting of ηb as a third-order polynomial function of 1/N. From Fig. S2(a), we obtain ηb = 0.7359. Similarly, we also obtain ηf = 0.0364. Figure S2(b) shows the fitting of λ2, which is necessary for calculating ηb in LPA’. The intercept of 2 the functions for λ2 is λ2 = 0.2763. Similarly, we obtain h = 2.2115. By substituting them into Eqs. (11)-(14) in the main text, we obtain ηb = 0.7357 and ηf = 0.0374. Both values are consistent with those obtained by the direct fitting discussed above.