Origin of Mott Insulating Behavior and Superconductivity in Twisted Bilayer Graphene

The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters.

Citation Po, Hoi Chun et al. "Origin of Mott Insulating Behavior and Superconductivity in Twisted Bilayer Graphene." Physical Review X 8, 3 (September 2018): 031089

As Published http://dx.doi.org/10.1103/PhysRevX.8.031089

Publisher American Physical Society

Version Final published version

Citable link http://hdl.handle.net/1721.1/118618

Terms of Use Creative Commons Attribution

Detailed Terms http://creativecommons.org/licenses/by/3.0 PHYSICAL REVIEW X 8, 031089 (2018)

Origin of Mott Insulating Behavior and Superconductivity in Twisted Bilayer Graphene

Hoi Chun Po,1 Liujun Zou,1,2 Ashvin Vishwanath,1 and T. Senthil2 1Department of Physics, Harvard University, Cambridge, Massachusetts 02138, USA 2Department of Physics, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139, USA

(Received 4 June 2018; revised manuscript received 13 August 2018; published 28 September 2018)

A remarkable recent experiment has observed Mott insulator and proximate superconductor phases in twisted bilayer graphene when electrons partly fill a nearly flat miniband that arises a “magic” twist angle. However, the nature of the Mott insulator, the origin of superconductivity, and an effective low-energy model remain to be determined. We propose a Mott insulator with intervalley coherence that spontaneously breaks Uð1Þ valley symmetry and describe a mechanism that selects this order over the competing magnetically ordered states favored by the Hund’s coupling. We also identify symmetry-related features of the nearly flat band that are key to understanding the strong correlation physics and constrain any tight- binding description. First, although the charge density is concentrated on the triangular-lattice sites of the moir´e pattern, the Wannier states of the tight-binding model must be centered on different sites which form a honeycomb lattice. Next, spatially localizing electrons derived from the nearly flat band necessarily breaks valley and other symmetries within any mean-field treatment, which is suggestive of a valley- ordered Mott state, and also dictates that additional symmetry breaking is present to remove symmetry- enforced band contacts. Tight-binding models describing the nearly flat miniband are derived, which highlight the importance of further neighbor hopping and interactions. We discuss consequences of this picture for superconducting states obtained on doping the valley-ordered Mott insulator. We show how important features of the experimental phenomenology may be explained and suggest a number of further experiments for the future. We also describe a model for correlated states in trilayer graphene heterostructures and contrast it with the bilayer case.

DOI: 10.1103/PhysRevX.8.031089 Subject Areas: Condensed Matter Physics

I. INTRODUCTION (at least approximately) a triangular lattice [8,9,11,14,15]. The electronic states near each valley of each graphene Superconductivity occurs proximate to a Mott insulator monolayer hybridize with the corresponding states from the in a few materials. The most famous are the cuprate high-T c other monolayer. When the twisting angle is close to certain materials [1]; others include layered organic materials [2], discrete values known as the magic angles, theoretical certain fullerene superconductors [3], and some iron-based superconductors [4]. In these systems, there is a complex calculations show that there are two nearly flat bands (per and often poorly understood relationship between the Mott valley per spin) that form in the middle of the full spectrum insulator and the superconductor, which has spurred that are separated from other bands [12]. When the carrier tremendous research activity in condensed matter physics density is such that the chemical potential lies within these in the past 30 years. Very recently, in some remarkable nearly flat bands, interaction effects are expected to be 1 4 3 4 ν −2 developments, both Mott insulating behavior and proxi- enhanced. At a filling of = or = (denoted ¼ and 2 ν 4 mate superconductivity have been observed in a very þ , respectively, with full band filling denoted ¼þ )of different platform: two layers of graphene that are rotated these nearly flat bands, Ref. [5] reports insulating behavior by a small angle relative to each other [5,6]. at very low temperatures. At such fillings, band insulation Twisted bilayer graphene (TBG) structures have been is forbidden, which leads naturally to the expectation that studied intensely in the past few years [7–18]. The charge these are correlation-driven (Mott) insulators. Doping the 1 4 — density is concentrated on a moir´e pattern which forms Mott insulator at = band filling with either electrons or holes—reveals superconductivity at low T [6]. A number of other striking observations are made in Refs. [5,6] about both the Mott insulator and the super- Published by the American Physical Society under the terms of conductor from transport studies in a magnetic field. The the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to Mott insulation is suppressed through the Zeeman coupling the author(s) and the published article’s title, journal citation, of the magnetic field at a low scale of approximately 5T— and DOI. roughly the same scale as the activation gap inferred from

2160-3308=18=8(3)=031089(31) 031089-1 Published by the American Physical Society PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018) zero field resistivity. Quantum oscillations are seen in the a fully insulating state but rather leads to a Dirac semimetal. hole-doped state with a frequency set (in the hole-doped The development of a true insulator needs a further side) by the density deviation from the Mott insulator. The symmetry breaking (or some more exotic mechanism) to degeneracy of the corresponding Landau levels is half of gap out the Dirac points. We show that, once the valley what might be expected from the spin and valley degrees of symmetry is spontaneously broken, the physics at lower freedom that characterize electrons in graphene. The super- energy scales can be straightforwardly formulated in terms conductivity occurs at temperatures that are high given the of a real-space honeycomb lattice tight-binding model with low density of charge carriers. Just like in other doped Mott a dominant cluster-charging interaction and other weaker insulators, there is a dome of superconductivity with Tc interactions. We outline a number of different possible reaching an “optimal” value at a finite doping. The super- routes in which a true insulator [19] can be obtained in such conductivity is readily suppressed in accessible magnetic an IVC-ordered system. A concrete example is a state that fields—both perpendicular and parallel to the plane. further breaks C3 rotational symmetry. We show how The observation of these classic phenomena in graphene doping this specific IVC insulator can explain the phe- gives new hope for theoretical progress in addressing old nomenology of the experiments. We present a possible questions on Mott physics and its relationship to super- pairing mechanism due to an attractive interaction mediated conductivity. They also raise a number of questions. What by Goldstone fluctuations of the IVC phase. We describe is the nature of the insulators seen at these fractional and contrast features of other distinct routes by which the fillings? How are they related to the observed super- IVC state can become a true insulator at ν ¼2.We conductivity? On the theoretical side, what is an appro- propose a number of future experiments that can distin- priate model that captures the essential physics of this guish between the different routes through which an IVC system?. can become a true insulator. In this paper, we make a start on addressing these In addition, very recently, a heterostructure of ABC- questions. The two nearly flat bands for each valley found stacked trilayer graphene and boron nitride (TLG/hBN), in the band structure have Dirac crossings at the moir´e K which also forms a triangular moir´e superlattice even at a points (but not Γ). We argue that these Dirac crossings are zero twist angle, was studied [20]. This system also features protected by symmetries of the TBG system. We show that nearly flat bands that are separated from the rest of the this protection precludes finding a real-space representation spectrum. Correlated Mott insulating states are seen at of the nearly flat bands in terms of Wannier orbitals fractional fillings of the nearly flat band. Unlike the TBG, localized at the triangular moir´e sites, in contrast to natural here the nearly flat band has no Dirac crossing, which expectations. Thus, a suitable real-space lattice model is makes the details of the two systems potentially rather necessarily different from a correlated triangular-lattice different. In particular, the nearly flat band of the TLG/hBN model with two orbitals (corresponding to the two valleys) can be modeled in real space as a triangular-lattice model per site. We instead show that a representation that is with two orbitals per site, supplemented with interactions. faithful to the Dirac crossings is possible on a honeycomb However, the hopping matrix elements are, in general, lattice with two orbitals per site, but even this representa- complex (but subjected to some symmetry restrictions). We tion has some subtleties. First, one cannot implement a describe some properties of this model and suggest that this natural representation of all the important symmetries in the system offers a good possibility to realize novel kinds of problem, which include spatial symmetries, time reversal, quantum spin-orbital liquid states. and a separate conservation of electrons of each valley [which we dub Uvð1Þ]. Second, since the charge density is II. ELECTRONIC STRUCTURE OF TWISTED concentrated at the moir´e triangular sites (which appear as BILAYER GRAPHENE: GENERAL the centers of the honeycomb plaquettes), the dominant CONSIDERATIONS interaction is not an on-site Coulomb repulsion on the A. Setup honeycomb sites. Rather, it is a “cluster-charging energy” that favors having a fixed number of electrons in each First, to establish the notation, let us consider a graphene honeycomb plaquette, which makes this model potentially monolayer, with lattice vectors A1 and A2 (see Appendix A rather different from more standard Hubbard models with for details). The honeycomb lattice sites are located at 1 1 on-site interactions. r1;2 ¼ 2 ðA1 þ A2Þ ∓ 6 ðA1 − A2Þ, where the − and þ signs Armed with this understanding of the microscopics, we are, respectively, for the sites labeled by 1 and 2. can begin to address the experimental phenomenology. We Now consider the moir´e pattern generated in the twisted propose that this system spontaneously breaks the valley bilayer problem. For concreteness, imagine we begin with a Uvð1Þ symmetry—we call the resulting order “intervalley pair of perfectly aligned graphene sheets, and consider coherent” (IVC). We discuss microscopic mechanisms that twisting the top layer by an angle θ relative to the bottom stabilize IVC symmetry breaking. We point out that, even one. Now we have two pairs of reciprocal lattice vectors, 0 when the IVC is fully polarized, it cannot, by itself, lead to the original ones Ba and Ba ¼ RθBa. Like Refs. [7,12],we

031089-2 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018) approximate the moir´e superlattice by the relative wave rotated Bloch Hamiltonian of a monolayer can then be vectors, leading to a periodic structure with reciprocal identified as HφðkÞ¼HðR−φkÞ. Linearizing about the 0 − − θ ˆ φ ˆ φ lattice vectors ba ¼ Ba Ba ¼ðRθ IÞBa. For small , rotated K point RφK obtains hð Þ, with hð Þ defined we can approximate this by b ¼ θzˆ × B . Thus, the moir´e ˆ a a by replacing σ ↦ σφ in h , where pattern also has triangular-lattice symmetry, but it is rotated by 90° and has a much larger lattice constant. Note that, in −iφσ3=2 iφσ3=2 σφ ≡ e σe : ð4Þ this dominant harmonic approximation, questions of com- mensuration or incommensuration are avoided, since no Focusing on a single valley, say, K, the continuum theory comparison is made between the other sets of harmonics of [7,12] of the twisted bilayer graphene system is described the moir´e superlattice that are not commensurate to the by the Hamiltonian Hˆ ¼ Hˆ þ Hˆ , where dominant one. cont Dirac T Let us now briefly review the low-energy electronic Hˆ ¼ hˆ ðφ Þþhˆ ðφ Þ; structure of monolayer graphene to set the notation. Dirac Zþ t þ b P Λ ∈ −1 2 1 2 † Parameterizing k ¼ j¼1;2 gjBj for gj ð = ; = , ˆ 2 ψˆ ψˆ HT ¼ d k þb;kTq1 þt;kþq1 þ H:c: the Bloch Hamiltonian for the nearest-neighbor hopping 0 model is þ symmetry-related terms; ð5Þ

− 2π 3 − − 2π 3 2 HðgÞ¼tðe ið = Þðg1 g2Þ þ e ið = Þðg1þ g2Þ t and b, respectively, denote the top and bottom layers, and 2π 3 2 − φ θ 2 φ −θ 2 þ eið = Þð g1þg2ÞÞσ þ H:c: ð1Þ we set t ¼ = and b ¼ = . Here, we introduce q1 ≡ R−θ=2K − Rθ=2K, which characterizes the momentum Note that, as a general property of our present choice transfer between the electronic degrees of freedom of the of Fourier transform, the Bloch Hamiltonian is not two layers [12]. Assuming Tq1 is real, the symmetries of the manifestly periodic in the Brillouin zone (BZ). Rather, system, which we discuss in the following subsection, − σ for any reciprocal lattice vector B,wehaveHðk þ BÞ¼ constrain Tq1 [21] to take the form Tq1 ¼ w0 w1 1, where † − η η η iB·ra w0;1 are real parameters [22]. Similarly, one can generate BHðkÞ B, where B ¼diagðe Þ. One can now pass to a continuum limit near each Dirac point K¼ð2B1 −B2Þ= the omitted symmetry-related terms by applying sym- 3 and −K. We then have the linearized Hamiltonian metries on Tq1 .

HðK þ kÞ¼−ℏvFk · σ; B. Symmetries of the continuum theory Hð−K þ kÞ¼ℏvFk · σ ; ð2Þ Let us discuss how the symmetries of the graphene ffiffiffi monolayer are modified in the twisted bilayer problem p within the dominant harmonic approximation. We see where ℏvF ¼ 3ta=2 in our simple nearest-neighbor model. Since HðqÞ is not periodic in the BZ, expanding that, in addition to the moir´e translation symmetry, we about the other equivalent Dirac points leads to a slightly have C6 rotation, time reversal, and a mirror symmetry. modified form of the Hamiltonian (due to conjugation by Furthermore, a U(1) valley symmetry that allows us to some η). In second quantized notation, we can write the assign valley charge to the electrons emerges in the low- continuum Hamiltonian: energy limit. The generator of C6 rotation and time reversal flips the valley charge, while reflection leaves it invariant. Z Microscopically, the stacking pattern of the two layers ˆ −ℏ −2 ψˆ † σ ψˆ ; hþ ¼ vF d k þ;kðk · Þ þ;k can be specified as follows [12,21,23]: First, we align the Z two layers perfectly in a site-on-site manner, corresponding ˆ ℏ −2 0ψˆ † 0 σ ψˆ to the “AA-stacking” pattern, and then rotate the top and h− ¼þ vF d k −;k0 ðk · Þ −;k0 ; ð3Þ bottom layers about a hexagon center by angles θ=2 and −θ=2 clockwise, respectively; second, we shift the top layer where the momentum integration is understood to be by a vector d parallel to the plane. For generic values of θ implemented near the Dirac point momentum by introduc- and d, one expects that almost all of the spatial symmetries ≤ Λ −2 2π 2 ing a cutoff jkj and d k ¼½ðdkxdkyÞ=ð Þ . The are broken. symmetry implementation on the continuum fields is However, within the dominant harmonic approximation, tabulated in Appendix A. For example, C3-rotation sym- it is found that, on top of possessing moir´e lattice trans- −1 ˆ ψˆ ˆ ∓ið2π=3Þσ3 ψˆ metry is represented as C3 μ;kC3 ¼ e μ;C3k, lation symmetries, the effective theory is also insensitive to where μ ¼ t, b is the layer index. d [12]. This finding implies that, given θ, the effective Next, we couple the degrees of freedom in the two layers theory will at least possess all the exact symmetries for any of graphene and arrive at a continuum theory for the twisted choice of d. A particularly convenient choice is when we bilayer graphene system [7,12]. First, we note that the take d ¼ 0. In this case, we can infer all the point-group

031089-3 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018)

(a) (b) [21]. We tabulate the explicit symmetry transformation of the electron operators in Appendix A. In the effective theory, the degrees of freedom arising from the microscopic K and K0 points are also essentially decoupled [7,12], which is because, for a small twist angle satisfying j sin θj ≪ 1,wehavejbaj ≪ jKj, and therefore the coupling between the K and K0 points is a very high- (c) (d) order process. Hence, on top of the usual electron-charge conservation, the effective theory has an additional, emer- gent Uvð1Þ conservation corresponding to the independent conservation of charge in the two valleys K and K0. Henceforth, we refer to this as “valley conservation.” (e) (f) The valley-charge operator is given by Z ˆ đ2 ψˆ † ψˆ − ψˆ † ψˆ Iz ¼ kð þ;k þ;k −;k −;kÞ: ð6Þ

FIG. 1. Effective symmetries and constraints on band struc- Note that, as time reversal T interchanges the K and K0 tures. (a) The effective symmetries of the twisted bilayer valleys, it is not a symmetry of a single valley. Similarly, C6 graphene system can be inferred by inspecting the point-group also interchanges the two valleys; thus, symmetry of a hexagon center in real space, taken to be the rotation axis of the layers. (b) Schematic band structure along a high-symmetry path in the moir´e Brillouin zone. (c)–(f) Effect of ˆ ˆ ˆ −1 ˆ T IzT ¼ −Iz; ð7Þ symmetry breaking. (c) Breaking the C2 rotation gaps out the Dirac points. (d) An external perpendicular electric field breaks the mirror My symmetry, which only modifies the energetics but ˆ ˆ ˆ −1 ˆ cannot open a band gap at charge neutrality [7]. (e) When C3 C6IzC6 ¼ −Iz; ð8Þ rotation is broken, but the combined symmetry of twofold rotation C2 and time reversal T is preserved, the Dirac points 0 remain protected, although unpinned from KM and KM. (f) When ˆ ˆ ˆ −1 ˆ MyIzMy ¼þIz: ð9Þ valley conservation Uvð1Þ symmetry is broken, one can no longer label the bands using their valley index. The gaplessness at charge neutrality is no longer symmetry required, although, depending We then see that their combined symmetry C6T is a on detailed energetics, there can still be remnant Dirac points. In contrast, at the quarter filling relevant for the observed Mott symmetry in the single-valley problem. In fact, one can physics, there are necessarily Dirac points present in this case. check that the symmetry of the single-valley problem is The other symmetry-breaking patterns listed above also do not described by the magnetic space group 183.188 (BNS open band gaps at quarter filling. notation; Ref. [25]). We tabulate the generating symmetries in Table I. symmetries of the system by focusing on the center of the hexagons [Fig. 1(a)]. Aside from the rotational symmetries TABLE I. Summary of key effective symmetries. From top to generated by the sixfold rotation C6, we see that there is an bottom, the listed symmetries are time reversal, moir´e lattice additional mirror plane My, which, in fact, combines a translation, a perpendicular 2D mirror, threefold rotation, com- mirror perpendicular to the 2D plane together with an in- bined symmetry of twofold rotation and time reversal, and valley plane mirror which flips the top and bottom layers. Strictly Uvð1Þ conservation. For any symmetry g, it either commutes η 1 η −1 speaking, this process leads to a twofold rotation in 3D ( g ¼þ ) or anticommutes ( g ¼ ) with the valley-charge ˆ space, but when restricting our attention to a 2D system it operator Iz. acts as a mirror. Symmetry η Remarks In summary, the effective theory will at least have the g following spatial symmetries: lattice translations, a sixfold T −1 Broken by valley polarization hIzi ≠ 0 1 rotation, and a mirror, which allows one to uniquely ta þ 1 identify its wallpaper group (i.e., 2D space group) as My þ Broken by perpendicular electric field 1 0 p6mm (numbered 17 in Ref. [24]). Having identified the C3 þ Pins Dirac points to KM and KM T 1 symmetries of the system, one can derive the model C2 þ Protects the local stability of the Dirac points following a phenomenological approach by systematically exp −iθIˆ þ1 incorporating all symmetry-allowed terms with some cutoff ð zÞ

031089-4 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018)

III. LOW-ENERGY THEORY: TWO-BAND contrasted with the case of Bernal-stacked bilayer gra- PROJECTION phene, whose quadratic band touching at charge neutrality can be gapped by an external electric field [27]. Alter- Formally, the continuum effective theory [7,12] we natively, if C3 symmetry is broken, the Dirac points are describe corresponds to an infinite-band problem for each unpinned from K and K0 [Fig. 1(e)]. As such, for a valley. However, near charge neutrality it is found that, for M M sufficiently strong C3 breaking, a band gap might open at some range of angles, the moir´e potential can induce charge neutrality if the Dirac points could meet their additional band gaps at a certain commensurate filling of oppositely charged partners and annihilate. (Though, as the moir´e unit cell. The “nearly flat bands” identified near we argue later, this situation is impossible without further the magic angle correspond to two bands per valley, symmetry breaking [28].) separated from all other bands by band gaps, that form Now consider the case when valley conservation is Dirac points at the K and K0 points in the moir´eBZ. M M spontaneously broken by an IVC, i.e., the valley charge These bands correspond to the relevant degrees of freedom ˆ for the correlated states observed in Refs. [5,6], and, in the Iz is no longer conserved. In this case, we should first consider the full four-band problem consisting of both following, we focus our attention to the properties of these T bands. In this section, we always focus on a single valley, valleys. At, say, Km, the combined symmetry of My say, that corresponding to the K point in the microscopic ensures that the Dirac points from the two valleys are description. degenerate. While such degeneracy is lifted in the presence of an IVC, as long as the remaining symmetries are all intact, we can only split the degeneracy according to A. Symmetry-enforced band contacts 4 ¼ 2 ⊕ 2 [Fig. 1(f)]. This remaining twofold degeneracy A salient feature of the effective theory is the presence of rules out an interpretation of the experimentally observed Dirac points at charge neutrality, whose velocity is strongly Mott insulator as a Slater insulator with a spatial-symmetry- renormalized and approaches zero near the magic angle respecting (ferro) IVC incorporated at the Hartree-Fock [7,12]. The stability of the Dirac points can be understood level. Instead, one must either introduce additional sym- from symmetries: For a single valley, KM is symmetric metry breaking, say, that of C3 or lattice translations, or under the (magnetic) point group generated by C6T .In consider an IVC which also breaks some additional spatial 2 particular, ðC6T Þ ¼ C3, and therefore we can label each symmetries. We elaborate on these points in Sec. VI.We band at KM by its C3 eigenvalue, which takes a value in also note that an essentially identical argument holds for the −ið2π=3Þ f1; ω ¼ e ; ω g. In particular, a band with C3 eigen- case of spontaneously ferromagnetic order leading to fully value ω is necessarily degenerate with another with spin-polarized bands of Iz ordering. In this way, it connects 2 eigenvalue ω ,asðC6T Þ ≠ 1 on these bands and enforces to the quarter-filled Mott insulator we are interested in. a Kramers-like degeneracy. The observed Dirac points at charge neutrality correspond precisely to this two- B. Triangular versus honeycomb lattice dimensional representation [Fig. 1(b)]. A conventional route for understanding the correlated T While we allude to the presence of C6 symmetry in states observed in Refs. [5,6] is to first build a real-space explaining the stability of the Dirac points, these band tight-binding model for the relevant bands and then contacts are actually locally stable so long as the symmetry 3 incorporate short-range interactions to arrive at, say, a ðC6T Þ ¼ C2T is kept. This stability can be reasoned by Fermi-Hubbard model. Typically, the orbital degrees of noting that C2T quantizes the Berry phase along any closed freedom involved in the tight-binding model can be loop to 0; π mod 2π, and a Dirac point corresponds identified from either applying chemistry insight or, more precisely to the case of a nontrivial π Berry phase [26]. systematically, studying the projected density of states for Let us now consider the effect of breaking the various the relevant bands, both of which are inapplicable to the symmetries (spontaneously or explicitly) in the system. current moir´e potential problem; furthermore, an under- First, as C2T is crucial in protecting the local stability of standing of the structure of the wave functions is required. the Dirac points, once it is broken, the Dirac points can be Indeed, as is noted in Refs. [9,23,29], the local density immediately gapped out [Fig. 1(c)]. However, as long as states for the flat bands are well localized to the AA regions C2T symmetry is preserved, a small breaking of any other of the moir´e pattern, which form a triangular lattice. This point-group symmetries will not lead to a gapped band theoretical prediction is also confirmed experimentally structure at charge neutrality. For instance, the mirror My [8,11,14]. Based on this observation, it is natural to 0 maps KM to KM, and its presence ensures only that the two consider a real-space model starting from effective orbitals inequivalent Dirac points are at the same energy. Therefore, centered at the AA sites, which corresponds to a tight- even when a perpendicular electric field is externally binding model defined on the triangular lattice [5,6].In applied such that My is broken, as in the setup in addition, by treating the two valleys separately, one Refs. [5,6], it can only induce an energy difference between envisions a model with two orbitals localized to each of the two Dirac points [7] [Fig. 1(d)]. This result should be the triangular sites (i.e., AA regions of the moir´e pattern).

031089-5 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018)

From symmetry representations, however, we can relating to the stability of band contacts under different immediately rule out such a model. This observation can symmetry assumptions, which, in turn, dictate whether an be readily inferred from the computed band structure insulator will result at particular fillings. In particular, we [7,12,21,23] [Fig. 4(j)]: While the two bands are non- find two symmetry obstructions to deriving a single-valley degenerate at Γ, as we have explained, they form sym- tight-binding model. The first concerns the symmetry 0 metry-protected Dirac points at KM and KM. Using such a representations of My: We find that the two bands have pattern of degeneracies, one can infer the possible sym- opposite My eigenvalues of 1, whereas, from a real-space metry representations at these high-symmetry points, and analysis [30–32], one can show that the two bands in a – from a real-space analysis [30 32] one finds that a tight-binding model must have the same My eigenvalue. triangular-lattice model always leads to the same symmetry There is a second, more serious, obstruction: Aside from representation at all three of the high-symmetry points; i.e., a quantized Berry phase of π for any closed loop encircling they are either all nondegenerate or are all Dirac points. a single Dirac point, one can further define a Z-valued This result is inconsistent with the observed pattern of winding number [28]. In contrast to the conventional case degeneracies, which rules out all triangular-lattice models. of graphene, the two inequivalent Dirac points in the single- In fact, the degeneracy pattern described is familiar— valley model are known to have the same winding number it corresponds exactly to the monolayer graphene [5,28,33]. As the net winding number of the Dirac points problem. One can further check that this is the only possible arising in any two-band tight-binding model would nec- solution using the methods described in Refs. [30,32]. essarily be zero, we can then conclude that there is an Symmetrywise, this correspondence implies that any obstruction for a symmetric real-space description; i.e., tight-binding model must correspond to orbitals forming there is an obstruction for constructing localized Wannier a honeycomb lattice. To reconcile with the predicted and functions that reproduces just the two bands of interest, observed local density of states [8,9,11,14,23], however, represents C2T naturally, and preserves valley quantum these orbitals must have nontrivial shapes: Although each numbers. A more detailed description of this obstruction, orbital is centered at a honeycomb site, which corresponds to by relating it to the anomalous surface state of a three- the AB=BA region of the moir´e pattern, the weight of the dimensional topological phase, is contained in Appendix B. orbitals is mainly localized to the AA sites. Therefore, we Essentially, this argument invokes three key ingredients: expect the shape of the orbitals to resemble a (three-lobed) (i) a two-band model, (ii) C2T symmetry, and (iii) net fidget spinner [Figs. 3(a) and 3(b)]. winding of the Dirac points in the Brillouin zone. We return to the question of tight-binding models in C. Obstructions to symmetric Wannier states Sec. VIII, but, for the discussion below, we work directly in Our symmetry analysis suggests that one should the momentum space in the manifold of states spanned by model the system by orbitals centered at the AB=BA the nearly flat bands. regions of the moir´e potential, which form a honeycomb lattice. A minimal tight-binding model of a single valley IV. INTERVALLEY COHERENT ORDER: would then be PHENOMENOLOGICAL MOTIVATION X iϕρ † We first describe some important clues from experiments ˆ i ˆ ˆ Hminimal ¼ tρi e cr crþρi þ H:c:; ð10Þ ρ [5,6] on the nature of both the Mott state and the super- i conductor. We begin with the observation that—at optimal † where cˆr is an electron-creation operator centered at a doping—an in-plane magnetic field suppresses the super- honeycomb site (for a single valley) and ρi connects two ith conductivity when the Zeeman energy scale is of the order nearest-neighbor sites. Given that this equation describes a of the zero field Tc. This observation shows that the single valley which breaks time-reversal symmetry, the superconductor has spin-singlet pairing. Upon hole hoppings are, in general, complex unless constrained by a doping the ν ¼ −2 insulator, quantum oscillations are seen space-group symmetry. with a frequency set by the density of doped holes in A pedestrian approach involves optimizing the param- perpendicular B fields exceeding approximately 1T. This ϕ “ ” eters ftρi ; ρi g to reproduce the energy eigenvalues result tells us that the normal metallic state and the obtained from the continuum description. Would this be superconductor that emerges from it should be regarded a good starting point for building up a real-space effective as doped Mott insulators: The charge carriers that are model upon which we can incorporate interaction terms? available to form the normal state Fermi surface or the Contrary to usual expectations, we argue that such an superconducting condensate are the doped holes. Thus, approach has a serious flaw in capturing certain essential the hole-doped superconductor retains information about properties. Specifically, we show that, while the energy the Mott insulator. In contrast, electron doping this Mott eigenvalues may be well approximated, the topology of the insulator leads very quickly to quantum oscillations with a resulting Bloch wave functions will necessarily be incor- high frequency that is set by the deviation of the charge rect. This result has important dynamical consequences, density from the charge neutrality point (ν ¼ 0). This result

031089-6 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018) may indicate a first-order transition between a metal and Interestingly, to treat this stage of the problem, it is Mott insulator on the electron-doped side. It will be sufficient to work within a momentum-space formulation, important to search for signs of hysteresis in transport which enables us to sidestep the difficulties elaborated in experiments as the gate voltage is tuned. As the super- Sec. III C with a real-space tight-binding formulation. conductor is better developed and characterized on the Consider the nearly flat bands in the limit of strong hole-doped side, we restrict our attention to hole doping Coulomb repulsion. Note that the dominant part of the from now on. interaction is fully SUð4Þ invariant. We expect that the A further important clue from the quantum oscillation Coulomb interaction prefers an SUð4Þ ferromagnetic data is that the Landau levels (per flux quantum) are state—similar to the SUð4Þ ferromagnetism favored by twofold degenerate, whereas one would expect fourfold Coulomb interaction in the zeroth Landau level in mono- degeneracy coming from the spin and valley degeneracy. layer graphene [35–37] or in the extensive literature on The doped holes have thus lost either their spin or valley flat-band ferromagnetism [38]. Indeed, the difficulties quantum numbers (or some combination thereof). Losing with Wannier localization of the nearly flat bands also spin makes it hard to reconcile with spin-singlet pairing that suggest that, when Coulomb interactions dominate, an can be suppressed with a Zeeman field. Thus, we propose SUð4Þ ferromagnetic ground state will be favored. The instead that the valley quantum number is lost. The simplest band dispersion, however, is not SUð4Þ symmetric, and hence option [34] then is that the valley quantum number is frozen there will be a selection of a particular direction of polarization due to symmetry breaking, i.e., hIi ≠ 0. Here, we may in the SUð4Þ space. To address this, we consider the energies define I using the electron operators cˆðkÞ for the nearly flat- of different orientations of the SUð4Þ ferromagnet within a band states: simple Hartree-Fock theory. Specifically, we compare a spin- X polarized state, a pseudospin Iz-polarized state, and the IVC † x I ¼ cˆanαðkÞ τabcˆbnαðkÞ; ð11Þ state with I polarization. a;b;n;α;k Assume a Hamiltonian where a; b correspond to the valley index, α is the spin ¼ H ¼ H0 þ V ð12Þ index, n labels the two bands for each valley, and τ denotes the standard Pauli matrices. with A nonzero expectation value for Iz breaks time-reversal X † symmetry, which leads to a sharp finite-temperature phase H0 ¼ ϵanðkÞcanαðkÞcanαðkÞ: ð13Þ transition in 2d and would likely have been detected in the anαk experiments. Given the absence of any evidence of a sharp Similarly to before, a is the valley index, α is the spin finite-temperature transition, we propose that the ordering index, and n labels the two bands for each valley. The is in the pseudospin xy plane. These phenomenological dispersion ϵ ðkÞ is independent of the spin, and, due to considerations therefore lead us to an IVC-ordered state. an time reversal, ϵ ðkÞ¼ϵ− ð−kÞ. We assume a simple We note that, for IVC ordering to be useful to explain an an form of interaction: the quantum oscillations, it has to occur at a scale that is X large compared to the scales set by the magnetic field. g † † V ¼ c αðk1 þ qÞc αðk1Þ · c 0 0 0 ðk2 − qÞc 0 0α0 ðk2Þ; Specifically, the band splitting due to IVC ordering must N an an a n α a n k1k2q be bigger than the Landau level spacing approximately 15–30 K at the biggest fields used (of the order of 5T), ð14Þ which means that the IVC order is much more robust than where N is the number of k points in the moir´e Brillouin the superconductivity and occurs at a higher temperature scale. We further need the IVC order to be present already zone. Repeated indices are summed over here. This interaction actually has an SUð8Þ symmetry, but it is in the Mott insulator, so that upon doping it can impact the 4 quantum oscillations. strongly broken down to SUð Þ by the difference in dispersion between the two bands and eventually down Thus, our view is that the first thing that happens as 2 2 the sample is cooled from a high temperature is IVC to Uð Þ × Uð Þ by the asymmetry of the dispersion under k → −k. Each Uð2Þ factor corresponds simply to indepen- ordering. This order then sets the stage for other phenom- 1 2 ena to occur at lower temperatures (the Mott insulation or dent Uð Þ charge and SUð Þ spin-conservation symmetries the superconductivity). of the two valleys. We also remark that Eq. (14) is overly simplified, for it does not incorporate form factors arising from the V. SIMPLE THEORY OF THE modulation of the Bloch wave functions over the BZ when IVC-ORDERED STATE projecting onto the nearly flat bands. With such form We now describe a mechanism that stabilizes IVC factors included, the interaction projected onto the nearly ordering and describe the properties of the resultant state. flat bands should be written as

031089-7 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018) X g a a0 is large, the four nearly flat bands split into two sets of two V ¼ Λ 0 ðk1 þ q;k1ÞΛ 0 ðk2 −q;k2Þ N nn mm k1k2q bands [Fig. 1(f)]. At quarter filling, we fill the bottommost † † band, which, however, results in not a Mott insulator but a α 0 − 0 0 0 ·can ðk1 þ qÞcan αðk1Þ · ca0mα0 ðk2 qÞca m α ðk2Þð15Þ Dirac semimetal. Thus, the IVC state by itself does not lead to a Mott insulator, and a further mechanism is needed. We with the form factors given by the Bloch wave functions of discuss this in the next section. We note that the semimetals the states in the nearly flat bands via obtained from planar valley order versus Iz order are rather a different, the latter being similar to spin-ordered states. Λ 0 k1; k2 u k1 u 0 k2 ; 16 nn ð Þ¼h anð Þj an ð Þi ð Þ Furthermore, while additionally breaking C3 symmetry alone can eventually gap the Dirac points of the IVC where juanðkÞi is the Bloch wave function of a state in the semimetal, the same is not true of spin- or Iz-ordered nearly flat bands labeled by valley index a, band index n, semimetals, which need further symmetry breaking due to and momentum k (it has no dependence on the spin their Dirac points carrying the same chirality. indices). These form factors are potentially important for Going beyond the mean field, the universal properties of the present problem due to the nontrivial band topology the IVC-ordered state are determined by its symmetry present in the valley-resolved band structure. Our prelimi- breaking. It has a Goldstone mode with linear dispersion at nary analysis suggests that the results of the Hartree-Fock the longest scales. Furthermore, it has a finite-temperature calculation are modified in the ultra-flat-band limit, i.e., Berezinskii-Kosterlitz-Thouless transition, which has weak when the interaction term overwhelms the kinetic energy, signatures in standard experimental probes [39]. whereas the key conclusions below are stable within a range of intermediate interaction strengths. In view of this VI. INTERVALLEY COHERENT MOTT analysis, in the following, we first pursue the simplified INSULATORS: GENERALITIES AND A Hartree-Fock theory and leave the task of settling down the CONCRETE EXAMPLE real ground state for future (numerical) studies; it is an interesting question answering whether the actual exper- We see that IVC ordering by itself gives us only a Dirac imental systems demand a more sophisticated treatment. semimetal and not a Mott insulator. We now consider the Details of the Hartree-Fock calculation are presented in physics below the IVC-ordering scale. First, we note that Appendix E. In summary, we find that the IVC state has a that, once Uvð1Þ symmetry is broken, there is no difficulty lower energy than both spin- and Iz-polarized states. The with writing down a real-space tight-binding model for the physical reason is that, for both the spin- and Iz-polarized two lowest bands. This model lives on the honeycomb states, the order parameter is conserved, and, hence, there lattice and must be supplemented with interactions. is a linear shift of the band when the order parameter Naively, we might imagine that the dominant interaction is nonzero. In contrast, due to the k → −k dispersion is an on-site Hubbard repulsion. However, we know that the anisotropy, the IVC order parameter does not commute orbital shapes are such that the actual charge density is with the Hamiltonian. IVC order thus does not simply shift concentrated at the original triangular sites, i.e., at the the band but modifies it more significantly. Assuming a center of the hexagons of the honeycomb lattice. Now, if near full polarization in the Hartree-Fock Hamiltonian, there is an electron at a honeycomb site r, its wave function the noncommutativity leads to an extra energy gain at is spread equally between the three hexagonal plaquettes second order in the IVC state compared to the spin- that the site r is a part of. The integral of the modulus square polarized or Iz-polarized states. of the wave function in any one such plaquette is 1=3. The Note that, in the presence of Uð2Þ × Uð2Þ symmetry, the total charge that is localized at the center R of each hexagon spin-singlet IVC state is degenerate with states that have (i.e., the triangular moir´e sites) is therefore spin-triplet IVC ordering with an order parameter IxS. The X X nα r selection between the singlet and triplet IVC order has to ð Þ QR ¼ 3 : ð17Þ occur due to other terms in the Hamiltonian that have been r∈hexagon rα ignored so far. We do not attempt to pin down the details of this selection in this paper and simply assume, as suggested Now, let us make the reasonable approximation that the by the phenomenology, that the spin-singlet IVC is primary Coulomb interaction is on site on the triangular- preferred and discuss its consequences. lattice sites R. Then, in terms of the honeycomb model, the “ ” Next, we turn to a description of the properties of the appropriate Coulomb interaction is a cluster Hubbard IVC state. We assume that the order parameter is large and term: X first study its effects on the band structure. In the absence of H U Q − 2 2: valley ordering, at the two Dirac points, there is a fourfold U ¼ ð R Þ ð18Þ R band degeneracy. As explained in Sec. III A, the valley ordering splits this fourfold degeneracy into two sets of We also specialize to ν ¼ −2 when this honeycomb lattice twofold degenerate Dirac points. When the order parameter is half filled. This interaction penalizes charge fluctuations

031089-8 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018)

each other [Fig. 1(e)] and eventually annihilate to produce a fully gapped insulator. This annihilation (and, correspond- ingly, the gap minimum just into the insulator) occurs at either the Γ or M point depending on the details of the dispersion. Note that, within this picture, the C3 breaking also occurs at a scale bigger than the approximately 5 K charge gap of the Mott insulator. Clearly, the excitations above the charge gap are ordinary electrons, and their gap can be readily closed by a Zeeman field. Upon doping this insulator, charge enters as ordinary holes and forms a small Fermi pocket. This pocket is centered at either Γ or M depending on the location of FIG. 2. A C3 symmetry-breaking order which preserves other the minimum insulating gap. In either choice, due to the symmetries. absence of C3 symmetry, there is just a single such Fermi pocket which will accommodate the full density of doped on each honeycomb plaquette. Thus, a suitable model holes. Because of the intervalley ordering, these holes are valley polarized in the Ix direction. Naturally, this explains Hamiltonian at scales much smaller than the intervalley — coherence scale takes the form the quantum oscillation experiments the frequency is set by the density of doped holes, and the Landau level degeneracy (per flux quantum) is only twofold (from H ¼ H þ H ; ð19Þ t U the spin). X X A natural pairing mechanism emerges from the coupling − † Ht ¼ trr0 crαcr0α þ H:c: ð20Þ of the holes to Goldstone fluctuations of the intervalley 0 α rr order, as we now elaborate. In the presence of an intervalley For the usual Hubbard model with a strong on-site condensate, an appropriate effective action is repulsion, the Mott insulating state has the usual two- ψ ψ θ sublattice N´eel order. However, when the cluster-charging S ¼ S0½ þS1½ ; ; ð21Þ energy is dominant, this is not obviously the case. We Z Z Z  2 2 therefore allow ourselves to consider a few different S0 ¼ dτ d xψ¯ ð∂τ þ μÞψ þ d kψ¯ khkψ k ; ð22Þ possibilities for the Mott insulator. Naturally, in all these options, the charge gap of the insulator is much lower than Z the scale of IVC ordering. In the experiments, the charge τ 2 Φ −iθψ¯ ψ S1 ¼ d d x 0ðe þ − þ c:c:Þ: ð23Þ gap is estimated to be about 5 K. The IVC ordering should then occur at a much higher scale, consistent with what we Here, ψ is a continuum electron field that represents the already concluded based on the phenomenology. In this θ section, to be concrete, we focus on a particular Mott electrons in the low-energy nearly flat bands, is the phase of the intervalley condensate, and Φ0 is its amplitude. insulator where the C3 rotation symmetry is spontaneously h ϵ k ϵ k τz 2 2 k broken while preserving other symmetries (Fig. 2). Note that k ¼ sð Þþ að Þ is a × matrix for each In passing, we note that insulators driven by cluster point [45]. We allow for slow Goldstone fluctuations of the charging have been considered in a number of different phase and obtain a convenient form of the electron-electron interaction induced by these fluctuations. To that end, we contexts before. For instance, Refs. [40,41] study models χ with extended Coulomb interactions as a route to access first define new fermion variables through Wigner-Mott insulators. Cluster-charging insulators have θτ 2 ψ ¼ ei z= χ: ð24Þ also been long studied [42–44] as a platform for various fractionalized insulating phases of matter. In contrast to This redefinition removes the θ dependence from S1,butS0 these earlier works, where a dominant cluster-charging now takes the form interaction is simply postulated and the resulting physics 0 explored, here we identify a natural mechanism for such an S0½ψ¼S0½χþS0½χ; θ; ð25Þ interaction. Z i 1 0 χ θ ∂ θχτ¯ zχ ∂ θ v S0½ ; ¼ τ þ i Ji ðxÞ: ð26Þ x;τ 2 2 A. C3-broken insulator

Breaking the C3 symmetry allows gapping out the Dirac Here, Jv is the contribution to the Uvð1Þ current from the points and leads to an insulator. As the C3 breaking order fermions. It is conveniently written down in momentum parameter increases, the two Dirac points move towards space as

031089-9 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018) Z ∂h v 2 χ¯ k χ above naturally leads to a spin-singlet superconductor. Ji ðqÞ¼ d k kþq k: ð27Þ “ ” ∂ki Furthermore, it forms out of a normal metal of ordinary holes through a BCS-like pairing mechanism. We expect Φ Now we assume that 0 is near maximum polarization and then that Zeeman fields of the order of Tc efficiently diagonalize the χ Hamiltonian obtained from S0½χþS1½χ. suppress the superconductivity except possibly at very low As discussed earlier, there are two sets of bands per spin doping (where eventually phase fluctuations kill Tc). At (corresponding to Ix ¼1) that are well separated from low doping, and when one is near a high-symmetry point of each other. The low-energy electrons are those that have the Brillouin zone (which is consistent with the fact that valley polarization Ix ≈ 1. We wish to obtain the coupling there is no additional degeneracy seen in quantum oscil- of these electrons to the θ fluctuations. For the bands with lations), the antisymmetric part of the dispersion is Ix ≈ 1, we write expected to be constrained by symmetry to be small. For example, near the Γ point, it vanishes as the cube of the χ χ ≡ þα ¼ −α dα: ð28Þ crystal momentum, which would lead to a small valley current (the derivative of the antisymmetric dispersion with z z It follows that χτ¯ χ≈0 and similarly χ¯f½∂ϵsðkÞ=∂kigτ χk ≈ respect to momentum) and, hence, a weakening of the 0. The only nonvanishing coupling, therefore, is to the coupling to valley Goldstone modes, as the doping is contribution from ϵaðkÞ. We get reduced. However, if C3 rotation symmetry is broken, the Z Z antisymmetric dispersion can include a term that is linear in ∂ϵ k v ≈ 2 ¯ að Þ ≡ 2 a ¯ momentum, leading to a nonvanishing valley current at Ji ðqÞ d kdkþq dk d kvi ðkÞdkþqdk: ð29Þ ∂ki small doping. It is also important to ask if a conventional pairing Now we assume we have integrated out the fermions mechanism due to coupling to phonons might be operative. everywhere except in the close vicinity of the Fermi We note that the bandwidths of the nearly flat bands surface, which gives a long-wavelength, low-frequency (approximately 10 meV) are much smaller than the typical effective action for the θ fluctuations of the form phonon energies in graphene. Thus, the magic-angle Z   twisted bilayer graphene system is far from a regime in ω2 θ 2 θ ω 2 which phonon effects can be treated within the usual Seff½ ¼ K 2 þ q j ðq; Þj : ð30Þ q;ω v adiabatic approximation. Furthermore, the observation of superconductivity only in the vicinity of the correlated Here, K is the phase stiffness of the θ field, and v is the insulator appears unnatural within a phonon-based theory. velocity of the linear dispersing θ fluctuations. We now Nevertheless, it is possible that phonon effects play a role in integrate out θ to get an effective interaction between the c various aspects of our physics and contribute to the electrons: effective interactions between the electrons. We leave for Z the future a proper treatment of the electron-phonon q2 − v ω 2 coupling in these systems and focus instead here on Sint ¼ ω2 2 jJi ðq; Þj ; ð31Þ q;ω 32 2 electron-electron interaction effects, which clearly must Kðv þ q Þ play an important role given the proximity of the super- which is an attractive interaction. Anticipating that the conductor to a correlation-driven Mott insulator. important regime for pairing is jωj ≪ vq for an approxi- ω 0 mate treatment, we set ¼ in the prefactor to get a VII. OTHER POSSIBLE MOTT simplified effective interaction: INSULATING STATES Z 1 The C3-broken insulator is a concrete example of how an − v ω 2 Sint ¼ jJi ðq; Þj : ð32Þ 32K q;ω intervalley condensate of the twisted bilayer system can eventually become a Mott insulator. However, given the We emphasize again that—within our approximate current experimental information, it is not clear that this is — v uniquely dictated. Therefore, we sketch a few different treatment the only contribution to Ji comes from the antisymmetric part of the “normal” state dispersion. This Mott insulating states and present some of their phenom- attractive interaction can now be treated within a BCS mean enological consequences. field and leads to a superconducting state. (1) Translation-broken insulator.—Broken moir´e Note that, in real space, since the large repulsion will be translations—for instance, Kekule ordering on the on the hexagon center and not on the honeycomb site, effective honeycomb lattice—can also gap out the there is no particular reason to disfavor on-site s-wave Dirac points. The properties of this state and its pairing. Though we do not give a detailed description of the evolution into the doped superconductor are similar pairing symmetry, the route to superconductivity sketched to the C3-broken insulator discussed above.

031089-10 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018)

(2) Antiferromagnetic insulator.—This state is the the earlier case with each valley considered separately. familiar Mott insulator of the usual honeycomb Instead, it is natural to demand each of the two orbitals to be Hubbard model. Upon doping, it is expected to a time-reversal singlet, which leads to a standard repre- evolve into a spin-singlet superconductor as seen in sentation for the symmetry group of p6mm together with numerical studies of the t − J honeycomb model time reversal. However, due to the aforementioned [46]. The pairing symmetry appears to be d þ id.It anomaly, the representation of valley Uvð1Þ is necessarily will be interesting to look for signatures of broken complicated, and we address that later. time-reversal symmetry if this scenario is realized. Forgetting about valley conservation for the time being, the Furthermore, this state is known to have quantized construction of Wannier functions becomes a rather standard spin and thermal Hall effects and associated gapless problem, and well-established protocols apply. In particular, edge states [47,48]. Other properties related to this we construct well-localized Wannier functions using the state are also discussed in the literature [49,50]. projection method [57], starting from a set of well-localized (3) Featureless Mott insulators.—Given that the honey- trial wave functions as the “seed” of the Wannier functions comb lattice features two sites in the unit cell, it (Appendix C). Specifically, we start with the continuum evades the Lieb-Shultz-Mattis theorem and allows for theory of Ref. [12],asdescribedaroundEq.(5), with the a featureless ground state (i.e., a gapped insulator parameters w0 ¼ 110 meV and w1 ¼ 120 meV. The success with neither topological order nor symmetry break- of the construction hinges crucially on having a nonsingular ing) at half filling [51–56]. Pictorially, this state is projection everywhere in the BZ, which can be monitored by viewed as a spin-singlet Cooper pair of electrons ensuring that the overlap between the seed and the actual being localized on orbitals composed of equal-weight Bloch wave functions neither vanish nor diverge anywhere in superpositions of the hexagons of the honeycomb the BZ [57]. Using this approach, we construct Wannier lattice. While model wave functions of this phase functions for a particular choice of parameters, detailed in have been constructed, the interactions that can drive Appendix C, for the four nearly flat bands near charge a system into this phase remain to be understood. neutrality (spin ignored). Our trial wave functions attain a (4) Quantum spin liquids.—The simplest possibility is a minimum and maximum overlap of 0.38 and 3.80, respec- fully gapped quantum spin liquid. In this case, there tively, indicating a satisfactory construction. Indeed, the are neutral spin-1=2 excitations (spinons) in the Wannier functions we obtain are quite well localized insulator. Upon doping, a natural possibility is that [Figs. 3(a) and 3(b)], with approximately 90% of their weight the charge goes in as bosonic holons (spinless contained within one lattice constant from the Wannier center. charge-e quasiparticles) whose condensation leads In addition, the Wannier functions we construct are exponen- to superconductivity. This process is the classic tially localized [Figs. 3(c) and 3(d)], as anticipated from the resonating valence bond (RVB) mechanism [1] for nonsingular trial wave-function projection. superconductivity in a doped Mott insulator. How- Having constructed the Wannier functions, one can ever, in this scenario, at low doping the super- readily extract an effective tight-binding model Hˆ WF by conducting Tc will not have anything to do with first projecting the full Bloch Hamiltonian into the Wannier the spin gap (measured by the Zeeman scale needed basis and then performing Fourier transform. Because of to suppress pairing). the exponential tail, however, the resulting tight-binding We do not attempt to decide between these different model has infinite-range hopping despite an exponentially options in this paper. However, we outline experiments that suppressed amplitude. To capture the salient behavior of the can distinguish between them in Sec. X. model, it is typically sufficient to keep only the bonds with strength larger than some cutoff tc. In other words, tc VIII. TIGHT-BINDING MODELS serves as a control parameter, and one recovers the exact band structure in the limit of t → 0, albeit at the cost of A. For TBG c admitting infinite-range hoppings. As we argue, there is an obstruction for writing down any The obtained band structures for different value of tc are tight-binding model for the single-valley problem. A plotted in Figs. 4(a)–4(d). We find that a fairly long-range natural way out, therefore, is to instead consider the model (see Appendix D for parameters), keeping terms that four-band problem consisting of both valleys. Our earlier connect sites up to two lattice constants apart, is needed to argument requiring the orbitals to be centered on the sites of capture all the salient features of the energetics. It should be the honeycomb lattice still applies, but now with two noted that the range of the approximate models generally orbitals associated with every site. In addition, to resolve depends on the localization of the Wannier function, and in the mirror-eigenvalue obstruction we describe, these two this work we have not optimized the Wannier functions. It orbitals should transform oppositely under the mirror is therefore possible that, by further optimization, one may symmetry. These orbitals, however, cannot have definite capture the energetics more faithfully using only shorter- valley charge, for otherwise the problem is reduced back to range terms.

031089-11 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018)

×10-3 ×10-3 (a)8 (b) 8 Although spatial and time-reversal symmetries are respected in the tight-binding model, valley conservation 6 6 is explicitly broken, which is because our Wannier func- tions cannot be chosen to represent valley conservation 4 4 naturally, similar to the case of topological insulators [58], and, therefore, any truncation of the transformed 2 2 Hamiltonian generically introduces explicit valley- conservation symmetry breaking. Furthermore, one can

(c)-1 (d) -1 ask how the operator Iz is represented. In particular, we -2 -2 want to construct the projection operators for the single- P 1 1 -3 -3 particle problem, ¼ 2 ð IzÞ, which project into the -4 -4 valleys. It would be most desirable if one can formulate Iz, P -5 -5 and hence , directly in real space. Given that Iz is also a -6 -6 free-electron Hermitian operator, this formulation amounts -7 -7 to finding a symmetric Wannier representation of Iz. -8 -8 However, we find that there are again obstructions, which 05100510mirror exactly the obstructions we face when attempting to construct Wannier functions for the single-valley two-band FIG. 3. Constructed Wannier functions. As both valleys are taken into account, there are four nearly flat bands (spin ignored), model of the nearly flat bands, i.e., a mismatch in the mirror giving rise to four Wannier functions per unit cell. Results shown eigenvalues, as well as a nonzero net charge of the Dirac are for a pair related by C6 symmetry; results for the other pair, points. Such an inheritance of the obstructions is presum- which look nearly identical, are presented in Appendix C. (a),(b) ably a manifestation of the underlying anomaly of the Amplitude of the Wannier function wðrÞ. Red stars indicate the single-valley description. AA regions in the moir´e potential. (c),(d) The amplitude decays Towards recovering valley conservation.—It is desirable exponentially when the distance from the Wannier center, δr to restore valley conservation even approximately, for our (measured in units of the moir´e lattice constant), is δr ≳ 3. truncated model, and we describe a method below. Recall that we denote the Hermitian valley-charge operator in the ˆ ˆ continuum effective theory by Iz [Eq. (6)]. Projecting Iz into the four-band subspace of the nearly flat bands and

(a)(b) (c) (d) (e) 1.5 1 1

0.5

E (meV) 0

-0.5 -1 -1 (f) (g) (h) (i) (j) 1.5

1

0.5

E (meV) 0

-0.5

-1

FIG. 4. Effective tight-binding model for the nearly flat bands. (a)–(d) Effective tight-binding models for the nearly flat bands are derived by projecting the Hamiltonian in the continuum theory into the Wannier basis. Bonds with strength

031089-12 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018) rotating into the Wannier function basis, we arrive at ˜ WF Ψ† WF Ψ Ψ† WF Ψ H ðkÞ¼ þ;kH ðkÞ þ;k þ −;kH ðkÞ −;k; ð35Þ another Hermitian operator defined on the honeycomb sites, which we can simply interpret as yet another giving an easy way to perform the described projection. Hamiltonian-like object in our problem. Similar to the As is shown in Figs. 4(f)–4(i), the projected effective ˆ earlier discussion for the Hamiltonian, the effective Iz tight-binding model reexhibits all the symmetry features of ˆWF operator Iz has infinite-range hopping with an exponen- the bands from the continuum theory [Fig. 4(j)], for any tially suppressed amplitude, and it is natural to approximate choice of truncation parameter tc. In particular, Figs. 4(a) it by truncation, keeping again only terms with a strength and 4(f) represent the simplest model which demonstrates 0 larger than some tc. Such a truncation, however, introduces the utility of our approach, with the valley projection alone ˆ converting an otherwise hopping-free Hamiltonian into one a deviation in the eigenvalues of Iz from the physical values of 1 [Fig. 4(e)]. To fix this deviation, therefore, we can exhibiting the charge-neutrality Dirac points. We further further perform spectral flattening of the corresponding remark that, although generically HWF does not respect 1 bands to 1 in momentum space. This procedure is well Uvð Þ, the effective Hamiltonian corresponding to Fig. 4(b) 1 defined so long as a band gap is sustained between the comes very close to being Uvð Þ symmetric in terms of its ˆWF energetics along the high-symmetry line. We provide an second and third bands of Iz , which is generically true as 0 explicit tabulation of the bonds in this HWF and that of IWF long as tc is chosen to be reasonably small. We denote this z ˜ˆWF in Appendix D. flattened version by Iz . Physically, this corresponds to an approximation of the actual representation of the valley- charge operator in our tight-binding model, and again our B. Nearly flat bands in trilayer graphene-boron 0 nitride moir´e superlattices approximation can be made exact in the limit of tc → 0. We can now restore valley conservation in our effective Recently, Mott insulating phases (but not superconduc- Hamiltonian. To this end, for n ∈ Z define the projection tivity, at the time of writing) were observed [20] in a operator heterostructure of ABC trilayer graphene encapsulated in X boron nitride (TLG/hBN), where a moir´e superlattice is Pˆ α α n ¼ jn; ihn; j; ð33Þ present even at zero twisting between the graphene layers. α Four minibands are observed close to neutrality, whose ˜ˆWF bandwidth and separation can be tuned by a vertical electric which projects into the sector with Iz eigenvalue n (in the ˆ ˆ ˆ field. Half filling one of the nearly flat bands results in a many-body Hilbert space) and satisfies PnPm ¼ Pnδn;m. We can now define Mott insulator. X We remark that the symmetry setup for this trilayer ˜ˆ WF ˆ ˆ WF ˆ heterostructure bears more resemblance to the Bernal- H ≡ PnH Pn; ð34Þ n∈Z stacked bilayer graphene than the TBG system we discuss above. In particular, the absence of Dirac points among the for which valley conservation is restored. In essence, minibands suggests that no C2 symmetry is present, and the through this procedure we introduce a pair of Hermitian system is potentially described by a wallpaper group for ˜ˆ WF ˜ˆWF operators ðH ; Iz Þ, corresponding, respectively, to the which the two sublattices of the honeycomb lattice are no Hamiltonian and the valley charge, that converge to the longer symmetry related (say, the wallpaper group p3m1, 0 exact operators in the limit tc, tc → 0. No. 14 [24]). If that is indeed the case, one expects the The valley projection procedure we describe applies valley-resolved band structure to admit a tight-binding ˆ WF equally well to an interacting Hamiltonian HU obtained model defined on the triangular lattice, although it remains by projecting the microscopic interaction terms to our to be checked whether or not the charge density profile Wannier basis, which again would not be automatically exhibits any nontrivial features, akin to that found for the valley conserving due to the truncation errors. For the free TBG system [Fig. 3(c)]. It will be of great interest to derive part of the Hamiltonian, however, the projection procedure a concrete real-space effective model for the TLG/hBN, but ˜ˆWF can be greatly simplified, because the Bloch states of Iz , we leave this derivation as a future work. ˆWF which equal those of Iz by definition, are known, and using which we can decompose HWFðkÞ into the valley- IX. MODEL FOR CORRELATED STATES IN conserving and valley-breaking parts. The projection then TRILAYER GRAPHENE HETEROSTRUCTURE proceeds simply by retaining only the valley-conserving In this section, we briefly consider the case of triangular “ ” part. More concretely, write the Bloch Hamiltonian of the moir´e superlattices in a trilayer graphene heterostructure. ˜ Ψ Ψ† − Ψ Ψ† valley-charge operator as IzðkÞ¼ þ;k þ;k −;k −;k, Correlated insulating states were observed very recently in Ψ 4 2 where ;k are × matrices. Note that the columns of this system [20]. Just like in the twisted bilayer, here, too, Ψ ;k are simply the -valley-charge eigenstates. We can there are nearly flat bands that are separated from the rest of then perform the projection by the spectrum. However, unlike the TBG, here there are no

031089-13 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018)

Dirac crossings in this nearly flat band, and the H ¼ H0 þ V; trilayer X X X low-energy degrees of freedom are in the trivial repre- † iϕ 0 τ H0 ¼ − t 0 c αðe rr z Þ 0 c 0 0α þ H:c:; sentation of C3. In addition, it is known that a vertical RR Ra aa R a hRR0i a¼ α electric field can induce a gap for ABC-stacked trilayer X graphene, and, depending on the direction of the electric U − 2 V ¼ 2 ðNR N0Þ ð39Þ field, the band structure can have a zero Chern for one R direction of the electric field or a nonzero Chern number for the other direction of the electric field [59].Itis with ϕ12 þ ϕ23 þ ϕ31 ¼Φ with a þ sign for up-facing thus reasonable that, in the case where the direction of triangles and a − sign for down-facing ones. Here, sites the electric field is such that the nearly flat bands 1, 2, and 3 are assumed to be arranged counterclockwise on possess no net Chern number, the nearly flat band can each triangle. NR is the total electron number at site R, and be modeled in real space by a triangular-lattice model N0 controls the filling factor. As in previous sections, this with two orbitals (corresponding to the two valleys) per Hamiltonian has a Uð2Þ × Uð2Þ symmetry corresponding site supplemented with interactions. However, some care to independent Uð2Þ rotations of each valley in addition to is still necessary. Time reversal and C2 both act by the discrete symmetries described above. The model thus flipping the valley index. Thus, the band dispersion needs to be supplemented with further weaker interactions ϵ ðkÞ within a single valley is not symmetric under that break the continuous symmetry down to Uð2Þ, though k → −k: we do not specify them here. Note that if the flux Φ ¼ 0, then the model actually has an even higher SUð4Þ sym- ϵ ≠ ϵ − ðkÞ ð kÞ; ð36Þ metry. Also, this model has an extra C2 rotational symmetry that flips the valleys, which should be viewed as an though emergent symmetry of the model defined above, which should be broken by other terms. In particular, it should be ϵ ϵ − þðkÞ¼ −ð kÞð37Þ differentiated from a microscopic C2 symmetry; if such a microscopic symmetry was present, it would combine with is satisfied. A real-space tight-binding description on the time-reversal symmetry and protect an odd number of Dirac triangular lattice therefore takes the form points in the single-valley trilayer graphene band structure, X X X which suffers from a parity anomaly [60,61]. Our effective † ϕ τ t − i rr0 z Htrilayer ¼ tRR0 cRaαðe Þaa0 cR0a0α þ H:c:; model does not suffer from this parity anomaly, because 0 α RR a¼ this microscopic C2 symmetry is absent. ð38Þ The minimal model above allows for a discussion of the Mott insulator in the strongly correlated regime of large with trr0 real and positive. The phases ϕrr0 are, in U at integer N0. In the experiments, Mott insulators at general, nonzero. The phase ϕ even on nearest-neighbor fillings N0 ¼ 1, 2 have been reported. In the large-U limit, bonds cannot be removed, as, in general, the symmetries the effective model takes the form of a “spin-orbital” permit a nonzero flux Φ for any single valley through an Hamiltonian on a triangular lattice that has four states per elementary triangle (and the opposite flux for the other site: two spins and two valleys. A systematic t=U expansion valley). is readily performed to yield this spin-orbital Hamiltonian. When the Coulomb interaction dominates, the SUð4Þ At Oðt2=UÞ, the “superexchange” is not sensitive to the flux ferromagnet with a further selection of IVC order is once Φ, and we end up with an SUð4Þ quantum antiferromagnet again a possibility. From a real-space point of view, the on the triangular lattice (This is not actually correct. As projection of the Coulomb interaction to the Wannier basis shown in Ref. [62], there is a second order process which used to formulate the tight-binding model leads to an depends on the phase of t. This leads to a term that breaks appropriate interaction Hamiltonian. If the Wannier orbitals SUð4Þ symmetry. However we anticipate that the more are not tightly confined to each triangular site, then there interesting dependence on flux comes from the third order will be a significant intersite ferromagnetic Hund’s term identified below.). For N0 ¼ 1, the SUð4Þ spins are in exchange which promotes SUð4Þ ferromagnetism with a the fundamental representation, while for N0 ¼ 2 they are in further selection of IVC ordering. the six-dimensional representation. It is interesting to consider the limit where the Wannier Antiferromagnetic models of SUð4Þ spins have been functions are sufficiently tightly localized that such a studied on a variety of lattices with different motivations ferromagnetic intersite exchange is weak and can be (for some representative recent papers, see Refs. [63–65]). It ignored. In that limit, to obtain a minimal model for this seems likely that they go into “paramagnetic” states that system we restrict the hopping to just be nearest neighbor preserve SUð4Þ symmetry. However, a new feature in the and include an on-site repulsion. The minimal model then present problem is the presence of the flux Φ in the underlying takes the form Hubbard model which breaks SUð4Þ to Uð2Þ × Uð2Þ.This

031089-14 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018) feature modifies the spin-orbital model at Oðt3=U2Þ.Inthe ordering strongly breaks this approximate short translation experiments, the ratio of Coulomb interactions to the band- symmetry. Within each moir´e site, the density of states is width of the nearly flat bands may be controlled by a uniform at the lattice scale when there is no intervalley perpendicular electric field, and it may thus be possible to ordering but oscillates once this order sets in. This difference tune the strength of these third-order terms relative to the may be detectable through scanning tunneling microscopy second-order ones. In Appendix F, we derive the spin-orbital (though, if the bilayer graphene is fully encapsulated by boron Hamiltonian to third order, showing how the flux Φ leads to nitride, it may be challenging to see the graphene layer). new terms not present in an SUð4Þ-invariant model. We, Assuming there is intervalley ordering, if the undoped however, leave for the future a detailed study of these Mott insulator develops antiferromagnetic order, it appears interesting spin-orbital models. likely that the doped superconductor is a spin-singlet At any rate, we emphasize that this trilayer system is thus dx2−y2 þ idxy superconductor, which spontaneously breaks qualitatively different from the twisted bilayer graphene time-reversal symmetry. In contrast, for a doped C3-broken where we argue that a real-space triangular-lattice descrip- state, either s-wave or d þ id spin-singlet superconductiv- tion is not possible due to Dirac crossings within the nearly ity seem possible. It is also useful to directly search for flat bands. broken C3 or moir´e translational symmetry in the experi- ments. Finally, the very different behavior in quantum X. PROPOSED FUTURE EXPERIMENTS oscillations between electron and hole doping away from the Mott insulator suggests that there may be a first-order As discussed in previous sections, the ideas presented in transition into the Mott state as it is approached from the this paper suggest a number of experiments which will be charge neutrality point, which will lead to a hysteretic extremely useful in revealing the physics. Here, we reiterate response as the gate voltage is tuned towards charge and elaborate on some of these suggestions. neutrality from the Mott insulator. A crucial clue from the existing experiments is that an in- plane field suppresses the superconductivity—at optimal XI. CONCLUSION doping—when the Zeeman energy is of the order of the zero-field Tc. This suppression indicates spin-singlet pairing In this paper, we addressed some of the theoretical and that Tc at optimal doping is associated with the loss of challenges posed by the remarkable observations of Mott pairing. It will be extremely useful to study this systemati- insulating states and proximate superconductivity in cally as a function of doping. For the doped C3-broken twisted bilayer graphene. insulator, the superconductivity may be driven by the pairing We proposed that both the Mott insulator and the super- of a small Fermi surface of electrons. Then (except perhaps conductor develop out of a state with spontaneous inter- at very small doping), Tc and the critical Zeeman scale valley coherence that breaks independent conservation of continue to track each other as the doping is decreased. In electrons at the two valleys. We described a mechanism for contrast, if the pairing (in the form of singlet valence-bond the selection of this order over other spin- or valley-polarized formation) already happens in the Mott insulator—as in the states owing to the peculiarities of the symmetry realization usual RVB theory, or with the featureless Mott insulator— in the band structure. We showed that intervalley ordering by then, with decreasing doping, Tc and the critical Zeeman itself does not lead to a Mott insulator and described possible field should part ways significantly. routes through which a Mott insulator can develop at a low A second crucial clue from the experiments is the temperature. A specific concrete example is a C3-broken 2; 4; 6; 8; … degeneracy pattern of the Landau fan emanat- insulator. We showed how doping such an insulator leads to ing from the Mott insulator. We propose that this pattern is an understanding of the quantum oscillation data and due to the freezing of the valley degree of freedom, which presented a possible pairing mechanism for the development can be distinguished from the alternate possibility that there of superconductivity. We described potentially useful is spin freezing by studying the quantum oscillations in a experiments to distinguish the various possible routes to a tilted field. Zeeman splitting, if it exists, should show up in Mott insulator from an intervalley coherent state. a characteristic way as a function of the tilt angle. Our work was rooted in a microscopic understanding of the Our proposal is the intervalley phase coherence at a scale twisted graphene bilayer. We showed that the momentum- higher than both the superconducting Tc ≈ 1.5 Kandthe space structure of the nearly flat bands places strong con- Mott insulating scale approximately 5 K. The valley sym- straints on real space descriptions. In particular, contrary to metry is, as usual, related to translational symmetry of the natural expectations, we showed that a real-space lattice microscopic graphene lattice. In the twisted bilayer, there is an model is necessarily different from a correlated triangular- approximate translation symmetry that holds at some short lattice model with two orbitals (corresponding to the two scale associated with translation by one unit cell of the valleys) per site. This difference is due to a symmetry- microscopic graphene lattices. Under this approximate trans- enforced obstruction to constructing Wannier functions cen- lation operation, electrons at the different valleys get different tered at the triangular sites that can capture the Dirac crossings phases, which is a Uvð1Þ rotation. Therefore, intervalley of the nearly flat bands. We showed that a honeycomb lattice

031089-15 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018) representation may be possible but requires a nonlocal 1 2 2 1 r1 ¼ A1 þ A2; r2 ¼ A1 þ A2: ðA2Þ implementation of valley Uvð1Þ symmetry. In our description 3 3 3 3 of the intervalley ordered state and its subsequent low- temperature evolution into the Mott or superconducting states, In momentum space, the K and K0 points are given by we sidestepped these difficulties by first treating the problem ðB1 þ B2Þ=3 or, for the equivalent ones lying on the x directly in momentum space and defining a real-space model axis, ð2B1 − B2Þ=3. Note that jKj¼4π=ð3aÞ,asiswell only at scales below the intervalley ordering (when the known. Furthermore, we take the Dirac speed vF to be 6 −1 obstruction to a honeycomb representation is gone). We also 10 ms . Besides, we choose the moir´e lattice vectors to be contrasted the bilayer system with trilayer graphene where pffiffiffi   pffiffiffi  Mott insulators have recently been observed. In the trilayer a 3 1 a 3 1 a1 ¼ ; ; a2 ¼ − ; : system, it is reasonable to construct a real-space triangular- 2sinðθ=2Þ 2 2 2sinðθ=2Þ 2 2 lattice two-orbital model, but the symmetries allow for A3 complex hopping (with some restrictions). We argue that this ð Þ system may offer a valuable platform to realize interesting In the main text, we list all the symmetries of the quantum spin-orbital liquids. continuum theory (Table I). Here, we tabulate explicitly the symmetry transformations of the electron operators, ACKNOWLEDGMENTS which follow from that of the Dirac points in the monolayer We thank Yuan Cao, Valla Fatemi, and Pablo Jarillo- problem: Herrero for extensive discussions of their data and sharing ˆ ψˆ ˆ−1 ∝ ik·ρψˆ ; their insights. We also thank Shiang Fang, Liang Fu, Gene tρ μ;ktρ e μ;k

Mele, Patrick Lee, Oskar Vafek,Feng Wang, Noah F. Q. Yuan, ˆ ˆ −1 ∓ið2π=3Þσ3 C6ψˆ μ; C6 ¼ σ1e ψˆ ∓μ; ; and Ya-Hui Zhang for interesting discussions or correspon- k C6k dence. T. S. is supported by U.S. Department of Energy Grant ˆ ψˆ ˆ −1 σ ψˆ ; My μ;kMy ¼ 1 My½μ;Myk No. DE-SC0008739 and in part by a Simons Investigator Tˆ ψˆ Tˆ −1 ψˆ award from the Simons Foundation. A. V. was supported by a μ;k ¼ ∓μ;−k; ðA4Þ Simons Investigator award and by NSF-DMR 1411343. where μ ¼ t, b. Note that My is the only symmetry which Note added.—Recently, Ref. [66] appeared, which has flips the two layers, i.e., My½t¼b and vice versa. significant differences from the present paper; Refs. [67– The symmetries listed in Eq. (A4) generate all the spatial 69], which discuss, in particular, the symmetries and symmetries of the continuum theory of the TBG [7,12,21] constructions of Wannier functions, also appeared. Note (in wallpaper group 17). In particular, we see that tρ and My that these discussions on Wannier functions disregard the preserve the valley index (K versus −K) whereas C6 and T presence of the (emergent) sixfold rotation symmetry, and, do not. However, their (pairwise) nontrivial products leave consequentially, the Dirac points observed at charge valley invariant, and it is helpful to also document their neutrality are not symmetry-protected features of these symmetry action explicitly (which are fixed by the above): models. A more detailed discussion can be found in our −1 ˆ ψˆ ˆ ∓ið2π=3Þσ3 ψˆ ; subsequent work reported in Ref. [70]. C3 μ;kC3 ¼ e μ;C3k

ˆ ˆ ˆ ˆ −1 ið2π=3Þσ3 ðC6T Þψˆ μ; ðC6T Þ ¼ σ1e ψˆ μ;− ; APPENDIX A: LATTICE AND SYMMETRIES k C6k ˆ ˆ ˆ ˆ −1 ðC2T Þψˆ μ; ðC2T Þ ¼ σ1ψˆ μ; : ðA5Þ In this Appendix, we document some details on the k k conventions and the symmetry transformations. Here, we make two remarks regarding the subtleties in Consider a monolayer of graphene. We let the primitive the symmetry representation documented here: First, the lattice vectors A and reciprocal lattice vectors B be momenta k appearing above are defined as the deviation  pffiffiffi  1 3 from the original Dirac points in the monolayer problem. ˆ − ˆ ˆ ; A1 ¼ ax; A2 ¼ a 2 x þ 2 y Generally, they correspond to different momenta in the pffiffiffi  moir´e BZ. For instance, the Dirac point labeled by ðþ;tÞ, 4π 3 1 4π ffiffiffi ˆ ˆ ffiffiffi ˆ i.e., that of the K point in the top layer, is mapped to KM, B1 ¼ p x þ y ; B2 ¼ p y; ðA1Þ − 0 3a 2 2 3a whereas ð ;tÞ is mapped to KM. Similarly, ðþ;bÞ and − 0 ð ;bÞ are, respectively, mapped to KM and KM. If desired, where a 2.46 Å is the lattice constant (some authors use ¼ pffiffiffi one can also rewrite Eqs. (A4) and (A5) using a common a to denote the C—C bond length, which is a factor of 3 set of momentum coordinates defined with respect to the smaller than the lattice constant we are using here). In this origin of the moir´eBZ. choice, we can choose the basis of the honeycomb lattice Second, the representation of the translation symmetry tˆρ sites to be has a subtlety in its definition, which is because the

031089-16 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018) microscopic translation effectively becomes an internal two Dirac points at the middle of this band structure have symmetry for the slowly varying fields appearing in the the same chirality. This ingredient allows us to write down continuum theory. As such, for a single layer one can the following effective Hamiltonian close to neutrality: deduce its representation only up to an undetermined phase, ∝ 0 − ∂ σ ∂ σ ⊗ 1 hence the appearance of in Eq. (A4). However, the Hlow ¼ ivF½ x 1 þ y 2 ; ðB4Þ relative momentum across the different slowly varying fields, say, the operators corresponding to the þ valley of wherewe now have a four-component structure to include the the top and bottom layers, is a physical quantity. two Dirac nodes. Note that there is no mass term that will gap Consequentially, there is really only one common ambi- out these nodes and also preserve the chiral condition (B3); guity across all the degrees of freedom appearing in the hence, this Hamiltonian corresponds to the surface of a three- continuum theory. dimensional topological phase in class AIII, with index ν ¼ 2, corresponding to the two Dirac nodes. Since this APPENDIX B: VALLEY SYMMETRY AND Hamiltonian is the surface state of a nontrivial 3D topological WANNIER OBSTRUCTION phase, it does not admit a Wannier representation. However, when combined with the opposite valley band structure, We argue here that the valley-symmetry-resolved band together the pair of band structures does admit LWFs but at structure does not admit a Wannier representation. Note, the price of losing valley conservation symmetry. since we ignore spin, that this is a two-band model, which Finally, let us address a conundrum that the careful will be crucial for what follows. If one were to include other reader may be puzzled by. The valley-resolved bands are bands, the arguments below would fail, although precisely stated to be the anomalous surface states of a 3D topo- what selection of bands would lead to localized Wannier logical phase; nevertheless, they appear as isolated bands, functions (LWFs) remains to be determined. In some ways, which seems to contradict the usual expectation that such it is not very surprising that a valley-resolved band structure anomalous bands cannot be separated in energy. The way does not admit a Wannier description, and a simple this conundrum is resolved in the present case is through example is a single valley of monolayer graphene, which the two-band condition, which further allows us to map the is just an isolated Dirac node. But in those cases the band problem to one with particle-hole symmetry (class AIII). structure does not terminate on raising the energy and, The later problem can have anomalous surface states that hence, does not form a band. In contrast, in our present are disconnected from the bulk bands, because they are problem for TBG, there is an isolated band, and so one may forced to stick at a zero chemical potential, and hence expect to capture the physics with LWFs. Nonetheless, we cannot be deformed into the bulk. This mapping to class argue that there is an obstruction, as can be seen as follows. AIII holds only for the two-band model; hence, if we add We begin with three ingredients: (i) a two-band model; bands or fold the Brillouin zone from translation symmetry T (ii) C2 symmetry; and a third ingredient which will be breaking, the presented arguments no longer hold. specified shortly. The two ingredients above enforce the following form on the momentum-space Hamiltonian: APPENDIX C: WANNIER FUNCTIONS H ϵ0 p ϵ1 p σ1 ϵ2 p σ2; B1 ¼ ð Þþ ð Þ þ ð Þ ð Þ We construct Wannier functions using the projection method [57]. The method proceeds by first specifying a where there is no condition on the function ϵ ðpÞ. Similar to i collection of well-localized, symmetric trial wave functions the main text, we implement C2T by σ1K, where K denotes in real space, which serves as the seed for constructing a complex conjugation. Check that C2T leaves the smooth gauge required in obtaining well-localized Wannier Hamiltonian above invariant. Now, if we are interested functions for the problem of interest. in the band wave functions, they are independent of the first Let us begin by considering the symmetry properties of a term in the Hamiltonian, and we could pass to the following real-space wave function in our present problem. For one by imposing a constraint: simplicity, we let β be a collective index for the valleys β − − 0 and layers, i.e., ¼ðþ;tÞ; ð ;tÞ; ðþ;bÞ; ð ;bÞ. Define the H ¼ ϵ1ðpÞσ1 þ ϵ2ðpÞσ2: ðB2Þ real-space electron operators Z The obvious constraint is to demand Λ † ¯ † ψˆ ∝ d2k¯e−ik·rψˆ ; β;r β;k¯ ðC1Þ 0 0 0 σ3H σ3 ¼ −H ; ðB3Þ where we do not keep track of the overall normalization of which is nothing but the chiral condition that specifies the operator. Here, k¯ is defined as the momentum measured class AIII, and one can show that this condition can be ψˆ † implemented as an on-site symmetry for a two-band with respect to the origin of the moir´e BZ. Note that β;r ψˆ † system. Now, we introduce the third ingredient: (iii) The inherits symmetry transformation from that of β;k¯ .

031089-17 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018)

þt To construct a collection of well-localized, symmetric only condition on w0 ðrÞ is that it is a two-component wave trial wave functions, one can follow the standard discussion function well localized to 0; however, we simply assume a concerning the symmetry representation associated with Gaussian form: such a real-space basis, say, as reviewed in Supplemental 2 2 Materials of Ref. [30]. We briefly sketch the main ideas wþtðrÞ¼e−jrj =2ξ ϕþt; ðC7Þ β 0 0 below. Let Wh0 ðrÞ be a two-component (column) vector localized to h0 (the two components here originate from the where ξ is the localization length and ϕ0 is a constant two- sublattice degree of freedom in the microscopic problem). component vector. Correspondingly, all the other two- Define component wave functions take a similar form, although ¯ 0 X Z they can be localized to a different point, say, C3 . Such a ˆ † ≡ 2 ψˆ † β choice is particularly convenient, as its Fourier transform Wh0 d r β;rWh0 ðrÞðC2Þ β can be readily evaluated: Z and its associated momentum-space operator β 2 −ik·r −jr−cj2=2ξ2 β −ik·c −jkj2ξ2=2 β γc ðkÞ¼ d re e ϕc ∝ e e ϕc ; Z 1 X ˆ † ik·a ˆ † 2 β Γ ≡ e W d kψˆ β; Γ k ; ðC8Þ h0;k h0þa ¼ k h0 ð Þ ðC3Þ V a which enables an efficient computation of the overlap where a is a moir´e lattice vector and we assume a periodic between our trial and the Bloch wave functions. system with V moir´e unit cells. Thus far, we have focused on only one well-localized ˆ † For our purpose, we want Wh0 to serve as our seed for the wave function in real space. In our problem, the two sites of construction of symmetric Wannier functions. To this end, the effective honeycomb lattice are related by C6; i.e., we suppose h0 ¼ða1 − a2Þ=3, which corresponds to a honey- simply construct the wave function localized to h1 ≡ C6h0 comb site in the moir´e potential, i.e., an AB=BA region. We through † demand Wˆ to be invariant under time reversal, the mirror h0 ˆ † ˆ ˆ † ˆ −1 W ≡ C6W C6 : C9 My, and the threefold rotation about h0, which we denote h1 h0 ð Þ ¯ by C3. In addition, recall that the previously predicted charge density profile [8,9,11,14,23] suggests that the Besides, to describe a four-band problem, we should have Wannier functions take the shape shown in Fig. 3(a). two orbitals on each of the honeycomb sites. These two ˆ † orbitals are not symmetry related. However, to reproduce Therefore, it is natural to consider a trial Wh0 taking the the My representation in momentum space, we have to take form ζ 1 the two orbitals to, respectively, correspond to My ¼þ † 2 2† ˆ † † ¯ˆ † ¯ˆ ¯ˆ † ¯ˆ and ζM ¼ −1. W ¼ wˆ 0 þ C3wˆ 0C3 þ C3wˆ 0C3 ; ðC4Þ y h0 In our numerical construction of the Wannier functions, 0 15 ˆ † we take the localization length to be . ja1j and the two- where w0 is localized to the unit-cell origin (an AA site). By ˆ † ¯ component vectors definition, W transforms trivially under C3, which one h0   can verify leads to the correct C3 representations for the þt;ζ ¼þ1 −0.416 þ 0.168i ϕ My ¼ ; nearly flat bands. 0 0 820 0 356 ˆ † T . þ . i It remains to ensure that w0 is symmetric underP and   † β† t;ζ −1 0.296 − 0.380i ˆ ˆ þ My ¼ My. In the spirit of Eq. (C2), we may write w0 ¼ βw0 for ϕ ¼ : ðC10Þ 0 0 776 0 407 β ¼ðþ;tÞ; ð−;tÞ; ðþ;bÞ; ð−;bÞ. From the symmetry trans- . þ . i formation in Eq. (A4), we set These choices are found simply by a search of the −t† ˆ t† ˆ −1 −b† ˆ b† ˆ −1 parameter space to optimize the minimum overlap between wˆ ¼ T wˆ þ T ; wˆ ¼ T wˆ þ T : ðC5Þ 0 0 0 0 the trial and the Bloch wave functions. We check the overlap for approximately 1180 momenta along the high- Similarly, to respect My symmetry, we set symmetry line MM − KM − ΓM − MM, as well as for an † † additional 1000 randomly sampled points in the BZ. The wˆ þb ¼ ζ Mˆ wˆ þt Mˆ −1; ðC6Þ 0 My y 0 y minimum and maximum over the overlap are, respectively, found to be 0.38 and 3.80, indicating a satisfactory where ζ ¼1. As such, we reduce our degree of My construction of the Wannier functions. In Fig. 3 in the freedom on the specification of theR trial wave function main text, we present the results for the two symmetry- ˆ † ˆ þt† 2 ψˆ † þt ζ 1 Wh0 down to our choice of w0 ¼ d r β;rw0 ðrÞ.Our related Wannier functions labeled by My ¼þ ; the

031089-18 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018)

×10-3 ×10-3 (a) (b) TABLE II. (Continued) 8 8 (a) HWF 6 6 To Fr a tðμeVÞ 4 4 4 1 (1,1) 76.2 3 3 (1,2) 66.4 2 2 4 4 (1,2) 66.4 1 1 (2,0) 81.2

(c)-1 (d) -1 2 2 (2,0) 81.2 3 3 (2,0) −67.2 -2 -2 4 4 (2,0) −67.2 -3 -3 1 4 (2,1) 76.2 -4 -4 3 2 (2,1) 76.2 -5 -5 3 3 (2,1) 66.4 -6 -6 4 4 (2,1) 66.4 -7 -7 1 1 (2,2) 81.2 −76 2 -8 -8 1 4 (2,2) . 051005102 2 (2,2) 81.2 3 2 (2,2) −76.2 FIG. 5. Localization of the other set of constructed Wannier 3 3 (2,2) −67.2 functions. 4 4 (2,2) −67.2

ζ −1 (b) IWF corresponding results for the pair with My ¼ are shown z in Fig. 5. Remarkably, the localization properties of the two To Fr a t sets are essentially identical. 1 2 (0,0) 0.451i 3 4 (0,0) −0.443i APPENDIX D: TIGHT-BINDING MODEL 1 2 (0,1) 0.451i 1 3 (0,1) −0.217i In this Appendix, we provide an explicit tabulation of the −0 217 WF 2 4 (0,1) . i bonds in the H corresponding to Figs. 4(b) and 4(g) in 3 1 (0,1) −0.217i WF the main text, as well as the associated Iz , whose 3 4 (0,1) −0.443i eigenvalues are plotted in Fig. 4(e). 4 2 (0,1) −0.217i 1 2 (1,0) −0.172i −0 217 WF 1 3 (1,0) . i TABLE II. Effective Hamiltonian H and valley-charge 0 172 WF 2 1 (1,0) . i operator Iz . We denote a lattice vector a ¼ l1a1 þ l2a2 by 2 4 (1,0) −0.217i ðl1;l2Þ. 3 1 (1,0) −0.217i 0 181 (a) HWF 3 4 (1,0) . i 4 2 (1,0) −0.217i To Fr a tðμeVÞ 4 3 (1,0) −0.181i 0 451 1 1 (0,0) 213.8 1 2 (1,1) . i 0 217 2 2 (0,0) 213.8 1 3 (1,1) . i 0 217 3 3 (0,0) −213.8 2 4 (1,1) . i 0 217 4 4 (0,0) −213.8 3 1 (1,1) . i −0 443 2 3 (0,1) −76.2 3 4 (1,1) . i 0 217 4 1 (0,1) −76.2 4 2 (1,1) . i −0 172 1 1 (0,2) 81.2 1 2 (1,2) . i 0 181 1 4 (0,2) 76.2 3 4 (1,2) . i 2 2 (0,2) 81.2 3 2 (0,2) 76.2 3 3 (0,2) −67.2 In the following, we parametrize a “bond” by 4 4 (0,2) −67.2 † 23ð1; −1Þ −76.2 tcˆ cˆ H:c:; D1 rToþa rFr þ ð Þ 33ð1; −1Þ 66.4 41ð1; −1Þ −76.2 where a is a moir´e lattice vector and To, Fr ¼ 1; …; 4 labels 1 −1 44ð ; Þ 66.4 the four Wannier functions localized to each unit cell. 2 3 (1,1) 76.2 Physically, orbital 1 corresponds to the one localized to h0 (Table continued) ζ 1 with My ¼þ ; orbital 2 is the one localized to h1

031089-19 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018) symmetry related to orbital 1; orbital 3 is localized to h0 We assume a simple form of interaction: with ζ ¼ −1; and orbital 4 is symmetry related to 3. My X The bonds in HWF with t 63μeV are tabulated in g † † c ¼ V ¼ c αðk1 þ qÞc αðk1Þc 0 0 0 ðk2 − qÞc 0 0α0 ðk2Þ; N an an a n α a n Table II(a). Note that we tabulated only half of the bonds, in k1k2q the sense that the Hermitian conjugates of the listed bonds E3 are not included. In particular, for consistency we also ð Þ halve the coefficient of the on-site chemical potential n α x a ∼ ˆ† ˆ where a ð Þ is the electron density with flavor and spin tcr cr, which is Hermitian by itself. We in addition α “ ” . Repeated indices are summed over here. This interaction subtract the trace part of the chemical potential (i.e., has an SUð8Þ symmetry. As discussed in the main text, the we remove a constant energy offset) from the model. The more complete form of the interaction that takes into 0 0 15 effective valley-charge operator defined with tc ¼ . is account the form factors arising from projecting the similarly tabulated in Table II(b). interactions onto the nearly flat bands should be X g a a0 APPENDIX E: HARTREE-FOCK THEORY V ¼ Λ 0 ðk1 þ q;k1ÞΛ 0 ðk2 − q;k2Þ N nn mm FOR SELECTION OF IVC ORDERING k1k2q † † α 0 − 0 0 0 Here, we discuss a simple mean-field treatment to ·can ðk1 þ qÞcan αðk1Þ · ca0mα0 ðk2 qÞca m α ðk2ÞðE4Þ illustrate that an IVC state is favored by the system at with the form factors given by the Bloch wave functions of ν ¼ −2, which is described by the following simplified the states in the nearly flat bands via Hamiltonian of Eq. (12): Λa nn0 ðk1; k2Þ¼huanðk1Þjuan0 ðk2Þi; ðE5Þ H ¼ H0 þ V; ðE1Þ where juanðkÞi is the Bloch wave function of a state in the where the free Hamiltonian is nearly flat bands labeled by valley index a, band index n, X and momentum k (it has no dependence on the spin † indices). However, for simplicity, we first present the result H0 ¼ ϵanðkÞcanαðkÞcanαðkÞðE2Þ anαk from analyzing the simplified interaction (E3) and com- ment on the preliminary result from analyzing (E4) at the with a the valley index, n the band index, and α the spin end of this Appendix. index. Notice that the dispersion ϵanðkÞ is independent of We factorize the interaction in a Hartree-Fock mean-field the spin and, due to time reversal, ϵanðkÞ¼ϵ−anð−kÞ. manner:

X g † † † † V ¼ ½hc αðk1 þ qÞc αðk1Þic 0 0 0 ðk2 − qÞc 0 0α0 ðk2Þþhc 0 0 0 ðk2 − qÞc 0 0α0 ðk2Þic αðk1 þ qÞc αðk1Þ MF N an an a n α a n a n α a n an an k1k2q † † † † − α 0 0 0 − − − α 0 0 0 hcan ðk1 þ qÞca n α ðk2Þica0n0α0 ðk2 qÞcanαðk1Þ hca0n0α0 ðk2 qÞcanαðk1Þican ðk1 þ qÞca n α ðk2Þ † † † † − α − 0 0 0 α 0 0 0 − hcan ðk1 þ qÞcanαðk1Þihca0n0α0 ðk2 qÞca n α ðk2Þi þ hcan ðk1 þ qÞca n α ðk2Þihca0n0α0 ðk2 qÞcanαðk1Þi: ðE6Þ

X X The first, second, and fifth terms are the Hartree 1 1 n1 ¼ n1 ðkÞ; ϕ1 ¼ ϕ1 ðkÞ: ðE8Þ contributions, while the other terms are the Fock contri- n N n n N n k k butions. The Hartree contribution is determined by the local total electron density alone and independent of ordering, so In state IZP, we assume for our purposes they can be simply dropped. We thus focus on the Fock terms. † hc αðk1Þc 0 0α0 ðk2Þi We would like to compare the energies of a spin- an a n z δ 0 δ 0 δ 0 ϕ τ δ polarized state (SP), a valley-Z-polarized state (IZP), and ¼ nn αα ½n2nðk1Þ aa þ 2ðk1Þ aa0 k1k2 : ðE9Þ a valley-XY-polarized state (IVC). In state SP, we assume The corresponding macroscopic quantities are defined as † hcanαðk1Þca0nα0 ðk2Þi 1 X 1 X z n2n ¼ n2nðkÞ; ϕ2 ¼ ϕ2ðkÞðE10Þ δ 0 δ 0 δ 0 ϕ σ δ ¼ aa nn ½n1nðk1Þ αα þ 1nðk1Þ αα0 k1k2 : ðE7Þ N N k k

The corresponding macroscopic quantities are defined as In the IVC state, we assume

031089-20 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018)

† hcanαðk1Þca0n0α0 ðk2Þi This is indeed a fully polarized state. x In this fully polarized state, and further assuming ¼ δ 0 δαα0 ½n3 ðk1Þδ 0 þ ϕ3 ðk1Þτ 0 δ : ðE11Þ nn an aa n aa k1k2 that the lower flat band is strictly lower than the higher flat band, the total energy of this state is Notice here that ϕ3n is a complex number, and changing its phase implies changing the valley order in the X E1 ¼ ϵa1ðkÞ − 2gN: ðE17Þ valley-XY plane. For simplicity, we take ϕ3n to be positive for both n. The corresponding macroscopic quantities are ak defined as (2) Valley Iz polarized.—Next, we turn to the state 1 X 1 X IZP. In this case, the interaction Hamiltonian is n3 ¼ n3 ðkÞ; ϕ3 ¼ ϕ3 ðkÞ: ðE12Þ an N an n N n replaced by k k X z † Below, we calculate the energies of these states by the −2ϕ τ α 0 VMF ¼ g ½ 2 aa0 can ðkÞca nαðkÞ mean-field approximation. k (1) Spin-polarized state.—We start with the SP state. In 2 2 2 þ 8ϕ2 − 4ðn21 þ n22Þ: ðE18Þ this case, the interaction Hamiltonian is replaced by its mean-field representative, which reads X The total mean-field Hamiltonian is −2ϕ σz † VMF ¼ g ½ 1n αα0 canαðkÞcanα0 ðkÞ H ¼ H0 þ V k MF X MF 4 ϕ2 ϕ2 − 2 − 2 ϵ − 2 ϕ † þ ð 11 þ 12 n11 n12Þ: ðE13Þ ¼ f½ þnðkÞ g 2cþnαðkÞcþnαðkÞ nαk The total mean-field Hamiltonian is then † þ½ϵ−nðkÞþ2gϕ2c−nαðkÞc−nαðkÞg 2 2 2 H ¼ H0 þ V þ gN½8ϕ2 − 4ðn21 þ n22Þ: ðE19Þ MF X MF ϵ − 2 ϕ † ¼ f½ anðkÞ g 1ncanþðkÞcanþðkÞ ank Suppose g is much larger than the bandwidth, the † þ½ϵanðkÞþ2gϕ1ncan−ðkÞcan−ðkÞg system wants to fully polarize along a ¼þ, and 2 2 2 2 both bands with n 1 and n 2 are occupied for þ 4gNðϕ þ ϕ − n − n Þ: ðE14Þ ¼ ¼ 11 12 1 2 a ¼þ. Self-consistency requires that X Consider the limit where g is much larger than the 1 † hc ðkÞc 1α0 ðkÞi ¼ δαα0 ¼ δ 0 ðn21 þ ϕ2Þ; bandwidth. In this limit, we expect the spin is fully N þ1α þ aa polarized. Without loss of generality, assume ϕ1 ≥ k n X 0 for both n. In this case, all electrons are in the state 1 † hc ðkÞc 2α0 ðkÞi ¼ δαα0 ¼ δ 0 ðn22 þ ϕ2Þ; with α ¼þ and n ¼ 1. Self-consistency requires N þ2α þ aa k that X 1 † hc ðkÞc−1α0 ðkÞi ¼ 0 ¼ δ 0 ðn21 − ϕ2Þ; X N −1α aa 1 † k hc ðkÞc 01 ðkÞi ¼ δ 0 ¼ δ 0 ðn11 þ ϕ11Þ; N a1þ a þ aa aa 1 X k † hc ðkÞc−2α0 ðkÞi ¼ 0 ¼ δ 0 ðn22 − ϕ2Þ; X N −2α aa 1 † k hc ðkÞc 01−ðkÞi ¼ 0 ¼ δ 0 ðn11 − ϕ11Þ; N a1− a aa k ðE20Þ 1 X † 0 δ ϕ hca2þðkÞca02þðkÞi ¼ ¼ aa0 ðn12 þ 12Þ; which yields N k X 1 1 † ϕ hc ðkÞc 02−ðkÞi ¼ 0 ¼ δ 0 ðn12 − ϕ12Þ; n21 ¼ n22 ¼ 2 ¼ : ðE21Þ N a2− a aa 2 k ðE15Þ This is indeed a fully valley-Z-polarized state. In this fully polarized case, the total energy of this which yields state is X 1 E2 ¼ 2 ϵ 1ðkÞ − 2gN ¼ E1: ðE22Þ n11 ¼ ϕ11 ¼ ;n12 ¼ ϕ12 ¼ 0: ðE16Þ þ 2 k

031089-21 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018) qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi (3) IVC state.—Finally, we discuss the IVC state. In this 2 2 2 E ðkÞ¼ϵ¯ ðkÞ δϵ ðkÞ þ 4g ϕ : ðE25Þ case, the interaction Hamiltonian is replaced by n n n 3n X † −2ϕ Denote the eigenmodes corresponding to E nðkÞ by VMF ¼ g f 3n½cþnαðkÞc−nαðkÞ k dnðkÞ, and they satisfy 0 1 † 2 2ϕ2 − 2 þ c−nαðkÞcþnαðkÞ þ ð 3n n3anÞg:   θ k θ k   c ðkÞ Bcos nð Þ −sin nð Þ C d ðkÞ þn @ 2 2 A þn ðE23Þ ¼ θ θ c− k nðkÞ nðkÞ d− k nð Þ sin 2 cos 2 nð Þ The total mean-field Hamiltonian is ðE26Þ H ¼ H0 þV MF X MF with † ¼ ϵ ðkÞcanαðkÞc αðkÞ δϵ an an pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffinðkÞ anαk cos θnðkÞ¼ ; X δϵ ðkÞ2 þ 4g2ϕ2 −2 ϕ † † n 3n g 3n½cþnαðkÞc−nαðkÞþc−nαðkÞcþnαðkÞ −2gϕ3 nαk θ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffin sin nðkÞ¼ 2 2 2 : ðE27Þ 2 2 δϵ 4 ϕ 2 2ϕ − nðkÞ þ g 3n þ gNð 3n n3anÞ: ðE24Þ

Again, consider the limit where g is much larger ϵ¯ ϵ ϵ 2 Denote nðkÞ¼f½ þnðkÞþ −nðkÞ= g and than the bandwidth; then the system tends to occupy δϵ ϵ − ϵ 2 nðkÞ¼f½ þnðkÞ −nðkÞ= g, and the spectrum the bands with energies E−1. Self-consistency re- of the above Hamiltonian is quires that

X X 1 † sin θnðkÞ † † ϕ3 ¼ hc αðkÞc− αðkÞi ¼ hd αðkÞd αðkÞ − d− αðkÞd− αðkÞi n N þn n 2N þn þn n n k k X 1 gϕ3n † † ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi hd− αðkÞd− αðkÞ − d αðkÞd αðkÞi; N δϵ 2 4 2ϕ2 n n þn þn k nðkÞ þ g 3n X  1 1 þ cos θnðkÞ † 1 − cos θnðkÞ † n3þn ¼ 2 hdþnαðkÞdþnαðkÞi þ 2 hd−nαðkÞd−nαðkÞi ; N k X  1 1 þ cos θnðkÞ † 1 − cos θnðkÞ † n3−n ¼ 2 hd−nαðkÞd−nαðkÞi þ 2 hdþnαðkÞdþnαðkÞi : ðE28Þ N k

In the limit where g is much larger than the where time-reversal symmetry of the noninter- ϕ → 1 2 bandwidth, 3n = , which means the system Pacting band structure is used in the last step: δϵ 0 tends to fully polarize along the valley-XY direction. k 1ðkÞ¼ . At the same time, nan → 1=2. The total energy of Therefore, if the interaction strength is much larger than this state in this case is the bandwidth, the system tends to be fully polarized. qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Within this analysis, among different fully polarized states, X the valley-XY-ordered state (IVC) has a lower energy than a 2 ϵ¯ − δϵ 2 2 E3 ¼ ½ 1ðkÞ 1ðkÞ þ g : ðE29Þ spin-polarized state and a valley-Z-ordered state. k Finally, we comment on the effect of taking into account the form factors into the interaction term, as in Eq. (E4).It This energy is lower than E2 in Eq. (E22): turns out that in this case which state has the lowest energy X qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi can potentially be changed from the IVC state if bands are − 2 ϵ¯ − δϵ 2 2 −ϵ too flat, and our preliminary calculations show that the E3 E2 ¼ ½ 1ðkÞ 1ðkÞ þ g þ1ðkÞþg k spin-polarized state and the valley-Z-ordered state have a X qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi lower energy compared to the IVC state when the inter- 2 2 ¼ 2 ½g − δϵ1ðkÞ − δϵ1ðkÞ þ g < 0; action strength is around 10 times the bandwidth of the k nearly flat bands. A more reliable way to settle down the ðE30Þ real ground state is to do a numerical calculation formulated

031089-22 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018) in momentum space, using the Hamiltonian given by where a ¼labels the flavor, α labels the spin, and ηa ¼ Eqs. (E1), (E2), and (E4). 1 if a ¼. The interaction Hamiltonian on each site is taken as APPENDIX F: SPIN-ORBITAL MODEL FOR MOTT INSULATORS IN TRILAYER GRAPHENE U − 2 V ¼ 2 ðN N0Þ ; ðF2Þ In this Appendix, we derive an effective spin-orbital P † model applicable to the trilayer graphene described in where N ¼ α caαðr Þc αðr Þ, and we are interested in ≫ a i a i Sec. IX, in the limit of U t. This effective model is both the cases with N0 ¼ 2 and N0 ¼ 1. All other terms of applicable for both the cases of ν ¼ −1 and ν ¼ −2, and it the many-body system can be generated by applying is obtained by a systematic expansion in the large-U limit to various symmetries. 3 2 the order of t =U . We are looking for an effective spin-orbital model at Besides the translation symmetry of the triangular lattice, large U to the order of t3=U2. We first present the result and the system is assumed to have Uð2Þ × Uð2Þ symmetries, discuss some simple physical consequences before pre- corresponding to charge and flavor Uð1Þ conservations, as senting the details of the derivation. The final result is well as spin conservation. In addition, there is a C6 ð2Þ ð3Þ symmetry that maps one flavor into the other and a Heff ¼ H þ H ðF3Þ time-reversal symmetry that also maps one flavor into the other (while leaving the spin unchanged, so T2 ¼ 1 for with this time reversal). 2 2 We start from a Hubbard model with nearest-neighbor 2t 1 4nt Hð2Þ ¼ ðn2 þ S · S Þðn2 þ I · I Þ − ðF4Þ hopping and on-site Hubbard interactions. Consider three U n2 A B A B U nearby sites (A, B, and C) forming an elementary triangle of this triangular lattice. The kinetic Hamiltonian is taken as and   X 3 3 η ϕ † † 3t 1 1 12nt − i a α α ð3Þ − ð3;1Þ ð3;2Þ − H0 ¼ t e ½ca ðBÞcaαðAÞþca ðCÞcaαðBÞ H ¼ 2 3 h þ 2 h 2 ; ðF5Þ aα U 4n 4n U † þ caαðAÞcaαðCÞ þ H:c:; ðF1Þ where

ð3;1Þ 8 2 3ϕ 2 2 z z z z z z h ¼ n cos ðn þ SA · SB þ SB · SC þ SC · SAÞðn þ IAIB þ IBIC þ ICIAÞ 4 2 2 iϕ þ − þ − þ − þ n ðn þ SA · SB þ SB · SC þ SC · SAÞ · ½e ðIA IB þ IB IC þ IC IBÞþH:c: 8 3ϕ z z z 2 z z z þ sin ½IAIBIC þ n ðIA þ IB þ ICÞ · ½SA · ðSB × SCÞ − 4 z iϕ þ − − −iϕ − þ → → → iSA · ðSB × SCÞ · ½IAðe IB IC e IBIC ÞþðA B C AÞ ðF6Þ and

3 2 kj jk 2 ϕ z 2 z −2 ϕ z 2 z jk kj ð ; Þ i ðIBþ ICÞ i ðIBþ ICÞ → → → h ¼ TB TC e þ e TB TC þðB C A BÞ 4 2 2 2 3ϕ 2 z z 3ϕ −iϕ þ − iϕ − þ → → → ¼ · ðn þ SA · SBÞ · ð n cos þ IAIB cos þ e IA IB þ e IAIB ÞþðB C A BÞ: ðF7Þ

1 In the above, n ¼ 2 for N0 ¼ 1, and n ¼ 1 for N0 ¼ 2. coefficients, which can potentially lead to interesting kinds We briefly comment on these results before turning to of topological order, such as a double-semion state. Last, the detailed derivation. First, we notice the effective model we note the system may develop an SUð4Þ antiferromag- indeed has the same set of symmetries as the original netic order in the limit where U ≫ t at N0 ¼ 2, although ð2Þ Hubbard model. Second, we note that H is actually there is evidence that the system is disordered at N0 ¼ 1. SUð4Þ invariant, and Hð3Þ is also SUð4Þ invariant at ϕ ¼ 0, Suppose the SUð4Þ antiferromagnetic Heisenberg model which is most easily seen by inspecting Eqs. (F19) and indeed results in an SUð4Þ-broken state; we would like to (F30). These SUð4Þ symmetric interactions can potentially understand how the SUð4Þ-breaking terms in the make the system an SUð4Þ antiferromagnet. Third, there is Hamiltonian affects the ground state. a spin chirality term for the two valleys with opposite To this end, we expand Hð3Þ for small ϕ and obtain

031089-23 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018)

ð3;1Þ 2 2 2 h ¼ 8n ðn þ SA · SB þ SB · SC þ SC · SAÞðn þ IA · IB þ IB · IC þ IC · IAÞ − 8½SA · ðSB × SCÞ · ½IA · ðIB × ICÞ 24ϕ z z z 2 z z z þ ½IAIBIC þ n ðIA þ IB þ ICÞ · ½SA · ðSB × SCÞ − 8ϕ 2 2 y x − x y → → → n ðn þ SA · SB þ SB · SC þ SC · SAÞ½IAIB IAIB þðA B C AÞ 4ϕ z þ − − þ → → → O ϕ2 þ SA · ðSB × SCÞ · ½IAðIB IC þ IBIC ÞþðA B C AÞ þ ð ÞðF8Þ and 1 − P D ¼ : ðF15Þ E0 − V ð3;2Þ 2 2 h ¼ 8ðn þSA ·SBÞðn þIA ·IBÞ → → → 8ϕ 2 y x − x y þðA B C AÞþ ðn þSA ·SBÞðIAIB IAIBÞ → → → O ϕ2 þðA B C AÞþ ð Þ: ðF9Þ 1. Mott insulator at N0 =1 We first discuss the effective Hamiltonian for N0 ¼ 1. 4 As we can see, the main effect of SUð Þ-breaking inter- We first write the results in terms of some SUð4Þ generators ϕ actions to the leading order of , besides giving rise to the and convert them into a form in terms of the S and I spin chirality terms, is to introduce terms roughly in the operators later. The final result in terms of SUð4Þ gen- following form to the Hamiltonian: erators is

y y δ ϕ x − x → → → 2 X4 2 H ¼ J ½IAIB IAIB þðA B C AÞ: ðF10Þ 2t 2t Hð2Þ ¼ TijTji − ; ðF16Þ U A B U Consider the I’s as classical spins on the XY plane and i;j¼1 x θ A θ parametrize IA ¼ cos A and Iy ¼ sin A; the above Ham- iltonian becomes where

ij † T ¼ ci cj; ðF17Þ δH ¼ Jϕ sinðθA − θBÞþðA → B → C → AÞ: ðF11Þ with i 1,2,3,4,and 1 ↑ , 2 ↓ , This term tends to introduce some canting of the I order to ¼ j i¼jþ i j i¼jþ i j3i¼j− ↑i, and j4i¼j− ↓i. In terms of these states, gain energy. ij Below, we derive effective spin-orbital Hamiltonians of the action of T is 3 2 such systems in the large-U limit to the order of t =U , ij T ¼ δjkjii; ðF18Þ for N0 ¼ 1 and N0 ¼ 2 separately. We use the standard Van Vleck perturbation theory to derive the effective and Hamiltonian, which involves two steps: writing down the matrix elements of the effective Hamiltonian within 3 X4 3 6t ji kj −2 ϕ z z z the low-energy manifold and expressing these matrix ð Þ − ik i ðIAþIBþICÞ H ¼ 2 ½TA TB TC e elements in terms of charge-neutral operators. U i;j;k¼1 In this problem, the effective Hamiltonian can be 2iϕðIz þIz þIz Þ ij jk ki written as þ e A B C TA TB TC 3 3 X4 t kj jk 2iϕðIz þ2Iz Þ −2iϕðIz þ2Iz Þ jk kj ð2Þ ð3Þ þ ½T T e B C þ e B C T T Heff ¼ H þ H þ ðF12Þ U2 B C B C j;k¼1 with 6t3 þðB → C → A → BÞ − cos 3ϕ: ðF19Þ U2 ð2Þ H ¼ PH0DH0P ðF13Þ The details of the calculations are below. and a. Order t2=U ð3Þ 2 H ¼ PH0DH0DH0P; ðF14Þ To get the effective Hamiltonian at the order t =U,itis sufficient to consider a single bond of the triangular where P is the projector into the ground-state manifold lattice, and all other terms can be generated by applying of V and symmetries.

031089-24 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018)

Now we first calculate the matrix elements of the t3 effective Hamiltonian at the order t2=U. Denote the two hi; i; jjHð3Þji; i; ji¼− ð−2 cos η ϕ þ 2 cos η ϕÞ¼0; U2 i j sites linked by this bond by A and B, and then ji; ji¼ 3 † † 3 3t η ϕ 2 η ϕ − η ϕ−2 η ϕ 0 ð Þ i i þ i j − i j i i ci ðAÞcj ðBÞj i. hi; j; ijH ji; i; ji¼ 2 ðe e Þ; ðF26Þ There are only two types of nonzero matrix elements: U 3 2 2t 2t ð3Þ − 3η ϕ 3η ϕ 3η ϕ hi; jjHð2Þji; ji¼− ; hi;j;kjH ji;j;ki¼ 2 ðcos i þ cos j þ cos k Þ U U 6 3 2t2 − t 3ϕ hj; ijHð2Þji; ji¼ ðF20Þ ¼ 2 cos ; U U 3t3 ð3Þ 2iηkϕþiηjϕ −2iηjϕ−iηkϕ ≠ hi;k;jjH ji;j;ki¼ 2 ðe þ e Þ; for i j. U These matrix elements can be recast into the effective 3 3 6t − η ϕ− η ϕ− η ϕ Hamiltonian hj;k;ijHð Þji;j;ki¼− e i i i j i k ; U2 3 4 6 2 2 X 2 2 3 t iη ϕ iη ϕ iη ϕ 2 t ij ji t k;i;j Hð Þ i;j;k − e i þ j þ k : F27 Hð Þ ¼ T T − ; ðF21Þ h j j i¼ 2 ð Þ U A B U U i;j¼1 Recasting these matrix elements into a compact form where yields

ij † 6 3 X4 T ¼ ci cj ðF22Þ 3 t ji kj −2 ϕ z z z ð Þ − ik i ðIAþIBþICÞ H ¼ 2 ½TA TB TC e U 1 such that i;j;k¼ 2iϕðIz þIz þIz Þ ij jk ki þ e A B C TA TB TC ij δ T jki¼ jkjii: ðF23Þ 3 3 X4 t kj jk 2iϕðIz þ2Iz Þ −2iϕðIz þ2Iz Þ jk kj þ ½T T e B C þ e B C T T U2 B C B C These operators satisfy the commutation relation of SUð4Þ j;k¼1 generators: 6 3 → → → − t 3ϕ þðB C A BÞ 2 cos : ðF28Þ ij kl il kj U ½T ;T ¼δjkT − δilT : ðF24Þ

2. Mott insulator at N0 =2 Now we discuss the effective Hamiltonian for N0 ¼ 2.In b. Order t3=U2 this case, there are six states on each site, which can be 3 2 Now we turn to the order t =U . A simple inspection of ≡ † † 0 − denoted as jiji ci cj j i¼ jjii. the model shows that, to calculate the effective Hamiltonian 4 3 2 Again, we first write the result in terms of some SUð Þ at the order of t =U in this problem, we need to consider generators and then convert it to a form in terms of the only a single elementary triangle; then all other terms can original S and I operators. The final result is be obtained by symmetries. 2 2 X 4 2 Now we first calculate the matrix elements of the ð2Þ t ij ji t 3 2 H ¼ T T − ; ðF29Þ effective Hamiltonian to the order t =U . To this end, A B U ij U we need to first have a systematic way to label the states in ij † the ground-state manifold of the three-site problem of a where T ¼ ci cj.And single elementary triangle. It turns out that we can classify the states into three classes: i; j; k , i; i; k , and i; i; i with 6 3 j i j i j i ð3Þ t lj jk kl 2iϕðI˜z þI˜z þI˜z Þ H ¼ − ½T T T e A B C i ≠ j ≠ k ≠ i, such that the matrix elements between states U2 A B C from two different classes always vanish. Now we need −2iϕðI˜z þI˜z þI˜z Þ jl kj lk þ e A B C T T T only to calculate the matrix elements between states within A B C 3 3 the same classes. t ij ji 2iϕðI˜z þ2I˜z Þ þ ½T T e B C þ H:c: þðB → C → A → BÞ The results are U2 B C 12 3 − t 3ϕ ð3Þ 0 2 cos ; ðF30Þ hi; i; ijH ji; i; ii¼ ; ðF25Þ U

031089-25 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018)

˜z From these matrix elements, we obtain Hð2Þ: where IA gives the flavor of the particle that is acted lj jk kl 2iϕðI˜z þI˜z þI˜z Þ by the T operators. For example, T T T e A B C ¼ 2 2 X 4 2 A B C ð2Þ t ij ji t lj jk kl iϕðη þη þη Þ 2iϕðI˜z þI˜z þI˜z Þ lj jk kl H ¼ T T − : ðF34Þ T T T e j k l ¼ e A B C T T T . A B A B C A B C U ij U The details of the calculations are below. 1 ij † As in the case of N0 ¼ , T ¼ ci cj that satisfies a. Order t2=U Eq. (F24), the commutation relations of the generators 2 4 To get the effective Hamiltonian at the order t =U,itis of SUð Þ. Now acting on two-particle states on each site, sufficient to consider a single bond of the triangular lattice, the actions of these operators are ij and all other terms can be generated by applying symmetries. T jkli¼δjkjiliþδjljkii: ðF35Þ Now we first calculate the matrix elements of the effective Hamiltonian at the order of t2=U. Denote the two sites linked by this bond by A and B, and then b. Order t3=U2 † † † † 0 3 2 jij; kli¼ci ðAÞcj ðAÞckðBÞcl ðBÞj i. Now we turn to the order t =U . A simple inspection of It is useful to distinguish three types of states: jij; kli, the model shows that, to calculate the effective Hamiltonian jij; iki, and jij; iji, where different letters denote different at the order of t3=U2 in this problem, we need to consider states. Clearly, there are no matrix elements between two only a single elementary triangle, and then all other terms states from two different types, and all we need is to can be obtained by symmetries. calculate the matrix elements between states within the Now we first calculate the matrix elements of the 3 2 same type. effective Hamiltonian to the order t =U . To this end, The independent matrix elements include we need to first have a systematic way to label the states in the ground-state manifold of the three-site problem of a hij; ijjHð2Þjij; iji¼0; ðF31Þ single elementary triangle. It turns out there are five types of states: jij; ij; iji, 2t2 hij; ikjHð2Þjij; iki¼− ; jij;jk;iji, jij;jk;iki, fjij;kl;ili;jij;kl;ijig, and jij; ik; ili, U where, for example, 2t2 hij; ikjHð2Þjik; iji¼ ; ðF32Þ † † † † † † 0 U jij;jk;kii¼ci ðAÞcj ðAÞcj ðBÞckðBÞckðCÞci ðCÞj i: ðF36Þ 2 2 4t All other states can be related to these states by certain hij; kljHð Þjij; kli¼− ; U permutations. 2 Now we calculate the matrix elements of the effective 2 2t hik; jljHð Þjij; kli¼ : ðF33Þ model between these states at the order of t3=U2. The U matrix elements between different types of the above states All other matrix elements at this order either vanish or can always vanish. So we need to calculate only the matrix be obtained from the above by permutations. elements between states within the same type:

hij; ij; ijjHð3Þjij; ij; iji¼0; ðF37Þ t3 hij; jk; ijjHð3Þjij; jk; iji¼− ½2 cosð3η ϕÞ − 2 cosð3η ϕÞ ¼ 0; U2 k i 3 3 3t η ϕ 2 η ϕ −2 η ϕ− η ϕ hij; ij; jkjHð Þjij; jk; iji¼− ðei k þ i i − e i k i i Þ; ðF38Þ U2 t3 6t3 hij; jk; ikjHð3Þjij; jk; iki¼− ½−2 cosð3η ϕÞ − 2 cosð3η ϕÞ − 2 cosð3η ϕÞ ¼ cos 3ϕ; U2 i j k U2 3 3 3t 2 η ϕ η ϕ −2 η ϕ− η ϕ hij; ik; jkjHð Þjij; jk; iki¼− ðe i i þi j þ e i j i i Þ; ðF39Þ U2 t3 hij; kl; iljHð3Þjij; kl; ili¼− ½2 cosð3η ϕÞþ2 cosð3η ϕÞ − 2 cosð3η ϕÞ − 2 cosð3η ϕÞ ¼ 0; U2 j k l i 3 3 3t 2 η ϕ η ϕ − η ϕ−2 η ϕ hij; kl; iljHð Þjij; il; kli¼ ðe i k þi i − e i k i i Þ; U2 t3 hij; kl; ijjHð3Þjij; kl; iji¼− ½2 cosð3η ϕÞþ2 cosð3η ϕÞ − 2 cosð3η ϕÞ − 2 cosð3η ϕÞ ¼ 0; U2 k l j i hij; kl; ijjHð3Þjij; ij; kli¼0; ðF40Þ

031089-26 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018)

3 3 3t η ϕ 2 η ϕ −2 η ϕ− η ϕ hij; kl; ijjHð Þjik; lj; iji¼− ðei k þ i j − e i k i j Þ; U2 t3 6t3 hij; ik; iljHð3Þjij; ik; ili¼− ½2 cosð3η ϕÞþ2 cosð3η ϕÞþ2 cosð3η ϕÞ ¼ − cos 3ϕ; U2 j k l U2 3 3 3t η ϕ 2 η ϕ − η ϕ−2 η ϕ hij; il; ikjHð Þjij; ik; ili¼ ðei k þ i l þ e i l i k Þ; U2 3 3 6t η ϕ η ϕ η ϕ hil; ij; ikjHð Þjij; ik; ili¼− ei j þi k þi l : ðF41Þ U2 Recasting these matrix elements into a compact form yields

6 3 ð3Þ t lj jk kl 2iϕðI˜z þI˜z þI˜z Þ −2iϕðI˜z þI˜z þI˜z Þ jl kj lk H ¼ − ½T T T e A B C þ e A B C T T T U2 A B C A B C 3 3 12 3 t ij ji 2iϕðI˜z þ2I˜z Þ t þ ½T T e B C þ H:c: þðB → C → A → BÞ − cos 3ϕ; ðF42Þ U2 B C U2

˜z where IA gives the flavor of the particle that is acted by the T operators.

3. Effective models in terms of spin and orbital operators As seen in the above, the effective Hamiltonian expressed in terms the operators Tij is relatively concise, and they are the same for both N0 ¼ −1 and N0 ¼ −2 up to some constants. However, to gain more intuition, it is helpful to express these effective Hamiltonians in terms of spin operators S and valley operator I, where

þ † † 12 34 − † † 21 43 S ¼ cþ↑cþ↓ þ c−↑c−↓ ¼ T þ T ;S¼ cþ↓cþ↑ þ c−↓c−↑ ¼ T þ T ; 1 1 z † † † † 11 33 22 44 S c c ↑ c c−↑ − c c ↓ − c c−↓ T T − T − T ; ¼ 2 ð þ↑ þ þ −↑ þ↓ þ −↓ Þ¼2 ð þ Þ þ † † 13 24 − † † 31 42 I ¼ cþ↑c−↑ þ cþ↓c−↓ ¼ T þ T ;I¼ c−↑cþ↑ þ c−↓cþ↓ ¼ T þ T ; 1 1 z † † † † 11 22 33 44 I c c ↑ c c ↓ − c c−↑ − c c−↓ T T − T − T ; F43 ¼ 2 ð þ↑ þ þ þ↓ þ −↑ −↓ Þ¼2 ð þ Þ ð Þ as well as filling fraction

1 † † † † 1 11 22 33 44 n c c ↑ c c ↓ c c−↑ c c−↓ T T T T : F44 ¼ 2 ð þ↑ þ þ þ↓ þ þ −↑ þ −↓ Þ¼2 ð þ þ þ Þ ð Þ

Here, n ¼ 1=2 means N0 ¼ 1 and n ¼ 1 means N0 ¼ 2. In the above, S and I form two decoupled SUð2Þ algebras, and n commutes with all others.

a. Effective Hamiltonian for N0 =1 ij To this end, we first reexpress the operators T in terms of S, I, and n.Forn ¼ 1=2 (N0 ¼ 1),

T11 ¼ðn þ SzÞðn þ IzÞ;T12 ¼ Sþðn þ IzÞ;T13 ¼ Iþðn þ SzÞ;T14 ¼ SþIþ; T22 ¼ðn − SzÞðn þ IzÞ;T23 ¼ S−Iþ;T24 ¼ Iþðn − SzÞ; T33 ¼ðn þ SzÞðn − IzÞ;T34 ¼ Sþðn − IzÞ; T44 ¼ðn − SzÞðn − IzÞ: ðF45Þ

Substituting these into Eqs. (F16) and (F19) yields

031089-27 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018)   2t2 U X Hð2Þ TijTji þ · 2 2 ¼ A B U t ij 1 þ − þ − þ − − þ þ − − þ 2 z z þ − − þ ¼ 2 ðSA SB þ IA IB þ H:c:ÞþðSA SB þ SASB ÞðIA IB þ IAIB Þþ IAIBðSA SB þ SASB Þ 2 z z þ − − þ 4 2 z z 2 z z þ SASBðIA IB þ IAIB Þþ ðn þ SASBÞðn þ IAIBÞ 2 2 ¼ 4 · ðn þ SA · SBÞ · ðn þ IA · IBÞðF46Þ and   6t3 U2 Hð3Þ þ cos 3ϕ ¼ −2hð3;1Þ þ hð3;2Þ ðF47Þ U2 3t3 with

ð3;1Þ 8 2 3ϕ 2 2 z z z z z z h ¼ n cos ðn þ SA · SB þ SB · SC þ SC · SAÞðn þ IAIB þ IBIC þ ICIAÞ 4 2 2 iϕ þ − þ − þ − þ n ðn þ SA · SB þ SB · SC þ SC · SAÞ · ½e ðIA IB þ IB IC þ IC IBÞþH:c: 8 3ϕ z z z 2 z z z þ sin ½IAIBIC þ n ðIA þ IB þ ICÞ · ½SA · ðSB × SCÞ − 4 z iϕ þ − − −iϕ − þ → → → iSA · ðSB × SCÞ · ½IAðe IB IC e IBIC ÞþðA B C AÞ ðF48Þ and

3 2 kj jk 2 ϕ z 2 z −2 ϕ z 2 z jk kj ð ; Þ i ðIBþ ICÞ i ðIBþ ICÞ → → → h ¼ TB TC e þ e TB TC þðB C A BÞ 4 2 2 2 3ϕ 2 z z 3ϕ −iϕ þ − iϕ − þ → → → ¼ · ðn þ SA · SBÞ · ð n cos þ IAIB cos þ e IA IB þ e IAIB ÞþðB C A BÞ: ðF49Þ

b. Effective Hamiltonian for N0 =2 1 2 ij kl δ il − † † For n ¼ (N0 ¼ ), it is useful to first consider the general relation T T ¼ jkT ci ckcjci. Restricting to two- particle states, we can use this general relation to write down

SþIþ ¼ 2T14;SþI− ¼ 2T32;SþIz ¼ T12 − T34; S−Iþ ¼ 2T23;S−I− ¼ 2T41;S−Iz ¼ T21 − T43; 1 z þ 13 − 24 z − 31 − 42 z z 11 − 22 − 33 44 S I ¼ T T ;SI ¼ T T ;SI ¼ 2 ðT T T þ T Þ: ðF50Þ

Using these, we can convert the relations and get

z z þ z þ z þ þ 11 ðn þ S Þðn þ I Þ 12 S ðn þ I Þ 13 I ðn þ S Þ 14 S I T ¼ 2 ;T¼ 2 ;T¼ 2 ;T¼ 2 ; z z − þ þ z 22 ðn − S Þðn þ I Þ 23 S I 24 I ðn − S Þ T ¼ 2 ;T¼ 2 ;T¼ 2 ; z z þ z 33 ðn þ S Þðn − I Þ 34 S ðn − I Þ T ¼ 2 ;T¼ 2 ; z z 44 ðn − S Þðn − I Þ T ¼ 2 ; ðF51Þ which differs from Eq. (F46) only by factors of 2.

031089-28 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018)

Substituting these into Eqs. (F29) and (F30) yields [12] R. Bistritzer and A. H. MacDonald, Moir´e Bands in Twisted   Double-Layer Graphene, Proc. Natl. Acad. Sci. U.S.A. 108, 2 X 12233 (2011). 2 4t U ij ji Hð Þ þ ¼ T T [13] J. M. B. Lopes dos Santos, N. M. R. Peres, and A. H. Castro U 2t2 A B ij Neto, Continuum Model of the Twisted Graphene Bilayer, ¼ðn2 þ S ·S Þ · ðn2 þ I · I ÞðF52Þ Phys. Rev. B 86, 155449 (2012). A B A B [14] D. Wong, Y. Wang, J. Jung, S. Pezzini, A. M. DaSilva, H.-Z. and Tsai, H. S. Jung, R. Khajeh, Y. Kim, J. Lee, S. Kahn, S. Tollabimazraehno, H. Rasool, K. Watanabe, T. Taniguchi,   A. Zettl, S. Adam, A. H. MacDonald, and M. F. Crommie, 12t3 U2 ð3Þ 3ϕ −2 ð3;1Þ ð3;2Þ Local Spectroscopy of Moir´e-Induced Electronic Structure H þ 2 cos · 3 ¼ h þ h ðF53Þ U 3t in Gate-Tunable Twisted Bilayer Graphene, Phys. Rev. B 92, 155409 (2015). 3 1 3 2 with 8hð ; Þ and 4hð ; Þ given by the same expressions as in [15] K. Kim, A. DaSilva, S. Huang, B. Fallahazad, S. Larentis, T. Eqs. (F48) and (F49), respectively, but notice that the value Taniguchi, K. Watanabe, B. J. LeRoy, A. H. MacDonald, of n is changed from 1=2 to 1. and E. Tutuc, Tunable Moir´e Bands and Strong Correla- tions in Small-Twist-Angle Bilayer Graphene, Proc. Natl. Acad. Sci. U.S.A. 114, 3364 (2017). [16] S. Huang, K. Kim, D. K. Efimkin, T. Lovorn, T. Taniguchi, K. Watanabe, A. H. MacDonald, E. Tutuc, and B. J. LeRoy, [1] P. A. Lee, N. Nagaosa, and X.-G. Wen, Doping a Mott Topologically Protected Helical States in Minimally Insulator: Physics of High-Temperature Superconductivity, Twisted Bilayer Graphene, Phys. Rev. Lett. 121, 037702 Rev. Mod. Phys. 78, 17 (2006). (2018). [2] B. J. Powell and R. H. McKenzie, Quantum Frustration in [17] P. Rickhaus, J. Wallbank, S. Slizovskiy, R. Pisoni, H. Organic Mott Insulators: From Spin Liquids to Unconven- Overweg, Y. Lee, M. Eich, M.-H. Liu, K. Watanabe, T. tional Superconductors, Rep. Prog. Phys. 74, 056501 Taniguchi, V. Fal’ko, T. Ihn, and K. Ensslin, Transport (2011). through a Network of Topological States in Twisted Bilayer [3] Y. Iwasa and T. Takenobu, Superconductivity, Mott- Hubbard States, and Molecular Orbital Order in Interca- Graphene, arXiv:1802.07317. Electrically Tunable lated Fullerides, J. Phys. Condens. Matter 15, R495 (2003). [18] A. Ramires and J. L. Lado, [4] Y. Song, Z. Yamani, C. Cao, Y. Li, C. Zhang, J. S. Chen, Gauge Fields in Tiny-Angle Twisted Bilayer Graphene, Q. Huang, H. Wu, J. Tao, Y. Zhu, W. Tian, S. Chi, H. arXiv:1803.04400. Cao, Y.-B. Huang, M. Dantz, T. Schmitt, R. Yu, A. H. [19] In the literature, a distinction is often made between Mott Nevidomskyy, E. Morosan, Q. Si, and P. Dai, A Mott and Slater insulators. The former is used to denote situations Insulator Continuously Connected to Iron Pnictide Super- in which there is a wide intermediate energy regime with conductors, Nat. Commun. 7, 13879 (2016). insulating charge transport and well-developed localized [5] Y. Cao, V. Fatemi, A. Demir, S. Fang, S. L. Tomarken, J. Y. magnetic moments. In the twisted bilayer system, it is not Luo, J. D. Sanchez-Yamagishi, K. Watanabe, T. Taniguchi, clear that such a regime exists. However, it is clear that the E. Kaxiras, R. C. Ashoori, and P. Jarillo-Herrero, Correlated insulating behavior is due to electron-electron interaction Insulator Behaviour at Half-Filling in Magic-Angle effects. Thus, to avoid potential confusion, it may be safer to Graphene Superlattices, Nature (London) 556, 80 (2018). refer to the insulator as a "correlated insulator" to emphasize [6] Y. Cao, V. Fatemi, S. Fang, K. Watanabe, T. Taniguchi, E. the importance of interaction effects without wading into the Kaxiras, and P. Jarillo-Herrero, Unconventional Supercon- Mott versus Slater debate. ductivity in Magic-Angle Graphene Superlattices, Nature [20] G. Chen, L. Jiang, S. Wu, B. Lv, H. Li, K. Watanabe, T. (London) 556, 43 (2018). Taniguchi, Z. Shi, Y. Zhang, and F. Wang, Gate-Tunable [7] J. M. B. Lopes dos Santos, N. M. R. Peres, and A. H. Castro Mott Insulator in Trilayer Graphene-Boron Nitride Moir´e Neto, Graphene Bilayer with a Twist: Electronic Structure, Superlattice, arXiv:1803.01985. Phys. Rev. Lett. 99, 256802 (2007). [21] E. J. Mele, Band Symmetries and Singularities in Twisted [8] G. Li, A. Luican, J. M. B. Lopes dos Santos, A. H. Castro Multilayer Graphene, Phys. Rev. B 84, 235439 (2011). Neto, A. Reina, J. Kong, and E. Y. Andrei, Observation of [22] Note that, due to a different choice of basis, our parameter- van Hove Singularities in Twisted Graphene Layers, Nat. ization is slightly different from that of Refs. [12,21].We Phys. 6, 109 (2010). therefore define w1 with an additional minus sign to account [9] G. T. de Laissardi`ere, D. Mayou, and L. Magaud, Locali- for the difference in basis choice. zation of Dirac Electrons in Rotated Graphene Bilayers, [23] S. Fang and E. Kaxiras, Electronic Structure Theory of Nano Lett. 10, 804 (2010). Weakly Interacting Bilayers, Phys. Rev. B 93, 235153 [10] E. J. Mele, Commensuration and Interlayer Coherence in (2016). Twisted Bilayer Graphene, Phys. Rev. B 81, 161405 (2010). [24] International Tables for Crystallography, Vol. A: Space- [11] A. Luican, G. Li, A. Reina, J. Kong, R. R. Nair, K. S. Group Symmetry, edited by T. Hahn, 5th ed. (Springer, Novoselov, A. K. Geim, and E. Y. Andrei, Single-Layer New York, 2006). Behavior and Its Breakdown in Twisted Graphene Layers, [25] D. B. Litvin, Magnetic Group Tables: 1-, 2- and 3- Phys. Rev. Lett. 106, 126802 (2011). Dimensional Magnetic Subperiodic Groups and Magnetic

031089-29 PO, ZOU, VISHWANATH, and SENTHIL PHYS. REV. X 8, 031089 (2018)

Space Groups (International Union of Crystallography, [40] C. Li and Z. Wang, Mott and Wigner-Mott Transitions in Chester, England, 2013). Doped Correlated Electron Systems: Effects of Superlattice [26] Y. Kim, B. J. Wieder, C. L. Kane, and A. M. Rappe, Dirac Potential and Intersite Correlation, Phys. Rev. B 80, Line Nodes in Inversion-Symmetric Crystals, Phys. Rev. 125130 (2009). Lett. 115, 036806 (2015). [41] A. Camjayi, K. Haule, V. Dobrosavljević, and G. Kotliar, [27] This can be understood by noting that, in the case of Bernal Coulomb Correlations and the Wigner-Mott Transition, stacking, the system has an effective C2 symmetry which Nat. Phys. 4, 932 (2008). interchanges the top and bottom layers. A perpendicular [42] O. I. Motrunich and T. Senthil, Exotic Order in Simple electric field will therefore break this symmetry, and hence Models of Bosonic Systems, Phys. Rev. Lett. 89, 277004 the band degeneracy at charge neutrality is lifted. (2002). [28] M. Goerbig and G. Montambaux, in Dirac Matter, edited by [43] O. I. Motrunich, Bosonic Model with Z3 Fractionalization, B. Duplantier, V. Rivasseau, and J.-N. Fuchs (Springer Phys. Rev. B 67, 115108 (2003). International Publishing, New York, 2017), pp. 25–53. [44] G. Chen, H.-Y. Kee, and Y. B. Kim, Cluster Mott Insulators [29] Y. Cao, J. Y. Luo, V. Fatemi, S. Fang, J. D. Sanchez- and Two Curie-Weiss Regimes on an Anisotropic Kagome Yamagishi, K. Watanabe, T. Taniguchi, E. Kaxiras, and P. Lattice, Phys. Rev. B 93, 245134 (2016). Jarillo-Herrero, Superlattice-Induced Insulating States and [45] Strictly speaking, we should allow for all four bands per spin Valley-Protected Orbits in Twisted Bilayer Graphene, Phys. and work with a four-component ψ and a corresponding Rev. Lett. 117, 116804 (2016). 4 × 4 Hamiltonian. However, for the present discussion, we [30] H. C. Po, A. Vishwanath, and H. Watanabe, Symmetry- eventually only are interested in the modes in the vicinity of Based Indicators of Band Topology in the 230 Space the Fermi surface after the flavor ordering. It is thus Groups, Nat. Commun. 8, 50 (2017). sufficient to focus on the two lower bands that are split [31] B. Bradlyn, L. Elcoro, J. Cano, M. G. Vergniory, Z. Wang, off by the flavor ordering. We therefore focus on just these C. Felser, M. I. Aroyo, and B. A. Bernevig, Topological two right from the start. Quantum Chemistry, Nature (London) 547, 298 (2017). [46] Z.-C. Gu, H.-C. Jiang, D. N. Sheng, H. Yao, L. Balents, and [32] H. Watanabe, H. C. Po, and A. Vishwanath, Structure and X.-G. Wen, Time-Reversal Symmetry Breaking Supercon- Topology of Band Structures in the 1651 Magnetic Space ducting Ground State in the Doped Mott Insulator on the Groups, Sci. Adv. 4, eaat8685 (2018). Honeycomb Lattice, Phys. Rev. B 88, 155112 (2013). [33] R. de Gail, M. O. Goerbig, F. Guinea, G. Montambaux, and [47] T. Senthil, J. B. Marston, and M. P. A. Fisher, Spin Quantum A. H. Castro Neto, Topologically Protected Zero Modes in Hall Effect in Unconventional Superconductors, Phys. Rev. Twisted Bilayer Graphene, Phys. Rev. B 84, 045436 (2011). B 60, 4245 (1999). [34] A more exotic option to explain the reduced Landau level [48] G. E. Volovik, On Edge States in Superconductors with degeneracy should also be kept in mind. Instead of losing Time Inversion Symmetry Breaking, JETP Lett. 66, 522 the valley quantum number by symmetry breaking, we lose (1997). it through fractionalization. For instance, the electron could [49] R. Nandkishore, L. S. Levitov, and A. V. Chubukov, Chiral split into a fermion that carries its charge and spin but not the Superconductivity from Repulsive Interactions in Doped valley quantum number and a charge-0, spinless boson that Graphene, Nat. Phys. 8, 158 (2012). carries its valley quantum number. If the boson is gapped [50] R. Nandkishore, R. Thomale, and A. V. Chubukov, Super- while the fermion forms a Fermi surface in the doped state, conductivity from Weak Repulsion in Hexagonal Lattice we get the reduced Landau level degeneracy. Of course, Systems, Phys. Rev. B 89, 144501 (2014). such fractionalization comes hand in hand with an emergent [51] I. Kimchi, S. A. Parameswaran, A. M. Turner, F. Wang, and gauge field. A. Vishwanath, Featureless and Nonfractionalized Mott [35] K. Nomura and A. H. MacDonald, Quantum Hall Ferro- Insulators on the Honeycomb Lattice at 1=2 Site Filling, magnetism in Graphene, Phys. Rev. Lett. 96, 256602 Proc. Natl. Acad. Sci. U.S.A. 110, 16378 (2013). (2006). [52] P. Kim, H. Lee, S. Jiang, B. Ware, C.-M. Jian, M. Zaletel, [36] J. Alicea and M. P. A. Fisher, Graphene Integer Quantum J. H. Han, and Y. Ran, Featureless Quantum Insulator on Hall Effect in the Ferromagnetic and Paramagnetic the Honeycomb Lattice, Phys. Rev. B 94, 064432 (2016). Regimes, Phys. Rev. B 74, 075422 (2006). [53] B. Ware, I. Kimchi, S. A. Parameswaran, and B. Bauer, [37] A. F. Young, C. R. Dean, L. Wang, H. Ren, P. Cadden- Topological Crystalline Bose Insulator in Two Dimensions Zimansky, K. Watanabe, T. Taniguchi, J. Hone, K. L. via Entanglement Spectrum, Phys. Rev. B 92, 195105 Shepard, and P. Kim, Spin and Valley Quantum Hall (2015). Ferromagnetism in Graphene, Nat. Phys. 8, 550 (2012). [54] M. A. Metlitski and R. Thorngren, Intrinsic and Emergent [38] H. Tasaki, From Nagaoka’s Ferromagnetism to Flat-Band Anomalies at Deconfined Critical Points, Phys. Rev. B 98, Ferromagnetism and Beyond: An Introduction to Ferro- 085140 (2018). magnetism in the Hubbard Model, Prog. Theor. Phys. 99, [55] C.-M. Jian, Z. Bi, and C. Xu, Lieb-Schultz-Mattis Theorem 489 (1998). and Its Generalizations from the Perspective of the Sym- [39] In principle, as discussed earlier, the Uvð1Þ symmetry is metry-Protected Topological Phase, Phys. Rev. B 97, only approximate and will be weakly broken by, say, small 054412 (2018). corrections to the band structures or by three-body inter- [56] J. Y. Lee, A. Turner, and A. Vishwanath, Building Symmetry action terms, which will modify the physics very close to the Enriched Topological Phases from a Bipartite Lattice phase transition. Construction and Anyon Condensation, arXiv:1802.02155.

031089-30 ORIGIN OF MOTT INSULATING BEHAVIOR AND … PHYS. REV. X 8, 031089 (2018)

[57] N. Marzari, A. A. Mostofi, J. R. Yates, I. Souza, and [65] M. G. Yamada, M. Oshikawa, and G. Jackeli, Emergent D. Vanderbilt, Maximally Localized Wannier Functions: SUð4Þ Symmetry in α-ZrCl3 and Crystalline Spin-Orbital Theory and Applications, Rev. Mod. Phys. 84, 1419 (2012). Liquids, Phys. Rev. Lett. 121, 097201 (2018). [58] A. A. Soluyanov and D. Vanderbilt, Wannier Representation [66] C. Xu and L. Balents, Topological Superconductivity in of z2 Topological Insulators, Phys. Rev. B 83, 035108 (2011). Twisted Multilayer Graphene, Phys. Rev. Lett. 121, 087001 [59] Y.-h. Zhang, D. Mao, Y. Cao, P. Jarillo-Herrero, and T. (2018). Senthil, arXiv:1805.08232. [67] N. F. Q. Yuan and L. Fu, Model for Metal-Insulator Tran- [60] M. Koshino and E. McCann, Trigonal Warping and Berry’s sition in Graphene Superlattices and Beyond, Phys. Rev. B Phase nπ in abc-Stacked Multilayer Graphene, Phys. Rev. 98, 045103 (2018). B 80, 165409 (2009). [68] J. Kang and O. Vafek, preceding paper, Symmetry, Max- [61] A. Kumar and R. Nandkishore, Flat Bands with Berry imally Localized Wannier States, and Low-Energy Model Curvature in Multilayer Graphene, Phys. Rev. B 87, for the Twisted Bilayer Graphene Narrow Bands, Phys. 241108 (2013). Rev. X 8, 031088 (2018). [62] Y.-H. Zhang and T. Senthil, Bridging Hubbard Model [69] M. Koshino, N. F. Q. Yuan, M. Ochi, K. Kuroki, and L. Fu, Physics and Quantum Hall Physics in Trilayer Gra- this issue, Maximally Localized Wannier Orbitals and phene/h-BN Moir´e Superlattice, arXiv:1809.05110. the Extended Hubbard Model for the Twisted Bilayer [63] P. Corboz, M. Lajkó, A. M. Läuchli, K. Penc, and F. Mila, Graphene, Phys. Rev. X 8, 031087 (2018). Spin-Orbital Quantum Liquid on the Honeycomb Lattice, [70] L. Zou, H. C. Po, A. Vishwanath, and T. Senthil, Band Phys. Rev. X 2, 041013 (2012). Structure of Twisted Bilayer Graphene: Emergent [64] R. K. Kaul, Spin Nematics, Valence-Bond Solids, and Spin Symmetries, Commensurate Approximants, and Wannier Liquids in SOðnÞ Quantum Spin Models on the Triangular Obstructions, Phys. Rev. B 98, 085435 (2018). Lattice, Phys. Rev. Lett. 115, 157202 (2015).

031089-31