ARTICLE https://doi.org/10.1038/s41467-020-19773-y OPEN Unconventional Hall response in the quantum limit of HfTe5 ✉ S. Galeski 1 , X. Zhao2,3,4, R. Wawrzyńczak1, T. Meng 5, T. Förster6, P. M. Lozano7,8, S. Honnali1, N. Lamba1, T. Ehmcke 5, A. Markou 1, Q. Li. 7,8,G.Gu8, W. Zhu2,4, J. Wosnitza6,9, C. Felser1, G. F. Chen 2,3,4 & ✉ J. Gooth 1,9 Interacting electrons confined to their lowest Landau level in a high magnetic field can form a 1234567890():,; variety of correlated states, some of which manifest themselves in a Hall effect. Although such states have been predicted to occur in three-dimensional semimetals, a corresponding Hall response has not yet been experimentally observed. Here, we report the observation of an unconventional Hall response in the quantum limit of the bulk semimetal HfTe5, adjacent to the three-dimensional quantum Hall effect of a single electron band at low magnetic fields. The additional plateau-like feature in the Hall conductivity of the lowest Landau level is accompanied by a Shubnikov-de Haas minimum in the longitudinal electrical resistivity and its magnitude relates as 3/5 to the height of the last plateau of the three-dimensional quantum Hall effect. Our findings are consistent with strong electron-electron interactions, stabilizing an unconventional variant of the Hall effect in a three-dimensional material in the quantum limit. 1 Max Planck Institute for Chemical Physics of Solids, Nöthnitzer Straße 40, 01187 Dresden, Germany. 2 Institute of Physics and Beijing National Laboratory for Condensed Matter Physics, Chinese Academy of Sciences, 100190 Beijing, China. 3 Songshan Lake Materials Laboratory, 523808 Dongguan, Guangdong, China. 4 School of Physics Science, University of Chinese Academy of Sciences, 100049 Beijing, China. 5 Institute of Theoretical Physics and Würzburg- Dresden Cluster of Excellence ct.qmat, Technische Universität Dresden, 01062 Dresden, Germany. 6 Hochfeld-Magnetlabor Dresden (HLD-EMFL) and Würzburg-Dresden Cluster of Excellence ct.qmat, Helmholtz-Zentrum Dresden-Rossendorf, 01328 Dresden, Germany. 7 Department of Physics and Astronomy, Stony Brook University, Stony Brook, NY 11794-3800, USA. 8 Condensed Matter Physics and Materials Science Department, Brookhaven National Laboratory, Upton, NY, USA. 9 Institut für Festkörper- und Materialphysik, Technische Universität Dresden, 01062 Dresden, Germany. ✉ email: [email protected]; [email protected] NATURE COMMUNICATIONS | (2020) 11:5926 | https://doi.org/10.1038/s41467-020-19773-y | www.nature.com/naturecommunications 1 ARTICLE NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-020-19773-y pplying a strong magnetic field to an electron gas confines could manifest themselves in a Hall response that should be the electrons motion in cyclotron orbits with a set of observable in the quantum limit of 3D semimetals10,13,14. A — discrete eigenenergies the Landau levels. In two- Inspired by these ideas, the possibility of finding a 3D QHE has dimensional (2D) systems, this quantization leads to a fully been explored in several material systems. For example, sig- gapped energy spectrum and to the emergence of the quantum natures of the integer quantum Hall effect (IQHE) have been Hall effect (QHE)1. In the limit where only the lowest Landau found in quasi-2D semiconducting multilayer lattices15, Bech- 16,17 18 19 20 level (LLL) is occupied (the so-called quantum limit), gaards salts , η-Mo4O11 , n-doped Bi2Te3 , and EuMnBi2 , electron–electron interactions can play a significant role, leading in which the layered crystal structure itself supplies the stack of to the appearance of correlated states, such as the fractional 2D systems. Very recently, the QHE has also been observed in 3D 2 fi 5 21 22 quantum Hall effect . In contrast, the Landau level spectrum of a graphite lms , bulk ZrTe5 , and HfTe5 samples . In graphite, three-dimensional (3D) electron gas is not fully gapped and the imposed periodic superstructure has been attributed to the becomes like that of a one-dimensional system. As a consequence, formation of standing electron waves. In ZrTe5 and HfTe5, the the electrons can still move along the field direction, which in IQHE was originally believed to arise from a charge density wave turn destroys the quantization of the Hall effect3–5. Nevertheless, (CDW), due to the scaling of plateau height with the Fermi it has been predicted that a generalized version of the QHE can wavevector. This scenario is, however, in contrast with thermo- emerge in 3D electron systems that exhibit a periodically dynamic and thermoelectric measurements on ZrTe5 that did not modulated superstructure6–8. Analogous to as in two dimensions, reveal any signatures of a field-induced CDW transition. Instead, in the vicinity of the quantum limit, 3D electron systems are also it was proposed that ZrTe5 should be considered a stack of weakly prone to form a variety of correlated electron states, including interacting Dirac 2DEGs with the plateau height scaling origi- Luttinger liquids; charge, spin and valley density waves; excitonic nating from the interplay of small carrier density and the pecu- insulators; Hall and Wigner crystals; or staging transitions in the liarities of Landau quantization of the Dirac dispersion23.In case of highly anisotropic layered systems3,4,6,9–13. It has been parallel to the search for the 3D QHE, there has been a long- theoretically pointed out that some of these states are related to standing experimental effort to observe field-induced correlated quantum Hall physics in three dimensions and likewise states in the quantum limit of three-dimensional materials. a b c d B || x B || y B || z 3 B 5 5 5 cm) I Vxx cm) cm) Ω z Ω Ω Vxy (m (m (m xx y x xx xx 3 K 3 K 3 K 2 0 0 0 cm) 0123 Ω 0369 0369 B (T) B (T) (m B (T) xx 1 e f g cm) cm) 4 5 B || y 2 Ω B || z Ω B || x cm) m m Ω 0 0 0 m –2 –1 0 50 100 150 200 250 300 0 ( xx (10 (10 –5 T (K) –4 xx xx Δ –2 0.0 0.2 0.4 0.6 0.8 0.0 0.2 0.4 0.6 01234 Δ Δ B–1 (T–1) B–1 (T–1) B–1 (T–1) h i j k k 15 15 z 7 z B z B B z 6 y x y x y x 12 12 2D 2D 5 9 9 4 (T) (T) N F F B B k 3 6 6 y 90° 80° kx 3D 3D 2 70° 60° 3 3 1 50° 40° 30° 20° 10° 0° 0 0 0 0123 0306090 0306090 B–1 (T–1) (°) (°) Fig. 1 Three-dimensional morphology of the Fermi surface in HfTe5.aLongitudinal electrical resistivity ρxx as a function of temperature T at zero magnetic field. Inset: Sketch of the measurement configuration in the three spatial directions x, y, and z. The bias current I is applied along x and the magnetic field B along y. The corresponding voltage responses are measured in x-(Vxx) and in y direction (Vxy). b ρxx as a function of B at 3 K with B applied along x, c along y and d along the z direction. e Variation of the longitudinal electrical resistivity Δρxx as a function of B at 3 K with B applied along x, f, along y, and g along the z direction. h Landau-index fan diagram for the integer Landau levels N for different angles α of B in the z–x plane (see inset; α is positive −1 from the z to the x direction) as a function of B . Data are obtained from the minima of ρxx in extended data Fig. 4a. i, Shubnikov-de Haas frequency as a function of angle α and j. β. β is the rotation angle of B in the z–y plane (see inset; β is positive from the z to the y direction). The black dots represent the measurement data. The red fitting curve represents a planar 2D Fermi surface, the blue fitting curve corresponds to an ellipsoidal 3D Fermi surface. k The Fermi surface of HfTe5 in momentum space along the kx, ky, and kz direction. 2 NATURE COMMUNICATIONS | (2020) 11:5926 | https://doi.org/10.1038/s41467-020-19773-y | www.nature.com/naturecommunications NATURE COMMUNICATIONS | https://doi.org/10.1038/s41467-020-19773-y ARTICLE Although those studies have provided signatures of field-induced in Fig. 1b–d (upper panels). In each field direction, we have 24,25 21,26 ρ B states in the longitudinal electrical resistivity of Bi , ZrTe5 observed maxima in xx that are periodic in 1/ , each of which and graphite5,27–30, correlated states with Hall responses have yet corresponds to the onset of a Landau level. In the associated to be observed. minima, ρxx(B) does not vanish, which is a consequence of the In this work, we present measurements of the low-temperature remaining dispersion in z direction in 3D systems and Landau longitudinal and Hall resistivities of the 3D semimetals HfTe5 level-broadening due to disorder (see Supplementary Note 4, 23 and ZrTe5. Previous studies have shown that HfTe5 is an iso- Supplementary Fig. S4–S7 and ref. for details). To determine 31 structural counterpart of ZrTe5 . Both materials share an the SdH oscillation frequency, we have subtracted the smooth orthorhombic crystal structure and a single elliptical 3D Fermi high-temperature (50 K)—ρxx(B) from the low-T-data, obtaining surface, comprising less than 1% of the Brillouin zone and the oscillating part of the longitudinal resistivity Δρxx(B).