First and Second Law of Quantum Thermodynamics: A Consistent Derivation Based on a Microscopic Definition of Entropy

Philipp Strasberg1 and Andreas Winter1,2 1F´ısica Te`orica: Informaci´oi Fen`omensQu`antics,Departament de F´ısica, Universitat Aut`onomade Barcelona, 08193 Bellaterra (Barcelona), Spain and 2ICREA – Instituci´oCatalana de Recerca i Estudis Avan¸cats, Pg. Lluis Companys, 23, 08010 Barcelona, Spain (Dated: August 31, 2021) Deriving the laws of thermodynamics from a microscopic picture is a central quest of statistical mechanics. This tutorial focuses on the derivation of the first and second law for isolated and open quantum systems far from equilibrium, where such foundational questions also become practically relevant for emergent nanotechnologies. The derivation is based on a microscopic definition of five essential quantities: internal energy, thermodynamic entropy, work, heat and temperature. These definitions are shown to satisfy the phenomenological laws of nonequilibrium thermodynamics for a large class of states and processes. The consistency with previous results is demonstrated. The framework applies to multiple baths including particle transport and accounts for processes with, e.g., a changing temperature of the bath, which is determined microscopically. An integral fluctuation theorem for entropy production is satisfied. In summary, this tutorial introduces a consistent and versatile framework to understand and apply the laws of thermodynamics in the quantum regime and beyond.

I. INTRODUCTION TO NONEQUILIBRIUM In equations, for any physical process THERMODYNAMICS: PHENOMENOLOGY ∆Suniv ≥ 0, (2a) Before we turn to any microscopic derivation of the where Suniv denotes the thermodynamic entropy of the laws of thermodynamics, it seems worthwhile to briefly universe, which should be distinguished from any infor- recall what we actually want to derive. mation theoretic notion of entropy at this point. Note Thermodynamics is an independent physical theory, that the terminology ‘universe’ does not necessarily refer whose principles have been applied with an enormous to the entire universe in the cosmological sense, but to success over a wide range of length, time and energy any system which is sufficiently isolated from the rest of scales. It arose out of the desire to understand trans- the world. For our purposes, this also includes a gas of formations of matter in chemistry and engineering in the ultracold atoms [5,6] or the system and the bath within 19th century [1]. The systems under investigation were the open quantum systems paradigm [7,8]. The change macroscopic and described by very few variables; for in- in entropy of the universe is often called the entropy pro- stance, temperature T , pressure p and volume V . These duction [3] and denoted by Σ = ∆Suniv. If Σ = 0, the macroscopic systems could exchange heat Q with their process is called reversible, otherwise irreversible. surroundings and mechanical work W could be supplied Focusing on the system-bath setup, e.g., as sketched to them. A prototypical example of a thermodynamic in Fig.1, the entropy of the universe is often additively setup partitioned into a system, a heat bath and a work decomposed into the entropy of the system and the en- reservoir is sketched in Fig.1. The theory is based on two central axioms, which are called the first and second law of thermodynamics [2,3]

(there is also a zeroth and a third law of thermodynamics, ⇝ which are not the topic of this paper). The first law states arXiv:2002.08817v6 [quant-ph] 30 Aug 2021 that the change ∆US in internal energy of the system is ⇝

balanced by heat Q and mechanical work W : ⇝

∆US = Q + W. (1) Figure 1. Thermodynamic setup where the system is a gas in Note that we define heat and work to be positive if they a container. By pushing a piston, the thermodynamic vari- increase the internal energy of the system. The first law ables (such as T , p or V ) can be changed in a mechanically is a consequence of conservation of energy applied to the controlled way, which is abstracted as the action of a ‘work system, the heat bath and the work reservoir. However, reservoir.’ Furthermore, through the walls of the container the fundamental distinction between heat and work be- the gas is in simultaneous contact with a heat bath, with comes only transparent by considering the second law. which it can exchange energy. This exchange of energy is ac- The second law, in its most general form, states that companied with an exchange of entropy, which is the defining “the entropy of the universe tends to a maximum” [4]. property to call this energy exchange ‘heat.’ 2 vironment: Suniv = SS + Senv. This is justified whenever temperature T and by computingdQ ¯ = CB(T )dT , where surface effects are negligible compared to bulk properties, CB(T ) is the heat capacity of the bath. Unfortu- which is often (but not always) the case for macroscopic nately, on phenomenological grounds it is not known how systems. Then, the second law becomes to construct such reversible transformations connecting nonequilibrium states in general, and it seems doubt- Σ = ∆SS + ∆Senv ≥ 0. (2b) ful that this is always possible. Widely accepted solu- Furthermore, the environment is typically assumed to tions to this problem seem to exist only in the linear re- be well-described by an equilibrium state with a time- sponse regime [10] or if the local equilibrium assumption dependent temperature T such that its change in entropy is valid [3]. R In this tutorial, we mostly focus on small systems, is ∆Senv = − dQ/T¯ . Here,dQ ¯ denotes an infinitesimal heat flow into the system. Then, the second law reads which can show quantum effects, are driven far from equi- librium, and are in contact with an environment. Such Z dQ¯ systems are called open quantum systems [7,8]. For Σ = ∆S − ≥ 0, (2c) S T many potential future technologies—such as thermoelec- tric devices, solar cells, energy efficient computers, refrig- which was introduced by Clausius, who called Σ uncom- erators that cool down to almost zero Kelvin, or quantum pensated transformations (“unkompensierte Verwandlun- computing, sensing or communication devices—these are gen”) [4]. In fact, the word ‘entropy’ was chosen by Clau- very interesting systems. Furthermore, we are also inter- sius based on the ancient greek word for ‘transformation’ ested in isolated quantum many-body systems such as ul- (τροπή). Equation (2c) is often referred to as Clausius’ tracold quantum gases [5,6]. In all of these cases, neither inequality. Finally, if the bath gets only slightly per- the local equilibrium assumption nor linear response the- turbed away from its initial temperature, here denoted ory can be applied in general. A thermodynamic descrip- by T0, then Eq. (2c) reduces to tion purely based on phenomenological grounds therefore Q appears challenging. Σ = ∆SS − ≥ 0 (2d) Moreover, the traditionally used classifications in ther- T0 modynamics of a heat bath and a work reservoir are be- with Q = R dQ¯ the total flow of heat from the bath. coming increasingly inadequate. Nowadays, experimen- These basic building blocks of phenomenological talists have access to information beyond simple macro- nonequilibrium thermodynamics can be further extended scopic parameters such as temperatures or chemical po- to, e.g., multiple heat baths or particle transport (above, tentials, they can engineer specifically tailored environ- we tacitly assumed that the system only exchanges en- ments and make use of more sophisticated external re- ergy but not particles with the bath). For most parts, sources, including quantum measurements and feedback we focus on the microscopic derivations of the laws above control loops. Accounting for all these possibilities in a and turn to these extensions only at the end. purely phenomenological way seems impossible. Finally, a microscopic derivation of the laws of thermo- dynamics gives us a better understanding about the phe- II. GOAL OF THIS TUTORIAL nomenological theory. The resulting theoretical frame- work, in which thermodynamic principles are explained A. The need for a microscopic derivation and supplemented by quantum mechanical and statistical considerations, is called quantum thermodynamics. While it is important to emphasize the status of ther- modynamics as an independent physical theory, its pre- cise scope is debated and problems appear when trying to B. Setting apply it far from equilibrium. It thus remains a subject of ongoing research [9]. We briefly recall the quantum mechanical setting we The difficulties one faces with a purely phenomenolog- are interested in. First, we consider the case of an iso- ical approach are perhaps best exemplified by the notion lated system. Its state at time t is described by a density of system entropy SS. How should this quantity—apart matrix ρ(t) and the Hamiltonian of the system is denoted from an unimportant additive constant—be defined out H(λt). Here, λt is some externally specified driving pro- of equilibrium? Clausius suggested to use Eq. (2c) by tocol (e.g., a changing electromagnetic field or the moving postulating that any two system states can be connected piston in Fig.1). The validity of modelling the dynamics by a reversible transformation [4]. If such a transforma- of a quantum system via a time-dependent Hamiltonian tion is found, inequality (2c) becomes an equality, rests on the assumption that the driving field is generated Z by a classical, macroscopic device. Finally, the dynam- dQ¯ ics of the system state obeys the Liouville-von Neumann ∆SS = , (3) R T equation (~ ≡ 1 throughout) where the subscript R means ‘reversible.’ Equation (3) ∂ ρ(t) = −i[H(λ ), ρ(t)], (4) allows to quantify ∆SS by measuring the time-dependent ∂t t 3 where [A, B] = AB − BA is the commutator. The time First, in Sec.I we saw that there are five important evolution starting from an initial state ρ(0) (we always concepts in phenomenological thermodynamics. These set the initial time to be t = 0) is therefore unitary: are the two state functions internal energy and thermo- dynamic entropy, the two process-dependent quantities ρ(t) = U(t, 0)ρ(0)U †(t, 0). (5) mechanical work and heat and the temperature appear- ing in Clausius’ inequality (2c). For all of them we like Here, the unitary time evolution operator U(t, 0) = to provide a microscopic definition, which is expressed in R t exp+[−i 0 dsH(λs)] is defined as the time-ordered ex- terms of the quantum mechanical Hamiltonian and the ponential of the Hamiltonian. Notice that we make no density matrix (or quantities derived from them). assumption about the specific form of H(λt) in the fol- Second, these quantities are supposed to satisfy the lowing, we only need to make one assumption about the first law (1) as well as the second laws (2a), (2b), (2c) initial state ρ(0), as explained in the next section. and (2d). As explained in the previous section, Eq. (2a) Next, we consider open quantum systems and use a is more general than Eq. (2b), which is more general than subscript SB (for system and bath) to denote the global Eq. (2c), which is more general than Eq. (2d), with each state and Hamiltonian. The latter is of the form one following from the previous one in its respective range of validity, and we demand that this hierarchy of second HSB(λt) = HS(λt) ⊗ 1B + 1S ⊗ HB + VSB (6) laws is reproduced in the microscopic derivation. We = HS(λt) + HB + VSB, remark that, due to the relations extablished by the laws of thermodynamics, the five thermodynamic quantities where we suppressed tensor products with the identity in we seek to define are not all independent. the notation of the second line. Here, H (H ) denotes S B Third, as an important consistency check, we demand the Hamiltonian of the unperturbed system (bath) and that the proposed definitions should reduce to well known V their interaction. Again, they are completely arbi- SB results derived previously in and out of equilibrium. trary in our framework. However, in view of Fig.1, we The above three criteria are certainly the most basic assumed that the external driving protocol λt only influ- ences the system Hamiltonian. It is also possible to con- desiderata we can have about any microscopic derivation of the laws of thermodynamics. As it turns out, it is sider time-dependent interactions VSB(λt) to model, e.g., the coupling and decoupling of the system and the bath. possible to strictly satisfy all of them for any Hamiltonian Our results continue to hold in this case, but for ease of the isolated system or the system-bath composite. We need, however, one assumption about the initial of presentation we assume VSB to be time-independent. state. This assumption is mathematically specified later Finally, while the joint system-bath state ρSB(t) evolves in time according to Eq. (4) with respect to the Hamil- on, but here we explain why we need one. The micro- tonian (6), the evolution of the reduced system state scopic equations of motion, such as Eq. (4) or Newton’s equation for classical systems, obey a property called

ρS(t) = trB{ρSB(t)} (7) time-reversal symmetry. Roughly speaking, this means that to any solution of the dynamics with a given initial (with trB{... } denoting the partial trace over the bath and final condition, it is possible to find a conjugate ‘twin degrees of freedom) is no longer unitary. In fact, the evo- solution’ with initial and final condition exchanged (Ap- lution of ρS(t) is markedly different and in general very pendixA gives a precise account of time-reversal sym- hard to compute [7,8]. The laws of thermodynamics metry). Thus, if thermodynamic entropy increases for derived below hold, however, regardless of these consid- the first solution, it must decrease for the conjugate twin erations. solution. Consequently, “the second law can never be A final word on terminology is useful to avoid confu- proved mathematically by means of the equations of dy- sion. In thermodynamics, a system is called (i) isolated, namics alone,” as Boltzmann stressed already [11]. (ii) closed or (iii) open if it can exchange (i) only work, The reason why we see no violations of the second law (ii) work and heat in form of energy or (iii) work and heat in our daily life comes from the fact that initial condi- in form of energy and particles with its surroundings. In tions, which generate a spontaneous entropy decrease, are contrast, in open quantum system theory the words iso- extremely hard to prepare experimentally, see Fig.2 for lated and closed are used interchangeably to describe case an illustration. Mathematically, these ‘unnatural’ states, (i), whereas cases (ii) and (iii) are both called open. We which are very hard to prepare, need to be excluded by indeed use the terminology open in the latter sense and, a proper choice of initial state specified later on. for definiteness, call case (i) isolated. We remark that this is not the only way to derive the second law microscopically. It is also possible to consider arbitrary initial states, but in this case the second law C. Desiderata and assumption can only be established by imposing restrictions on the Hamiltonian. We do not consider this approach in detail We here precisely specify what we mean by a consis- here, but see Refs. [12–18] for an exposition of historical tent microscopic derivation of the laws of thermodynam- and recent ideas and discussions in this direction. We ics and what we need to assume to accomplish it. remark that the five thermodynamic quantities defined 4

III. INTERNAL ENERGY AND MECHANICAL WORK IN ISOLATED SYSTEMS

For an isolated system we identify the expectation value of its Hamiltonian with the internal energy appear- ing in phenomenological thermodynamics,

Figure 2. Time evolution of gas particles in a box with per- U(t) ≡ tr{H(λt)ρ(t)}. (8) fectly reflecting walls. Left: Initially, all gas particles have a velocity pointing to the left such that in the next time step We remark that definition (8) is an assumption, but we none of them is reflected to the right. This is an extremely are not aware of any attempt to define internal energy unlikely state and an experimental preparation of it requires differently. ˙ precise control about every single gas particle. Right: Given If the system is not driven (λt = 0), its internal energy the initial state on the left, the state of the gas after the time is conserved since the Hamiltonian is a constant of mo- step is characterized by a lower entropy, in seeming violation tion: ∆U(t) = 0. Here and in the following, we use the to the second law of thermodynamics. notation ∆X(t) = X(t) − X(0) to denote the change of any time-dependent state function X(t). If the system is driven, its internal energy can change in time: below nonetheless remain meaningful for this different ∆U(t) = tr{H(λt)ρ(t)} − tr{H(λ0)ρ(0)}. (9) approach. Finally, one might wonder how the second law can Since the system is isolated (i.e., only coupled to a work emerge at all in a universe with time-reversal symmet- reservoir), no heat is flowing (Q = 0) and the phenomeno- ric evolution equations. The most likely explanation is logical first law (1) forces us to identify the change in that the universe started off in a state with extremely internal energy with the work supplied to the system: low entropy. Thus, the second law seems to be a con- ∆U(t) = W (t). (10) sequence of the boundary conditions. This conjecture is known as the past hypothesis. An informal discussion of This is the first law of thermodynamics for an isolated the microscopic origin of the second law and the arrow system. A quick calculation, using Eq. (4) and that the of time is given in Ref. [19]. trace is cyclic, reveals that the work can be expressed as

Z t d W (t) = ds tr{H(λ )ρ(s)} ds s 0 (11) Z t ∂H(λ )  Z t D. Outline = dstr s ρ(s) = dsW˙ (s) 0 ∂s 0

We start with the definition of internal energy and me- with the instantaneously supplied power W˙ (s). chanical work in isolated systems in Sec.III, which are To conclude, the identifcation of mechanical work in an clearly the most uncontroversial definitions. Afterwards, isolated system follows solely from the phenomenological we review various microscopic notions of thermodynamic first law together with the assumption (8). entropy for an isolated system in Sec.IV and we argue for a concept called observational entropy as the most appropriate candidate. Equipped with this concept, we IV. MICROSCOPIC NOTIONS OF then establish the second law of thermodynamics for iso- THERMODYNAMIC ENTROPY lated systems in Sec.V. Also the notion of an effective nonequilibrium temperature is introduced there. This is The central concept in both, thermodynamics and sta- followed by a detailed derivation of the laws of thermody- tistical mechanics alike, is entropy. Unfortunately, it is namics in open systems in Sec.VI. In Sec.VII we report also the most debated concept, which got constantly mys- on further extensions of our framework, including the tified during the history of science. Here, we review three treatment of multiple heat baths and particle exchanges. important entropy concepts in statistical mechanics: the SectionVIII is devoted to the derivation of fluctuation Gibbs-Shannon-von Neumann entropy, the Boltzmann theorems, which generalize previous results by extending entropy and observational entropy. The last candidate the notion of entropy production to single stochastic tra- unifies the previous two concepts and it will be our micro- jectories recorded in an experiment. The final Sec.IX scopic choice for thermodynamic entropy in the following, contains some concluding reflections. Two appendices in and out of equilibrium. We argue that this choice res- about time-reversal symmetry (AppendixA) and basic onates with recent findings in nonequilibrium statistical information theory concepts (AppendixB) accompany mechanics and extends ideas expressed by Boltzmann, this tutorial for self-containedness. Gibbs, von Neumann, Wigner, Jaynes, among others. 5

A. Gibbs-Shannon-von Neumann entropy Gibbs-Shannon entropy) coincides with thermodynamic entropy even out of equilibrium. This became consensus The Gibbs-Shannon-von Neumann entropy of a state in classical stochastic [28–30] and quantum thermody- ρ, regardless whether it is in or out of equilibrium, reads namics [31, 32]. It was subject to a direct experimental test [33] and experimental confirmations of Landauer’s SvN(ρ) ≡ −tr{ρ ln ρ}. (12) principle further support this hypothesis [34–39].

Since we are interested in quantum systems throughout this manuscript, we used a subscript ‘vN’ and mostly call B. Boltzmann entropy it von Neumann entropy for brevity. The success of Eq. (12) for applications in (classical The second well-known microscopic candidate for ther- and quantum) information and communication theory is modynamic entropy is Boltzmann’s entropy. To define undeniable [20–22]. It has many useful properties and it precisely, we consider a special case of later rele- many information theory concepts are directly related to vance. Let H be the Hamiltonian of an isolated system, it; those which are useful for the purposes of the present where we dropped any dependence on external parame- manuscript are reviewed in AppendixB. ters λt for notational simplicity. We write the stationary One of these properties says that the von Neumann Schr¨odingerequation as entropy is invariant under unitary evolution, i.e., for any state ρ and any unitary U H|Ei, `ii = Ei|Ei, `ii, (15)

† SvN(ρ) = SvN(UρU ). (13) where |Ei, `ii denotes an energy eigenstate with eigenen- ergy Ei and `i labels possible exact degeneraries. Now, Consequently, if we were to interpret von Neumann en- imagine an isolated system with many components such tropy as thermodynamic entropy (times the Boltzmann that its associated Hilbert space is extremely large. For factor kB, which we set to one in the following), then, all practical purposes, it is then impossible that a mea- from Eq. (5), we would conclude that the thermody- surement of the energy is so precise that it yields a unique namic entropy of every isolated system is always con- eigenenergy Ei. Instead, any measurement has a finite stant. This conflicts with empirical facts, which show resolution or uncertainty δ, which can be mathematically that most spontaneous processes are accompanied with captured by a projector of the form a strict increase in thermodynamic entropy (e.g., the free expansion of a gas, the mixing of liquids or the evolution X X ΠE ≡ |Ei, `iihEi, `i|. (16) of the cosmological universe). E ∈[E,E+δ) ` Thus, von Neumann entropy can not correspond to i i thermodynamic entropy in general. This fact was clearly These projectors form a complete and orthogonal set recognized by von Neumann himself, who confessed that P {ΠE}, i.e., ΠE = 1 (with 1 the identity operator) Eq. (12) is “not applicable” to problems in statistical E and ΠEΠE0 = δE,E0 ΠE (with δE,E0 the Kronecker delta). mechanics as it is “computed from the perspective of an This describes a coarse-grained measurement. observer who can carry out all measurements that are Now, if such a coarse-grained measurement yields out- possible in principle” [12] (translated in Ref. [13]). come E, the Boltzmann entropy of the system is Importantly, we did not say that von Neumann entropy never coincides with thermodynamic entropy. In fact, it SB(E) ≡ ln VE, (17) does so in two important cases. The first case corresponds to a system at equilibrium, where VE = tr{ΠE} is the rank of the projector, called which can be described by the Gibbs ensemble in the following also a volume term. Thus, the Boltz- mann entropy counts all possible microstates compatible e−βH π(β) ≡ , Z(β) ≡ tr{e−βH }, (14) with the constraint of knowing E, and then takes the Z(β) logarithm of it (remember that kB ≡ 1). Clearly, if information about further macrocopic vari- or a generalization thereof, e.g., the grand canonical en- ables is available, e.g., the particle number N, then the semble if particle numbers are important. In this case, Boltzmann entropy becomes SvN[π(β)] coincides with thermodynamic entropy. We re- mark that this conclusion is only valid if the system obeys SB(E,N,... ) ≡ ln VE,N,..., (18) the equivalence of ensembles [23]. Beyond that, even the foundations of equilibrium statistical mechanics remain where VE,N,... counts all possible microstates compatible debated (see, e.g., Refs. [24–27] for recent research on the with the constraints E, N, etc. We remark that the pre- correct definition of equilibrium temperature). cise definition of VE,N,... is subtle if the corresponding The second case is given by small open systems, which observables do not commute. However, for the majority are in weak contact with a large thermal bath. Then, of applications in macroscopic thermodynamics, the cor- von Neumann entropy (or its classical counterpart, the responding observables commute at least approximately. 6

A distinctive feature of Boltzmann’s entropy compared to the von Neumann entropy is that it is nonzero even for a pure state ρ = |ψihψ|. For instance, if the pure state is confined to the energy shell [E,E + δ), i.e., ΠE|ψi = |ψi, one confirms that

SB(E) = ln VE 6= SvN[|ψihψ|] = 0. (19)

Moreover, Boltzmann’s concept easily allows to explain the second law, even without the need to introduce any notion of ensembles. For an isolated system it is much Figure 3. Thought experiment with a gas in a box with per- more probable to evolve from a region of small volume fectly reflecting boundaries. Initially, the gas is confined to towards a region of large volume and to reside for long the left half of the box. Then, at t0 a small hole is opened in times in the region with the largest volume, which is iden- the wall such that gas particles can diffuse to the right. Since tified with thermodynamic equilibrium. This explains the gas needs time to diffuse, it seems sensible to demand the increase of entropy after lifting a constraint and the that thermodynamic entropy should smoothly interpolate be- tendency to find systems in a state of maximum entropy. tween the lower initial and higher final value (dashed line). The power and simplicity of Boltzmann’s concept is so In contrast to this desideratum, von Neumann entropy stays appealing that many researchers have univocally adapted constant for all times (thick blue line). A naive application of Boltzmann’s entropy S (E) captures the correct initial and the idea to identify Boltzmann’s entropy with the phe- B final value, but contains a sudden discontinous jump at t0 nomenological thermodynamic entropy for macroscopic even if the hole in the wall is very small. Thus, it misses systems, even out of equilibrium. Perhaps surprisingly, some relevant dynamical information. For a similar and more also Jaynes was a proponent of it [40–42]. detailed discussion see Ref. [44]. In his words, “we feel quickly that [Eq. (18)] must be correct, because of the light that this throws on our prob- lem. Suddenly, the mysteries evaporate; the meaning of Carnot’s principle, the reason for the second law, and the Independent of the reader’s opinion on that matter justification for Gibbs’ variational principle, all become (even the present authors do not fully agree on it), we obvious” (stated below Eq. (17) in Ref. [40]) and “the find it important to point out that also Boltzmann’s ap- above arguments make it clear that [...] any macrostate— proach faces deficiencies in light of current experiments. equilibrium or nonequilibrium—has an entropy [(18)]” In fact, as stressed above, there is an agreement in fa- (stated above Eq. (25) in Ref. [41]). vor of von Neumann’s (or Shannon’s) entropy for small Indeed, it is easy to recognize that Boltzmann’s en- systems in weak contact with a thermal bath. Since to- tropy fits well Jaynes’ epistemological view on the second days nanotechnologies allow to make very precise mea- law for two resons. First, for the computation of Boltz- surements on small systems, the volume term appearing mann’s entropy “it is necessary to decide at the outset in Boltzmann’s entropy can be one and hence, it’s log- of a problem which macroscopic variables or degrees of arithm is zero. Therefore, Boltzmann entropy seems to freedom we shall measure and/or control” [42], where be inadequate to take into account microscopic informa- the “macrosopic variables” in Jaynes’ language are our tion, which is available to us now, but was not available observables E, N, etc. Second, if these observables are a hundred years ago. fixed to a given accuracy, then the state reflecting max- To conclude, whereas von Neumann entropy appears imum ignorance about the situation (i.e., maximum en- too fine-grained for all systems, which have more than tropy in the information theory sense), is given by the a few degrees of freedom, Boltzmann’s entropy appears microcanonical ensemble. If only the energy E is known, too coarse-grained to account for today’s experimental this microcanonical ensemble reads capabilities. This is also once more exemplified in Fig.3. It therefore seems desirable to have a flexible concept for Π ω(E) ≡ E , (20) entropy, which can interpolate between these two ideas. VE which satisfies SvN[ω(E)] = ln VE = SB(E). Albeit also favouring the Boltzmann entropy, the C. Observational entropy purely epistemological nature of the second law is denied in Refs. [19, 43] by pointing out that the flow of heat from hot to cold in macroscopic systems is a fact, which does We now review a third concept, which is called observa- not depend on the observer’s state of knowledge. That tional (or coarse-grained) entropy and which overcomes is to say, one expects the laws of thermodynamics to be the problems mentioned above. We begin with its formal generically true, either on a distant planet (about which definition, followed by the recapitulation of some useful we have no knowledge) or in an isolated many-body sys- mathematical facts needed later on, and we end with re- tem (where we might be able to prepare pure states). marks about its appearance in the literature. 7

1. Formal definition 2. Some elementary mathematical properties

We ignore any thermodynamic considerations for the We now list a couple of mathematical facts as lem- moment and consider some coarse-graining X = {Πx} mas, which appear scattered throughout the literature defined by a complete set of orthogonal projectors sat- (see the next subsection for a list of references). These P isfying x Πx = 1 and ΠxΠx0 = δx,x0 Πx. This coarse- lemmas add further appeal to the definition of observa- graining can be associated to a measurement of a suit- tional entropy. They hold for any coarse-graining {Πx} able observable, but the eigenvalues of the observable are and therefore might be of interest even outside thermo- unimportant for us. Instead, if the system is in state ρ, dynamic considerations. Below, we also make use of the we only need the probability px = tr{Πxρ} to observe quantum relative entropy D(ρkσ) ≡ tr{ρ(ln ρ − ln σ)}, outcome x and the volume term Vx ≡ tr{Πx}. Then, see AppendixB for further details. observational entropy with respect to a coarse-graining First, observational entropy can be bounded from X is defined as above and below. X X Lemma IV.1. If dim H < ∞ denotes the dimension of Sobs(ρ) ≡ px(− ln px + ln Vx). (21) x the Hilbert space of the isolated system, then X To convince ourselves that observational entropy inter- SvN(ρ) ≤ Sobs(ρ) ≤ ln dim H. (23) polates between the notions of von Neumann and Boltz- Second, observational entropy is extensive in the limit mann entropy, we consider the following two cases. where one expects it to be extensive. First, assume that we are an observer who, in von Neu- mann’s words [12, 13], “can carry out all measurements Lemma IV.2. Consider a composite system in the that are possible in principle.” Then, we could choose a decorrelated state ρ = ρ1 ⊗ · · · ⊗ ρn and a compos- coarse-graining X = {|xihx|}, which matches the eigen- ite coarse-graining X = X1 ⊗ · · · ⊗ Xn with projectors P basis of the state ρ = x λx|xihx|. We immediately Πx1 ⊗ · · · ⊗ Πxn . Then, X reveal that in this case Sobs(ρ) = SvN(ρ). n Second, observe that we can write observational en- X X Xj S (ρ) = S (ρj). (24) tropy as obs obs j=1 X X Sobs(ρ) = SSh(px) + pxSB(x), (22) Of course, one expects Eq. (24) to remain approxi- x mately true for weakly correlated system, which describe multiple macroscopic systems in contact with each other. where SSh(px) is the Shannon entropy of the probabilities p and the second term presents an averaged Boltzmann If surface properties are negligible compared to their bulk x properties, this then implies the usual notion of extensiv- entropy. Thus, if p = δ 0 , i.e., we are certain that x x,x ity known from thermodynamics. the state ρ is confined in the ‘macrostate’ Πx0 , we ob- X 0 Next, we note a useful rewriting of observational tain Sobs(ρ) = SB(x ). Depending on the coarse-graining X, this allows us to reproduce, e.g., Boltzmann’s en- entropy. For that purpose, we introduce the nota- tropy (17) associated to an imprecise energy measure- tion ρ(x) ≡ ΠxρΠx/px, which describes the post- ment. measurement state given outcome x, and ω(x) ≡ Πx/Vx, The last point about the correct choice of coarse- which denotes a generalized ‘microcanonical ensemble’ graining is very important. Definition (21) formally holds given the constraint x. for any coarse-graining. To connect observational en- Lemma IV.3. We have tropy to thermodynamic entropy, we need to make the " # right choice of coarse-graining just as Jaynes indicated X X X Sobs(ρ) = SvN pxρ(x) + pxD[ρ(x)kω(x)]. (25) by saying that “it is necessary to decide at the out- x x set of a problem which macroscopic variables or degrees of freedom we shall measure and/or control” [42]. The Proof. Since the states ρ(x) have support on orthogonal difference is that observational entropy is not restricted subspaces, it follows from Theorem 11.10 in Ref. [21] that to “macroscopic variables [= coarse-grainings],” but can " # X X take into account more detailed information as well. Note SvN pxρ(x) = px {SvN[ρ(x)] − ln px} . (26) that the correct choice of variables (energy, polarization, x x particle number, etc.) is often determined by the physi- Using this insight in Eq. (25) yields cal setup itself, whereas the level of ‘coarse-grainedness’ quantified by the volume terms Vx depends experimen- X X Sobs(ρ) = − px [ln px + tr{ρ(x) ln ω(x)}] tally on the precision of the measurement. Theoretically, x this precision is a free parameter in principle, which has X (27) to be chosen reasonably. Luckily, one also expects that = − px [ln px − tr{ρ(x) ln Vx}] , the qualitative picture does not sensitively depend on the x precise choice of Vx. which is identical to Eq. (21) since tr{ρ(x)} = 1. 8

The next lemma characterizes the states ρ which have For classical systems, where one needs to coarse-grain the same von Neumann and observational entropy. the phase space into cells, variants of Eq. (21) appear

X already in the work of Gibbs [45] and Lorentz [46], see Lemma IV.4. We have SvN(ρ) = Sobs(ρ) if and only if also Sec. 23a of the treatise about statistical mechanics X of the Ehrenfests [47]. In this context, Eq. (21) is also ρ = pxω(x) (28) known as “coarse-grained entropy” (see Wehrl [48], who x connects it to ergodicity and mixing and cites further references). for an arbitary set of probabilities px. For quantum systems, Eq. (21) can be traced back to Proof. Using Eq. (25), we can write von Neumann, who attributes it to a personal communi- " # cation from Wigner and clearly acknowledges its useful- X X ness for problems in statistical mechanics [12]. In fact, S (ρ) − SvN(ρ) = SvN pxρ(x) − SvN(ρ) obs von Neumann proves a remarkable ‘H-theorem’ in his x (29) X work, which we summarize here informally (see also the + pxD[ρ(x)kω(x)]. accompanying article [14] of the English translation [13] x for further details). For this purpose consider an iso- X lated system with time-independent Hamiltonian H and We see that Sobs(ρ) − SvN(ρ) is given as the sum of two non-negative terms because it follows from Theorem 11.9 suppose that the Hamiltonian has no degenerate energy in Ref. [21] that gaps. Furthermore, the orientation between the eigen- vectors of H and the eigenvectors of the coarse-graining " # {Π } is assumed random. Finally, it is assumed that the X x SvN pxρ(x) − SvN(ρ) ≥ 0 (30) projectors Πx are sufficiently ‘coarse,’ i.e., the number x of elements in the set {Πx} must be much smaller than dim H. Then, von Neumann found that for all initial with equality if and only if ρ = P p ρ(x). Furthermore, x x states |ψ i the observational entropy SX (|ψ ihψ |) will be D[ρ(x)kω(x)] = 0 if and only if ρ(x) = ω(x). Hence, 0 obs t t close to its upper bound ln dim H for most times t. In par- Eq. (28) follows. X ticular, if the initial observational entropy Sobs(|ψ0ihψ0|) The next lemma can be seen as a precursor of the is small, this proves the increase of entropy after waiting a sufficient amount of time. Von Neumann’s H-theorem second law, albeit the coarse-graining {Πx} is still ar- bitrary and not necessarily of thermodynamic relevance. can be regarded as complementary to what we establish It applies to any isolated system with time evolution (5). below: Whereas we rely on a special class of initial states, Since we are now interested in changes in observational no assumption about the Hamiltonian H is used. Finally, Xt Xt we remark that von Neumann’s approach was recently entropy, we write Sobs(t) = Sobs[ρ(t)] for the observa- tional entropy at time t and indicate that also the chosen extended [49–51], but the focus was on equilibration of expectation values, whereas his H-theorem related to ob- coarse-graining X = Xt can depend on time. servational entropy found no further attention. X0 Lemma IV.5. If Sobs(0) = SvN[ρ(0)], then We continue by pointing out that a second-law-like in- crease of observational entropy similar to Lemma IV.5 ∆SXt (t) = SXt (t) − SX0 (0) ≥ 0. (31) obs obs obs was proven for quantum systems in §106 of Tolman’s Proof. From the unitarity of time evolution we deduce book [52] and in Sec. 1.3.1 of the book by Zubarev et S [ρ(0)] = S [ρ(t)] and hence, al. [53] (the proof in the classical case seems to date vN vN back to Gibbs, see again the Ehrenfests [47]). It is, Xt Xt however, interesting to note that both books refuse to ∆S (t) = S (t) − SvN[ρ(t)]. (32) obs obs use Eq. (21) as a definition of thermodynamic entropy This term is easily shown to be positive using Eqs. (25) for out-of-equilibrium processes: Tolman discusses the and (30). connection to thermodynamic entropy only at equilib- rium and prefers to use the Gibbs-Shannon-von Neu- mann entropy [compare with Eq. (122.10) therein] and 3. Historical remarks Zubarev et al. prefer the Gibbs-Shannon-von Neumann entropy of a generalized out-of-equilibrium Gibbs ensem- Definition (21) or, more often, similar but less general ble. Other sources, where observational entropy was forms of it appear scattered throughout the literature sometimes more and sometimes less clearly identified as on statistical mechanics. Not seldomly Eq. (21) is used thermodynamic entropy, are Refs. [54–61]. in various computations without explicitly identifying it The present tutorial was in particular inspired by the with thermodynamic entropy, in particular not out of recent work of Safr´anek,Deutsch˘ and Aguirre [62], who equilibrium. Our efforts to trace back the origin and use coined the terminology “observational entropy” and pro- of definition (21) has yielded the following results, which pose it as a generally valid definition of thermodynamic shall not imply that the given list is exhaustive. entropy for isolated nonequilibrium quantum systems. 9

Further arguments for it are also given in their subse- independent of the unitary time evolution operator. quent work [44, 63–65], where also the case of multiple Equation (35) is the entropy production of an isolated non-commuting coarse-grainings is treated. In our expo- homogenous system for an energy coarse-graining. sition, we only deal with single or multiple but commut- ing coarse-grainings. B. Reversible case

V. SECOND LAW AND EFFECTIVE It is instructive to consider the reversible case of TEMPERATURE IN ISOLATED SYSTEMS Et E0 Eq. (35), defined by: Sobs[ρ(t)] = Sobs[ρ(0)]. While we are mostly interested in nonequilibrium situations in this We now consider the first thermodynamic application article, the reversible case is an important limiting case of observational entropy in isolated systems, thereby in- in thermodynamics and typically (approximately) gener- troducing concepts that turn out to be important for the ated by changing the protocol λt very slowly. open system paradigm studied in Sec.VI. For simplicity, The goal of this section is to prove that reversible pro- we focus on a homogeneous isolated system with energy cesses are characterized by the fact that they are easy as the only relevant macrovariable. We beginn by study- to time-reverse from a macroscopic point of view, in uni- ing entropy production in general followed by a discussion son with our knowledge from thermodynamics. For that of the reversible case. We then introduce the important purpose, we need the notion of time-reversal symmetry, concept of an effective nonequilibrium temperature. Fi- which is introduced in greater detail in AppendixA. nally, we briefly discuss possible extensions. We recall how to time-reverse a unitary process in principle. Let ρ(t) = U(t, 0)ρ(0)U †(t, 0) be the time evolved state in the ‘forward process.’ We denote by A. Entropy production in a homogeneous system Θ the anti-unitary time-reversal operator. Consequently, † −1 UΘ(t, 0) = ΘU (t, 0)Θ becomes the unitary time evolu- We consider a driven isolated system with Hamilto- tion operator generated by the Hamiltonian HΘ(λs,B) = nian H(λt) and imagine the total energy of the isolated H(λt−s, −B) with a time-reversed driving protocol and system to be the only relevant (or accessible) thermo- a reversed magnetic field B. Finally, let Θρ(t)Θ−1 de- dynamic quantity. We call such a system homogenous note the time-reversed final state of the forward process. as we ignore any spatial irregularities. Thus, our coarse- Then, time-reversal symmetry guarantees that we can graining is defined by {Π }, where Π is obtained from Et Et recover the initial state ρ(0) by the previously introduced projector (16) by replacing the eigenenergies Ei and eigenstates |Ei, `ii by Ei(λt) and −1 −1 † ρ(0) = Θ UΘ(t, 0)Θρ(t)Θ UΘ(t, 0)Θ. (36) |Ei(λt), `i(λt)i to take into account the external driving. The time evolution of the system is described by In words, we recover the initial state of the forward pro- Eq. (5). Denoting p (t) = tr{Π ρ(t)} and V = Et E(λt) Et cess if we time-reverse the final state, let the protocol run tr{Π }, the observational entropy reads E(λt) backward (and perhaps reverse a magnetic field), and ap- X ply the inverse time-reversal on the state. Here, the ex- SEt [ρ(t)] ≡ p (t)[− ln p (t) + ln V ], (33) obs Et Et Et perimentally easy part is to reverse the driving protocol Et and a magnetic field. The hard part instead corresponds which we use as our microscopic definition of thermody- to time-reversing the state ρ(t) (for instance, classically namic entropy in this section. this requires to flip all momenta p → −p, which already Next, we consider the set of states ρ(t) that satisfy for a single particle is hard to achieve accurately). More- Et Sobs[ρ(t)] = SvN[ρ(t)]. From Lemma IV.4 we know that over, since Θ is anti-unitary, it can not be implemented in this set is a lab in general. An implementation of Eq. (36) therefore ( ) remains experimentally out of reach in most cases. X Ω(λt) = pE ω(Et) pE arbitrary . (34) There is, however, one important class of exceptions: t t −1 Et the operation Θρ(t)Θ is easy to achieve if the states ρ(t) are symmetric under time-reversal. These states are These states correspond to a somewhat larger set of equi- precisely the set of states characterized by Eq. (34). Sym- librium states than conventionally considered in statisti- −1 bolically, we can denote this by ΘΩ(λt)Θ = Ω(λt) or cal mechanics, but they share the same feature: they −1 ΘΩ(λt,B)Θ = Ω(λt, −B) in presence of a magnetic are invariant in time for a fixed Hamiltonian H(λt) and, field. In words, an equilibrium state is invariant under given a distribution p , they maximize the von Neu- Et time-reversal and hence, there is actually no need to im- mann entropy as a measure about our ‘ignorance’ of the plement the cumbersome time-reversal operation (apart state. from perhaps flipping B). Moreover, whenever we start with a state ρ(0) ∈ Ω(λ0), Now, we return to the reversible case of Eq. (35). From the second law follows from Lemma IV.5, Et E0 E0 Sobs[ρ(t)] = Sobs[ρ(0)] and Sobs[ρ(0)] = SvN[ρ(0)] = Et E0 Σ(t) = Sobs[ρ(t)] − Sobs[ρ(0)] ≥ 0, (35) SvN[ρ(t)] we can conlude (cf. Lemmas IV.4) that also the 10

final state must be an equilibrium state: ρ(t) ∈ Ω(λt). Assuming the Hamiltonian to be time-independent for ∗ Thus, our approach based on observational entropy shows a moment, another property of βt follows by recalling that reversible processes are characterized by the fact that the canonical ensemble π(β) with internal energy that they are simple to time-reverse from a macrocopic U(β) = tr{Hπ(β)} satisfies point of view. C(β) This statement is not a trivial tautology. If we had dU(β) = C(T )dT = − dβ, (39) started with a different entropy concept, it is unclear β2 whether this would imply the same statement. For in- stance, if we had identified the von Neumann entropy where dU(β) = U(β + dβ) − U(β) and C(β) = 2 2 2 with thermodynamic entropy, then we would need to call β [tr{H π(β)} − tr{Hπ(β)} ] denotes the heat capacity, all processes ‘reversible’ despite being very complicated which is non-negative. Thus, by definition of the effective ∗ ∗ to time-reverse in practice. inverse temperature we can conclude that β = β (U) is monotonically decreasing as a function of the internal en- ergy U, stretching from β∗ = ∞ if the system is in its ∗ C. Effective nonequilibrium temperature ground state to β = −∞ if the system is in its highest excited state (assuming the Hamiltonian is bounded from above, otherwise β∗ remains positive). Up to now, we have introduced microscopic notions for ∗ internal energy, mechanical work and thermodynamic en- Finally, β allows us to establish a remarkable connec- tropy. Another important quantity is temperature. This tion between energy and entropy, even out of equilibrium. concept also plays an important role in the next section, Let S(β, λ) = SvN[π(β, λ)] denote the von Neumann en- tropy of a Gibbs state at inverse temperature β. It fol- where our goal is to provide a microscopic derivation of ∗ ∗ the phenomenological Clausius inequality (2c), which re- lows that Tt dS(βt, λt) = dU(t) − tr{[dH(λt)]πt(βt )}. Here, dU(t) is the change of the nonequilibrium inter- mains valid even out of equilibrium. ∗ We remark that the definition of meaningful nonequi- nal energy in Eq. (37), and the term tr{[dH(λt)]πt(βt )} librium temperatures has a long history [66]. The def- can be interpreted as the work done on the system during a (fictitious) equilibrium process. Let us label inition we adapt here has appeared in phenomenolog- ∗ ical nonequilibrium thermodynamics under the name Tt dS(βt, λt) ≡ dQ¯ (t) as a heat flux for reasons that will “nonequilibrium contact temperature” more than 40 become clear in the next section. Then, years ago [67, 68]. It has also appeared at various places Z dQ¯ (s) in the statistical mechanics literature (see, e.g., Refs. [69– S(β∗) − S(β∗) = (40) t 0 T ∗ 72]) without, however, enjoying a wider popularity. s For an arbitrary nonequilibrium state ρ(t) we define ∗ Thus, the entropy production (35) can be written as: the inverse nonequilibrium temperature βt by demanding Z dQ¯ (s) ∗ Et ∗ U(t) = tr{H(λt)ρ(t)} ≡ tr{H(λt)π(βt )}, (37) Σ(t) = Sobs[ρ(t)] − S(βt , λt) + ∗ Ts (41) i.e., we ask which inverse temperature does a fictitious ∗ E0 + S(β0 , λ0) − Sobs[ρ(0)] ∗ Gibbs state π(βt ) need to have such that its internal en- ergy matches the true internal energy. In terms of our In particular, if the isolated system is prepared in a Gibbs coarse-grained energy measurement (16), a Gibbs state state, the last line vanishes. Furthermore, since the Gibbs is approximatively described by probabilities πEt (β) ≈ state maximizes entropy with respect to a fixed energy, −βEt P −βEt Et ∗ V e /Z(β, λ ) with Z(β, λ ) = V e . Def- we can conclude S [ρ(t)] ≤ S(β , λt). Consequently, Et t t Et Et obs t inition (37) can then be also expressed as Z dQ¯ (s) ≥ Σ(t) ≥ 0, (42) X X ∗ T ∗ EtpEt (t) ≡ EtπEt (βt ). (38) s E E t t which we will use in the next section. Of course, definitions (37) and (38) match only if the measurement uncertainty (or thickness of the energy shell) δ is chosen sufficiently small such that U(t) ≈ D. Extensions P E p (t). For ease of presentation, we assume this Et t Et in the following. The simple description of an isolated system in terms of An alternative way to describe the meaning of the a homogenous coarse-grained energy variable covers only ∗ ∗ nonequilibrium temperature Tt = 1/βt is as follows [67, a small fraction of thermodynamically interesting situa- 68]. Suppose that we have a collection of superbaths at tions. For instance, an accurate description of ultracold our disposal, which are prepared at different equilibrium atoms experiments [5,6] likely requires further variables ∗ temperatures T . Then, Tt is defined to be the tempera- (particle number, magnetization, polarization, etc.) and, ture T of a superbath, which causes no net heat exchange perhaps, variables with spatial resolution (e.g., energy or when coupling the system to it. particle densities). Since it seems impossible to cover all 11 these experiments in a tutorial article, we focused only our microscopic definition for thermodynamic entropy in on the basics above. They can be generalized by refining the following. the coarse-graining and illustrative examples have been We believe that the coarse-graining above most accu- already investigated by using observational entropy [62– rately reflects the current spirit of open quantum system 64]. theory and many nanotechnological platforms. We re- mark, however, that all the following identities—unless otherwise stated—are valid for a system and a bath of VI. FIRST AND SECOND LAW IN OPEN any size. Moreover, by choosing a coarse-graining for the SYSTEMS bath as in Sec.V, we implicitly assumed the bath to be a homogeneous object from a macroscopic point of view. In this section, we derive the hierarchy of second Perhaps not too far in the future, it might be necessary laws (2a), (2b), (2c) and (2d) for a suitable coarse- to extend the present approach to take into account fur- graining reflecting the degree of control an external agent ther information about the bath in form of, e.g., spatial has about the open quantum system. To derive Clausius’ irregularities. Furthermore, if the system itself becomes inequality (2c) we use the microscopic definition (37) of large (say, larger than 10 qubits), the description in terms temperature and introduce definitions for heat and in- of fine-grained rank-1 projectors |stihst| might no longer ternal energy of an open system. The present treatment be adequate. Whatever information is necessary to ac- presents a significant extensions of earlier work using ob- curately describe the experiment, the present approach servational entropy [73]. Further generalizations of this can be adapted accordingly. approach (initially correlated states, multiple baths, par- Finally, we fix the initial state. In unison with the ticle transport) are treated in Sec.VII. conventional open quantum systems approach [7,8], we consider in this section an initial state of the form

A. Relevant coarse-graining and initial state ρSB(0) = ρS(0) ⊗ πB(β0). (44)

This describes a system state ρS(0) initially decorrelated The central idea of system-bath theories is to divide from a bath described by a Gibbs state at inverse temper- the universe into relevant degrees of freedom (the ‘sys- ature β0. Since we are allowed to choose any {|s0ihs0|}, tem’) and irrelevant degrees of freedom (the ‘bath’) [7,8]. 0 0 we assume hs0|ρS(0)|s0i = 0 for all s0 6= s0 in the follow- The relevant degrees of freedom are assumed to be ac- ing. Indeed, if the experimenter initially performs a mea- cessible by experiment, whereas only limited informa- surement with projectors {|s0ihs0|}, then this assump- tion is available about the irrelevant degrees of free- tion holds automatically. Furthermore, as in Sec.VC, dom. Many current experimental platforms—such as we assume the resolution δ of the energy measurement of cavity or ciruit QED setups, optomechanical or nano- the bath to be sufficiently small such that electromechanical systems, quantum dots or nitrogen va- cancy (NV) centers—show such a separation between EB SB(β0) ≡ SvN[πB(β0)] ≈ S [πB(β0)]. (45) precisely measurable system quantities and coarse infor- obs mation about the bath. This implies that we are consistent at equilibrium, with Our definition of observational entropy is supposed observational entropy coinciding with the standard equi- to reflect this situation and, therefore, we choose the librium entropy of a canonical ensemble. coarse-graining {|sihs| ⊗ ΠEB }. Here, {|sihs|} is a set Extensions of the initial state (44) to take into ac- of rank-1 projectors acting on the system Hilbert space, count system-bath correlations or a bath not prepared whereas {ΠEB } is a set of coarse-grained energy projec- in a canonical ensemble are treated in Sec.VII. tors for the bath, i.e., ΠEB is constructed as in Eq. (16), but with respect to the bath Hamiltonian HB. Thus, a measurement yielding outcome (s, EB) with probability B. General second law ps,EB = trSB{|sihs|⊗ΠEB ρSB} gives us complete knowl- edge about the microstate of the system, but reveals only Using the properties of the initial state discussed partial information about the energy of the bath (which S0,EB above, we confirm that SvN[ρSB(0)] = Sobs [ρSB(0)]. is related to its temperature). We remark that the ba- Thus, from Lemma IV.5 we directly find sis for the system coarse-graining is arbitrary and might change in time. Therefore, we write |si = |s i in the St,EB S0,EB t Σa(t) = S [ρSB(t)] − S [ρSB(0)] ≥ 0. (46) following. Then, the observational entropy follows as obs obs This quantifies the entropy production in our setup in ps ,E (t) St,EB X t B its most general form. The subscript ‘a’ on Σa(t) shall Sobs [ρSB(t)] ≡ − pst,EB (t) ln , (43) VEB remind us that this entropy production corresponds to st,EB the phenomenological second law of Eq. (2a). Furthermore, the decorrelated initial state (44) implies where VEB = trB{ΠEB } counts the number of bath mi- S0,EB S0 EB crostates compatible with outcome EB. Equation (43) is Sobs [ρSB(0)] = Sobs[ρS(0)] + Sobs [ρB(0)]. At any later 12

∗ ∗ time we can write Here, we used that dSB(βs ) = βs dUB(s) and dUB(s) = dQ¯ (s) because the bath Hamiltonian is time-independent. St,EB St EB Sobs [ρSB(t)] = Sobs[ρS(t)] + Sobs [ρB(t)] EB ∗ (47) Furthermore, we used Sobs [ρB(t)] ≤ SB(βt ) as also done St,EB − Iobs [ρSB(t)]. below Eq. (41). The idea to identify the change in internal energy of Here, the classical mutual information (see AppendixB) the bath with the heat flux appears very convincing at this point. Following our convention to count the energy p (t) St,EB X st,EB flux into the system positive, we setdQ ¯ (s) ≡ −dU (s). Iobs [ρSB(t)] = pst,EB (t) ln (48) B pst (t)pEB (t) st,EB It follows from Eqs. (49) and (52) that characterizes the correlations in the final measurement Z St dQ¯ (s) Σc(t) = ∆S [ρS(t)] − ≥ 0. (53) result (st,EB). Since it is non-negative, we obtain obs ∗ Ts

St EB Σb(t) = ∆Sobs[ρS(t)] + ∆Sobs [ρB(t)] ≥ 0, (49) This constitutes a microscopic derivation of Clauisus’ in- equality (2c). It extends an earlier analysis [76] by not which is the microscopic analogue of the phenomenolog- assuming the bath to be at equilibrium at each time step. ical second law (2b). It follows that It is instructive to discuss the consequences of the iden- St,EB tificationdQ ¯ (s) ≡ −dUB(s) further. First, one finds Σb(t) − Σa(t) = Iobs [ρSB(t)] ≥ 0. (50)

∗ EB The relevance of the mutual information term for the Σc(t) − Σb(t) = SB(βt ) − Sobs [ρB(t)] ≥ 0. (54) (thermo)dynamics of open quantum systems still needs further elucidation. In general, it obeys the inequalities As expected, this difference is zero if the bath is also at later times well described by an equilibrium state St,EB ∗ 0 ≤ Iobs [ρSB(t)] ≤ IS:B[ρSB(t)] ≤ 2 ln dim HS, (51) with temperature Tt . In general, however, Σc overes- timates Σb by neglecting potential nonequilibrium re- where we assumed the Hilbert space dimension dim HS sources stored in the distribution of bath energies EB of the system to be smaller than the Hilbert space dimen- at time t. Such nonequilibrium resources were indeed sion of the bath. Furthermore, numerical results [74, 75] recently studied in Refs. [77, 78]. suggest that the mutual information can be large in view Second, the present identification of heat forces us, by of these bounds: if st denotes measurements of the sys- virtue of the first law (1), to identify the internal energy tem energy and if the system is undriven (λt = con- of the open system as stant), then—as a result of the microscopic conservation of energy—strong system-bath correlations can build up. US(t) ≡ trSB{[HS(λt) + VSB]ρSB(t)}. (55) But if the system is driven, correlations seem to dimin- ish [75] and the entropy production will be dominated In fact, based on this definition it is easy to microscop- EB by changes in the bath entropy ∆Sobs [ρB(t)], which can ically verify the first law ∆US(t) = Q(t) + W (t). Here, grow proportional with time t in contrast to the mutual R t Q(t) = 0 dQ¯ (s) is the total heat flow into the system information [74]. and the mechanical work W (t) was defined in Eq. (11). We concluded in Sec.III that this definition of work is unambiguous for a driven system. Using the form of the C. Heat, internal energy and Clausius inequality system-bath Hamiltonian (6), Eq. (11) simplifes to

We now derive Clausius’ inequality (2c). It quanti- Z t   ∂HS(λs) fies the entropy production of a system undergoing a W (t) = dstrS ρS(s) . (56) ∂s nonequilibrium process while being in contact with a 0 bath, whose temperature changes due to the flow of heat. The definition (55) seems to naturally follow in the Recall Sec.VC where we defined a nonequilibrium present framework and it has also appeared in different temperature for any isolated system. This definitions ∗ earlier approaches [79–85]. It is, however, important to also applies equally well to any subsystem. Thus, let Tt point out that Eq. (55) can not be computed by only denote the time-dependent nonequilibrium temperature knowing the reduced system state ρ (t) as it includes the of the bath, obtained from Eq. (37) by adding a sub- S interaction Hamiltonian VSB, which is a disadvantage of script B to all quantities. Then, from Eqs. (41) and (45) the present definition. Only in the weak coupling regime, we infer that the change in bath entropy is where the effect of VSB is assumed negligible compared Z to HS and HB, we have US(t) ≈ trS{HS(λt)ρS(t)}. EB EB ∗ dUB(s) ∆Sobs [ρB(t)] = Sobs [ρB(t)] − SB(βt ) + ∗ In fact, beyond weak coupling the correct definition Ts Z of heat and internal energy is fiercly debated. Different dUB(s) proposals exist for quantum systems [86–97] and also the ≤ ∗ . (52) Ts classical case remains debated [98–106]. The goal of this 13 tutorial is not to advertise Eq. (55) as the only mean- since ∆UB(t) is itself of order . ingful candidate. We believe, however, that the present Thus, for a weakly perturbed bath we can conclude framework helps to advance the debate for two reasons. Q(t) First, we established a link between heat and entropy St Σc(t) ≈ Σd(t) = ∆Sobs[ρS(t)] − ≥ 0. (61) changes in the bath in Eq. (52). It indicates that heat T0 remains a meaningful concept even if the bath is not at This finishes our derivation of the hierarchy of sec- equilibrium, but it no longer is the only contribution to ond laws. Since the above inequality holds for all sys- the change in bath entropy. This important link has not tem coarse-grainings {|stihst|}, we can also choose it been established in previous approaches. to coincide with the eigenbasis of ρS(t). Then, we get Second, our identification of heat results from our ∆SSt [ρ (t)] = ∆S [ρ (t)] and the second law becomes choice to view the coarse-grained energy of the bath as obs S vN S the relevant variable. We emphasized already that dif- Q(t) ferent, more refined choices are possible. This might in Σc(t) ≈ Σd(t) = ∆SvN[ρS(t)] − ≥ 0. (62) T0 particular be relevant at strong coupling. Checking which of the many different proposals above can be explained This expression of the second law if often found in the by using observational entropy with respect to a different context of open quantum system theory [7, 31]. We con- coarse-graining would add further appeal and additional clude this section by putting our results in context of two insights to them. other findings. First, Eq. (62) is often written for an infinitesimal time step as D. Weakly perturbed bath d Q˙ (t) Σ˙ d(t) = SvN[ρS(t)] − , (63) We complete the derivation of the laws of thermody- dt T0 namics by deriving Eq. (2d). From the phenomenological where Σ˙ d(t) is the entropy production rate and Q˙ (t) = description we expect Eq. (2d) to emerge out of the sec- −dUB(t)/dt. Whereas the non-negativity of Eq. (62) is ond law (2c) whenever the bath can be approximated as guaranteed, the non-negativity of the entropy produc- static such that its thermodynamic parameters do not tion rate Σ˙ d(t) is not. However, one has Σ˙ d(t) ≥ 0 if change. This is often justified if the bath is very large the dynamics of the open system state ρS(t) is described and the system very small. by the so-called Born-Markov-secular master equation, To reflect this idea in our framework, we start by ex- which has become—despite its many approximations panding the probabilities p (t) to measure the bath en- EB involved—a widely used tool in the field [7, 31, 107]. ergy EB at time t as Similar approximations can be also used to derive a master equation for the probabilities p (t). Then, p (t) = π (β )[1 + q (t)]. (57) st,EB EB EB 0 EB in analogy to the previous case, one can confirm that St,EB −β0EB Σ˙ a(t) = dS [ρSB(t)]/dt ≥ 0 [75]. We remark that Here, πEB (β0) = VEB e /ZB(β0) denotes the ini- obs Markovianity alone is not sufficent to guarantee the non- tial probability to measure EB and qEB (t) ∈ [−1, 1] is a correction term, which, due to normalization, satisfies negativity of the entropy production rate in general [96]. P π (β )q (t) = 0. A weakly perturbed bath is now Second, Eq. (62) emerged out of the more general ver- EB EB 0 EB described by the situation where the parameter  is small sion (53) of the second law for a weakly perturbed bath enough such that terms of order O(2) are negligible. in unison with the phenomenological theory. Somewhat We now apply this idea to compute the change in bath remarkably, it is possible to show that Eq. (62) always entropy. By using Eq. (57), we get holds for the initial condition (44), regardless of how far the bath is pushed away from equilibrium [80, 81, 83– EB 2 85]. To distinguish this case from the regime of validity ∆Sobs (t) = β0∆UB(t) + O( ), (58) of Eq. (62), we denote this inequality by where the change in coarse-grained bath energy is ˜ Q(t) X Σd(t) ≡ ∆SvN[ρS(t)] − ≥ 0. (64) ∆UB(t) = EB[pEB (t) − πEB (β0)] T0 EB Importantly, for a bath far from equilibrium it has not X (59) =  EBπEB (β0)qEB (t). been possible to link Q(t)/T0 to an entropy change.

EB Strictly speaking, Eq. (64) therefore coincides with the second law only if the bath is weakly perturbed, whereas Likewise, Eq. (57) also implies that the final nonequilib- Eq. (53) is consistent with the second law for a larger rium temperature must be -close to the initial tempera- class of transformations not restricted to the isothermal ∗ ture: |Tt − T0| = O(). We then obtain case. Furthermore, it was recently found [108] that Σ˜ d(t) Z t is an upper bound on the entropy production since dUB(s) ∆UB(t) 2 ∗ ds = + O( ) (60) ˜ ∗ 0 Ts T0 Σd(t) − Σc(t) = D[πB(βt )kπB(β0)] ≥ 0, (65) 14 which has consequences for the efficiency of heat engines large. In this case, the system reaches a periodic steady in contact with finite baths [108]. state and constantly dissipates energy into the bath. We therefore expect that the entropy production scales with time such that Σa(t) ∼ t. Our conjecture is that the lines VII. FURTHER EXTENSIONS in Eq. (67) have been ordered in decreasing relevance:

Z dQ¯ (s) We here extend the previous framework to cover a St ∆Sobs[ρS(t)] − ∗ (68) larger class of initial states (Sec.VIIA), multiple baths Ts

(Sec.VIIB) and particle transport (Sec.VIIC). EB ∗ EB ∗  Sobs [ρB(t)] − SB(βt ) − Sobs [ρB(0)] + SB(β0 )

S0,EB St,EB  I [ρSB(0)] − I [ρSB(t)] A. Generalized initial states obs obs We justify this conjecture as follows. First, if the As promised above, the second law can be shown to system reaches a periodic steady state maintained by a strictly hold for a much larger class of initial states than constant uptake of mechanical work, the total heat flux those described by Eq. (44). In fact, by Lemma IV.5 we Q(t) ∼ t has to scale proportional to t by the first law. know that Eq. (46) holds for all initial states satisfying St Thus, although ∆S [ρS(t)] becomes negligible as it is S0,EB obs Sobs [ρSB(0)] = SvN[ρ(0)]. By Lemma IV.4 and by bounded by ln(dim HS), the first line in Eq. (67) is ex- choosing the coarse-graining from the previous section, pected to scale as t. Furthermore, the last line can not these states are given by scale with t and must reach a constant, which is at most X 2 ln(dim HS). Therefore, it is negligible for long times. ρSB(0) = ps0,EB (0)|s0ihs0| ⊗ ωB(EB), (66) The really challenging question concerns the second line. s0,EB We can not exclude that this contribution scales with t, albeit we believe that its rate of growth should be in most with arbitrary probabilities p (0). This generalizes √ s0,EB cases sublinear (e.g., t). This believe is motivated by the previous initial state (44) in two ways. First, the the fact that the microscopic dynamics of a typical heat bath need not be described by a Gibbs state—a micro- bath are often very complex, characterized by (close to) canonical state or any convex combination thereof can chaotic behaviour, such that it becomes hard to distin- also be considered. Second, the initial state does not guish its true state from an idealized Gibbs ensemble. need to be decorrelated. It can have arbitrary classical This idea is indeed supported by research on equilibration correlations with respect to the chosen coarse-graining. and thermalization in isolated many-body systems [109– In view of what we said at the end of Sec.VIC, it is 114]. In any case, while the behaviour of the first and also possible to imagine coarse-grainings different from third line in Eq. (67) appears universal, the behaviour of the one chosen in Sec.VIA. In particular, by going be- the second line will be model-dependent. yond a coarse-graining with a system-bath tensor product structure as considered here, quantum correlations could be included in the description. B. Multiple baths Finally, we explicitly decompose the entropy produc- tion (46) for an initial state of the form (66) into all its In many relevant situations, in particular to study contributions: transport process, the open system is coupled to multiple Z dQ¯ (s) baths, labeled by ν ∈ {1, . . . , n}, see Fig.4 for a sketch. Σ (t) = ∆SSt [ρ (t)] − (67) a obs S ∗ The system-bath Hamiltonian (6) is then generalized to Ts EB ∗ EB ∗ + Sobs [ρB(t)] − SB(βt ) − Sobs [ρB(0)] + SB(β0 ) X h (ν) (ν)i HSB(λt) = HS(λt) + VSB + HB . (69) S0,EB St,EB + Iobs [ρSB(0)] − Iobs [ρSB(t)] ν The first line describes the Clausius contribution to the We denote the global system-bath state at time t by entropy production, obtained by neglecting system-bath ρSB(t) and the marginal state of bath ν by ρν (t). In the correlations and by assuming the bath to be well de- following, we show that our framework can be extended scribed by its effective temperature only. The second line to this situation in a straightforward way. takes into account nonequilibrium features of the bath First, as our relevant coarse-graining we choose state in comparison with a fictitious Gibbs ensemble at {|stihst| ⊗ ΠE1 ⊗ · · · ⊗ ΠEn }, where ΠEν corresponds to a the same energy. The third line quantifies the influence coarse-grained measurement of the energy of bath ν. For of system-bath correlations on the second law. notational simplicity, we write E ≡ (E1,...,En). Then, To estimate the influence of each of these terms, we the observational entropy is generalized to consider a small system, which is coupled to a large bath p (t) and subject to a, say, periodic driving protocol with pe- St,E X st,E Sobs [ρSB(t)] = − pst,E(t) ln (70) VE riod τ. Furthermore, we consider times t = nτ with n st,E 15

Q P (ν) with VE ≡ ν trBν {ΠEν }. The initial state of our setup where US(t) ≡ trSB{[HS(λt) + ν VSB ]ρSB(t)} general- is described by a generalization of the initial state (44), izes Eq. (55) and W (t) is still given by Eq. (56). Finally, we consider the case of very large baths or, ρSB(0) = ρS(0) ⊗ π1(β1) ⊗ · · · ⊗ πn(βn), (71) alternatively, times t that a short enough such that the baths are only weakly perturbed. Then, (T ∗) ≈ T and assuming each bath to be prepared in a Gibbs ensemble t ν ν Eq. (75) reduces to at inverse temperature βν . Clearly, from Sec.VIIA we know that a larger class of initial states is admissible. X Qν (t) Σ (t) = SSt [ρ (t)] − ≥ 0. (77) From these considerations, a non-negative change in d obs S T thermodynamic entropy quantified by Eq. (70) follows: ν ν Of course, one could also imagine situations where the Σ (t) = SSt,E[ρ (t)] − SS0,E[ρ (0)] ≥ 0. (72) a obs SB obs SB baths have different sizes and need to be treated accord- Since the initial state is decorrelated, we also confirm ingly. Moreover, in the long run and if the bath size is finite, one expects all baths to equilibrate to the same St X Eν temperature. This behaviour is captured by Eq. (75), Σb(t) = ∆Sobs[ρS(t)] + ∆Sobs[ρν (t)] ≥ 0. (73) ν but not by Eq. (77). Importantly, the difference C. Particle exchanges St X Eν Σb(t) − Σa(t) = Sobs[ρS(t)] + Sobs[ρν (t)] ν (74) Energy is not the only quantity, which gets exchanged St,E − Sobs [ρSB(t)] between different subsystems. Also particles are ex- changed and the most relevant particle species for cur- is now given by the non-negative total information, rent nanotechnological applications are probably elec- which—even if dim H is small—can be large as it also S trons. To avoid notational clutter, the exposition below quantifies the correlations between the different baths. is adapted to the case of a single particle species only. Next, we use our definition of temperature, Eq. (37), We start with equilibrium considerations for an iso- for each bath separately. Then, if (T ∗) describes the t ν lated system with Hamiltonian H and particle num- effective nonequilibrium temperature of bath ν at time t, ber operator Nˆ. We use a ‘hat’ for the particle num- manipulations identical to those of Sec.VIC yield ber operator to distinguish it from its expectation value Z N ≡ tr{Nρˆ }. At equilibrium, the theory is constructed X dQ¯ ν (s) Σ (t) = SSt [ρ (t)] − ≥ 0. (75) c obs S (T ∗) by using the grand canonical ensemble ν s ν e−β(H−µNˆ) Here,dQ ¯ ν (t) = −dUBν (t) is minus the infinitesimal Ξ(β, µ) ≡ , (78) change in energy of bath ν. This identification of heat Z(β, µ) implies the first law where µ is the chemical potential and Z(β, µ) ≡ X −β(H−µNˆ) ∆U (t) = Q (t) + W (t), (76) tr{e } the grand canonical partition function. S ν An infinitesimal change in the equilibrium entropy ν S(β, µ) = SvN[Ξ(β, µ)] can be expressed as 1 µ dS = dU − dN. (79) T T In unison with our definition of an effective nonequi- librium temperature, we can also introduce an effective chemical potential for any state ρ by demanding that its particle number expectation value matches the one of the grand canonical ensemble. Thus, the following two equations determine β∗ and µ∗:

∗ ∗ Figure 4. Sketch of a system in contact with an infinite bath U = tr{Hρ} ≡ tr{HΞ(β , µ )}, (80) at temperature T1 and chemical potential µ1, a finite bath at N = tr{Nρˆ } ≡ tr{NˆΞ(β∗, µ∗)}. (81) T2 and µ2 and a work reservoir (sketched by a weight m). The system experiences a change in internal energy ∆US and sys- To connect the equilibrium entropies of two states with tem entropy ∆SS due to heat flows Q1 and Q2 from the bath ∗ ∗ ∗ ∗ (βt , µt ) and (β0 , µ0), we find from Eq. (79) and in accor- and the mechanical work W supplied to it. Since the second dance with Eq. (40) that bath is finite, its temperature and chemical potential change with time t whereas T1 and µ1 remain unchanged (sketched Z ∗ ∗ ∗ ∗ ∗ dU(s) − µsdN(s) on the right). S(βt , µt ) − S(β0 , µ0) = ∗ . (82) Ts 16

Finally, we define observational entropy with respect to Again, it is possible to quantify the difference between a coarse-graining of energy and particles. Since [H, Nˆ] = Σa(t) and Σb(t) by the total information and between 0, we can jointly measure both quantities. Then, Σb(t) and Σc(t) by nonequilibrium features in the bath distribution. Finally, by considering the limit of a weakly E,N X ∗ ∗ Sobs (ρ) = pE,N (− ln pE,N + ln VE,N ). (83) perturbed bath described by (Tt )ν ≈ Tν and (µt )ν ≈ µν , E,N we obtain from Eq. (91)

Each quantity is defined by analogy with the previous Q (t) Σ (t) ≈ Σ (t) = ∆SSt [ρ (t)] − ν ≥ 0 (92) case: pE,N = tr{ΠEΠN ρ} and VE,N = tr{ΠEΠN }. c d obs S Tν Note that both, ΠE and ΠN , describe in general again a measurement with a finite resolution or uncertainty. with Qν (t) = −[∆Uν (t) − µν ∆Nν (t)]. Equation (92) As before, however, we demand that the uncertainty is quantifies the entropy production for transport processes, small enough to be consistent at equilibrium such that where the baths are kept at a fixed temperature and E,N Sobs [Ξ(β, µ)] ≈ S(β, µ). chemical potential. The scope of Eq. (91) is wider and After these preliminary consideration, we can now re- captures dynamical features in the bath, already ob- turn to the case of a system coupled to multiple baths, served in experiments [115, 116], in a self-contained way. exchanging energy and particles with them, see Fig.4. Equations (79) and (82) suggest to define the infinitesi- mal heat flux from bath ν at time t as VIII. FLUCTUATION THEOREMS ∗ dQ¯ ν (t) ≡ −[dUν (t) − µν dNν (t)], (84) Fluctuation theorems present important refinements even if the state ρν (t) of bath ν is out of equilibrium. on our view on the second law. They are exact rela- From this definition it follows that the first law needs to tions, which constrain the fluctuations in thermodynam- be generalized to ics quantities such that, among other consequences, the second law can be formulated as an equality. Fluctuations X ∆US(t) = Qν (t) + W (t) + Wchem(t). (85) theorems play an important role in classical nonequilib- ν rium statistical mechanics [117–119], stochastic thermo- dynamics [29, 30] and quantum thermodynamics based Here, a new contribution appears known as chemical R on the so-called ‘two-point measurement scheme’ [120]. work. It is defined as Wchem(t) = dW¯ chem(s) with The goal of this section is to show that the entropy P ∗ ∗ dW¯ chem(s) = ν (µs)ν dNν , where (µs)ν denotes the production as defined by the change of observational en- chemical potential of bath ν at time s. This form of tropy also satisfies fluctuations theorems. We do so in an work is associated with particle exchanges and quantifies abstract way as in Sec.IVC, assuming the entire system the ability of, e.g., electrons to charge a battery, whose is isolated and evolves according to Eq. (5). We believe energy can be converted back into mechanical work. that the derivation below captures the essence of fluctua- Finally, by choosing the coarse-graining tion theorems from a technical point of view. Particular applications can be then worked out by following the lines {|s ihs | ⊗ Π Π ⊗ · · · ⊗ Π Π }, (86) t t E1 N1 En Nn of Secs.V,VI andVII, which we will not do here. observational entropy becomes To approach the problem, we first define fluctuations of observational entropy. From definition (21) we see that

St,E,N X pst,E,N(t) observational entropy can be written as an average of Sobs [ρSB(t)] = − pst,E,N(t) ln . (87) Vst,E,N(t) st,E,N sobs(x, t) ≡ − ln px(t) + ln Vx, (93) The definition of each term should be obvious as it follows by analogy with the previous cases. Furthermore, by where the average is carried out with respect to the prob- restricting our considerations to the initial state abilities px(t):

X X ρSB(0) = ρS(0) ⊗ Ξ(β1, µ1) ⊗ · · · ⊗ Ξ(βn, µn), (88) Sobs[ρ(t)] = px(t)sobs(x, t). (94) x the following hierarchy of second laws also follows by analogy: Thus, sobs(x, t) is a random variable, whose construction requires knowledge of the probabilities px(t). St,E,N 0 ≤ Σa(t) ≡ ∆Sobs [ρSB(t)] (89) Next, we look at fluctuations in the change of sobs(x, t). X To this end, we use the two-point measurement scheme, ≤ Σ (t) ≡ ∆SSt [ρ (t)] + ∆SEν ,Nν [ρ (t)] (90) b obs S obs ν first used in Refs. [69, 121, 122]. Imagine that we perform ν Z initially a measurement of X0, giving rise to outcome x0, S dQ¯ ν (s) t and finally a measurement of Xt with outcome xt. The ≤ Σc(t) ≡ ∆Sobs[ρS(t)] − ∗ . (91) (Ts )ν fluctuations of the random variable (93) in this process 17 are time-reversed process is then defined by starting with a measurement of ΠΘ , followed by an evolution accord- xt ∆sobs(xt, t; x0, 0) ≡ sobs(xt, t) − sobs(x0, 0) Θ ing to UΘ(t, 0), and ending with a measurement of Πx . p (0)V (95) 0 = ln x0 xt . The probability to observe the sequence of measurement Vx0 pxt (t) results (x0, xt) in the time-reversed process is

† Moreover, the probability to observe outcomes (xt, x0) is ptr ≡ tr{ΠΘ U (t, 0)ΠΘ ρ (t)ΠΘ U (t, 0)}, (100) x0,xt x0 Θ xt tr xt Θ p = tr{Π U(t, 0)Π ρ(0)Π U †(t, 0)}. (96) xt,x0 xt x0 x0 where ρtr(t) is the initial state in the time-reversed pro- P cess. Note that we count time ‘backwards’ in the time- Finally, let us denote by h...i = . . . px ,x an av- xt,x0 t 0 reversed process, starting at t and ending at 0, which erage over this process. is convenient from a notational perspective. We empha- Then, if the condition SX0 [ρ(0)] = S [ρ(0)] is obs vN size, however, that in any experimental realization of that satisifed (which we also assumed to derive our second process time runs ‘as always’ forward. laws, cf. Lemma IV.5), we find the following integral fluc- As done multiple times before, we assume again that tuation theorem: the initial states in the forward and time-reversed pro- D E cess obey SX0 [ρ (0)] = S [ρ (0)] and SXt [ρ (t)] = e−∆sobs(xt,t;x0,0) = 1, (97) obs fw vN fw obs tr SvN[ρtr(t)]. This implies that (see Lemma IV.4) ρfw(0) = P fw P tr Θ p Πx /Vx and ρtr(t) = p Π /Vx for arbi- where here and in the following we tacitly assume p 6= 0 x0 x0 0 0 x0 xt xt t x0 trary probabilities pfw and ptr . We now make the impor- for all x to avoid ‘dividing by zero,’ which is related to x0 xt 0 tant choice that the phenomenon of absolute irreversibility [123]. The proof goes as follows. From Eq. (95) and ptr = pfw, (101) the assumption ρ(0) = P p Π /V , which implies xt xt x0 x0 x0 x0 Πx0 ρ(0)Πx0 = px0 Πx0 /Vx0 , we get the chain of equali- i.e., the initial probabilities in the time-reversed process ties: coincide with the final measurement statistics of the for- D E X px ward process. Note that this does not imply that ρtr(t) is e−∆sobs(xt,t;x0,0) = tr{Π U(t, 0)Π U †(t, 0)} t xt x0 the time-reversed final state of the forward process, i.e., Vxt xt,x0 ρ (t) 6= Θρ (t)Θ−1 with ρ (t) = U(t, 0)ρ (0)U †(t, 0). p tr fw fw fw X † xt Taken together, these assumptions and our special choice = tr{Πxt U(t, 0)U (t, 0)} Vxt xt imply the central relation px X t fw fw tr = tr{Πxt } = 1. (98) p = exp[∆s (x , t; x , 0)]p . (102) xt,x0 obs t 0 x0,xt Vxt xt This result follows from the relation P † For the last steps we used x Πx0 = 1, U(t, 0)U (t, 0) = P 0 † 1, tr{Πxt } = Vxt , and x pxt = 1. tr{Πx0 U (t, 0)Πxt U(t, 0)} t y (103) By using the inequality e ≥ 1 + y for y ∈ R, we Θ Θ † = tr{Π UΘ(t, 0)Π U (t, 0)} confirm that the integral fluctuation theorem (97) implies x0 xt Θ the formal second law (31). An even more general class which is a consequence of Eqs. (A2) and (A5) and of integral fluctuation theorems was derived in Ref. [124]. † tr{Πx0 U (t, 0)Πxt U(t, 0)} ∈ R. Finally, there is also a detailed fluctuation theorem, Now, we return to Eq. (99). From Eq. (102) we imme- which makes the connection with time-reversal symme- diately obtain try (see AppendixA) particularly transparent and im- plies the integral fluctuation theorem. To derive it, we ∆s X fw tr Pfw(∆s) = e δ[∆s − ∆s (xt, t; x0, 0)]p , start with the probability P (∆s) to observe a change in obs x0,xt observational entropy ∆s in the forward process: xt,x0 (104) X fw fw where we used the Dirac-delta function to pull the fac- Pfw(∆s) = δ[∆s − ∆s (xt, t; x0, 0)]p , (99) ∆s obs xt,x0 tor e out of the summation. Next, we note that xt,x0 fw fw ∆sobs(xt, t; x0, 0) = −∆sobs(x0, 0; xt, t), which implies where δ(·) denotes the Dirac-delta function. Here, we X P (∆s) = e∆s δ[∆s + ∆sfw (x , 0; x , t)]ptr have been particularly cautious and indicated by ‘fw’ fw obs 0 t x0,xt quantities associated to the forward process. xt,x0 ∆s Next, we introduce the ‘time-reversed’ process. To this ≡ e Qtr(−∆s). (105) end, we use time-reversal symmetry (see AppendixA) Θ and denote the time-reversed projectors by Πx ≡ Here, Qtr(∆s) is the probability that the quantity −1 fw ΘΠxΘ and by UΘ(t, 0) the unitary time evolution op- ∆sobs(x0, 0; xt, t) takes on the value ∆s in the time- erator associated to the time-reversed dynamics. The reversed process with respect to the choice (101). This 18 choice also reveals that tribution of entropy production during the time-reversed process and we find the detailed fluctuation theorem ptr (0)V tr x0 xt ∆s (x , 0; x , t) = ln (106) Pfw(∆s) obs 0 t tr = e∆s. (107) Vx0 px (t) t Ptr(−∆s) pfw(0) x0 fw = ln tr + ∆sobs(x0, 0; xt, t). px (0) 0 IX. CONCLUDING REFLECTIONS

In general, pfw(0) 6= ptr (0) and then Q (∆s) can not be x0 x0 tr This tutorial was devoted to the understanding, deriva- interpreted as the distribution of the stochastic entropy tion and quantification of the laws of thermodynamics in production associated to the time-reversed process. In- open and isolated quantum systems based on microscopic stead, Qtr(∆s) quantifies the distribution of entropy pro- definitions for internal energy, heat, work, (thermody- duction associated to the forward dynamics, according to namic) entropy, temperature and chemical potentials. a fixed choice of pfw(0), but which is observed during the x0 Summarizing the situation for open systems without par- time-reversed process. Nevertheless, Eq. (105) implies ticle exchanges, the first law reads ∆US(t) = Q(t)+W (t). the integral fluctuation theorem (97) and, therefore, pro- Moreover, using Eq. (43) as our entropy definition and vides a stronger constraint on Pfw(∆s) than Eq (97). Eq. (37) as our definitions for the (nonequilibrium) tem- Finding the conditions for which pfw(0) = ptr (0) holds perature of the bath, we found that the second law for x0 x0 is not simple in our general setting and involves addi- a system initially decorrelated from a thermal bath can tional assumptions (e.g., steady state regime or relax- be summarized by the following hierarchy of inequalities, ation to equilibrium after the driving) [29, 30]. However, where each member reflects the degree of control or in- if pfw(0) = ptr (0), then Q (∆s) ≡ P (∆s) is the dis- formation taken into account in an experiment: x0 x0 tr tr

St,EB 0 ≤ ∆Sobs [ρSB(t)] most general version of the second law (108)

St EB St,EB ≤ ∆Sobs[ρS(t)] + ∆Sobs [ρB(t)] disregard SB correlations Iobs [ρSB(t)] (109) Z St dQ¯ (s) ∗ EB ≤ ∆Sobs[ρS(t)] − ∗ disregard noneq. bath distribution SvN[πB(βt )] − Sobs [ρB(t)] (110) Ts

St Q(t) ∗ ≤ ∆Sobs[ρS(t)] − disregard finite-size effects D[πB(βt )kπB(β0)] (111) T0

This exactly matches the hierarchy of phenomenological present approach is suitable to study a variety of ther- second laws (2a), (2b), (2c) and (2d) if one identifies ob- modynamic processes at the nanoscale and can be readily servational entropy with thermodynamic entropy, as also applied to many experiments. done in Refs. [12, 13, 44, 55, 57, 58, 61–65, 73, 75, 108]. Thus, by starting with a microscopic definition of ther- Of course, not in every experiment will it be possible to modynamic entropy, a conceptually clear and consistent measure the variables or observables that we have cho- approach emerges, which covers diverse applications such sen above to derive the second law. Moreover, even if as multiple baths, initially correlated or non-Gibbsian it is possible to measure those observables, the coarse- bath states, small baths with changing temperatures, etc. graining might not be ‘fine’ enough and deviations of the actual initial state from our choice above, Eqs. (44) In fact, our growing nanotechnological abilities also en- or (66), could become visible. In these cases, there is hance our abilities to control and measure a bath. Finite no guarantee that the change in observational entropy size effects and changing thermodynamic parameters are remains always non-negative and satisfies a second law. already reality in experiments [115, 116, 125–130]. Fur- Still, we believe that one should not be afraid of such thermore, ultrasensitive thermometers were developed to ‘violations’ of the second law. Instead, one should view track small changes in bath energies [131–134]. Observa- them as a welcome feature as they reveal something un- tional entropy explicitly takes into account experimental expected about the experimental setup. To quote Jaynes (in)capabilities in its definition from the start. Further- again [42]: “recognizing this should increase rather than more, notice that the initial states chosen in this tutorial, decrease our confidence in the future of the second law, e.g., in Eqs. (44) or (66) or see Lemma IV.4, satisfy the because it means that if an experiment ever sees an ap- requirement that an initial measurement of the chosen parent violation, then instead of issuing a sensational an- coarse-graining does not disturb the state. Hence, the nouncement, it will be more prudent to search for that 19 unobserved degree of freedom.” requires further research. It also seems that the current Despite experimental applications, we believe that also framework could inspire research in open quantum sys- much fruitful theoretical work lies ahead of us. For in- tem theory (see, e.g, Ref. [75]) and it has much poten- stance, an observer’s (in)capabilities to precisely know or tial to be fruitfully combined with methods reviewed in control an experimental setup are also a central element Refs. [109–114] to study the equilibration and thermal- of the resource theory of thermodynamics [112, 135–137]. ization dynamics of isolated many-body systems. In there, additional constraints on the global unitary dy- Finally, one can question whether—out of the plethora namics are imposed, for instance, by demanding that of possible candidates—our choice to use observational [U(t, 0),HS + HB] = 0. In the present approach, we entropy as a microscopic definition for thermodynamic have put no constraints on the dynamics, but rather fo- entropy was correct. We believe that this is not definitely cused on constraints on the available knowledge. For answered by the present tutorial, but we also believe that instance, if the bath energy is only known up to a small it has added significant appeal to this definition. There- uncertainty, it is experimentally impossible to determine fore, it seems that the final answer to that question can whether [U(t, 0),HS +HB] = 0 strictly holds. Combining not be too far from the present considerations. both aspects could prove very fruitful to equip the present Thus, to conclude, observational entropy is a versatile framework with more predictive power while making the concept, which provides a link between problems studied resource theory approach more applicable in practice. In in the field of equilibration and thermalization in isolated fact, the latter seems still in search of practical experi- quantum systems, quantum thermodynamics and open mental applications [138]. quantum systems theory. Therefore, it has the potential It is also desirable to extend the present approach in to provide an overarching framework for many problems other directions. For instance, the role of non-commuting studied in nonequilibrium statistical mechanics. coarse-grainings does not yet appear to be fully under- stood. Moreover, the tutorial focused on ‘two-time statis- tics’ characterized by a non-disturbing initial and a fi- Acknowledgements nal measurement. It remains unclear what is the ef- fect of multiple sequential measurements [139], but see We are grateful to Kavan Modi, Juan Parrondo, Fe- Ref. [60] for preliminary results. It is also interesting lix Pollock, Andreu Riera-Campeny, Dominik Safr´anek,˘ to extend the present framework to more generalized Anna Sanpera, and Joan Vaccaro for many interest- measurements characterized by positive operator-valued ing discussions. We also thank Miquel Saucedo Cuesta measures (‘POVMs’). In fact, strict projective measure- and Pablo Torron Perez for useful comments on the ments are hard to realize in an experiment. More likely is manuscript. PS further acknowledges various stimulat- that a measurement outcome x0 corresponds to applying ing discussions with Massimiliano Esposito about the 0 a Gaussian weight of projectors Πx fixed around x ≈ x . nature of entropy production over the years. The au- Interestingly, for an arbitrary set of POVM elements thors were partially supported by the Spanish Agencia P {Px}, which always satisfy x Px = 1, the main defini- Estatal de Investigaci´on(project PID2019-107609GB-I00 tion (21) of observational entropy remains: the probabil- and IJC2019-040883-I), the Spanish MINECO (FIS2016- ity to observe outcome x is given by px = tr{Pxρ} and the 80681-P, AEI/FEDER, UE), and the Generalitat de volume term becomes Vx = tr{Px}. Therefore, it seems Catalunya (CIRIT 2017-SGR-1127). PS is financially that the same qualitative picture should emerge, but this supported by the DFG (project STR 1505/2-1).

[1] Sometimes it is asserted that thermodynamics played [2] Michael Flanders and Donald Swann, “First and Second an important role for the industrial revolution to de- Law,” in At the Drop of Another Hat (Parlophone Ltd., sign efficient heat engines. Historically speaking, this is 1964) 2nd ed. incorrect. The industrial revolution is associated with [3] D. Kondepudi and I. Prigogine, Modern Thermodynam- the period from 1760 to (at most) 1840 (the steam en- ics: From Heat Engines to Dissipative Structures (John gine of Watt was introduced in 1776). The first mod- Wiley & Sons, West Sussex, 2007). ern work on thermodynamics is perhaps due to Carnot [4] R. Clausius, “Ueber verschiedene f¨urdie Anwendung be- in 1824, who, however, was not read by his contem- queme Formen der Hauptgleichungen der mechanischen poraries. The first law of thermodynamics was estab- W¨armetheorie,” Ann. Phys. 201, 353–400 (1865). lished around 1850 and the modern formulation of the [5] I. Bloch, J. Dalibard, and W. Zwerger, “Many-body second law goes back to Clausius in 1865. Even then, physics with ultracold gases,” Rev. Mod. Phys. 80, 885– however, it took time until engineers were inspired by 964 (2008). theoretical insights from thermodynamics. To the best [6] M. Lewenstein, A. Sanpera, and V. Ahufinger, Ultracold of our knowledge, Diesel (at the end of the 19th cen- Atoms in Optical Lattices: Simulating quantum many- tury) patented the first engine which was based on the body systems (Oxford University Press, Oxford, 2012). insight that a high temperature gradient increases the [7] H.-P. Breuer and F. Petruccione, The Theory of Open efficiency of the engine. Quantum Systems (Oxford University Press, Oxford, 20

2002). [30] C. Van den Broeck and M. Esposito, “Ensemble and tra- [8] I. de Vega and D. Alonso, “Dynamics of non-Markovian jectory thermodynamics: A brief introduction,” Physica open quantum systems,” Rev. Mod. Phys. 89, 015001 (Amsterdam) 418A, 6–16 (2015). (2017). [31] R. Kosloff, “Quantum thermodynamics: A dynamical [9] G. Lebon, D. Jou, and J. Casas-V´azquez, Under- viewpoint,” Entropy 15, 2100–2128 (2013). standing Non-equilibrium Thermodynamics: Founda- [32] S. Vinjanampathy and J. Anders, “Quantum thermo- tions, Applications, Frontiers (Springer, Berlin Heidel- dynamics,” Contemp. Phys. 57, 545–579 (2016). berg, 2008). [33] M. Gavrilov, R. Ch´etrite, and J. Bechhoefer, “Direct [10] N. Pottier, Nonequilibrium Statistical Physics - Linear measurement of weakly nonequilibrium system entropy Irreversible Processes (Oxford University Press, New is consistent with Gibbs-Shannon form,” Proc. Natl. York, 2010). Acad. Sci. USA 114, 11097–11102 (2017). [11] L. Boltzmann, “On Certain Questions of the Theory of [34] A. O. Orlov, C. S. Lent, C. C. Thorpe, G. P. Boech- Gases,” Nature 51, 413–415 (1895). ler, and G. L. Snider, “Experimental test of Landauer’s [12] J. von Neumann, “Beweis des Ergodensatzes und des H- principle at the sub-kbT level,” Jpn. J. Appl. Phys. 51, Theorems in der neuen Mechanik,” Z. Phys. 57, 30–70 06FE10 (2012). (1929). [35] A. B´erut, A. Arakelyan, A. Petrosyan, S. Ciliberto, [13] J. von Neumann, “Proof of the ergodic theorem and the R. Dillenschneider, and E. Lutz, “Experimental ver- H-theorem in quantum mechanics,” European Phys. J. ification of Landauer’s principle linking information H 35, 201–237 (2010). and thermodynamics,” Nature (London) 483, 187–189 [14] S. Goldstein, J. L. Lebowitz, R. Tumulka, and (2012). N. Zanghi, “Long-time behavior of macroscopic quan- [36] Y. Jun, M. Gavrilov, and J. Bechhoefer, “High- tum systems,” Eur. Phys. J. H 35, 173–200 (2010). precision test of Landauer’s principle in a feedback [15] S. Goldstein, T. Hara, and H. Tasaki, “The second law trap,” Phys. Rev. Lett. 113, 190601 (2014). of thermodynamics for pure quantum states,” arXiv: [37] J. P. S. Peterson, R. S. Sarthour, A. M. Souza, I. S. 1303.6393 (2013). Oliveira, J. Goold, K. Modi, D. O. Soares-Pinto, and [16] F. Jin, R. Steinigeweg, H. De Raedt, K. Michielsen, L. C. C´eleri,“Experimental demonstration of informa- M. Campisi, and J. Gemmer, “Eigenstate thermaliza- tion to energy conversion in a quantum system at the tion hypothesis and quantum Jarzynski relation for pure Landauer limit,” Proc. R. Soc. A 472, 20150813 (2016). initial states,” Phys. Rev. E 94, 012125 (2016). [38] J. Hong, B. Lambson, S. Dhuey, and J. Bokor, “Ex- [17] E. Iyoda, K. Kaneko, and T. Sagawa, “Fluctuation the- perimental test of Landauer’s principle in single-bit op- orem for many-body pure quantum states,” Phys. Rev. erations on nanomagnetic memory bits,” Sci. Adv. 2, Lett. 119, 100601 (2017). e1501492 (2016). [18] J. Gemmer, L. Knipschild, and R. Steinigeweg, “Com- [39] L. L. Yan, T. P. Xiong, K. Rehan, F. Zhou, D. F. ment on: ”Fluctuation theorem for many-body pure Liang, L. Chen, J. Q. Zhang, W. L. Yang, Z. H. Ma, quantum states” and reply to ”arxiv:1712.05172”,” and M. Feng, “Single-atom demonstration of the quan- arXiv: 1712.02128v2 (2017). tum Landauer principle,” Phys. Rev. Lett. 120, 210601 [19] J. L. Lebowitz, “Boltzmann’s Entropy and Time’s Ar- (2018). row,” Phys. Today 46, 32 (1993). [40] E. T. Jaynes, “Maximum-Entropy and Bayesian Meth- [20] T. M. Cover and J. A. Thomas, Elements of Information ods in Science and Engineering,” (Springer, Dordrecht, Theory (John Wiley & Sons, New York, 1991). 1988) Chap. The Evolution of Carnot’s Principle, pp. [21] M. A. Nielsen and I. L. Chuang, Quantum Computa- 267–281. tion and Quantum Information (Cambridge University [41] E. T. Jaynes, “Maximum Entropy and Bayesian Meth- Press, Cambridge, 2000). ods,” (Springer, Dordrecht, 1989) Chap. Clearing up [22] S. Mancini and A. Winter, A Quantum Leap in Infor- Mysteries; the Original Goal, pp. 1–27. mation Theory (World Scientific, Singapore, 2020). [42] E. T. Jaynes, “Maximum Entropy and Bayesian Meth- [23] H. Touchette, “Equivalence and nonequivalence of en- ods,” (Springer, Dordrecht, 1992) Chap. The Gibbs sembles: Thermodynamic, macrostate, and measure Paradox, pp. 1–21. levels,” J. Stat. Phys. 159, 987–1016 (2015). [43] S. Goldstein, J. L. Lebowitz, R. Tumulka, and [24] J. Dunkel and S. Hilbert, “Consistent thermostatistics N. Zangh`ı, “Statistical Mechanics and Scientific Ex- forbids negative absolute temperatures,” Nat. Phys. 10, planation,” (World Scientific, 2020) Chap. Gibbs and 67–72 (2014). Boltzmann Entropy in Classical and Quantum Mechan- [25] M. Campisi, “Construction of microcanonical entropy ics, pp. 519–581. on thermodynamic pillars,” Phys. Rev. E 91, 052147 [44]D. Safr´anek,A.ˇ Aguirre, and J. M. Deutsch, “Classical (2015). dynamical coarse-grained entropy and comparison with [26] E. Abraham and O. Penrose, “Physics of negative ab- the quantum version,” Phys. Rev. E 102, 032106 (2020). solute temperatures,” Phys. Rev. E 95, 012125 (2017). [45] J. W. Gibbs, Elementary principles in statistical me- [27] R. H. Swendsen, “Thermodynamics of finite systems: a chanics (Charles Scribner’s Sons, New-York, 1902). key issues review,” Rep. Prog. Phys. 81, 072001 (2018). [46] H. A. Lorentz (Teubner, Leipzig, 1906) Chap. Uber¨ [28] K. Sekimoto, Stochastic Energetics, Vol. 799 (Lect. den zweiten Hauptsatz der Thermodynamik und dessen Notes Phys., Springer, Berlin Heidelberg, 2010). Beziehung zu den Molekulartheorien, pp. 202–298. [29] U. Seifert, “Stochastic thermodynamics, fluctuation [47] P. Ehrenfest and T. Ehrenfest, “Begriffliche Grundlagen theorems and molecular machines,” Rep. Prog. Phys. der statistischen Auffassung in der Mechanik,” (Teub- 75, 126001 (2012). ner, Leipzig, 1911) pp. 3–90. 21

[48] A. Wehrl, “General properties of entropy,” Rev. Mod. [70] F. Ritort, “Resonant nonequilibrium temperatures,”J. Phys. 50, 221–260 (1978). Phys. Chem. B 109, 6787–6792 (2005). [49] S. Goldstein, J. L. Lebowitz, C. Mastrodonato, [71] R. S. Johal, “Quantum heat engines and nonequilibrium R.Tumulka, and N. Zhanghi, “Normal typicality and temperature,” Phys. Rev. E 80, 041119 (2009). von Neumann’s quantum ergodic theorem,” Proc. R. [72] U. Seifert, “Entropy and the second law for driven, or Soc. A 466, 3203–3224 (2010). quenched, thermally isolated systems,” Physica A 552, [50] M. Rigol and M. Srednicki, “Alternatives to Eigenstate 121822 (2020). Thermalization,” Phys. Rev. Lett. 108, 110601 (2012). [73] P. Strasberg, “Entropy production as change in obser- [51] P. Reimann, “Generalization of von Neumann’s Ap- vational entropy,” arXiv 1906.09933 (2019). proach to Thermalization,” Phys. Rev. Lett. 115, [74] K. Ptaszy´nskiand M. Esposito, “Entropy production 010403 (2015). in open systems: The predominant role of intraenvi- [52] R. C. Tolman, The Principles of Statistical Mechanics ronment correlations,” Phys. Rev. Lett. 123, 200603 (Oxford University Press, London, 1938). (2019). [53] D. Zubarev, V. Morozov, and G. R¨opke, Statistical Me- [75] A. Riera-Campeny, A. Sanpera, and P. Strasberg, chanics of Nonequilibrium Processes, Vol. 1 (Akademie “Quantum Systems Correlated with a Finite Bath: Verlag, Berlin, 1996). Nonequilibrium Dynamics and Thermodynamics,” PRX [54] W. Pauli, “Festschrift zum 60. Geburtstage A. Som- Quantum 2, 010340 (2021). merfeld,” (Hirzel, Leipzig, 1928) Chap. Uber¨ das H- [76] C. Jarzynski, “Microscopic analysis of Clausius-Duhem Theorem vom Anwachsen der Entropie vom Standpunkt processes,” J. Stat. Phys. 96, 415–427 (1999). der neuen Quantenmechanik, pp. 30–45. [77] R. S´anchez, J. Splettstoesser, and R. S. Whitney, [55] I. C. Percival, “Almost periodicity and the quantual h “Nonequilibrium system as a demon,” Phys. Rev. Lett. theorem,” J. Math. Phys. 2, 235 (1961). 123, 216801 (2019). [56] O Penrose, “Foundations of statistical mechanics,” Rep. [78] F. Hajiloo, R. S´anchez, R. S. Whitney, and Prog. Phys. 42, 1937–2006 (1979). J. Splettstoesser, “Quantifying nonequilibrium thermo- [57] V. Latora and M. Baranger, “Kolmogorov-Sinai En- dynamic operations in a multiterminal mesoscopic sys- tropy Rate versus Physical Entropy,” Phys. Rev. Lett. tem,” Phys. Rev. B 102, 155405 (2020). 82, 520–523 (1999). [79] I. M. Bassett, “Alternative derivation of the classical [58] M. Nauenberg, “The evolution of radiation toward ther- second law of thermodynamics,” Phys. Rev. A 18, 2356– mal equilibrium: A soluble model that illustrates the 2360 (1978). foundations of statistical mechanics,” Am. J. Phys. 72, [80] G. Lindblad, Non-Equilibrium Entropy and Irreversibil- 313–323 (2004). ity (D. Reidel Publishing, Dordrecht, Holland, 1983). [59] N. G. van Kampen, Stochastic Processes in Physics and [81] A. Peres, Quantum Theory: Concepts and Methods, Chemistry (North-Holland Publishing Company, Ams- Fundamental Theories of Physics, Vol. 57 (Springer terdam, 3rd ed., 2007). Netherlands, 2002). [60] J. Gemmer and R. Steinigeweg, “Entropy increase in k- [82] D. Andrieux, P. Gaspard, T. Monnai, and S. Tasaki, step Markovian and consistent dynamics of closed quan- “The fluctuation theorem for currents in open quantum tum systems,” Phys. Rev. E 89, 042113 (2014). systems,” New J. Phys. 11, 043014 (2009). [61] J. Lee, “Derivation of Markov processes that violate de- [83] M. Esposito, K. Lindenberg, and C. Van den Broeck, tailed balance,” Phys. Rev. E 97, 032110 (2018). “Entropy production as correlation between system and [62]D. Safr´anek,J.ˇ M. Deutsch, and A. Aguirre, “Quantum reservoir,” New J. Phys. 12, 013013 (2010). coarse-grained entropy and thermodynamics,” Phys. [84] T. Sagawa and M. Ueda, “Sagawa and Ueda Reply:,” Rev. A 99, 010101 (2019). Phys. Rev. Lett. 104, 198904 (2010). [63]D. Safr´anek,J.ˇ M. Deutsch, and A. Aguirre, “Quan- [85] K. Takara, H.-H. Hasegawa, and D. J. Driebe, “Gen- tum coarse-grained entropy and thermalization in closed eralization of the second law for a transition be- systems,” Phys. Rev. A 99, 012103 (2019). tween nonequilibrium states,” Phys. Lett. A 375, 88–92 [64] D. Faiez, D. Safr´anek,J.ˇ M. Deutsch, and A. Aguirre, (2010). “Typical and extreme entropies of long-lived isolated [86] M. F. Ludovico, J. S. Lim, M. Moskalets, L. Arrachea, quantum systems,” Phys. Rev. A 101, 052101 (2020). and D. S´anchez, “Dynamical energy transfer in ac- [65] J. Schindler, D. Safr´anek,ˇ and A. Aguirre, “Quantum driven quantum systems,” Phys. Rev. B 89, 161306 correlation entropy,” Phys. Rev. A 102, 052407 (2020). (2014). [66] J. Casas-V´azquezand D. Jou, “Temperature in non- [87] M. Esposito, M. A. Ochoa, and M. Galperin, “Nature of equilibrium states: a review of open problems and cur- heat in strongly coupled open quantum systems,” Phys. rent proposals,” Rep. Prog. Phys. 66, 1937 (2003). Rev. B 92, 235440 (2015). [67] W. Muschik, “Empirical foundation and axiomatic [88] P. Strasberg, G. Schaller, N. Lambert, and T. Brandes, treatment of non-equilibrium temperature,” Arch. Ra- “Nonequilibrium thermodynamics in the strong cou- tion. Mech. Anal. 66, 379–401 (1977). pling and non-Markovian regime based on a reaction [68] W. Muschik and G. Brunk, “A concept of non- coordinate mapping,” New. J. Phys. 18, 073007 (2016). equilibrium temperature,” Int. J. Eng. Sci. 15, 377–389 [89] A. Bruch, M. Thomas, S. V. Kusminskiy, F. von Op- (1977). pen, and A. Nitzan, “Quantum thermodynamics of the [69] H. Tasaki, “Jarzynski relations for quantum systems driven resonant level model,” Phys. Rev. B 93, 115318 and some applications,” arXiv: cond-mat/0009244 (2016). (2000). [90] A. Kato and Y. Tanimura, “Quantum heat current under non-perturbative and non-Markovian conditions: Applications to heat machines,” J. Chem. Phys. 145, 22

224105 (2016). [110] L. D’Alessio, Y. Kafri, A. Polkovnikov, and M. Rigol, [91] D. Newman, F. Mintert, and A. Nazir, “Performance “From quantum chaos and eigenstate thermalization to of a quantum heat engine at strong reservoir coupling,” statistical mechanics and thermodynamics,” Adv. Phys. Phys. Rev. E 95, 032139 (2017). 65, 239–362 (2016). [92] M. N. Bera, A. Riera, M. Lewenstein, and A. Winter, [111] C. Gogolin and J. Eisert, “Equilibration, thermalisa- “Generalized laws of thermodynamics in the presence of tion, and the emergence of statistical mechanics in correlations,” Nat. Comm. 8, 2180 (2017). closed quantum systems,” Rep. Prog. Phys. 79, 056001 [93] P. Strasberg, G. Schaller, T. L. Schmidt, and M. Es- (2016). posito, “Fermionic reaction coordinates and their appli- [112] J. Goold, M. Huber, A. Riera, L. del Rio, and cation to an autonomous Maxwell demon in the strong- P. Skrzypzyk, “The role of quantum information in ther- coupling regime,” Phys. Rev. B 97, 205405 (2018). modynamics – a topical review,” J. Phys. A 49, 143001 [94] M. F. Ludovico, L. Arrachea, M. Moskalets, and (2016). D. S´anchez, “Probing the energy reactance with adi- [113] J. M. Deutsch, “Eigenstate thermalization hypothesis,” abatically driven quantum dots,” Phys. Rev. B 97, Rep. Prog. Phys. 81, 082001 (2018). 041416 (2018). [114] T. Mori, T. N. Ikeda, E. Kaminishi, and M. Ueda, [95] W. Dou, M. A. Ochoa, A. Nitzan, and J. E. Subot- “Thermalization and prethermalization in isolated nik, “Universal approach to quantum thermodynamics quantum systems: a theoretical overview,” J. Phys. B in the strong coupling regime,” Phys. Rev. B 98, 134306 51, 112001 (2018). (2018). [115] J.-P. Brantut, J. Meineke, D. Stadler, S. Krinner, [96] P. Strasberg and M. Esposito, “Non-Markovianity and and T. Esslinger, “Conduction of Ultracold Fermions negative entropy production rates,” Phys. Rev. E 99, Through a Mesoscopic Channel,” Science 337, 1069– 012120 (2019). 1071 (2012). [97] A.´ Rivas, “Strong Coupling Thermodynamics of Open [116] J.-P. Brantut, C. Grenier, J. Meineke, D. Stadler, Quantum Systems,” Phys. Rev. Lett. 124, 160601 S. Krinner, C. Kollath, T. Esslinger, and A. Georges, (2020). “A Thermoelectric Heat Engine with Ultracold Atoms,” [98] U. Seifert, “First and second law of thermodynamics at Science 342, 713–715 (2013). strong coupling,” Phys. Rev. Lett. 116, 020601 (2016). [117] D. J. Evans and D. J. Searles, “The fluctuation theo- [99] P. Talkner and P. H¨anggi,“Open system trajectories rem,” Adv. Phys. 51, 1529–1585 (2002). specify fluctuating work but not heat,” Phys. Rev. E [118] L. P. Pitaevskii, “Rigorous results of nonequilibrium 94, 022143 (2016). statistical physics and their experimental verification,” [100] C. Jarzynski, “Stochastic and macroscopic thermody- Phys. Usp. 54, 625–632 (2011). namics of strongly coupled systems,” Phys. Rev. X 7, [119] C. Jarzynski, “Equalities and inequalities: irreversibil- 011008 (2017). ity and the second law of thermodynamics at the [101] P. Strasberg and M. Esposito, “Stochastic thermody- nanoscale,” Annu. Rev. Condens. Matter Phys. 2, 329– namics in the strong coupling regime: An unambiguous 351 (2011). approach based on coarse graining,” Phys. Rev. E 95, [120] M. Esposito, U. Harbola, and S. Mukamel, “Nonequi- 062101 (2017). librium fluctuations, fluctuation theorems and counting [102] H. J. D. Miller and J. Anders, “Entropy production and statistics in quantum systems,” Rev. Mod. Phys. 81, time asymmetry in the presence of strong interactions,” 1665 (2009). Phys. Rev. E 95, 062123 (2017). [121] B. Piechocinska, “Information erasure,” Phys. Rev. A [103] P. Strasberg and M. Esposito, “Measurability of 61, 062314 (2000). nonequilibrium thermodynamics in terms of the Hamil- [122] J. Kurchan, “A quantum fluctuation theorem,” arXiv: tonian of mean force,” Phys. Rev. E 101, 050101 (2020). cond-mat/0007360 (2000). [104] P. Talkner and P. H¨anggi,“Colloquium: Statistical me- [123] Y. Murashita, K. Funo, and M. Ueda, “Nonequilibrium chanics and thermodynamics at strong coupling: Quan- equalities in absolutely irreversible processes,” Phys. tum and classical,” Rev. Mod. Phys. 92, 041002 (2020). Rev. E 90, 042110 (2014). [105] P. Talkner and P. H¨anggi,“Comment on “Measurabil- [124] H.-J. Schmidt and J. Gemmer, “A framework for se- ity of nonequilibrium thermodynamics in terms of the quential measurements and general Jarzinsky equa- Hamiltonian of mean force”,” Phys. Rev. E 102, 066101 tions,” Z. Naturforsch. A 75, 265 (2020). (2020). [125] S. Trotzky, Y.-A. Chen, A. Flesch, I. P. McCulloch, [106] P. Strasberg and M. Esposito, “Reply to “Comment U. Schollw¨ock, J. Eisert, and I. Bloch, “Probing the re- on ‘Measurability of nonequilibrium thermodynamics in laxation towards equilibrium in an isolated strongly cor- terms of the Hamiltonian of mean force’ ”,” Phys. Rev. related one-dimensional Bose gas,” Nat. Phys. 8, 325– E 102, 066102 (2020). 330 (2012). [107] G. Schaller, Open Quantum Systems Far from Equilib- [126] M. Gring, M. Kuhnert, T. Langen, T. Kitagawa, rium (Lect. Notes Phys., Springer, Cham, 2014). B. Rauer, M. Schreitl, I. Mazets, D. Adu Smith, [108] P. Strasberg, M. G. D´ıaz, and A. Riera-Campeny, E. Demler, and J. Schmiedmayer, “Relaxation and “Clausius inequality for finite baths reveals universal Prethermalization in an Isolated Quantum System,” efficiency improvements,” Phys. Rev. E 104, L022103 Science 337, 1318–1322 (2012). (2021). [127] G. Clos, D. Porras, U. Warring, and T. Schaetz, “Time- [109] J. Gemmer, M. Michel, and G. Mahler, Quantum Ther- resolved observation of thermalization in an isolated modynamics (Lect. Notes Phys., Springer, Heidelberg, quantum system,” Phys. Rev. Lett. 117, 170401 (2016). 2004). [128] A. M. Kaufman, M. Eric Tai, A. Lukin, M. Rispoli, R. Schittko, P. M. Preiss, and M. Greiner, “Quan- 23

tum thermalization through entanglement in an isolated the same initial state at time 2t after flipping the momen- many-body system,” Science 353, 794–800 (2016). tum again. This is at least true for all classical Hamilto- ˙ [129] S. Krinner, T. Esslinger, and J.-P. Brantut, “Two- nian systems in absence of any driving protocol (λt = 0) terminal transport measurements with cold atoms,”J. and in absence of any magnetic field B. If a magnetic Phys. Condens. Matter 29, 343003 (2017). field is present, the above statement remains true if we [130] M. Bohlen, L. Sobirey, N. Luick, H. Biss, T. Enss, also flip the magnetic field from B to −B during the T. Lompe, and H. Moritz, “Sound Propagation and Quantum-Limited Damping in a Two-Dimensional time window [t, 2t]. This is intuitively appealing if one Fermi Gas,” Phys. Rev. Lett. 124, 240403 (2020). recalls that a magnetic field is caused by moving charges, [131] J. Govenius, R. E. Lake, K. Y. Tan, V. Pietil¨a,J. K. which changes sign when the charges move in the oppo- Julin, I. J. Maasilta, P. Virtanen, and M. M¨ott¨onen, site direction. The correct treatment of time-dependent ˙ “Microwave nanobolometer based on proximity Joseph- Hamiltonians (λt 6= 0) is revealed below. The picture son junctions,” Phys. Rev. B 90, 064505 (2014). above describes the essence of time-reversal symmetry, [132] S. Gasparinetti, K. L. Viisanen, O.-P. Saira, T. Faivre, which might be better called “reversal of the direction of M. Arzeo, M. Meschke, and J. P. Pekola, “Fast electron motion” according to Wigner [140]. thermometry for ultrasensitive calorimetric detection,” In quantum mechanics, one introduces a time-reversal Phys. Rev. Applied 3, 014007 (2015). operator Θ [141]. Quite strangely, it turns out that the [133] D. Halbertal, J. Cuppens, M. Ben Shalom, L. Embon, time-reversal operator has the property of being anti- N. Shadmi, Y. Anahory, H. R. Naren, J. Sarkar, A. Uri, Y. Ronen, Y. Myasoedov, L. S. Levitov, E. Joselevich, unitary, which means that A. K. Geim, and E. Zeldov, “Nanoscale thermal imag- ∗ ing of dissipation in quantum systems,” Nature 539, hΘψ|Θφi = hψ|φi for all |ψi, |φi ∈ H. (A1) 407–410 (2016). [134] B. Karimi, F. Brange, P. Samuelsson, and J. P. Pekola, Therefore, Θ is not an operator in the conventional sense “Reaching the ultimate energy resolution of a quantum (one should not intend to write it as a matrix). How- detector,” Nat. Comm. 11, 367 (2020). ever, it follows from the above property that Θ neverthe- [135] G. Gour, M. P. M¨uller, V. Narasimhachar, R. W. less leaves all probabilities unchanged since |hΘψ|Θφi| = Spekkens, and N. Y. Halpern, “The resource theory |hψ|φi|. It is also easy to show that anti-unitarity implies of informational nonequilibrium in thermodynamics,” trace conjugation: Phys. Rep. 583, 1–58 (2015). [136] N. H. Y. Ng and M. P. Woods, “Thermodynamics in the tr{ΘOΘ−1} = tr{O}∗ for all O. (A2) quantum regime,” (Springer, Switzerland, 2018) Chap. Resource Theory of Quantum Thermodynamics: Ther- Furthermore, if O is an observable, then ΘOΘ−1 is also mal Operations and Second Laws, pp. 625–650. an observable with the same eigenvalues as O but poten- [137] M. Lostaglio, “An introductory review of the resource tially different eigenvectors. theory approach to thermodynamics,” Rep. Prog. Phys. As a simple example we consider the quantum me- 82, 114001 (2019). chanical treatment of particles without spin, which is in [138] N. Y. Halpern, “Information and Interaction,” (Springer International Publishing, Switzerland, 2017) close analogy to the classical case. It then turns out Chap. Toward Physical Realizations of Thermodynamic that Θ can be identified with complex conjugation of the wavefunction in position representation. In equations, if Resource Theories, pp. 135–166. R [139] S. Milz and K. Modi, “Quantum Stochastic Pro- |ψi = drψ(r)|ri, where |ri are the eigenstates of the cesses and Quantum non-Markovian Phenomena,” PRX position operatorr ˆ (here denoted with a hat to be un- Quantum 2, 030201 (2021). ambiguous), then [140] E. P. Wigner, Group Theory and Its Application to Z the Quantum Mechanics of Atomic Spectra (Academic ∗ Press, New York, 1959). Θ|ψi = drψ (r)|ri. (A3) [141] J. J. Sakurai, Modern Quantum Mechanics, revised ed. ed. (Addison-Wesley, Reading, MA, 1994). Without too much effort, one confirms that [142] F. Haake, Quantum Signatures of Chaos (Springer- Verlag, Berlin Heidelberg, 2010). Θ2 = I, ΘˆrΘ−1 =r, ˆ ΘˆpΘ−1 = −p,ˆ (A4)

wherep ˆ denotes the momentum operator. The proper- ties (A4) are the ones we expect by analogy with the Appendix A: The time-reversal operator classical case. Note in particular that Θ is an involu- tion, i.e., Θ = Θ−1, which is not always the case (more In classical mechanics, it is clear from intuition that complicated systems are treated, e.g., in Ref. [142]). the trajectory of a particle is traversed in the opposite From what we said initially, we expect |ψ(0)i = −1 direction if one flips the momentum p of the particle to Θ UΘ(t, 0)ΘU(t, 0)|ψ(0)i for any initial state |ψ(0)i. In −p. More precisely, if one follows the trajectory in phase words: if we propagate any initial state forward in time space during a time window [0, t], then flips the momen- using U(t, 0), then time-reverse it, then propagate it for- tum at time t and follows the trajectory in phase space ward in time with respect to the time-reversed propaga- further during the time window [t, 2t], one ends up with tor UΘ(t, 0), and finally time-reverse it again, then we end 24 up with the same initial state. Written as an operator where we introduced the Shannon entropy of a classical identity, we have probability distribution λx. Since a unitary transforma- tion U leaves the eigenvalues of any operator invariant, −1 † Θ UΘ(t, 0)Θ = U (t, 0). (A5) we obtain

This is obviously the result one would expect mathemat- † ically, but its physical interpretation reveals an impor- SvN(UρU ) = SvN(ρ). (B2) tant symmetry. In fact, directly implementing the right hand side of this equation, i.e., U †(t, 0), in a lab is not Moreover, the von Neumann entropy is bounded by 0 ≤ possible as it requires to map t 7→ −t. In contrast, as SvN(ρ) ≤ ln d, where d = dim H is the Hilbert space dimension. demonstrated below, UΘ(t, 0) corresponds to a legitimate ‘forward’ evolution of a physical system. Unfortunately, Many other quantities in quantum (classical) infor- however, the operator Θ, being anti-unitary, can not be mation theory are closely related to the von Neumann implemented in a lab in general. We now turn to the (Shannon) entropy. For us important is the quantum question how to define UΘ(t, 0) microscopically. mutual information of a bipartite state ρXY We first consider the case of a time-independent Hamil- −iHt tonian H and set U(t, 0) = e and UΘ(t, 0) = IX:Y (ρXY ) ≡ SvN(ρX ) + SvN(ρY ) − SvN(ρXY ), (B3) −iHΘt e with HΘ still unknown. To infer HΘ, we use the fact that anti-unitarity implies anti-linearity, which which measures the amount of correlations in the state ∗ means Θc|ψi = c Θ|ψi for any complex number c. ρXY . It is bounded by † −1 From UΘ(t, 0) = ΘU (t, 0)Θ , we then deduce HΘ = −1 ΘHΘ . If H denotes a Hamiltonian of interacting par- 0 ≤ IX:Y (ρXY ) ≤ 2 ln min{dX , dY }. (B4) ticles in absence of any magnetic field, then HΘ = H, i.e., the time-reversed motion is generated by the same By analogy, the classical mutual information for a joint Hamiltonian. This follows from the fact that the momen- probability distribution pxy with marginals px and py is tum enters quadratically the Hamiltonian: Θˆp2Θ−1 =p ˆ2. If H = H(B) depends on an external magnetic field, then IX:Y (pxy) ≡ SSh(px) + SSh(py) − SSh(pxy). (B5) HΘ = H(−B), which follows from the fact that for a par- 2 ticle with charge q a term (ˆp − qA/c) enters the Hamil- It is bounded by tonian, where c is the speed of light and A the vector potential, which gives rise to the magnetic field. 0 ≤ IX:Y (pxy) ≤ ln min{dX , dY }. (B6) Finally, we consider the case with driving protocol λs, s ∈ [0, t], and approximate the time evolution operator Note the missing factor 2 for the upper bound compared as to Eq. (B4). Furthermore, there are multiple ways to U(t, 0) ≈ e−iH(λN−1)δt/~ . . . e−iH(λ0)δt/~, (A6) extend the mutual information to more than two parties with probability distribution pxyz.... In the main text, we where we divided the time interval into steps of size δt = make twice use of the total information defined as t/N and implicitly keep in mind the limit N → ∞ in which Eq. (A6) becomes exact. We can then infer for the Itot(pxyz...) ≡ SSh(px) + SSh(py) + SSh(pz) + ... (B7) time-reversed time evolution operator − SSh(pxyz...), U (t, 0) = ΘU †(t, 0)Θ−1 Θ which is always non-negative. = ΘeiH(λ0)δt/~ . . . eiH(λN−1)δt/~Θ−1 (A7) A final concept used in the main text is quantum rel- = e−iHΘ(λ0)δt/~ . . . e−iHΘ(λN−1)δt/~, ative entropy. It is defined as

−1 where we again set HΘ(λt) = ΘH(λt)Θ . Thus, the D(ρkσ) = tr{ρ(ln ρ − ln σ)} (B8) time-reversed dynamics are defined by changing the pro- tocol backwards in time from λt to λ0 with respect to the and measures the statistical ‘distance’ between two states time-reversed Hamiltonian. ρ and σ. However, quantum relative entropy is not a metric since it is not symmetric: D(ρkσ) 6= D(σkρ). It satisfies D(ρkσ) ≥ 0 with equality if and only if ρ = σ. Appendix B: Basic information theory concepts Furthermore, quantum relative entropy allows to express quantum mutual information as The basic concept in quantum information theory is the von Neumann entropy SvN(ρ) = −tr{ρ ln ρ}. For IX:Y (ρXY ) = D[ρXY kρX ⊗ ρY ], (B9) P ρ = x λx|xihx| the von Neumann entropy reads which confirms its interpretation as a measure of corre- X SvN(ρ) = − λx ln λx ≡ SSh(λx), (B1) lations. x