Number of hidden states needed to physically implement a given conditional distribution Jeremy A. Owen,1 Artemy Kolchinsky,2 and David H. Wolpert2, 3, 4, ∗ 1Physics of Living Systems Group, Department of Physics, Massachusetts Institute of Technology, 400 Tech Square, Cambridge, MA 02139. 2Santa Fe Institute 3Arizona State University 4http://davidwolpert.weebly.com† (Dated: October 15, 2019) We consider the problem of how to construct a physical process over a finite state space X that applies some desired conditional distribution P to initial states to produce final states. This problem arises often in the thermodynamics of computation and nonequilibrium statistical physics more generally (e.g., when designing processes to implement some desired computation, feedback controller, or Maxwell demon). It was previously known that some conditional distributions cannot be implemented using any master equation that involves just the states in X. However, here we show that any conditional distribution P can in fact be implemented—if additional “hidden” states not in X are available. Moreover, we show that is always possible to implement P in a thermodynamically reversible manner. We then investigate a novel cost of the physical resources needed to implement a given distribution P : the minimal number of hidden states needed to do so. We calculate this cost exactly for the special case where P represents a single-valued function, and provide an upper bound for the general case, in terms of the nonnegative rank of P . These results show that having access to one extra binary degree of freedom, thus doubling the total number of states, is sufficient to implement any P with a master equation in a thermodynamically reversible way, if there are no constraints on the allowed form of the master equation. (Such constraints can greatly increase the minimal needed number of hidden states.) Our results also imply that for certain P that can be implemented without hidden states, having hidden states permits an implementation that generates less heat. I. INTRODUCTION conditional distribution of the system state at the final time given its state at the initial time. The entry Pij is Master equation dynamics over a discrete state space the probability that the system is in state i at the final play a fundamental role in nonequilibrium statistical time given that it was in state j at the initial time. physics and stochastic thermodynamics [1–3], and are The problem of characterizing the set of all stochas- used to model a wide variety of physical systems. Such tic matrices P that can arise this way from a (time- dynamics can arise after coarse-graining the continu- inhomogeneous) continuous-time master equation is ous phase space of a Hamiltonian system coupled to a known as the embedding problem for Markov chains [8– heat bath [4–6], or as semiclassical approximations of 13], which can be viewed as the classical and time- the evolution of an open quantum system with discrete inhomogeneous analogue of the Markovianity problem states [1, 7]. studied in quantum information [14, 15]. The linearity of the master equation implies a linear Despite the ubiquity of master equation dynamics in relationship between the distribution over the system’s models of physical systems, it turns out that many P are states at the initial time t = 0 and some final time (here non-embeddable, i.e., they cannot be implemented with taken to be t = 1 without loss of generality). This is any master equation. As we discuss below, examples true even if the master equation is time-inhomogeneous of non-embeddable matrices include common operations (i.e., non-autonomous). We can express this relation as such as bit erasure and the bit flip (a logical NOT), p(1) = P p(0) 1 1 0 1 P = ,P = . (1) erase 0 0 flip 1 0 where p(0) and p(1) are column vectors whose entries sum arXiv:1709.00765v4 [cond-mat.stat-mech] 14 Oct 2019 to one, representing probability distributions at times t = While neither of these operations are embeddable, 0 and t = 1 respectively. If our system has n states, the there is a crucial difference between them. Bit erasure is matrix P is an n × n stochastic matrix, representing the infinitesimally close to being embeddable, meaning that it can be implemented by a master equation if one is will- ing to tolerate some finite, but arbitrarily small, probabil- ∗ Massachusetts Institute of Technology ity of error. We call such matrices limit-embeddable. † This is a post-peer-review version of an article published in New On the other hand, as we show below, many stochas- Journal of Physics. The final, published version is available at tic matrices P , such as the bit flip, are not even limit- https://doi.org/10.1088/1367-2630/aaf81d embeddable. In light of this, suppose we encounter a 2 physical system with n “visible states” that we know while generating less heat, as we illustrate in detail in an obeys a continuous-time master equation, but is ob- example below. served to evolve according to a P that is not even limit- Note that our results assume complete freedom to use embeddable between t = 0 and t = 1. In such a situation any master equation to implement a given P . However, in we can conclude that the master equation dynamics must the real world there will often be major constraints on the actually take place over some larger state space, including form of the master equation that can be considered, e.g., some unseen hidden states in addition to the n visible due to known properties of an observed system we wish states. to model using a master equation, or due to limitations Alternatively, we can imagine that instead of observ- on what kind of system we can build. An example of the ing some stochastic dynamics, we attempt to build a sys- former is if we know that the system’s Hamiltonian can tem that carries out a specified P using master equation only couple degrees of freedom in certain restricted ways. dynamics. This is the challenge we might face, for ex- An example of the latter is if P must be implemented ample, if P is the update function of the logical state using a digital circuit made out of separate gates. In of a computer we wish to build. In this scenario, the cases where there are such constraints on the form of the minimal number of hidden states needed to implement master equation, our results can be considered as upper P with master equation dynamics emerges as a funda- bounds on lower bounds of the number of hidden states mental “state space cost” of the computation P . (See that are really required. (See Section VI C below.) also [16].) Previous research has shown how to carry out arbitrary In fact, by adding hidden states one can construct mas- P thermodynamically reversibly [18–21]. Moreover, the ter equations that meet more desiderata than just imple- constructions in those papers can all be formulated in menting a given P . Specifically, below we show that for terms of master equation dynamics.2 In addition, they all any given P and initial distribution p(0), one can con- exploit what in our terminology are called hidden states. struct a master equation that implements P on p(0) while However, none of those papers considered the issue of the being arbitrarily close to thermodynamically reversible.1 minimal number of hidden states needed to implement P , In this paper, we establish upper bounds on the num- i.e., the state space cost of implementing P , which is the ber of hidden states required to implement any stochas- focus of this paper. tic matrix P using a master equation, and also establish In a companion paper [16] we focus on the special case bounds when we require that P be implemented in a where the stochastic matrix P is “single-valued”, i.e., it thermodynamically reversible way. We do this using ex- represents a deterministic function. It turns out that for plicit constructions that show any stochastic matrix P that case at least, there is a second kind of cost aris- can be implemented by composing some appropriate set ing in master equations dynamics that implement P , in of fundamental transformations that we call local relax- addition to the state space cost which is the focus of ations, defined in detail below. this paper. Roughly speaking, that second cost is the Our main result is that any n × n stochastic matrix minimal number of times that the set of allowed state- P can be implemented with no more than r − 1 hidden to-state transitions changes (which in a physics context states, where r is the nonnegative rank of P . Because may correspond to raising or lowering infinite energy bar- r ≤ n, this result implies a simple corollary that one riers between states). This can be viewed as a “timestep additional binary degree of freedom (which doubles the cost” of implementing P . Interestingly, there is a tradeoff number of available states) is sufficient to carry out any between the timestep cost of implementing any (single- P in a thermodynamically reversible way. We also derive valued) P and the state space cost of implementing that exact results for some particular kinds of P , including P . those representing single-valued functions, which show This paper focuses on the general problem of bounding that the number of required hidden states can be much the state space cost of arbitrary (not necessarily single- smaller than r − 1. valued) stochastic matrices and the relationship of this cost to thermodynamic reversibility.