The Gibbs Paradox

gibbs, 7/8/1996

THE GIBBS PARADOX

E. T. Jaynes

DepartmentofPhysics. Washington University,

St. Louis, Missouri 63130 USA.

Abstract :We p oint out that an early work of J. Willard Gibbs 1875 contains a correct analysis

of the \Gibbs Paradox" ab out entropy of mixing, free of any elements of mystery and directly

connected to exp erimental facts. However, it app ears that this has b een lost for 100 years, due to

some obscurities in Gibbs' style of writing and his failure to include this explanation in his later

Statistical Mechanics. This \new" understanding is not only of historical and p edagogical interest;

it gives b oth classical and quantum statistical mechanics a di erent status than that presented in

our textb o oks, with implications for current research.

CONTENTS

1. INTRODUCTION 2

2. THE PROBLEM 3

3. THE EXPLANATION 4

4. DISCUSSION 6

5. THE GAS MIXING SCENARIO REVISITED 7

6. SECOND LAW TRICKERY 9

7. THE PAULI ANALYSIS 10

8. WOULD GIBBS HAVE ACCEPTED IT? 11

9. GIBBS' STATISTICAL MECHANICS 13

10. SUMMARY AND UNFINISHED BUSINESS 17

11. REFERENCES 18

In Maximum Entropy and Bayesian Methods , C. R. Smith, G. J. Erickson, & P. O. Neudorfer, Editors,

Kluwer Academic Publishers, Dordrecht, Holland 1992; pp. 1{22.

2 1. Intro duction 2

1. Intro duction

J. Willard Gibbs' Statistical Mechanics app eared in 1902. Recently, in the course of writing a

textb o ok on the sub ject, the present writer underto ok to read his monumental earlier work on

Heterogeneous Equilibrium 1875{78. The original purp ose was only to ensure the historical accu-

racy of certain background remarks; but this was sup erseded quickly by the much more imp ortant

unearthing of deep insights into current technical problems.

Some imp ortant facts ab out thermo dynamics have not b een understo o d by others to this

day, nearly as well as Gibbs understo o d them over 100 years ago. Other asp ects of this \new"

developmenthave b een rep orted elsewhere Jaynes 1986, 1988, 1989. In the present note we

consider the \Gibbs Paradox" ab out entropy of mixing and the logically inseparable topics of

reversibility and the extensive prop ertyofentropy.

For 80 years it has seemed natural that, to nd what Gibbs had to say ab out this, one should

turn to his Statistical Mechanics.For 60 years, textb o oks and teachers including, regrettably, the

present writer have impressed up on students how remarkable it was that Gibbs, already in 1902,

had b een able to hit up on this paradox which foretold { and had its resolution only in { quantum

theory with its lore ab out indistinguishabl e particles, Bose and Fermi statistics, etc.

It was therefore a sho ck to discover that in the rst Section of his earlier work whichmust

have b een written by mid{1874 at the latest, Gibbs displays a full understanding of this problem,

and disp oses of it without a trace of that confusion over the \meaning of entropy" or \op erational

distinguishabili ty of particles" on which later writers have stumbled. He go es straight to the heart

of the matter as a simple technical detail, easily understo o d as so on as one has grasp ed the full

meanings of the words \state" and \reversible" as they are used in thermo dynamics. In short,

quantum theory did not resolveany paradox, b ecause there was no paradox.

Why did Gibbs fail to give this explanation in his Statistical Mechanics ?We are inclined to

see in this further supp ort for our contention Jaynes, 1967 that this work was never nished.

In reading Gibbs, it is imp ortant to distinguish b etween early and late Gibbs. His Heterogeneous

Equilibrium of 1875{78 is the work of a man at the absolute p eak of his intellectual p owers; no

logical subtlety escap es him and we can nd no statement that app ears technically incorrect to day.

In contrast, his Statistical Mechanics of 1902 is the work of an old man in rapidly failing health,

with only one more year to live. Inevitably, some arguments are left imp erfect and incomplete

toward the end of the work.

In particular, Gibbs failed to p oint out that an \integration constant" was not an arbitrary

constant, but an arbitrary function. But this has, as we shall see, nontrivial physical consequences.

What is remarkable is not that Gibbs should have failed to stress a ne mathematical p oint in almost

the last words he wrote; but that for 80 years thereafter all textb o ok writers except p ossibly Pauli

failed to see it.

Today, the universally taught conventional wisdom holds that \Classical mechanics failed to

yield an entropy function that was extensive, and so statistical mechanics based on classical the-

ory gives qualitatively wrong predictions of vap or pressures and equilibrium constants, whichwas

cleared up only by quantum theory in which the interchange of identical particles is not a real

event". We argue that, on the contrary, phenomenological thermo dynamics, classical statistics,

and quantum statistics are all in just the same logical p osition with regard to extensivity of en-

tropy; they are silent on the issue, neither requiring it nor forbidding it.

Indeed, in the phenomenological theory Clausius de ned entropyby the integral of dQ=T over

a reversible path; but in that path the size of a system was not varied, therefore the dep endence

of entropy on size was not de ned. This was p erceived byPauli 1973, who pro ceeded to give the

correct functional equation analysis of the necessary conditions for extensivity. But if this is required

already in the phenomenological theory, the same analysis is required a fortiori in b oth classical

3 3

and quantum statistical mechanics. As a matter of elementary logic, no theory can determine the

dep endence of entropy on the size N of a system unless it makes some statement ab out a pro cess

where N changes.

In Section 2 b elowwe recall the familiar statement of the mixing paradox, and Sec. 3 presents

the explanation from Gibbs' Heterogeneous Equilibrium in more mo dern language. Sec. 4 discusses

some implications of this, while Sections 5 and 6 illustrate the p oints by a concrete scenario. Sec.

7 then recalls the Pauli analysis and Sec. 8 reexamines the relevant parts of Gibbs' Statistical

Mechanics to showhow the mixing paradox disapp ears, and the issue of extensivityofentropyis

cleared up, when the aforementioned minor oversight is corrected byaPauli typ e analysis. The

nal result is that entropy is just as much, and just as little, extensive in classical statistics as in

quantum statistics. The concluding Sec. 9 p oints out the relevance of this for current research.

2. The Problem

We rep eat the familiar story, already told hundreds of times; but with a new ending. There are

n moles of an ideal gas of typ e 1, n of another noninteracting typ e 2, con ned in twovolumes

1 2

V ;V and separated by a diaphragm. Cho osing V =V = n =n ,wemayhave them initially at the

1 2 1 2 1 2

same temp erature T = T and pressure P = P = n RT =V . The diaphragm is then removed,

1 2 1 2 1 1

allowing the gases to di use through each other. Eventually we reach a new equilibrium state

with n = n + n moles of a gas mixture, o ccupying the total volume V = V + V with uniform

1 2 1 2

comp osition, the temp erature, pressure and total energy remaining unchanged.

If the gases are di erent, the entropychange due to the di usion is, by standard thermo dy-

namics,

S = S S = nR log V n R log V + n R log V

f inal initial 1 1 2 2

or,

S = nR[f log f +1 f log1 f ] 1

where f = n =n = V =V is the mole fraction of comp onent 1. Gibbs [his Eq. 297] considers

1 1

particularly the case f =1=2, whereup on

S = nR log 2:

What strikes Gibbs at once is that this is indep endent of the nature of the gases, \::: except that

the gases which are mixed must b e of di erent kinds. If we should bring into contact two masses

of the same kind of gas, they would also mix, but there would b e no increase of entropy." He then

pro ceeds to explain this di erence, in a very cogentway that has b een lost for 100 years. But to

understand it, wemust rst surmount a diculty that Gibbs imp oses on his readers.

Usually, Gibbs' prose style conveys his meaning in a suciently clear way, using no more

than twice as manywords as Poincar e or Einstein would have used to say the same thing. But

o ccasionally he delivers a sentence with a p onderous unintelligibi l ity that seems to challenge us to

make sense out of it. Unfortunately, one of these app ears at a crucial p oint of his argument; and

this may explain why the argument has b een lost for 100 years. Here is that sentence:

\Again, when such gases have b een mixed, there is no more imp ossibility of the separation

of the two kinds of molecules in virtue of their ordinary motions in the gaseous mass

without any esp ecial external in uence, than there is of the separation of a homogeneous

gas into the same two parts into which it has once b een divided, after these have once

b een mixed."

The decipherment of this into plain English required much e ort, sustained only by faith in Gibbs;

but eventually there was the reward of knowing how Champ ollion felt when he realized that he had

4 3. The Explanation 4

mastered the Rosetta stone. Suddenly, everything made sense; when put backinto the surrounding

context there app eared an argument so clear and simple that it seemed astonishing that it has not

b een rediscovered indep endently dozens of times. Yet a library search has failed to lo cate any trace

of this understanding in any mo dern work on thermo dynamics or statistical mechanics.

We pro ceed to our rendering of that explanation, which is ab out half direct quotation from

Gibbs, but with considerable editing, rearrangement, and exegesis. For this reason we call it \the

explanation" rather than \Gibbs' explanation". We do not b elieve that we deviate in anyway from

Gibbs' intentions, whichmust b e judged partly from other sections of his work; in anyevent, his

original words are readily available for comparison. However, our purp ose is not to clarify history,

but to explain the technical situation as it app ears to day in the light of this clari cation from

history; therefore we carry the explanatory remarks slightly b eyond what was actually stated by

Gibbs, to take note of the additional contributions of Boltzmann, Planck, and Einstein.

3. The Explanation

When unlike gases mix under the conditions describ ed ab ove, there is an entropy increase S =

nR log 2 indep endent of the nature of the gases. When the gases are identical they still mix, but

there is no entropy increase. But wemust b ear in mind the following.

When wesay that two unlike gases mix and the entropy increases, we mean that the gases could

b e separated again and brought back to their original states by means that would leavechanges in

external b o dies. These external changes might consist, for example, in the lowering of a weight, or

a transfer of heat from a hot b o dy to a cold one.

But by the \original state" we do not mean that every molecule has b een returned to its

original p osition, but only a state which is indistinguishabl e from the original one in the macroscopic

prop erties that we are observing. For this we require only that a molecule originally in V returns

to V . In other words, we mean that we can recover the original thermodynamic state, de ned

for example by sp ecifying only the chemical comp osition, total energy,volume, and number of

moles of a gas; and nothing else. It is to the states of a system thus incompletely de ned that the

prop ositions of thermo dynamics relate.

But when wesay that two identical gases mix without change of entropy,we do not mean

that the molecules originally in V can b e returned to V without external change. The assertion

1 1

of thermo dynamics is that when the net entropychange is zero, then the original thermodynamic

state can b e recovered without external change. Indeed, wehave only to reinsert the diaphragm;

since all the observed macroscopic prop erties of the mixed and unmixed gases are identical, there

has b een no change in the thermo dynamic state. It follows that there can b e no change in the

entropyorinany other thermo dynamic function.

Trying to interpret the phenomenon as a discontinuous change in the physical nature of the

gases i. e., in the b ehavior of their microstates when they b ecome exactly the same, misses

the p oint. The principles of thermo dynamics refer not to any prop erties of the hyp othesized mi-

crostates, but to the observed prop erties of macrostates; there is no thought of restoring the original

microstate. We might put it thus: when the gases b ecome exactly the same, the discontinuityisin

what you and I mean by the words \restore" and \reversible".

But if such considerations explain why mixtures of like and unlike gases are on a di erent

fo oting, they do not reduce the signi cance of the fact that the entropychange with unlike gases is

indep endent of the nature of the gases. Wemay, without doing violence to the general principles

of thermo dynamics, imagine other gases than those presently known, and there do es not app ear

to b e any limit to the resemblance which there mightbebetween two of them; but S would b e

indep endent of it.

5 5

Wemayeven imagine two gases which are absolutely identical in all prop erties which come

into play while they exist as gases in the di usion cell, but which di er in their b ehavior in some

other environment. In their mixing an increase of entropywould take place, although the pro cess,

dynamically considered, might b e absolutely identical in its minutest details even the precise path

of each atom with another pro cess which might take place without any increase of entropy.In

such resp ects, entropy stands strongly contrasted with energy.

A thermo dynamic state is de ned by sp ecifying a small numb er of macroscopic quantities such

as pressure, volume, temp erature, magnetization, stress, etc. { denote them by fX ;X ;:::;X g

1 2 n

{ which are observed and/or controlled by the exp erimenter; n is seldom greater than 4. Wemay

contrast this with the physical state, or microstate, in whichwe imagine that the p ositions and

velo cities of all the individual atoms p erhaps 10 of them are sp eci ed.

All thermo dynamic functions { in particular, the entropy { are by de nition and construction

prop erties of the thermo dynamic state; S = S X ;X ;:::; X . A thermo dynamic variable may

1 2 n

or may not b e also a prop erty of the microstate. We consider the total mass and total energy to

b e \physically real" prop erties of the microstate; but the ab ove considerations show that entropy

cannot b e.

To emphasize this, note that a \thermo dynamic state" denoted by X fX :::X g de nes a

1 n

large class C X of microstates compatible with X . Boltzmann, Planck, and Einstein showed that

wemayinterpret the entropy of a macrostate as S X = k log W C , where W C is the phase

volume o ccupied by all the microstates in the chosen reference class C .From this formula, a large

mass of correct results may b e explained and deduced in a neat, natural way Jaynes, 1988. In

particular, one has a simple explanation of the reason for the second law as an immediate conse-

quence of Liouville's theorem, and a generalization of the second law to nonequilibriu m conditions,

which has useful applications in biology Jaynes, 1989.

This has some interesting implications, not generally recognized. The thermo dynamic entropy

S X is, by de nition and construction, a prop erty not of any one microstate, but of a certain

reference class C X of microstates; it is a measure of the size of that reference class. Then if two

di erent microstates are in C ,wewould ascrib e the same entropy to systems in those microstates.

But it is also p ossible that two exp erimenters assign di erententropies S; S to what is in fact the

same microstate in the sense of the same p osition and velo cityofevery atom without either b eing

in error. That is, they are contemplating a di erent set of p ossible macroscopic observations on

the same system, emb edding that microstate in two di erent reference classes C; C .

Two imp ortant conclusions follow from this. In the rst place, it is necessary to decide at the

outset of a problem which macroscopic variables or degrees of freedom we shall measure and/or

control; and within the context of the thermo dynamic system thus de ned, entropy will b e some

function S X ;:::;X of whatever variables wehavechosen. We can exp ect this to ob ey the second

1 n

law TdS dQ only as long as all exp erimental manipulations are con ned to that chosen set. If

someone, unknown to us, were to vary a macrovariable X outside that set, he could pro duce what

n+1

would app ear to us as a violation of the second law, since our entropy function S X ;:::;X might

1 n

decrease sp ontaneously, while his S X ;:::;X ;X increases. [We demonstrate this explicitly

1 n n+1

b elow].

Secondly,even within that chosen set, deviations from the second law are at least conceivable.

Let us return to the mixing of identical gases. From the fact that they mix without change of

entropy,wemust not conclude that they can b e separated again without external change. On the

contrary, the \separation" of identical gases is entirely imp ossible with or without external change.

If \identical" means anything, it means that there is no way that an \unmixing" apparatus could

determine whether a molecule came originally from V or V , short of having followed its entire

1 2

tra jectory.

6 4. Discussion 6

It follows a fortiori that there is no waywe could accomplish this separation repro ducibly by

manipulation of any macrovariables fX g. Nevertheless it might happ en without anyintervention

on our part that in the course of their motion the molecules which came from V all return to it at

some later time. Suchanevent is not imp ossible; we consider it only improbable.

Now a separation that Nature can accomplish already in the case of identical molecules, she

can surely accomplish at least as easily for unlike ones. The sp ontaneous separation of mixed unlike

gases is just as p ossible as that of like ones. In other words, the imp ossibili ty of an uncomp ensated

decrease of entropy seems to b e reduced to improbability.

4. Discussion

The last sentence ab ove is the famous one which Boltzmann quoted twentyyears later in his

reply to Zermelo's Wiederkehreinwand and to ok as the motto for the second 1898 volume of

his Vorlesungenub er Gastheorie. Note the sup eriority of Gibbs' reasoning. There is none of the

irrelevancy ab out whether the interchange of identical particles is or is not a \real physical event",

which has troubled so many later writers { including even Schrodinger. As we see, the strong

contrast b etween the \physical" nature of energy and the \anthrop omorphic" nature of entropy,

was well understo o d by Gibbs b efore 1875.

Nevertheless, we still see attempts to \explain irreversibility" by searching for some entropy

function that is supp osed to b e a prop erty of the microstate, making the second law a theorem of

dynamics, a consequence of the equations of motion. Such attempts, dating back to Boltzmann's

pap er of 1866, have never succeeded and never ceased. But they are quite unnecessary; for the

second law that Clausius gaveuswas not a statement ab out any prop erty of microstates. The

di erence in S on mixing of like and unlike gases can seem paradoxical only to one who supp oses,

erroneously, that entropy is a prop erty of the microstate.

The imp ortant constructive p oint that emerges from this is that thermo dynamics has a greater

exibility in useful applications than is generally recognized. The exp erimenter is at lib ertyto

cho ose his macrovariables as he wishes; whenever he cho oses a set within which there are exp eri-

mentally repro ducible connections like an equation of state, the entropy appropriate to the chosen

set will satisfy a second law that correctly accounts for the macroscopic observations that the

exp erimenter can makeby manipulating the macrovariables within his set.

Wemay drawtwo conclusions ab out the range of validity of the second law. In the rst place,

if entropy can dep end on the particular wayyou or I decide to de ne our thermo dynamic states,

then obviously, the statement that entropy tends to increase but not decrease can remain valid only

as long as, having made our choice of macrovariables, we stick to that choice.

Indeed, as so on as wehave Boltzmann's S = k log W , the reason for the second law is seen

immediately as a prop osition of macroscopic phenomenology, true \almost always" simply from

considerations of phase volume. Two thermo dynamic states of slightly di erententropy S S =

1 2

10 =393 cal/deg, corresp onding to one micro calorie at ro om temp erature, exhibit a phase volume

ratio

W =W = exp [S S =k ] = exp[10 ]: 2

1 2 1 2

Macrostates of higher entropy are sometimes said to `o ccupy' overwhelmingly greater phase volumes;

put more accurately, macrostates of higher entropymaybe realized byanoverwhelmingly greater

numb er, or range, of microstates. Because of this, not only all repro ducible changes b etween

equilibrium thermo dynamic states, but the overwhelming ma jority of all the macrostate changes

that could p ossibly o ccur { equilibrium or nonequilibrium { are to ones of higher entropy, simply

b ecause there are overwhelmingly more microstates to go to, by factors like 2. We do not see why

any more than this is needed to understand the second law as a fact of macroscopic phenomenology.

7 7

However, the argument just given showing howentropy dep ends, not on the microstate, but

on human choice of the reference class in which itistobeemb edded, may app ear so abstract that

it leaves the reader in doubt as to whether we are describing real, concrete facts or only a particular

philosophyofinterpretation, without physical consequences.

This is so particularly when we recall that after the aforementioned pap er of 1866, Boltzmann

sp ent the remaining 40 years of his life in inner turmoil and outward controversy over the second

law, rep eatedly changing his p osition. The details are recounted by Klein 1973. In the end this

degenerated into nit{picking arguments over the exact conditions under which his H {function did

or did not decrease. But none of this was ever settled or related to the real exp erimental facts

ab out the second law { which make no reference to anyvelo city distribution! It b eho oves us to b e

sure that we are not following a similar course.

Fortunately, the concrete reality and direct exp erimental relevance of Gibbs' arguments is easily

shown. The actual calculation following is probably the most common one found in elementary

textb o oks, but the full conditions of its validityhave never, to the b est of our knowledge, b een

recognized. The scenario in whichwe set it is only slightly fanciful; the discovery of isotop es was

very nearly a realization of it. As examination of several textb o oks shows, that discovery prompted

a great deal of confusion over whether entropy of mixing of isotop es should or should not b e included

in thermo dynamics.

5. The Gas Mixing Scenario Revisited.

Presumably, nob o dy doubts to day that the measurable macroscopic prop erties of argon i.e. equa-

tion of state, heat capacity,vap or pressure, heat of vap orization, etc. are describ ed correctly by

conventional thermo dynamics which ascrib es zero entropychange to the mixing of two samples of

argon at the same temp erature and pressure. But supp ose that, unknown to us to day, there are two

di erent kinds of argon, A1 and A2, identical in all resp ects except that A2 is soluble in Whifnium,

while A1 is not Whifnium is one of the rare sup erkalic elements; in fact, it is so rare that it has

not yet b een discovered.

Until the discovery of Whifnium in the next Century,we shall havenoway of preparing argon

with controllably di erent prop ortions of A1 and A2. And even if, by rare chance, we should

happ en to get pure A1involume V 1, and pure A2in V 2, wewould havenowayofknowing

this, or of detecting any di erence in the resulting di usion pro cess. Thus all the thermo dynamic

measurements we can actually maketoday are accounted for correctly by ascribing zero entropyof

mixing to argon.

Now the scene shifts to the next Century, when Whifnium is readily available to exp erimenters.

What could happ en b efore only by rare chance, we can now bring ab out by design. Wemay,at

will, prepare b ottles of pure A1 and pure A2. Starting our mixing exp eriment with n = fn moles

of A1 in the volume V = fV , and n =1 f n moles of A2in V =1 f V , the resulting actual

1 2 2

di usion may b e identical in every detail, down to the precise path of each atom, with one that

could have happ ened by rare chance b efore the discovery of Whifnium; but b ecause of our greater

knowledge we shall now ascrib e to that di usion an entropy increase S given by Eq 1, which

we write as:

S =S +S 3

1 2

where

S = nRf log f 4a

S = nR1 f log 1 f 4b

But if this entropy increase is more than just a gment of our imagination, it oughttohave

observable consequences, such asachange in the useful work that we can extract from the pro cess.

8 5. The Gas Mixing Scenario Revisited. 8

There is a scho ol of thought which militantly rejects all attempts to p oint out the close relation

between entropy and information, claiming that such considerations have nothing to do with energy;

or even that they would makeentropy \sub jective" and it could therefore could have nothing to do

with exp erimental facts at all. Wewould observe, however, that the numb er of sh that you can

catch is an \ob jective exp erimental fact"; yet it dep ends on howmuch \sub jective" information

you have ab out the b ehavior of sh.

If one is to condemn things that dep end on human information, on the grounds that they are

\sub jective", it seems to us that one must condemn all science and all education; for in those elds,

human information is all wehave. We should rather condemn this misuse of the terms \sub jective"

and \ob jective", which are descriptive adjectives, not epithets. Science do es indeed seek to describ e

what is \ob jectively real"; but our hyp otheses ab out that will have no testable consequences unless

it can also describ e what human observers can see and know. It seems to us that this lesson should

have b een learned rather well from relativity theory.

The amount of useful work that we can extract from any system dep ends { obviously and

necessarily { on howmuch \sub jective" information wehave ab out its microstate, b ecause that tells

us whichinteractions will extract energy and which will not; this is not a paradox, but a platitude.

If the entropywe ascrib e to a macrostate did not represent some kind of human information ab out

the underlying microstates, it could not p erform its thermo dynamic function of determining the

amountofwork that can b e extracted repro ducibly from that macrostate.

But if this is not obvious, it is easily demonstrated in our present scenario. The di usion will

still take place without anychange of temp erature, pressure, or internal energy; but b ecause of

our greater information we shall now asso ciate it with a free energy decrease F = T S . Then,

according to the principles of thermo dynamics, if instead of allowing the uncontrolled irreversible

mixing we could carry out the same change of state reversibly and isothermally,we should b e able

to obtain from it the work

W = F = T S: 5

Let us place b eside the diaphragm a movable piston of Whifnium. When the diaphragm is removed,

the A2 will then di use through this piston until its partial pressure is the same on b oth sides,

after whichwemove the piston slowly to maintain equal A2 pressure and to allow heat to owin

to maintain constant temp erature, in the direction of increasing V .From this expansion of A1

we shall obtain the work

W = P dV = n RT log V=V

1 1 1 1

or from 4a,

W = T S 6

1 1

The term S in the entropy of mixing therefore indicates the work obtainable from reversible

isothermal expansion of comp onent A1into the full volume V = V + V . But the initial di usion

1 2

of A2 still represents an irreversible entropy increase S from whichwe obtain no work.

Spurred by this partial success, the other sup erkalic element Whafnium is discovered, which

has the opp osite prop erty that it is p ermeable to A1 but not to A2. Then we can make an apparatus

with two sup erkalic pistons; the Whifnium moves to the right, giving the work W = T S , while

1 1

the Whafnium moves to the left, yielding W = T S . Wehave succeeded in extracting just

2 2

the work W = T S predicted by thermo dynamics. The entropy of mixing do es indeed represent

human information; just the information neededtopredict the work available from the mixing.

In this scenario, our greater knowledge resulting from the discovery of the sup erkalic elements

leads us to assign a di erententropychange to what may b e in fact the identical physical pro cess,

down to the exact path of each atom. But there is nothing \unphysical" ab out this, since that

9 9

greater knowledge corresp onds exactly to { b ecause it is due to { our greater capabilities of control

over the physical pro cess. Possession of a sup erkalic piston gives us the ability to control a new

thermo dynamic degree of freedom X , the p osition of the piston. It would b e astonishing if this

n+1

new technical capability did not enable us to extract more useful work from the system.

This scenario has illustrated the aforementioned greater versatility of thermo dynamics { the

wider range of useful applications { that we get from recognizing the strong contrast b etween the

natures of entropy and energy, that Gibbs p ointed out so long ago.

To emphasize this, note that even after the discovery of sup erkalic elements, we still have the

option not to use them and stick to the old macrovariables fX :::X g of the 20'th Century. Then

1 n

wemay still ascrib e zero entropy of mixing to the interdi usion of A1 and A2, and we shall predict

correctly, just as was done in the 20'th Century, all the thermo dynamic measurements that we can

make on Argon without using the new technology. Both b efore and after discovery of the sup erkalic

elements, the rules of thermo dynamics are valid and correctly describ e the measurements that it is

p ossible to makeby manipulating the macrovariables within the set that we have chosen to use.

This useful versatility { a direct result of, and illustration of, the \anthrop omorphic" nature

of entropy{ would not b e apparent to, and p erhaps not b elieved by, someone who thought that

entropywas, like energy,aphysical prop erty of the microstate.

6. Second LawTrickery.

Our scenario has veri ed another statement made ab ove; a p erson who knows ab out this new degree

of freedom and manipulates it, can play tricks on one who do es not know ab out it, and make him

think that he is seeing a violation of the second law. Supp ose there are two observers, one of whom

do es not know ab out A1;A2, and sup erkalic elements and one who do es, and we present them with

two exp eriments.

In exp eriment 1, mixing of a volume V of A1 and V of A2 takes place sp ontaneously, without

1 2

sup erkalic pistons, from an initial thermo dynamic state X to a nal one X without anychange

i f

of temp erature, pressure, or internal energy and without doing anywork; so it causes no heat ow

between the di usion cell and a surrounding heat bath of temp erature T .To b oth observers, the

initial and nal states of the heat bath are the same, and to the ignorant observer this is also true

of the argon; nothing happ ens at all.

In exp eriment2we insert the sup erkalic pistons and p erform the same mixing reversibly,

starting from the same initial state X . Again, the nal state X of the argon has the same

i f

temp erature, pressure, and internal energy as do es X . But nowwork W is done, and so heat

Q = W ows into the di usion cell from the heat bath. Its existence and magnitude could b e

veri ed by calorimetry. Therefore, for b oth observers, the initial and nal states of the heat bath

are now di erent. To the ignorant observer, an amount of heat Q has b een extracted from the heat

bath and converted entirely into work: W = Q, while the total entropy of the system and heat

bath has decreased sp ontaneously byS = Q=T , in agrant violation of the second law!

To the informed observer, there has b een no violation of the second law in either exp eriment.

In exp eriment 1 there is an irreversible increase of entropy of the argon, with its concomitant loss

of free energy; in exp eriment 2, the increase in entropy of the argon is exactly comp ensated by the

decrease in entropy of the heat bath. For him, since there has b een no change in total entropy, the

entire pro cess of exp eriment 2 is reversible. Indeed, he has only to move the pistons back slowly to

their original p ositions. In this he must give back the work W , whereup on the argon is returned

to its original unmixed condition and the heat Q is returned to the heat bath.

Both of these observers can in turn b e tricked into thinking that they see a violation of the

second lawby a third one, still b etter informed, who knows that A2 is actually comp osed of two

comp onents A2a and A2b and there is a subkalic element Who ofnium { and so on ad in nitum.

10 7. The Pauli Analysis 10

Aphysical system always has more macroscopic degrees of freedom b eyond what we control or

observe, and by manipulating them a trickster can always make us see an apparent violation of the

second law.

Therefore the correct statement of the second law is not that an entropy decrease is imp ossible

in principle, or even improbable; rather that it cannot be achievedreproducibly by manipulating the

macrovariables fX ;:::;X g that we have chosen to de ne our macrostate.Any attempt to write

1 n

a stronger law than this will put one at the mercy of a trickster, who can pro duce a violation of it.

But recognizing this should increase rather than decrease our con dence in the future of the

second law, b ecause it means that if an exp erimenter ever sees an apparent violation, then instead of

issuing a sensational announcement, it will b e more prudent to search for that unobserved degree of

freedom. That is, the connection of entropy with information works b oth ways; seeing an apparent

decrease of entropy signi es ignorance of what were the relevant macrovariables.

7. The Pauli Analysis

Consider now the phenomenological theory. The Clausius de nition of entropy determines the

di erence of entropyoftwo thermo dynamic states of a closed system no particles enter or leave

that can b e connected by a reversible path:

7 S S =

2 1

Many are surprised when we claim that this is not necessarily extensive; the rst reaction is:

\Surely,two bricks havetwice the heat capacity of one brick; so how could the Clausius entropy

p ossibly not b e extensive?" To see how, note that the entropy di erence is indeed prop ortional

to the number N of molecules whenever the heat capacity is prop ortional to N and the pressure

dep ends only on V=N ; but that is not necessarily true, and when it is true it is far from making

the entropy itself extensive.

For example, let us evaluate this for the traditional ideal monoatomic gas of N molecules and

consider the indep endent thermo dynamic macrovariables to b e T; V; N . This has an equation

of state PV = NkT and heat capacity C =3=2 Nk, where k is Boltzmann's constant. From

this all elementary textb o oks nd, using the thermo dynamic relations @ S=@ V =@ P =@ T and

T V

T @ S=@ T = C :

V v

@S @S

S T ;V ;N S T ;V ;N= dV + dT

2 2 1 1

@V @T

T V

3 T V

2 2

+ Nk log = Nk log

V 2 T

1 1

It is evident that this is satis ed byanyentropy function of the form

S T; V; N= k N log V + N log T + kf N 9

where f N is not an arbitrary constant, but an arbitrary function. The p oint is that in the

reversible path 7 wevaried only T and V ; consequently the de nition 7 determines only the

dep endence of S on T and V . Indeed, if N varied on our reversible path, then 7 would not b e

dN =T would b e needed. correct an extra `entropy of convection' term

11 11

Pauli 1973 noticed this incompleteness of 7 and saw that if we wish entropy to b e extensive,

then that is logically an additional condition, that wemust imp ose separately. The extra condition

is that entropy should satisfy the scaling law

S T; qV; qN = qS T; V; N; 0

Then, substituting 9 into 10, we nd that f N must satisfy the functional equation

f qN = qf N qN log q 11

Di erentiating with resp ect to q and setting q = 1 yields a di erential equation Nf N = f N N ,

whose general solution is

f N = Nf1 N log N: 12

[alternatively, just set N = 1 in 11 and we see the general solution]. Thus the most general

extensiveentropy function for this gas has the form

V 3

S T; V; N= Nk log + log T + f 1 : 13

N 2

It contains one arbitrary constant, f 1, which is essentially the chemical constant. This is not

determined by either the Clausius de nition 7 or the condition of extensivity 10; however, one

3=2

more fact ab out it can b e inferred from 13. Writing f 1 = log Ck , wehave

3=2

CV kT

S T; V; N= Nk log : 14

The quantity in the square brackets must b e dimensionless, so C must have the physical dimensions

1 3=2 3=2 3

of v ol ume ener g y =mass action .Thus on dimensional grounds we can rewrite

13 as

V 3 mk T

S T; V; N= Nk log : 15 + log

N 2

where m is the molecular mass and is an undetermined quantity of the dimensions of action.

It might app ear that this is the limit of what can b e said ab out the entropy function from phe-

nomenological considerations; however, there may b e further cogent arguments that have escap ed

our attention.

8. Would Gibbs Have Accepted It?

Note that the Pauli analysis has not demonstratedfrom the principles of physics that entropy

actual ly should be extensive ; it has only indicated the form our equations must take if it is. But

this leaves op en two p ossibilities:

1 All this is a temp est in a teap ot; the Clausius de nition indicates that only entropy

di erences are physically meaningful, so we are free to de ne the arbitrary additive

terms in anywaywe please. This is the view that was taught to the writer, as a

student manyyears ago.

2 The variation of entropy with N is not arbitrary; it is a substantive matter with

exp erimental consequences. Therefore the Clausius de nition of entropy is logical ly

12 8. Would Gibbs Have Accepted It? 12

incomplete, and it needs to b e supplemented either by exp erimental evidence or fur-

ther theoretical considerations.

The original thermo dynamics of Clausius considered only closed systems, and was consistent with

conclusion 1. Textb o oks for physicists have b een astonishingly slowtomovebeyond it; that of

Callen 1960 is almost the only one that recognizes the more complete and fundamental nature of

Gibbs' work.

In the thermo dynamics of Gibbs, the variation of entropy with N is just the issue that is

involved in the prediction of vap or pressures, equilibrium constants, or any conditions of equilibrium

with resp ect to exchange of particles. His invention of the chemical p otential has, according to

all the evidence of physical chemistry, solved the problem of equilibrium with resp ect to exchange

of particles, leading us to conclusion 2; chemical thermo dynamics could not exist without it.

Gibbs could predict the nal equilibrium state of a heterogeneous system as the one with

maximum total entropy sub ject to xed total values of energy, mole numb ers, and other conserved

quantities. All of the results of his Heterogeneous Equilibrium follow from this variational principle.

He showed that by a mathematical Legendre transformation this could b e stated equally well

as minimum Gibbs free energy sub ject to xed temp erature and chemical p otentials, which form

physical chemists have generally found more convenient however, it is also less general; in exten-

sions to nonequilibrium phenomena there is no temp erature or Gibbs free energy, and one must

return to the original maximum entropy principle, from which many more physical predictions may

b e derived.

This preference for one form of the variational principle over the other is largely a matter of

our conditioning from previous training. For example, from our mechanical exp erience it seems

intuitively obvious that a liquid sphere has a lower energy than any other shap e with the same

entropy; yet to most of us it seems far from obvious that the sphere has higher entropy than any

other shap e with the same energy.

It is interesting to note how the various classical textb o oks deal with the question of extensivity;

and then how Gibbs dealt with it. Planck 1926, p. 92, Epstein 1937; p. 136 and Zemansky 1943;

p. 311 do not give the scaling law 10, but simply write entropy as extensive from the start,

apparently considering this to o obvious to b e in need of any discussion. Callen 1960; p. 26 do es

write the scaling law, but do es not derive it from anything or deriveanything from it; thereafter

he pro ceeds to write entropy as extensive without further discussion.

In other words, all of these works simply assume entropy to b e extensive without investigating

the question whether that extensivity follows from, or is even consistent with, the Clausius de nition

of entropy. More recent textb o oks tend to b e even more careless in the logic; the only exception

known to us is that of Pauli. But let us note howmuch is lost thereby:

a The question of extensivity cannot haveany universally valid answer; for there are systems,

for example some spin systems and systems with electric charge or gravitational forces, for which

the scaling law 10 do es not hold b ecause of long{range interactions; yet the Clausius de nition

of entropy is still valid as long as N is held constant. Such systems cannot b e treated at all bya

formalism that starts by assuming extensivity.

b This reminds us that for any system extensivityofentropy is, strictly sp eaking, only an ap-

proximation which can hold only to the extent that the range of molecular forces is small compared

to the dimensions of the system. Indeed, for virtually all systems small deviations from exact

extensivity are observable in the lab oratory; the exp erimentalist calls them \surface e ects" and

we note that Gibbs' Heterogeneous Equilibrium gives b eautiful treatments of surface tension and

electro capillarity, all following from his single variational principle.

13 13

Obviously, then, Gibbs did not assume extensivity as a general prop erty, as did the others noted

ab ove. But of course, Gibbs was in agreement with them, that entropyisvery nearly extensive for

most single homogeneous systems of macroscopic size. But he is careful in saying this, adding the

quali cation \- - - , for many substances at least, - - -". Then he gives such relations as G = n

i i

which hold in the cases where entropy can b e considered extensive to sucient accuracy.

But Gibbs never do es give an explicit discussion of the circumstances in whichentropyisor

is not extensive! He app ears at rst glance to evade the issue by the device of talking only ab out

the total entropy of a heterogeneous system, not the entropies of its separate parts. Movementof

particles from one part to another is disp osed of by the observation that, if it is reversible, then the

total entropyisunchanged; and that is all he needs in order to discuss all his phenomena, including

surface e ects, fully.

But on further meditation we realize that Gibbs did not evade any issue; rather, his far

deep er understanding enabled him to see that there was no issue. He had p erceived that, when

two systems interact, only the entropy of the whole is meaningful.Todaywewould say that the

interaction induces correlations in their states which makes the entropy of the whole less than the

sum of entropies of the parts; and it is the entropy of the whole that contains full thermo dynamic

information. This reminds us of Gibbs' famous remark, made in a supp osedly but p erhaps not

really di erent context: \The whole is simpler than the sum of its parts." How could Gibbs have

p erceived this long b efore the days of quantum theory?

This is one more example where Gibbs had more to say than his contemp oraries could absorb

from his writings; over 100 years later, we still have a great deal to learn by studying how Gibbs

manages to do things. For him, entropy is not a lo cal prop erty of the subsystems; it is a \global"

prop erty like the Lagrangian of a mechanical system; i.e. it presides over the whole and determines,

by its variational prop erties, all the conditions of equilibrium { just as the Lagrangian presides over

all of mechanics and determines, by its variational prop erties, all the equations of motion.

Up to this p ointwehave b een careful to consider only the phenomenology and exp erimental

facts of thermo dynamics, in order to make their indep endent logical status clear. Most discussions

of these matters mix up the statistical and phenomenological asp ects from the start, in a way

that we think generates and p erp etuates confusion. In particular, this has obscured the fact that

the fundamental op erational de nitions of such terms as equilibrium, temp erature, and entropy

{ and the statements of the rst and second laws{involve only the macrovariables observed in

the lab oratory. They make no reference to microstates, much less to anyvelo city distributions,

probability distributions, or correlations. As Helmholtz and Planck stressed, this much of the eld

hasavalidity and usefulness quite indep endent of whether atoms and microstates exist.

9. Gibbs' Statistical Mechanics

Nowwe turn to the nal work of Gibbs, which app eared 27 years after his Heterogeneous Equilib-

rium, and examine the parts of it which are relevant to these issues. We exp ect that a statistical

theory might supplement the phenomenology in twoways. Firstly, if our present microstate theory

is correct, wewould get a deep er interpretation of the basic reason for the phenomenology, and a

b etter understanding of its range of validity; if it is not correct, we might nd contradictions that

would provide clues to a b etter theory. Secondly, the theoretical explanation would predict gen-

eralizations; the range of p ossible nonequilibrium conditions is many orders of magnitude greater

than that of equilibrium conditions, so if new repro ducible connections exist they would b e almost

imp ossible to nd without the guidance of a theory that tells the exp erimenter where to lo ok.

It is generally supp osed that Gibbs coined the term \Statistical Mechanics" for the title of this

b o ok. But in reading Gibbs' Heterogeneous Equilibrium we slowly develop ed a feeling, from the

ab ove handling of entropy and other incidents in it, that his thermo dynamic formalism corresp onds

14 9. Gibbs' Statistical Mechanics 14

so closely to that of Statistical Mechanics { in particular, the grand canonical ensemble { that he

must have known the main results of the latter already while writing the former.

This suspicion was con rmed in part by the discovery that the term \Statistical Mechanics"

app ears already in an Abstract of his dated 1884, of a pap er read at a meeting but which, to the b est

of our knowledge, was never published and is lost. The abstract states that he will b e concerned

with Liouville's theorem and its applications to astronomy and thermo dynamics; presumably, the

latter is what app ears in the rst three Chapters of his Statistical Mechanics,18years later.

It seems likely, then, that Gibbs found the main results of his Statistical Mechanics very early;

p erhaps even as early as his attending Liouville's lectures in 1867. But he delayed nishing the

b o ok for manyyears, probably b ecause of mysteries concerning sp eci c heats that he kept hoping

to resolve, but could not b efore the days of quantum mechanics; his remarks in the preface indicate

howmuch this b othered him. The parts that seemed unsatisfactory would have b een left unwritten

in nal form until the last p ossible moment; and those parts would b e the ones where we are most

likely to nd small errors or incomplete statements.

Gibbs' concern ab out sp eci c heats is no longer an issue for us to day, but wewant to understand

why classical statistical mechanics app eared at least to readers of the b o ok to fail badly in

the matter of extensivityofentropy.Heintro duces the canonical ensemble in Chapter IV, as a

probability density P p :::q in phase space, and writes his Eq. 90; hereafter denoted by SM.90,

1 n

etc.:

= = log P SM:90

where = p :::q is the Hamiltonian same as the total energy, since he considers only con-

1 n

servative forces, is the \mo dulus" of the distribution, and is a normalization constantchosen

so that Pdp ;:::;dq = 1. He notes that has prop erties analogous to those of temp erature,

1 n

and strengthens the analogy byintro ducing externally variable co ordinates a ;a with the mean-

1 2

ing of volume, strain tensor comp onents, heightinagravitational eld, etc. with their conjugate

forces A = @ =@ a . On an in nitesimal change i.e., comparing two slightly di erent canonical

i i

distributions he nds the identity

d = d A da SM:114

i i

where the bars denote canonical ensemble averages. He notes that this is identical in form with a

as the analog of entropy, which thermo dynamic equation if we neglect the bars and consider

he denotes, incredibly,by .

Here Gibbs anticipates { and even surpasses { the confusion caused 46 years later when Claude

Shannon used H for what Boltzmann had called H . In addition, the prosp ective reader is warned

that from p. 44 on, Gibbs uses the same symbol to denote b oth the \index of probability" as in

SM.90, and the thermo dynamic entropy, as in SM.116; his meaning can b e judged only from

the context. In the following we deviate from Gibbs' notation by using Clausius' symbol S for

thermo dynamic entropy.

Note that Gibbs, writing b efore the intro duction of Boltzmann's constant k , uses , with the

dimensions of energy, for what we should to day call kT ; consequently his thermo dynamic analog

of entropy is what we should to day call S=k , and is dimensionless.

In this resp ect Gibbs' notation is really neater formally { and more cogentphysically { than ours; for

Boltzmann's constant is only a correction factor necessitated by our custom of measuring temp erature in

arbitrary units derived from the prop erties of water. A really fundamental system of units would have

k 1 by de nition.

15 15

The thermo dynamic equation analogous to SM.114 is then

d = TdS A da SM:116

i i

Now Gibbs notes that in the thermo dynamic equation, the entropy S \is a quantity whichis

only de ned by the equation itself, and incompletely de ned in that the equation only determines

its di erential, and the constantofintegration is arbitrary. On the other hand, the in the

statistical equation has b een completely de ned." Then interpreting asentropy leads to the

familiar conclusion, stressed by later writers, that entropy is not extensive in classical statistical

mechanics.

Right here, we suggest, two ne p oints were missed, but the Pauli analysis conditions us to

recognize them at once. The rst is that normalization requires that P has the physical dimen-

sions action , while the argument of a logarithm should b e dimensionless. To obtain a truly

,we should rewrite SM.90 as = log P where is some quantity of the dimensionless

dimensions of action. This p oint has, of course, b een noted b efore many times.

The second ne p oint is more serious and do es not seem to have b een noted b efore, even by

Pauli; the question whether is the prop er statistical analog of thermo dynamic entropy cannot

b e answered merely by examination of SM.114. Let us denote by the correct dimensionless

statistical analog of entropy that Gibbs was seeking. The trouble is again that in SM.114 we are

varying only and the a ; consequently it determines only how varies with and the a .Asin

i i

9, from Gibbs' SM.114 we can infer only that the correct statistical analog of entropymust have

the form

= + g N ; 16

where N = n=3 denotes as b efore the numb er of particles. Again, the \constantofintegration" is

not an arbitrary constant, but an arbitrary function g N . Clearly, no de nite \statistical analog"

of entropy has b een de ned until the function g N is sp eci ed.

However, in defense of Gibbs, we should note that at this p oint he is not discussing extensivity

of entropy at all, and so he could reply that he is considering only xed values of N , and so is in

fact concerned only with an arbitrary constant. Thus one can argue whether it was Gibbs or his

readers who missed this ne p oint it is only 160 pages later, in the nal two paragraphs of the

b o ok, that Gibbs turns at last to the question of extensivity.

In anyevent, a p oint that has b een missed for so long deserves to b e stressed. For 60 years, all

scientists have b een taught that in the issue of extensivityofentropywehave a fundamental failure,

not just of classical statistics, but of classical mechanics, and a triumph of quantum mechanics.

The present writer was caught in this error just as badly as anyb o dy else, throughout most of his

teaching career. But now it is clear that the trouble was not in any failure of classical mechanics,

but in our own failure to p erceive all the freedom that Gibbs' SM.114 allows. If we wish entropy

to b e extensive in classical statistical mechanics, wehave only to cho ose g N accordingly, as was

necessary already in the phenomenological theory. But curiously, nob o dy seems to have noticed,

much less complained, that Clausius' de nition S dQ=T also failed in the matter of extensivity.

But exactly the same argument will apply in quantum statistical mechanics; in making the

connection b etween the canonical ensemble and the phenomenological relations, if we follow Gibbs

and use the Clausius de nition of entropy as our sole guide in identifying the statistical analog of

entropy, it will also allow an arbitrary additive function hN and so it will not require entropyto

b e extensiveany more than do es classical theory. Therefore the whole question of in what way{ or

indeed, whether { classical mechanics failed in comparison with quantum mechanics in the matter

of entropy,now seems to b e re{op ened.

16 9. Gibbs' Statistical Mechanics 16

Recognizing this, it is not surprising that entropy has b een a matter of unceasing confusion

and controversy from the day Clausius discovered it. Di erent p eople, lo oking at di erent asp ects

of it, continue to see di erent things b ecause there is still un nished business in the fundamental

de nition of entropy, in b oth the phenomenological and statistical theories.

In the case of the canonical ensemble, the oversight ab out extensivity is easily corrected, in

away exactly parallel to that indicated byPauli. In the classical case, considering a gas de ned

by the thermo dynamic variables A = p = pressure, a = V =volume, Gibbs' statistical analog

1 1

equation SM.114 may b e written

= d pdV 17 d

The statistical analog of entropymust have the form 16; and if wewant it to b e extensive, it must

also satisfy the scaling law 10. Now from the canonical ensemble for an ideal gas, with phase

space probability density

p 1

exp 18 P =

3N=2 N

2m V

whichwe note is dimensionally correct and normalized, although it do es not contain Planck's

constant, wehave from the dimensionally corrected SM.90,

= N log V + [log2m + 1] 3N log 19

Then, substituting 19 and 16 into 10, we nd that g N must satisfy the same functional

equation 11, with the same solution 13. The Gibbs statistical analog of entropy 16 is now

3 3 2m V

+ + log + g 1 : 20 = N log

N 2 2

The extensive prop erty 9 now holds, and the constants g 1 and combine to form essentially the

chemical constant. This is not determined by the ab ove arguments, just as it was not determined

in the phenomenological arguments.

Therefore it might app ear that the shortcoming of classical statistical mechanics was not any

failure to yield an extensiveentropy function, but only its failure to determine the numerical value

of the chemical constant. But even this criticism is not justi ed at present; the mere fact that

it is b elieved to involve Planck's constant is not conclusive, since e and c are classical quantities

and Planck's constant is only a numerical multiple of e =c.We see no reason why the particular

numb er 137.036 should b e forbidden to app ear in a classical calculation. Since the problem has not

b een lo oked at in this way b efore, and nob o dy has tried to determine that constant from classical

physics, we see no grounds for con dent claims either that it can or cannot b e done.

But the apparentinvolvementofe and c suggests, as Gibbs himself noted in the preface

to his Statistical Mechanics, that electromagnetic considerations may b e imp ortanteven in the

thermo dynamics of electrically neutral systems. Of course, from our mo dern knowledge of the

electromagnetic structure of atoms, the origin of the van der Waals forces, etc., such a conclusion is

in no way surprising. Electromagnetic radiation is surely one of the mechanisms by which thermal

equilibrium is maintained in any system whose molecules have dip ole or quadrup ole moments,

rotational/vibrational sp ectrum lines, etc. and no system is in true thermal equilibrium until it is

bathed in black{b o dy radiation of the same temp erature.

At this p oint the reader maywonder: Whydowe not turn to the grand canonical ensemble, in

which all p ossible values of N are represented simultaneously? Would this enable us to determine

17 17

byphysical principles howentropyvaries with N ? Unfortunately, it cannot do so as long as we try

to set up entropy in statistical mechanics by nding a mere analogy with the Clausius de nition of

entropy, S = dQ=T .For the Clausius de nition was itself logically incomplete in just this resp ect;

the information needed is not in it.

The Pauli correction was an imp ortant step in the direction of getting \the bulk of things" right

pragmatically; but it ignores the small deviations from extensivity that are essential for treatment

of some e ects; and in anyevent it is not a fundamental theoretical principle. A truly general

and quantitatively accurate de nition of entropymust app eal to a deep er principle which is hardly

recognized in the current literature, although we think that a cogent sp ecial case of it is contained

in some early work of Boltzmann, Einstein, and Planck.

10. Summary and Un nished Business.

Wehave shown here that the Clausius de nition of entropywas incomplete already in the phe-

nomenological theory; therefore it was inadequate to determine the prop er analog of entropyin

the statistical theory. The phenomenological theory, classical statistical mechanics, and quantum

statistical mechanics, were all silent on the question of extensivityofentropy as long as one triedto

identify the theoretical entropy merely by analogy with the Clausius empirical de nition. The Pauli

typ e analysis is a partial corrective, which applies equally well and is equally necessary in all three

cases, if we wish entropy to b e extensive.

It is curious that Gibbs, who surely recognized the incompleteness of Clausius' de nition, did

not undertaketogive a b etter one in his Heterogeneous Equilibrium; he merely pro ceeded to use

entropy in pro cesses where N changes and never tried to interpret it in such terms as phase volume,

as Boltzmann, Planck, and Einstein did later. In view of Gibbs' p erformance in other matters, we

cannot supp ose that this was a mere oversight or failure to understand the logic. More likely,it

indicates some further deep insight on his part ab out the diculty of such a de nition.

A consequence of the ab ove observations is that the question whether quantum theory really

gave an extensiveentropy function and determined the value of the chemical constant, also needs

to b e re{examined. Before a quantum theory analog of entropy has b een de ned, one must consider

pro cesses in which N changes. If we merely apply the scaling lawaswe did ab ove, the quantum

analog of entropy will have the form 20 in which is set equal to Planck's constant; but a new

arbitrary constant f 1 is intro duced that has not b een considered heretofore in quantum

statistics. If we continue to de ne entropyby the Clausius de nition 7, quantum statistics do es

not determine .

This suggests that, contrary to common b elief, the value of the chemical constant is not

determined by quantum statistics as currently taught, any b etter than it was by classical theory.

It seems to b e a fact of phenomenology that quantum statistics with = 0 accounts fairly well

for a numb er of measurements; but it gives no theoretical reason why should b e zero. The

Nernst Third Law of Thermo dynamics do es not answer this question; we are concerned rather

with the exp erimental accuracy of such relations as the Sackur{Tetro de formula for vap or pressure.

Exp erimental vap or pressures enable us to determine the di erence . Therefore we

gas liq uid

wonder how go o d is the exp erimental evidence that is not needed, and for how many substances

wehave such evidence.

For some 60 years this has not seemed an issue b ecause one thought that it was all settled

in favor of quantum theory; any small discrepancies were ascrib ed to exp erimental diculties and

held to b e without signi cance. Now it app ears that the issue is reop ened; it may turn out that

is really zero for all systems; or it may b e that giving it nonzero values may improve the accuracy

of our predictions. In either case, further theoretical work will b e needed b efore we can claim to

understand entropy.

18 REFERENCES 18

There is a conceivable simple resolution of this. Let us conjecture that the present common

teaching is correct; i.e., that new precise exp eriments will con rm that present quantum statistics

do es, after all give the correct chemical constants for all systems. If this conjecture proves to b e

wrong, then some of the following sp eculations will b e wrong also.

We should not really exp ect that a phenomenological theory, based necessarily on a nite

numb er of observations that happ ened to feasible in the time of Clausius { or in our time { could

provide a full de nition of entropy in the greatest p ossible generality. The question is not an

empirical one, but a conceptual one; and only a de nite theoretical principle, that rises ab ove all

temp orary empirical limitations, can answer it. In other words, it is a mistake to try to de ne

entropy in statistical mechanics merely by analogy with the phenomenological theory. The only

truly fundamental de nition of entropymust b e provided directly by the statistical theory itself;

and comparison with observed phenomenology takes place only afterward.

This is how relativity theory was constituted: empirical results like the Michelson{Morley

exp eriment mightwell suggest the principle of relativity; but the deduction of the theory from

this principle was made without app eal to exp eriment. After the theory was develop ed, one could

test its predictions on such matters as abb eration, time dilatation, and the orbits of fast charged

particles.

The answer is rather clear; for b oth Clausius and Gibbs the theoretical principles that were

missing did not lie in quantum theory that was only a quantitative detail. The fundamental

principles were the principles of probability theory itself; how do es one set up a probability distri-

bution over microstates { classical or quantum { that represents correctly our information ab out a

macrostate? If that information includes all the conditions needed in the lab oratory to determine

a repro ducible result { equilibrium or nonequilibriu m { then the theory should b e able to predict

that result.

We suggest that the answer is the following. For any system the entropy is a prop ertyof

the macrostate more precisely, it is a function of the macrovariables that we use to de ne the

macrostate, and it is de ned byavariational prop erty: it is the upp er b ound of k Tr log

over all density matrices that agree with those macrovariables. This will agree with the Clausius

and Pauli prescriptions in those cases where they were valid; but it automatically provides the

extra terms that Gibbs needed to analyze surface e ects, if we apply it to the nite sized systems

that actually exist in the lab oratory.

When thermo dynamic entropy is de ned by this variational prop erty, the long confusion ab out

order and disorder which still clutters up our textb o oks is replaced by a remarkable simplicity

and generality. The conventional Second Law follows a fortiori : since entropy is de ned as a

constrained maximum, whenever a constraint is removed, the entropy will tend to increase, thus

paralleling in our mathematics what is observed in the lab oratory. But it provides generalizations

far b eyond that, to many nonequilibrium phenomena not yet analyzed byany theory.

And, just as the variational formalism of Gibbs' Heterogeneous Equilibrium could b e used

to derive useful rules of thumb like the Gibbs phase rule, whichwere easier to apply than the full

formalism, so this generalization leads to simple rules of thumb like the phase volume interpretation

S = k log W of Boltzmann, Einstein, and Planck, which are not limited to equilibriu m conditions

and can b e applied directly for very simple applications like the calculation of muscle eciency in

biology Jaynes, 1989.

REFERENCES

Callen, H. B. 1960, Thermodynamics, J. Wiley & Sons, Inc., New York.

Epstein, P. S. 1937, Textbook of Thermodynamics, J. Wiley & Sons, Inc., New York.

19 19

Gibbs, J. Willard 1875{78 \On the Equilibrium of Heterogeneous Substances", Connecticut Acad.

Sci. Reprinted in The Scienti c Papers of J. Wil lard Gibbs,byDover Publications, Inc., New

York 1961.

Gibbs, J. Willard 1902, Elementary Principles in Statistical Mechanics,Yale University Press,

New Haven, Conn. Reprinted in The Col lected Works of J. Wil lard Gibbs,Vol.2byDover

Publications, Inc., New York 1960.

Jaynes, E. T. 1965, \Gibbs vs Boltzmann Entropies", Am. Jour. Phys. 33, 391{398.

Jaynes, E. T. 1967, \Foundations of Probability Theory and Statistical Mechanics", in Delaware

Seminar in Foundations of Physics, M. Bunge, Editor, Springer{Verlag, Berlin. Reprinted in

Jaynes 1983.

Jaynes, E. T. 1983, Papers on Probability, Statistics, and Statistical Physics, D. Reidel Publishing

Co., Dordrecht, Holland, R. D. Rosenkrantz, Editor. Reprints of 13 pap ers.

Jaynes, E. T. 1986 \Macroscopic Prediction", in Complex Systems { Operational Approaches,H.

Haken, Editor; Springer{Verlag, Berlin.

Jaynes, E. T. 1988, \The Evolution of Carnot's Principle", in Maximum Entropy and Bayesian

Methods in Science and Engineering ,Vol. 1, G. J. Erickson and C. R. Smith, Editors; Kluwer

Academic Publishers, Dordrecht, Holland.

Jaynes, E. T. 1989, \Clearing up Mysteries; the Original Goal", in Pro ceedings of the 8'th Inter-

national Workshop in Maximum Entropy and Bayesian Metho ds, Cambridge, England, August

1{5, 1988; J. Skilling, Editor; Kluwer Academic Publishers, Dordrecht, Holland.

Klein, M. J. 1973, \The Development of Boltzmann's Statistical Ideas", in E. C. D. Cohen & W.

Thirring, eds., The Boltzmann Equation , Springer{Verlag, Berlin; pp. 53{106

Pauli, W. 1973, Thermodynamics and the Kinetic Theory of Gases ,Pauli Lectures on Physics,

Vol. 3; C. P. Enz, Editor. MIT Press, Cambridge MA.

Planck, M. 1926, Treatise on Thermodynamics,Dover Publications, Inc., N. Y.

Zemansky, M. W. 1943 Heat and Thermodynamics , McGraw{Hill Bo ok Co., New York.