Edited by Emanuel Lotem

Copyright © 2013 Kafri NihulVe’Hashkaot Ltd.

All rights reserved.

ISBN-13: 978-1482687699eBook ISBN: 978-1-63003-433-7Library of Congress Control Number(LCCN): 201390681

“Life is much more successfullylooked at from a single windowafter all.”

-F. Scott Fitzgerald, TheGreat Gatsby

Contents

Prologue

Part I Physical Entropy

Chapter 1 Entropy and the Flowof Energy

Carnot’s Efficiency

The Clausius Entropy

Chapter 2 Statistical Entropy

The Boltzmann Entropy

The Gibbs Entropy

The Maxwell-BoltzmannDistribution

Chapter 3 The Entropy ofElectromagnetic Radiation

Electromagnetic Radiation

Black body Radiation

Black Holes

Planck’s Equation

Physical Entropy – A Summary

Part II Logical Entropy

Chapter 4 The Entropy of

Information

Shannon’s Entropy

Benford’s Law

Chapter 5 Entropy in SocialSystems

The Distribution of Links in SocialNetworks – Zipf’s Law

The Distribution of Wealth – ThePareto Law

Polls and Economics – The Planck-Benford Distribution

Conclusion

Appendices

Appendix A-1 Clausius’s Derivation of Entropy fromCarnot’s Efficiency

Appendix A-2 Planck’s Derivation of Boltzmann’sEntropy

Appendix A-3 Gibbs’s Entropy

Appendix A-4 Exponential Energy Decay and theMaxwell-Boltzmann Distribution

Appendix A-5 The Temperature of an Oscillator

Appendix A-6 Energy Distribution in Oscillators

Appendix B-1 File Compression

Appendix B-2 Distribution of Digits in Decimal Files –Benford’s Law

Appendix B-3 Planck-Benford Distribution – Zipf’s law

Biographical Sketches

Bibliography

Endnotes

Acknowledgements

This book is dedicated to the lateOfra Meir, our sister and sister-in-law, who had taken part innumerous discussions before theactual writing of this book began,

and passed away untimely beforeediting the Hebrew manuscript.

We would like to thank the editorof the this book, the indefatigablyintense and knowledgeableEmanuel Lottem, who left no stoneunturned, fixed, polished, checked,and even added examples of hisown. He belongs to a rare breed indanger of extinction.

One of us (O. K.) would like tothank his colleagues for discussionsin which much learned was beforewriting this book: Rafi Levine,Joseph Agassi, Yehuda Band, YarivKafri, and especially Joseph Oreg,who even corrected a derivation.

We would like to thank those thatread the original manuscript andmade significant suggestions: AlinaTal, Joshua Maor, Daniel Rosenne,Noa Meir-Leverstein, RachelAlexander, Zvi Aloni, MoshePereg, Galit Kafri, and Ehud Kafri.Special thanks to Dan Weiss for thenumerical calculations and figures.

Prologue

Mathematician Arnold Sommerfeld(1868–1951), one of the foundingfathers of atomic theory, said about

thermodynamics, the branch ofphysics that incorporates theconcept of entropy:

Thermodynamics is a funny subject.The first time you go through it, youdon’t understand it at all. Thesecond time you go through it, youthink you understand it, except forone or two small points. The thirdtime you go through it, you knowyou don’t understand it, but by thattime you are so used to it, it doesn’tbother you anymore.1

This is true of many more thingsthan we would like to imagine, but

it is especially true of our subjectmatter, namely entropy.

Entropy is a physical quantity, yetit is different from any otherquantity in nature. It is definite onlyfor systems in a state ofequilibrium, and it tends toincrease: in fact, entropy’s tendencyto increase is the source of allchange in our universe.

Since the concept of entropy wasfirst discovered during a study ofthe efficiency of heat engines, thelaw that defines entropy and itsproperties is called the second lawof thermodynamics. (This law, asits name implies, is part of the

science called thermodynamics,which deals with energy flows.)Despite its nondescript name,however, the second law is actuallysomething of a super-law,governing nature as it does.

It is generally accepted thatthermodynamics has four laws,designated by numbers, although notnecessarily in order of importance,and not in order of their discovery.

1. The first law, also knownas the law of conservationof energy, states that theenergy in an isolated

system remains constant.

2. The second law is the onewith which this book isconcerned, and we shallsoon go back to discuss it.

3. The third law states that atthe temperature of-273°C, an object isdevoid of all energy(meaning that it isimpossible to coolanything to a temperaturelower than that).

4. There is another law,called the zero law, whichdeals with the meaning ofthermal equality betweenbodies.

The second law ofthermodynamics concerns theinherent uncertainty in nature, whichis independent of forces. Becausethe nature of particles and the natureof forces are not intrinsically partsof the second law, we may concludethat any calculation basing on thislaw that will be made in some areaof physics, will also apply to other

areas as well.The main quantity behind the

second law of thermodynamics isentropy – which is a measure ofuncertainty. The forces deployed byentropy are not as strictlydetermined as those described byother laws of nature. They areexpressed as a tendency of changesto occur, in a manner somewhatanalogous to human will. In orderfor entropy to increase, energy mustflow; and any flow of energy thatleads to change, increases entropy.This means that any action in nature– whether “natural” or man-made –increases entropy. In other words,

entropy’s tendency to increase is thetendency of nature, including ushumans, to make energy flow.

The importance of entropy isimmeasurable, yet it is a safe betthat the average educated person,who is familiar without doubt withsuch terms as relativity, gravity,evolution and other scientificconcepts, may never have heard ofit, or else misunderstands it. Foreven those familiar with the conceptof entropy admit that while theymay understand its mathematicalproperties, they cannot alwayscomprehend its meaning. So what,then, is entropy? We hope that after

reading this book you willunderstand what it is, why it issignificance, and how it is reflectedin everything around us.

This book has two main sections:The first section deals with the

historical development of thesecond law of thermodynamics,from its beginning in the early 19th

century to first decades of the 20th.Here, the physical aspects of theconcept of entropy will bediscussed, along with the energydistributions that arise due toentropy’s propensity to increase.

The second part of this book

deals with the effects of entropy incommunications, computers andlogic (studied since the beginning ofthe 1940s), along with the influenceentropy has had on various socialphenomena.

The first section is concernedwith thermodynamics, which is, inessence, the study of energy flows.Every physical entity, whatever itsnature, ultimately involves energy.We consume energy in order to liveand we release energy when wedie. Our thoughts are energyflowing through our neurons. We putin energy to communicate andconsume energy to keep warm.

Since energy cannot be created (ordestroyed), this means it has tocome, or flow, from elsewhere.Why does energy flow, and whatprinciples guide it? Well, the reasonenergy flows is this: there is atheoretical quantity that is notdirectly measurable, called entropy,and it tends to increase. In order forit to increase, energy must flow.

The first person who quantifiedthis flow of energy, in 1824, was ayoung French aristocrat andengineer called Sadi Carnot. Heunderstood that in order to obtainwork – for example, to apply aforce in order to move an object

from one place to another in space– heat must flow from a hot area toa colder one. He calculated themaximum amount of work that canbe obtained by transferring a givenamount of energy between objectshaving two different temperatures,and thus laid the foundations ofthermodynamics.

About forty years later, in 1865,physicist Rudolf Clausius, a fanaticPrussian nationalist, formulated thelaws of thermodynamics anddefined a mysterious quantity thathe named “entropy.” Entropy, asClausius defined it, is the ratiobetween energy and temperature.

About twelve years later, in1877, an eccentric Austrian,Ludwig Boltzmann, and anunassuming American, JosiahWillard Gibbs, derived an equationfor entropy – concurrently butindependently. They describedentropy as the lack of informationassociated with a statistical system.At that point, the second lawobtained some unexpectedsignificance: it was now understoodthat uncertainty in nature has atendency to increase.

At approximately the same time,Scottish estate-owner James ClerkMaxwell, an illustrious figure in the

history of modern science –comparable only to Galileo,Newton, or Einstein – calculatedenergy distribution in gases anddemonstrated how energy isdistributed among the gas particlesin accordance with the second law.

Some forty years have passed,the 20th century began, sciencefound itself addressing the problemof electromagnetic radiation that isemitted by all objects, anddiscovered that it is determinedsolely by their temperature. Butthere was an anomaly: although theradiation equation had already beenformulated by Maxwell, the

intensity of the radiation as afunction of temperature refused tobehave according the laws ofthermodynamics, as understood atthe time.

This “black body radiation”problem, as it was known, wassolved in 1901 by Max Planck, aGerman intellectual. He showedthat if the energy distribution iscalculated according to the secondlaw of thermodynamics whileassuming that radiation energy isquantized (that is, comes in discreteamounts), the distribution observedexperimentally is the oneanticipated theoretically. Planck’s

discovery started the quantumrevolution, a storm that raged in theworld of physics over the nextdecades. Planck’s calculated energydistribution was such thatMaxwell’s distribution turned out tobe a special case of the Planckdistribution. Later on, Planck’sdistribution was generalized evenmore, and today it is known as theBose–Einstein distribution(unfortunately, discussing it liesbeyond the scope of this book).

Up to this point, entropy had beendiscussed in purely physicalcontexts. Part Two of this book

discusses “logical entropy” andshows how the boundaries ofentropy were expanded to includethe field of signals transmission incommunication. This began when,during the Second World War, anAmerican mathematician andgadgets aficionado called ClaudeShannon demonstrated that theuncertainty inherent in a data file isits entropy. What was unique herewas that the entropy that Shannoncalculated was purely logical: thatis, it lacked any physicaldimension. In the second part of ourbook we shall explain under whatconditions physical entropy

becomes logical entropy, anddevote a special section tonumerical files and Benford’s Law,which states that the distribution ofdigits within a random numericalfile is uneven. We shall see that thisuneven distribution results from thesecond law.

The significance of the tendencyfor logical entropy to increase ismanifested in a spontaneousincrease of information, analogousto the spontaneous increases ofordered phenomena (life, buildings,roads, etc.) around us. Now, if thesecond law is indeed responsiblefor this creation of order, it should

show up as logical distribution. Weshall examine the distribution oflinks in networks, of wealth amongpeople, and of answers to polls,and we shall see that calculationsobtained from logicalthermodynamics do in fact firm upactual rules of thumb that have beenknown for some time. In economics,the Pareto rule (also known as the80:20 Law) is quite famous. Inlinguistics, there is Zipf’s Law.These rules, which physicistsclassify under the general name of“power laws,” are all derived fromcalculations made by Planck, basingon Boltzmann’s work, more than a

century ago.So it seems that contrary to

Einstein’s famous quote that “Goddoes not play dice with the world,”logical entropy and its expressionin probability distributions provesthat God does, indeed, play dice,and actually plays quite fairly.

This book has three layers:historical background, intuitiveexplanation and formal analysis.

The historical aspect includes thebiographies of the heroes of thesecond law – those scientists whosenames are perhaps not as familiar tothe general public as those of

singers, actors, artists and othercelebrities are, but as long ashumankind aspires to know more,they will be engraved forever inscience’s hall of fame. Our heroesare Carnot (a Frenchman), Clausius(a Prussian), Boltzmann (anAustrian), Maxwell (a Scotsman),Planck (a German), and Gibbs andShannon (Americans). Theirbiographies reveal the historicalmotives that informed thesescientists in their various researchefforts: the industrial revolution,science’s golden age, and the age ofinformation. It is important howeverto remember that these persons did

not do their work in a vacuum, so afew brief biographical sketches ofothers who worked alongside themor were otherwise affiliated withtheir work are given at the end ofthis book.

The second layer in this bookpresents intuitive explanations,those which scientists sometimescontemptuously dismiss as “handwaving arguments.” Yet theseintuitive explanations represent theessence of science and the essenceof culture. Of course, there is noalternative in science to formalmathematical analysis, but this hasvalue only so long as that it

quantifies the verbal explanationwhich consists of words, mentalimages and often “gut feelings” –which is the reason why they arecalled “intuitive.” Everymathematician knows that behindhis or her pure mathematicalequations resides intuitiveunderstanding, and everyoneaccepts that the laws under whichwe live are guided by intuition.

Finally, there is the third, formallayer, contained in the appendixesat the end of the book, providingmathematical derivations forreaders with technical andscientific background. If you

studied engineering, economics orthe physical sciences, you will beable to examine these mathematicalanalyses directly, without having tobother with monographs on thevarious subjects. The derivationswere chosen so that they wouldmatch the intuitive explanations inthe book.

This book was written jointly bya team of husband and wife. He – ascientist in the relevant fields:thermodynamics, optics andinformation science. She – aneducator and welfare worker,translator and editor. The processwhich led to this work was an

ongoing dialogue between twodifferent worlds, which, we hope,will make our book an enjoyable,educational, and enlighteningexperience. It is intended not onlyfor the general public interested inthe nature of things, but also forstudents, engineers and scientists.

Part I

Part I

Physical Entropy

Chapter 1

Entropy and the Flow ofEnergy

Carnot’s Efficiency

Thermodynamics came into beingduring the industrial revolution,which began in England and thenspread to Europe and America,from the mid-18th to the mid-19th

century. The industrial revolutionwas propelled by the developmentof the coal-burning steam engine.Initially this engine was used onlyin mining, but followingimprovements by James Watt(between 1781 and 1788), it spreadinto the textile industry, agricultureand transportation. This

technological revolution literallytransformed society and economy,first in Europe and then all over theglobe. Hitherto agricultural andcommercial societies, which usedwater, wind and animals as sourcesof power, became now urban-industrial societies using theenormously more powerful steamengines. Engineering, which untilthen had been concerned either withthe construction of buildings,bridges, and other structures, or formilitary purposes armsmanufacturing and fortificationbuilding, was now expanding toinclude new applications: machines

and engines for mining, metallurgyand agriculture, allowing quick andefficient production. Interestingly,until about 1840, the inventors ofearly technologies were actuallycraftsmen; only toward the latterpart of the 19th century did sciencebecome involved in industry, in apartnership that still goes on today.

One result of the mechanizationof agriculture was a reduction of theworkforce required, leading to aflow of people from rural areas tothe cities, launching the vastprocess of urbanization.2

It was only a matter of time

before the steam engine wasintroduced into the world oftransportation: first on land, then onsea. Indeed, Richard Trevithick’sfirst steam locomotive hadappeared back in 1804, but twenty-one years passed before this formof transportation came intopractical use in 1825, when the firsttracks for passenger-cargo trainswere laid between Stockton andDarlington, two industrial centers inEngland. By 1829, railways werelaid also in the United States andFrance (using steam locomotivesmanufactured in England).3

At sea, the process was more

gradual and the transition from sailto steam only took place by 1865.This was mainly because, initially,the amounts of coal steamships hadto carry for their engines were sohuge that it made them cumbersomeand inefficient.

The mechanical principles of asteam engine’s operation areactually very simple: a boiler fullof water is heated by burning coal,thus transforming water into steam.A valve installed in the boilerreleases the steam into a chamber,where it pushes against a piston thatmoves and turns a wheel. Thiswheel, depending on how it is set

up, is capable of moving trains orpropelling boats. In effect, thesteam pressure which drives thepiston is similar to the windpressure that drives a windmill,except that steam is controllable,hence always available – a greatadvantage for ships, for instance:now they were no longer dependingat the mercy of the wind.

The vast possibilities opened upby the invention of the steam engineseized the imagination of SadiCarnot, a twenty-eight-year-oldFrench engineer, and led to one ofthe most exciting and importantdevelopments which paved the way

to the science of thermodynamics.As Carnot wrote in a little book hepublished in 1824, which made himworld famous, Reflections on theMotive Power of Heat and onMachines Fitted to Develop thatPower:

Every one knows that heat canproduce motion. That it possessesvast motive-power no one candoubt, in these days when thesteam-engine is everywhere so wellknown.

To heat also are due the vastmovements which take place on theearth. It causes the agitations of the

atmosphere, the ascension of theclouds, the fall of rain andmeteors,* the currents of waterwhich employed but a smallportion. Even earthquakes andvolcanic eruptions are the result ofheat.

From this immense reservoir wemay draw the moving forcenecessary for our purposes. Nature,in providing us with combustibleson all sides, has given us the powerto produce, at all times and in allplaces, heat and the impellingpower which is the result of it. Todevelop this power, to appropriateit to our uses, is the object of heat-

engines.The study of these engines is of

the greatest interest, theirimportance is enormous, their use iscontinually increasing, and theyseem destined to produce a greatrevolution in the civilized world.

Already the steam-engine worksour mines, impels our ships,excavates our ports and our rivers,forges iron, fashions wood, grindsgrains, spins and weaves our cloths,transports the heaviest burdens, etc.It appears that it must some dayserve as a universal motor, and besubstituted for animal power,waterfalls, and air currents.

Over the first of these motors ithas the advantage of economy, overthe two others the inestimableadvantage that it can be used at alltimes and places withoutinterruption.

If, some day, the steam-engineshall be so perfected that it can beset up and supplied with fuel atsmall cost, it will combine alldesirable qualities, and will affordto the industrial arts a range theextent of which can scarcely bepredicted.

The safe and rapid navigation bysteamships may be regarded anentirely new art due to the steam-

engine. Already this art permittedthe establishment of prompt andregular communications across thearms of the sea, and on the greatrivers of the old and newcontinents. It has enabled us tocarry the fruits of civilization overportions of the globe where theywould else have been wanting foryears. Steam navigation bringsnearer together the most distantnations. In fact, to lessen the time,the fatigues, the uncertainties, andthe dangers of travel – is not this thesame as greatly to shortendistances?4

Sadi Carnot“Wherever there exists a difference oftemperature, motive-power can beproduced.”

– Sadi Carnot5

Sadi Nicolas Leonard Carnot,

1796-1832

Sadi Nicolas Leonard Carnot wasborn on June 1, 1796 in the Palacedu Petit-Luxembourg in Paris. Hisfather Lazare Carnot, a brilliantscientist in his own right, was afascinating and highly influentialfigure during the time of the FrenchRevolution. A statesman endowedwith a vast global outlook, hebelieved in the importance ofeducation and was one of the first toadvocate compulsory education forall. (Unfortunately, he was notsuccessful in convincing hisrevolutionary colleagues.) Carnot

senior deeply believed in involvingintellectuals, scientists andindustrialists in the policy makingprocess and making them nationalleaders, and encouraged many suchpersons to enlist for the good ofFrance. He strove to strengthenFrance from within, including theimprovement of her economy, as aprerequisite to influencing all ofEurope. When he took upon himselfresponsibility of the army in 1793,he quickly mobilized the leadingscientists and industrialists of theperiod and applied their ideas asquickly as he could to military,education and economic affairs.

Within five months of hisappointment, the failing and feebleFrench army was transformed, andfour years later it was to becomethe scourge of Europe. Hisinfluence on military strategy wasinvaluable, and his innovations,which were proven on thebattlefield, changed the face ofmodern warfare. He believed thatofficers should be appointed basedon qualifications rather than socialstatus, and had no qualms aboutdismissing officers born to thenobility who lacked ingenuity,replacing them with more qualifiedindividuals, even if the latter had no

social standing to speak of. Inkeeping with his vision, it wasLazare Carnot who bestowed uponthe lowly Corsican Napoleoncommand of the French military andsupported him in his rapid rise topower. In 1794 Carnot founded theÉcole Polytechnique, which withina few years became one of the mostoutstanding scientific institutions inEurope.

Before becoming involved inmilitary and political affairs,Carnot senior had written the“Essai sur les machines en general”(“An Essay about Machines inGeneral”, 1786), describing the

principles of energy involved in afalling mass, and gave the firstproof that kinetic energy is lostwhen semi-elastic bodies collide.He was a follower of the greatGerman scientist Gottfried WilhelmLeibniz (1646-1716), and appliedLeibniz’s principles about theconservation of mechanical energy.For example, he claimed that in animaginary perfect waterwheel, theenergy embodied in the waterwould neither be lost nor dispersed,and its movement would bereversible (that is, if the waterwheel was moved in reverse, itshould become a perfect pump).

Small wonder, then, that his sonSadi adopted his idea, expanded itto steam engines and opened up anew and important branch ofscience, as will be describedbelow.

In 1797 Lazare Carnot wasremoved from power, losing hisseat in the French Directorate thathad been established in 1795, andwas forced into exile for a shortwhile – first to Switzerland andthen to Germany. He returned in1799, when Napoleon, now FirstConsul of the French FirstRepublic, called Carnot back toFrance and made him Minister of

War. Carnot resigned this position ayear later due to difference ofopinions with Napoleon, and eventhough he continued to fill variouspolitical positions until 1807, hefound that his influence ingovernment declined, mainlybecause he was one of the fewindividuals who opposedNapoleon’s imperial aspirations.With time on his hands, he returnedto devote his time to science –publishing articles on geometry andworking in the scientific section ofthe engineering school that he hadfounded in 17946 – and to theeducation of his son, Sadi, who

would later become one of the mostbrilliant scientists of all time. Hehad named his son after the classicPersian poet whom he muchadmired, Sadi of Shiraz (c. 1213 –c. 1291).

Sadi Carnot entered the ÉcolePolytechnique in 1812, when hewas only sixteen. Among histeachers were the eminent Joseph-Louis Gay-Lussac, one of thediscoverers of gas theory; Siméon-Denis Poisson, a founder ofstatistical distribution studies; andAndré-Marie Ampère, one of thefounding fathers ofelectrodynamics. In 1814, upon

graduation at the Polytechnique,Carnot went on to study militaryengineering for two years at theÉcole du Génie in Metz.

In 1815 Napoleon returned torule France for the Hundred DaysReign, appointing Lazare Carnot asminister of the interior. But inOctober of that year, followingNapoleon’s defeat at Waterloo,Lazare was exiled to Magdeburg,Germany, where he remained forthe rest of his life.

Sadi Carnot finished his studiesand became an engineer in theFrench army. His job consisted ofinspecting fortifications, counting

bricks, repairing walls, drawing upplans and writing reports whichwere promptly buried in archives.Carnot despised his job, not onlybecause his reports were not giventhe attention he felt they deserved,but mainly because he was routinelyrefused promotion and granted jobsthat did not befit his talents. He nowrealized that his name, which upuntil a short while ago had madesycophants beat a path to hisdoorstep, was now a stumblingblock to his military career.

Carnot’s frustration and hisinability to satisfy his passion forcontinuing his education made him

take the entrance examinationrequired for position in the GeneralStaff Corps in Paris in 1818. Hepassed the tests and on January 20,1819 became a lieutenant. Almostimmediately, though, he took aleave of absence to devote himselfto his studies, which wereinterrupted only once, in January1821, when he traveled to Germanyto visit his father.

Carnot began to attend courses atvarious institutions of highereducation in Paris, including theSorbonne and the Collège deFrance. He plunged enthusiasticallyinto mathematics, chemistry,

technology and political economics.Besides, he was passionate aboutmusic and other fine arts, and alsopractices various sports, includingswimming, fencing and even iceskating, paying close attention to thetheories underlying these pursuits.At that time he became intriguedwith industrial issues and alsostudied the theory of gases. Duringhis visit to his father in 1821, thetwo undoubtedly discussed thesteam engine. The first engine hadreached Magdeburg three yearsbefore and had greatly interestedLazare. Sadi Carnot returned toParis full of enthusiasm and

determination to develop a theoryof steam engines.7

Carnot went deeper and deeperinto the research that wouldeventually lead him to the discoveryof a mathematical theory of heat –the precursor of modernthermodynamics. By 1822 he hadcarried out his first major work: anattempt to find a mathematicalexpression for the work producedby one kilogram of steam. It isobvious from the style of hiswriting that he had intended it forpublication, but it was neverpublished, and only discovered in1966. The publication that actually

made him famous was the one inwhich he calculated the efficiencyof the ideal steam engine – theCarnot cycle.

In 1824, at the tender age of 28,Carnot published the results of hissteam engines studies in a slimvolume whose title was typical ofthat period, almost two hundredyears ago: Réflexions sur lapuissance motrice du feu et sur lesmachines propres à développercette puissance (Reflections on theMotive Power of Fire and on theMachines Fitted to Develop thisPower). Hippolyte, his brother,who returned to Paris after their

father had died in exile in 1823,helped Sadi rewrite his ideas so asto make them more accessible to thegeneral public, more popular instyle and using just the absoluteminimum of basic equations;detailed calculations were suppliedin footnotes. The most importantpart of the book was an abstractdescription of the ideal engine,which explained and clarified thebasic principles which underlie anyheat engine, of whatevermechanical structure.

Carnot’s book did not gain theattention it deserved, and after itwas sold out a short time after its

publication in 1824, was notreprinted and for a while wasextremely hard to find. Even LordKelvin, a great leader in theformulators of the laws ofthermodynamics, had difficultyobtaining a copy. The work becameknown to a wider audience in 1834,after physicist Benoît Clapeyronpublished its analytic version, andit gained the respect it deservedonly after Kelvin quoted it a numberof times in 1848 and 1849.Fortunately, this work (with someadditional paragraphs not includedin the printed version) was saved ina printout of The Motive Power

made by Hippolyte in 1878. Whatlittle we know about Carnot’spersonal life is due to his brother,who wrote about him with love anddeep respect.

The title page of Carnot’sReflections on the Motive Powerof Fire and on the Machines Fittedto Develop this Power.

Carnot continued his research.Although he never published any ofit, some of his notes werepreserved. Judging by what hewrote between 1824 and 1826, hewas beginning to question the“caloric” theory of heat that wasaccepted at the time. He describedin his notes details of experimentsthat he intended to perform in orderto examine the influence oftemperature on liquids. Some of

these experiments were identical tothose that James Prescott Joule andLord Kelvin would carry out manyyears later.

Toward the end of 1826, therewas a reorganization of the GeneralStaff Corps in Paris and Carnot wasrecalled to full service again. Heserved for a year as a militaryengineer with the rank of captain,first in Lyon and later in Auxonne,but he was not happy in the militaryand felt it limited his freedom. Hetherefore retired completely andreturned to Paris, intending topursue his research further into thetheory of heat.

Carnot, who shared his father’spolitical views as an adamantrepublican, was happy with thedirection in which France wasmoving after the July Revolution of1830. At that time he becameinvolved in public life, especiallywith the improvement of publiceducation. He was offered agovernment position but declined,and with the restoration of themonarchy, returned to his scientificwork.

His health, which has never beengood, began to deteriorate. In thewords of Hippolyte:

His excessive application affectedhis health towards the end of June,1832. Feeling temporarily better, hewrote gaily to one of his friendswho had written several letters tohim: “My delay this time is notwithout excuse. I have been sick fora long time, and in a verywearisome way. I have had aninflammation of the lungs, followedby scarlet-fever. (Perhaps you knowwhat this horrible disease is.) I hadto remain twelve days in bed,without sleep or food, without anyoccupation, amusing myself withleeches, with drinks, with baths,and other toys out of the same shop.

This little diversion is not yetended, for I am still very feeble.”

There was a relapse, then brainfever; then finally, hardly recoveredfrom so many violent illnesseswhich had weakened him morallyand physically, Sadi was carried offin a few hours, August 24, 1832, byan attack of cholera. Towards thelast, and as if from a darkpresentiment, he had given muchattention to the prevailing epidemic,following its course with theattention and penetration that hegave to everything.8

Carnot was only thirty-six when he

died of cholera. For fear ofcontagion, his belongings, includingmany of his writings, were buriedwith him. Aside from his book, onlya handful of his writings remain.

Thus, the road leading to thescience of thermodynamics,including the formulation of itssecond law, began with Carnot andhis study of the efficiency of steamengines in 1824. It was not Carnotwho formulated the second law, norwas he aware of entropy.Nevertheless, he understoodintuitively – in all likelihood,

following on the conversations hehad had with his illustrious father –that if a machine, of any type,transferred energy from a hot placeto a colder one, then the maximumamount of mechanical work that onecould expect to obtain would be theamount of energy removed from thehotter object, less the amountabsorbed by the colder one. Indeed,this result was identical with thesolution to the problem which hadintrigued Lazare Carnot: What is themaximum work that can be obtainedfrom water that drives a turbine?The answer is that the amount ofwork expected cannot be greater

than the energy released when thewater falls down from a higherlevel to a lower one.

Mechanical work is defined as aforce applied to an object along agiven distance: this force willchange the object’s motion – itsspeed and/or its direction, namelyits velocity. Moving objects can beused for various purposes: intransportation, for example, it isused to change the spatial locationof an object; in a mixer, to kneaddough; in the circulatory system, totransport the oxygen required forour bodily functions. Indeed,movement is a characteristic of life

itself.Therefore we can say that motion

is an expression of energy. In fact,the amount of energy stored in themotion of an object is directlyproportional to the product of itsmass and the square of its velocity.Today we call this energy of motion“kinetic energy.” The first to definethe energy of a moving object, in1689, was Gottfreid WilhelmLeibniz, a German mathematicianand influential philosopher, whowas also the first one(simultaneously with Newton, butindependently) to develop calculus.Leibniz called this energy “vis

viva” – Latin for “live energy.”It is not too difficult to

understand intuitively that kineticenergy somehow corresponds tomechanical work, but in fact, it alsocorresponds to heat energy – in thiscase, not with respect to the objectas a whole, but rather to itsmolecules, atoms, electrons andother particles which constitutematter. In Carnot’s day, however,there was still no knowledge of theparticles of the matter; Atomictheory was way in the future; also,the prevailing belief was that heatwas some form of primeval liquidthat, when absorbed by any

material, caused it to be hot(somewhat the way an alcoholicbeverage causes a warm sensationin our throats). This liquid wascalled caloric (by AntoineLavoisier, in 1787).

There is another form of energy,one that is not linked at all tomotion, which is called “potentialenergy.” This energy is stored in anobject as a result of some force thathas acted (or is acting) upon it. Forexample, water situated up highcontains more energy than waterdown low, because work must havebeen done to overcome earth’sgravity in order to raise the water to

a higher level. If we allow thewater to flow back down, itspotential energy is released and canbe used to drive a waterwheel. Andeven though Carnot was notinterested in the amount of workthat could be obtained from thepotential energy of a waterfall, butrather the work energy that could beextracted from a hotter object, theresults that Carnot obtained (withrespect to the amount of work thatcan be obtained when heat passesfrom a hotter object to a colder one)were, amazingly, similar to thoseobtained from the study of fallingwater. (It is worth noting that there

are other forms of energy that haveno connection to mass, for examplethe energy of light.)

Carnot’s showed the extent of hisgenius in the calculation he made tofind the efficiency of the process.As we shall see, this calculation isno more complicated than the trivialconclusion that he reached,presumably, intuitively. Thisefficiency, which is called todayCarnot’s efficiency, is one way ofexpressing the second law ofthermodynamics.

So, what exactly is the secondlaw of thermodynamics? Thegeneral answer is amazingly

simple: heat tends to flow from ahotter object to a colder one. But infact, this needs to be stated a bitmore carefully: heat will neverflow spontaneously from a colderobject to a warmer one, without anapplication of work. Indeed, thesecond law is no more complicatedthan this simple statement, yet as weshall see, it has many facets,meanings and implications – someof them even affecting how wedecide what to buy in a store.

One need not be a scientist inorder to know that when we want towarm ourselves, we cuddle next tosomething warmer than us, or go out

into the sunshine. And when ourhome is too warm and we want tocool it (that is, to emit energy fromour home into the warmersurroundings outside), we turn onour air conditioning. Heat tends toflow from a hotter object to acolder one for the same reason thatwater tends to flow down. Andwhile both phenomena are similar,there are obvious differencesbetween them. For example,everyone understands what heightis. In principle, we can measure theheight of any object that we wantwith a ruler. But what istemperature and how do we

measure that? Of course, everyoneunderstands what a hot object is,but even though the thermometerwas invented hundreds of yearsago, it is still difficult to reallycomprehend what, exactly, istemperature. We understandtemperature as a value thatrepresents the internal energy storedin an object’s particles (whichmakes it is similar to potentialenergy). But as we shall see, part ofthe difficulty inherent inunderstanding the second law andits accompanying evasive quantity,entropy, stems from the difficulty inproperly understanding the concept

of temperature (or, alternatively,from actually misunderstanding itsnature).

Carnot’s contemporaries did notknow about atoms and did notcomprehend the true nature of heat.In those days caloric was theaccepted theory explaining heat: hotobject was a combination of thisobject with caloric, somewhatanalogous to a compound or analloy. The smaller/greater theamount of caloric absorbed in theobject, the colder/warmer it was. Itwas also believed that caloricinherently tended to flow fromhotter objects to colder ones. To

modern readers, the differencebetween this material calledcaloric and the concept of thekinetic energy of an object’sparticles may not be so obvious.Surely we may assume that thereaders of this book are aware thatEinstein showed that matter is aform of energy, and vice versa. Inaddition, even in today’sterminology we can say that theconcept of the caloric in relation tothe steam engine is identical withthe way we think of energy, andthus, Carnot’s caloric and theenergy of motion (that is,mechanical work) are different

forms of essentially the same thing.In Carnot’s time, it was already

known that gases behave accordingto a simple set of rules, knowntoday as the “ideal gas laws,”*

according to which, the product ofthe gas’s volume and pressure is indirect proportion to its temperature.(In other words, the higher thetemperature, the larger the volumeand/or pressure.) This essentiallymeans that a gas at a highertemperature has the ability do morework, since an increase intemperature represents an increasein the internal energy of the gas.

Carnot’s aim was to calculate the

efficiency of the steam engine, or, inother words, to find the maximumamount of mechanical work that canbe expected to be produced by onekilogram of steam. The calculationsthat he did were amazingly simple,basing on the fact that a gas can becompressed or expanded undercontrolled conditions using a pistonin a cylinder. The piston does workin order to compress the gas, whichsubsequently heats up. When thecompressed gas is allowed toexpand, it now produces some workand, usually, cools down. Usingthese facts, Carnot described ahypothetically ideal engine, what is

called today the Carnot heat engine,or the Carnot cycle. Carnot’s heatengine – a gas-filled cylinder – issituated between two objects thatare kept at constant temperatures,one of the hotter than the other. Thegas is allowed to come into contactwith the hotter object and to heat upto its temperature. (Here Carnotassumed that the hotter object isinfinitely large, and thus can heat upthe gas without having its owntemperature reduced; such an objectis called a hot infinite reservoir.)The heated gas expands andproduces work, at the same timecooling down to the temperature of

the colder object. Thus, the energyof the hot gas is reduced: some of itpassed on to the colder object andsome used to do the work. Now wehave a cold gas in contact with thecolder object. In order for this gasto reach again the temperature of thehotter object, energy must be usedto compress it again, and the cycleis then repeats. To summarize theprocess, Carnot moved energy(“caloric”) from the hotter object tothe colder one, and obtained workin the process. This action could berepeated ad infinitum in a cyclicfashion. The equation that Carnotcame up with was the following:

where W represents the amount ofwork that can be obtained from thegas, Q is the amount of energy(“caloric”) that is extracted fromthe hotter object, TH is thetemperature of the hotter object, andTL is the temperature of the colderobject.

Because the above-mentionedinequality has importantimplications in the worlds ofchemistry, biology, computers andeven the human mind, as we shallsee in due course, it would be

worth our while to try to grasp thefull significance of these symbols.

Let us start with the mostproblematic quantity – temperature.Almost all over the world,temperature is measured on a scalenamed after its inventor, Swedishphysicist Anders Celsius (1701–1744), as indicated by the letter C.On this scale, the freezing point ofpure water (at standard atmosphericpressure) is 0°C and the boilingpoint (under the same conditions) is100°C. On the other hand, in theUnited States temperature ismeasured on a scale named afterGerman scientist Daniel Gabriel

Fahrenheit (1686–1736), indicatedby F. The zero point in Fahrenheit’sscale was the freezing point of aparticular brine solution (0°F isequal to – 18°C); and the 96°Fpoint was the “blood-heat”temperature that can be measuredunder the armpit of a healthyperson.

The problem with these scales istwofold:

Putting into Carnot’s efficiencyequation temperatures measured onthe Celsius scale would yield avalue different from what we wouldobtain using the Fahrenheit scale.

This would mean that efficiency is,incongruously, dependent on whichscale we choose.

Since negative values mayappear on both scales, using anegative value in the equationwould result in an efficiency valuegreater than one – obviouslyillogical, since this would mean thatthe amount of work obtained isgreater than the heat consumed.

These two problems can beeliminated by using a temperaturescale that is “absolute.” If yourecall, according to the gas laws,the product of the volume and

pressure of a (ideal) gas is directlyproportional to its temperature. Putdifferently, the state that the gas isin (volume and pressure) is aproduct of the work needed (or“taken out” of it) to bring the gas tothat state. If we cool down a gas toa temperature at which it has zerovolume, this would mean that thetemperature would also have to bezero. We can thus conclude thatthere exists some temperature suchthat both the pressure and the energyof a gas is zero. Now we assignsome temperature scale such that itspoint of “zero temperature”coincides exactly with the gas’s

state of “zero energy” – obviously,this scale can never have anynegative value. We have thus solvedboth problems in one stroke. Notethat there is no real significance tothe size of the intervals between thegradations (degrees) on thetemperature scale – it is arbitrary –meaning that the value TL/THbecomes a pure ratio, that is, theratio between the absolutetemperatures of the hotter andcolder objects.

Such a scale has indeed beendefined, and it is now called theKelvin scale. Its starting point iscalled absolute zero, and the

gradations (derived directly fromthe Celsius scale, i.e., eachgradation represents precisely1/100th of the difference betweenthe freezing and boiling points ofpure water at standard atmosphericpressure) are called kelvins; theirsymbol is the K. On the Kelvinscale, the freezing point of water is273 K (≡ 0° C).

Let us return now to the Carnot heatengine. Calculating its efficiency isnot merely a matter of finding theefficiency of some archaic machine,since it leads to a very importantlaw of nature that states that it is

impossible to build any machinewhose efficiency will be greaterthan its Carnot efficiency. Stateddifferently but equivalently, themaximum efficiency possible forany machine will be Carnot’sefficiency.

Figure 1: An illustration of the energyand temperature in a Carnot heatengine shows its similarity to awaterfall, except that instead ofdifferences in height, there are

differences in temperature (indicativeof the energy in the hotter and colderobjects.) In this case, the temperaturedifferences are fully exploited in theconversion of thermal energy intomechanical work.

Looking at Figure 1, we see that

it follows from the gas laws thatabsolute temperature is directlyproportional to the energy stored inthe gas (the product of pressure andvolume is proportional to theenergy of the gas). Thus, the ratiobetween the work that can beobtained and the energy removedfrom the hotter object (W/Q) is

identical to the ratio between thedifference in temperature betweenthe hotter object and the colder one,and the temperature of the hotterobject:

According to the above equation,all the energy of the hotter objectcan, in principle, be converted intowork (making the efficiency equalto one) – provided that thetemperature of the colder object isabsolute zero.

Carnot’s theorem is a

generalization of the results ofspecific calculations made for aspecial case. Anyone who wouldlike to disprove his generalizationmust perform an experiment whichwill show that it is possible tobuild a heat engine with efficiencyhigher than Carnot’s. Till now,nobody has successfully built amachine whose efficiency greaterthan Carnot’s heat engine. It isworth noting that Carnot’s theoremexhibits its profound importanceprecisely because the Carnot heatengine does not – and actuallynever will – work at its maximumefficiency. This is important

because, as we shall see, in anycase whose efficiency is smallerthan the maximum value of Carnot’sefficiency, entropy – the drivingforce of our universe – necessarilyincreases.

In order to understand thesignificance of Carnot’s theorem,two crucial concepts must beexplained, since they determinewhen maximum efficiency cantheoretically be obtained from aCarnot heat engine. These conceptsare equilibrium and reversibility.

Equilibrium is a “steady state.” Asystem in a steady state will notshow any changes when left on its

own over a period of time. (Sincetoday’s accepted theories state thateven the universe is not infinitelysteady time-wise, it is clear that thevery nature of a definition of asteady state requires reference totime.) It is also important tounderstand that dynamic systems, ifobserved macroscopically ratherthan microscopically, also exhibitequilibrium. A good example of thisis the temperature of earth’satmosphere. The atmosphere’saverage temperature is almostconstant: even the most alarmingforecasts of the ecologicalcatastrophe heading our way as a

result of global warming predict achange of only about 2 °C from thebeginning of the 20th century to theend of the 21st (a change whichmay, small though it appears to be,still have dire ecologicalconsequences). Yet this stability ofthe atmosphere’s averagetemperature does not contradict thefact that at any given moment onecan find temperatures above earth’ssurface ranging from less than-40°C to more than 40°C,depending on location.

This example shows whatthermodynamics is all about. If we

observe that gas-filled cylinder inCarnot’s heat engine and attempt tomeasure the gas’s pressure ortemperature at different places in it,we shall find that the measuredvalues vary from time to time andfrom place to place – similar to theway that air pressure andtemperature vary in the atmosphere,depending on where they aremeasured.

Returning to the ideal gas laws, abasic assumption is that the valuesof pressure and temperature arewell-defined and identicaleverywhere in the cylinder. Thesystem is assumed to be

homogenous, and as we havepointed out just now, this does notnecessarily occur in reality.Therefore, when Carnot calculatedthe efficiency of his heat engine, heunderstood that in basing hiscalculations on the gas laws he wasassuming a hypothetical, theoreticalequilibrium.

One of the most importantproperties of equilibrium is thatevery system tends to reach it. Ifwe throw a stone into a lake, forinstance, ripples form – but theywill disappear in due course. Thestate with no ripples, in which thesurface of the water is uniformly

level, is its state of equilibrium.Here, the tendency to reachequilibrium seems obvious. By thesame token, if we could observe thebehavior of gas particles in acontainer, it would be similarlyobvious that the “ripples” of gasatoms will tend to flow from areasof high to low concentration(pressure), until the pressurebecomes uniform. And this is alsoexactly what happens when thereare differences of temperature in asystem: heat will flow from hotterarea to cooler one. This tendency ofa system towards equilibrium, thispropensity that is so intuitively

obvious, is actually the profoundmeaning of the second law ofthermodynamics. Furthermore, itwould not be an exaggeration to saythe universe looks the way it does –complex, spectacular, and so full ofdiversity – precisely because of thisproperty.

Let us turn now to reversibility.A change will be reversible if it canbe “cancelled” in such a way that itwould be impossible to observethat the change has ever happened.Picking an apple from a tree isirreversible. But taking an apple outof a basket – so long as we don’tbite it – is reversible. We can return

it to the same place, and there willremain no evidence to the fact thatwe had moved it in the first place.The moment we sink our teeth intothe apple, however, we perform anaction that can never be reversed tothe apple’s original state. Mostactions in our day-to-day world areirreversible.

Interestingly, even though mostactions in this world areirreversible, this fact is not a directconsequence of any of the laws ofnature. On the contrary, the laws ofnature are reversible. Gravity, forexample, compels two masses toapproach each other, resulting in a

release of energy which can bestored – perhaps in the stretching orcompression of a spring. Byreleasing that spring, the storedenergy can force those objectsapart, back to their originalpositions. Another commonexample of a reversible process isa battery cell which is used toconvert stored chemical energy intoelectrical energy. There is nothingin the law of conservation of energythat prevents us from makingelectrical energy flow back throughthe battery, thus converting it backinto chemical energy, and indeedthis is what happens in

rechargeable batteries.This line of reasoning, that the

laws of nature enable a perfectback-and-forth cycling of anyreaction, is called the deterministicapproach: natural forces are soprecisely defined that it is possibleto determine – even in a systemcontaining an enormous number ofatoms – what state that system willbe in at any given time in the future,provided we know the presentcondition of its components and thevectors of their velocities.Moreover, if we believe in thedeterministic approach, we mayconclude that by reversing the

direction of the velocity vectors ofparticles (a difficult task, ofcourse!), the system will movebackward in time to its previousstate. In other words, it seems thatwith precise information about thestate of a system at a given momentin time, it is possible to know boththe system’s past and future. Yet thisis obviously not the case. Andindeed, the second law ofthermodynamics, as we shall showin detail later on, disproves thisdeterministic approach, provingrather that such an occurrence isimpossible.

So, to restate Carnot’s theorem: a

reversible engine which is in a stateof equilibrium during its operationwill have a Carnot efficiency of

That is, the maximum amount ofwork that can be produced byremoving a quantity of heat Q froman object whose temperature is THto a colder object at whosetemperature is TL, is thetemperature difference divided bythe temperature of the hotter object.However, if the engine is notworking reversibly, the efficiency

will be smaller than Carnot’sefficiency.

What is the meaning of areversible Carnot heat engine? ACarnot heat engine transferring anamount of heat Q from a hotterobject to a colder one producesmechanical work. If we can applythat exact same amount or work thatwe obtained and cause the engine totransfer the exact same amount ofheat, Q, back from the colder objectto the hotter one, we could say thatthe machine is reversible.Therefore, the gas in the cylindermust be in equilibrium at all times –a practical impossibility, to say the

least – otherwise the engine’sefficiency will be smaller.

Viewed historically, against thebackground of the industrialrevolution, is it is easy tounderstand what were Carnot’smotives for calculating theefficiency of the steam engine. It isharder to understand why thisproblem should interest you, ourreaders. Even so, assuming you areinterested, for reasons best knownto yourselves, there still remains thequestion, why is Carnot consideredby historians of science to be one ofthe greatest scientists of the

nineteenth century?The reason is the universality and

simplicity of his findings. Byrevealing that the efficiency of hisheat engine is dependent only on itshigh and low temperatures (and hasnothing to do with the way it isconstructed) he proved that it isimpossible to build a machinewhose efficiency is greater thanCarnot’s efficiency, and thereforemaximum efficiency attainable byany machine, whatever itsconstruction, is its Carnotefficiency. Furthermore, theimportance of Carnot’s theorem liesin the fact that it was the first

formulation of the second law ofthermodynamics – a law that, as weknow now, guides the nature ofthings in our universe. This law,that can be expressed verbally in anumber of ways (some of whichwill be stated later on), is not onlyimmensely important, but also,considering what was known at thetime, represents a tremendousintellectual achievement.

It is interesting to note that the“caloric” theory became obsoleteonly in 1850; nearly twenty-fiveyears after Carnot had published hiswork. Scientists were persuadedthat this theory was invalid mainly

by a simple experiment performedby English physicist James PrescottJoule (1818–1889). Hedemonstrated that when two chunksof ice were rubbed against eachother, the ice melted into water.Since water was supposed to havemore “caloric” than ice, it could nothave come from the ice, which ledto the conclusion that motion energycan be converted to “caloric”. Thisexperiment, alongside with thetheories independently offered byGerman physicists Julius Robertvon Mayer (1814–1878) andHermann von Helmholtz (1821–1894), laid the foundations for the

first law of thermodynamics, alsoknown as the law of conservation ofenergy. This law states that in anisolated system, energy maychange its form, but its totalamount must remain constant. Forexample, if an acid and a metal areplaced in a sealed isolatedcontainer, heat will be produced asa result of the chemical reactionbetween the acid and the metal,which produces a salt and a gas.The amount of heat released will beexactly the same as the energy thatthe molecules of the metal and acidlost during this transformation. Ifwe wish now to change these salt

and gas back to metal and acid, wemust supply an input of energywhich is at least equivalent toamount of the heat that wasreleased.

A furious scandal surrounded thediscovery of the law ofconservation of energy: In 1847,Julius von Mayer read Joule’sarticle on the conversion ofmechanical energy into heat andclaimed that he had actuallypreceded Joule in discovering thisphenomenon. And indeed, sevenyears earlier, while serving as adoctor on a Dutch ship in the IndianOcean, he began to question the

nature of heat and its production(inspired by his observations thatstorm-whipped waves werewarmer than water in a calm sea).He wrote a paper that discussed theconversion of mechanical work intoheat (and vice versa), but it wentlargely unnoticed by his colleagues.He later rewrote the paper andrepublished it elsewhere. Joulerefused to acknowledge Mayer’sclaim. Mayer, realizing that Joulenow had the credit that should havebeen his by rights, threw himself outof his third-floor window andended up hospitalized in an insaneasylum.9 Later on, Hermann von

Helmholtz was also to lay a claimto the discovery of the first law ofthermodynamics, since he hadpublished in 1847 a book whosetitle was On the Conservation ofForce10 (the term energy was not inuse at the time in its modernmeaning).

Be this as it may, let us go backto Carnot and his awareness of thenotion of conservation of energy:even though Carnot still used theterm “caloric”, the idea behind hisheat engine was based on his beliefthat “caloric” can be converted intowork, and work into “caloric”.Today we can appreciate that

Carnot understood the first law ofthermodynamics, despite his use ofa now obsolete term. In his book,Reflections on the Motive Powerof Fire, he wrote:

One can therefore posit a generalthesis that the motive power is anunchangeable quantity in nature;that, strictly speaking, it is nevercreated nor destroyed. In truth, itchanges form; that is to say that itsometimes produces one kind ofmotion, sometimes another, but it isnever annihilated.11

This is just amazing. True enough,

Carnot’s heat engine and Carnot’stheorem were defined in terms ofthe “caloric” (being at the timesomething everyone believed in;today, we suppose, many scientistsare not even familiar with thisterm). Yet over two centuries thathave passed since, during which anamazing number of machines havebeen invented, and Carnot’stheorem still stands firm and is stillrelevant.

It is no wonder that Carnot isconsidered one of science’s greats.

The Clausius Entropy

Forty years have passed sinceCarnot’s had written hismonumental paper on the efficiencyof heat engines before the scientificcommunity finally gave him therecognition he deserved. The firstto do so was Prussian nationalistand war hero Rudolph Clausius –the physicist who gave formaldefinition to the laws ofthermodynamics. Clausiusidentified within Carnot’sefficiency a quantity of strange andmysterious properties, and called itentropy. This new quantity wasquite unusual: first, because itsamount in the universe can only get

larger, never smaller – it is notconserved as many other physicalquantities are; and second, becauseentropy, through its tendency toincrease, is the cause of any flow ofenergy.

Contrary to other forces in naturethat are applied in precise andmeasurable ways, entropy’s effectscan only be predicted as tendencies.According to thermodynamics, heattends to flow from a hotter object toa colder one; however, entropyneither initiates the process nordefines the length of time it takes.Rather, it seems to stimulate apropensity that can be compared in

some limited ways to human will.For example, water at the top of ahill has a tendency to flow down tothe valley. Will it flow? Thatdepends. If it is contained in aconcrete pool, it will stay in place.But if there is a crack in theconcrete, even the smallest one,water will begin oozing its waydown. And given a more substantialopportunity, it will gush down themountain with gusto. Its entropywill increase – a truly human-likebehavior – and as we shall seefurther, this is not an idle analogy.

Clausius’s work, based entirelyon his studies of Carnot’s

efficiency, marked the beginning ofthe science of thermodynamics. Yetunlike Carnot, whose work wentlargely unrecognized in his day,Clausius’ work was immediatelyaccepted, probably thanks to theprominent position he had alreadyoccupied in the scientificcommunity during this golden age ofscience.

The significance of thediscoveries and inventions whichoccurred during a mere twenty fiveyears in the latter half of the 19th

century cannot be overstated. Inthose few years, breakthroughs tookplace that totally changed science’s

concepts: in 1855, Neanderthal manwas discovered; in 1856, RobertBunsen and Gustav RobertKirchhoff founded spectroscopy; in1859, Charles Darwin publishedhis book On the Origin of Speciesand started a controversy thatamazingly continues to this veryday; in 1861, Louis Pasteurinvented pasteurization and AlfredNobel developed dynamite; in1864, Maxwell published hiscomplete theory ofelectromagnetism, a centrallandmark in the history of science;in 1865, Gregor Mendel publishedhis seminal paper on genetics

(which was totally ignored!). Alsoin 1865, Clausius formulated thelaws of thermodynamics anddefined entropy and its properties.In 1869, Dmitri Mendeleev createdthe periodic table of the elements,Johannes Friedrich Miescheridentified the DNA molecule(claiming but not proving that it wassomehow connected to heredity),and Georg Cantor developed themathematical concept of infinity. Itis hard to explain why such anexplosion of human genius occurredover such a short time span, andonly in Europe.

Rudolph Clausius“The entropy of the universe tends to amaximum.”

– RudolphClausius, 186712

Rudolf Clausius, 1822–1888

Rudolf Julius Emanuel Clausiuswas born in Koslin, Prussia (todayKoszalin, Poland), the sixth son of aGerman school principal. As achild, he attended the school hisfather ran and then studied at thegymnasium (high school) in Stettin(today Szczecin, Poland). In 1840,having completed his high schoolstudies, he entered the University ofBerlin, studying mainly mathematicsand physics. He graduated in 1844,and in 1847 submitted to theUniversity of Halle his doctoraldissertation, which discussed theproblem of why the sky appearedblue during the day but red at

sunrise and sunset. He based hisexplanation on the phenomena oflight reflection and refraction;however, this was later proven tobe erroneous.*

Clausius was the first scientistwho identified the entity known asentropy in Carnot’s efficiency anddescribed its macroscopicproperties, and this was reasonenough to earn him a place in thepantheon of science. Unfortunately,he tended to ignore the works of hiscontemporaries, and even whenthey acknowledged the significanceof his insights, Clausius himself didnot reciprocate in kind. His

biographer, E. E. Daub, found thispuzzling:

Clausius’s great legacy to physics isundoubtedly his idea of theirreversible increase in entropy,and yet we find no indication ofinterest in Josiah Gibbs’ work onchemical equilibrium orBoltzmann’s views onthermodynamics and probability,both of which were utterlydependent on his idea. It is strangethat he himself showed noinclination to seek a molecularunderstanding of irreversibleentropy or to find further

applications of the idea; it isstranger yet, and even tragic, that heexpressed no concern for the workof his contemporaries who wereaccomplishing those very tasks.13

Perhaps his ardent patriotismmade him ignore work done outsidePrussia, even to the point of disputewith Lord Kelvin on priority in thediscovery of the first law ofthermodynamics: Kelvin attributedit to Joule (an Englishman), whileClausius insisted that the samediscovery was made by Helmholtz(a German). However, it is far morereasonable to attribute it to Carnot

(a Frenchman).In 1869, Clausius moved to

Bonn. This was a period ofpolitical transformation inGermany, and it has had anenormous impact on his personallife. Bismarck, who had succeededin uniting the northern Germanstates into a confederation, waslooking for ways and means toencourage the southern states to joinin, and came to the conclusion thatwar with France could bring aboutGerman unity. France, for its part,was confident that it could easilydefeat the new German state, and itsreaction to Bismarck’s

provocations was just as he hadexpected. As it turned out, in thewar that followed (1870–1871), theGermans proved far more powerfulthan the French had expected, andmade short shrift of their enemies.Clausius, by then almost 50 yearsold and a well-respected scientist,still felt obliged to serve hishomeland. He enlisted and wasgiven command of an ambulancecorps consisting of students fromBonn. During the war, he sustainedleg injuries and remained painfullydisabled for the rest of his life. Forhis services, he was awarded theIron Cross. Since then, in an attempt

to relieve his pain and on hisdoctor’s advice, he used to makehis way to the university onhorseback.

In 1872, following thepublication Maxwell’s paper, TheTheory of Heat (which will bediscussed later, in connection withthe Maxwell-Boltzmanndistribution), a bitter controversydeveloped between Clausius andScottish physicist Peter Guthrie Taitover claims of priority for thistheory. Tait, best known for hisphysics textbook, Treatise onNatural Philosophy, which hewrote with Kelvin in 1860, was one

of the pioneers of thermodynamics,and had written a book on thehistory of thermodynamics in 1865.Unfortunately, Tait too got carriedaway by patriotic fervor thatwarped his judgment andcompromised his scientificintegrity. Clausius was adamant thatthe British were claiming credit farbeyond whatever they deserved forthe theory of heat, and it seems thathe was quite right about this. Itshould be noted that in later years,Maxwell fully recognizedClausius’s contribution.

In 1875, Clausius’s wife diedwhile giving birth to their sixth

child. He continued teaching at theuniversity, but his academic careersuffered because he had to spendmuch more time taking care of hischildren. His brother Robertremarked that Clausius was adevoted, loving father who investedmuch in their education.

Clausius won many honors forhis work. From 1884 to 1885, heserved as Rector at BonnUniversity. In 1886 he remarriedand had another child, but departedfrom this world only two yearslater, at the age of sixty-six.

In 1850, Clausius published hismost famous paper, On the MovingForce of Heat, and the LawsRegarding the Nature of HeatItself Which are DeducibleTherefrom,14 in which he discussedthe mechanical theory of heat. Thispaper included his first version ofthe second law of thermodynamics.During the next fifteen years,Clausius published more than eightpapers in which he attempted toformulate this law mathematically,in a more simple and general way.In 1865 he published a collection ofthese papers in a book entitled The

Mechanical Theory of Heat.15

In 1867, Clausius moved toWürzburg. Only then did he use theterm “entropy” for the“transformation equivalence” thathe had described in his papers, andthis was the first appearance inprint of its mathematical equation(which he had actually defined in1850):

That is, the change in entropy S isequal to or greater than the ratio ofheat (Q, energy added or removed)

to temperature, T.*Clausius explained how he came

up with the word “entropy”:

If we wish to designate S by aproper name we can say of it that itis the transformation content of thebody, in the same way that we sayof the quantity Q that it is the heatand work content of the body.However, since I think it is better totake the names of such quantities asthese, which are important forscience, from the ancient languages,so that they can be introducedwithout change into all the modernlanguages, I proposed to name the

magnitude S the entropy of the body,from the Greek word [etrope], a transformation. I haveintentionally formed the wordentropy so as to be as similar aspossible to the word energy, sinceboth these quantities, which are tobe known by these names, are sonearly related to each other in theirphysical significance that a certainsimilarity in their names seemed tome advantageous.16

Clausius did not specify why hechose the symbol “S” to represententropy, but it is plausible tosuggest that he did this in honor of

Sadi Carnot.17

Clausius’s inequality (the valueQ/T is smaller than entropy) occursin two instances:

when a process is irreversible;when a system is not in equilibrium.

Under such conditions, themaximum value of Q/T can be S,such that

or, entropy is always greater thanQ/T, unless the system is inequilibrium, in which case they are

equal. This inequality, now called“Clausius’s inequality,” is theformal expression of the secondlaw.

Why did Clausius specify that itwas Q/T in equilibrium thatdefined the system’s entropy? Thereason is that Q/T can be exactlydefined only when it is in a state ofequilibrium. When we measuresome physical quantity, we preferthat the measurement be unique.That is, no matter when and wherethe same quantity is measured, itsvalue should be the same. In thecase of an ideal gas, thetemperature at equilibrium is

homogenous throughout its volume,and thus the value of S will beidentical throughout. However, in agas that is not in equilibrium, someareas may be hotter or cooler thanothers and thus Q/T will not beuniform throughout, and the entropywill not be well defined.

Take, for example, a sugar cubeand water in a cup. Each on its ownis in equilibrium, and the entropy ofeach can be calculated. But if weplace the sugar into water and waitfor a period of time, the sugar willdissolve in the water. Sugardissolves in water spontaneouslybecause the entropy of the solution

is higher than the entropy of thesugar and the entropy of the water –indeed, this is the reason the sugartends to dissolve in water.However, at this point we have noway of knowing the entropy of thesolution itself, at least not until thesugar has completely and uniformlydissolved and the solution iscompletely homogenous. Then thesystem will again be in equilibriumand its entropy can be determinedbased on the difference intemperatures between the solutionand pure water. It will be observedthat the entropy did, indeed,increase.

Another, somewhat differentexample is the process ofspontaneous separation of milk intocream, which floats on top, andskim milk beneath it. The reasonthis happens is that the entropies ofboth cream and skim milk,separately, are higher than that ofwhole milk. (In this industrial age,natural milk undergoeshomogenization to prevent thisspontaneous separation.)

Another important property ofentropy is that it is additive, or asthe physicists call it, “extensive.”This means that the addition of thevalues of the same property in two

different objects yields a value thatis the sum of both. Weight, forexample, is an extensive property: akilogram of tomatoes and akilogram of cucumbers placedtogether yield a total weight of twokilograms. On the other hand,temperature is not: if we mix waterat 50 °C with water at 90 °C, weshall not end up with water at 140°C. Thus temperature is not anextensive property.

Entropy, on the other hand, isextensive. If two boxes, A and B,are in equilibrium, and if box A hasentropy SA and box B has entropy

SB, the two boxes together willhave an amount of entropy that isequal to SA + SB. This propertygives entropy a physicalsignificance that is preferable tothat of temperature, because itdescribes a well-defined quantity,so to speak, whereas temperature isnot more of a property, rather thanquantity.

However, if the system understudy is not in equilibrium, thenquantity Q/T is not extensive. Thus,because most systems in the realworld are not in equilibrium – eventhough they always tend toward it –

it follows that in general:

That is, Q/T of A and B combinedwill not be equal to the sum of both.

As we shall see later, S can, inprinciple, be calculated for anywell-defined system. And sinceevery system naturally tends toreach equilibrium, the value of Q/Ttends to increase over time.

And what about entropy? If itwere possible to reduce its amount,that would mean that a heat enginewith an efficiency greater than one

could be achieved. A Carnot heatengine with an efficiency greaterthan one is called a perpetualmotion machine (perpetuummobile) of the second kind, becauseit violates the second law ofthermodynamics. (By the sametoken, a machine that produceswork without any energy input iscalled a perpetual motion machineof the first kind, because it violatesthe first law of thermodynamics,that is, the law of conservation ofenergy.) Because the laws ofthermodynamics are consideredproven, established and impossibleto violate, this negates any

possibility, ever, of creating of aperpetual motion machine of anykind. Indeed, the US Patent Officewill reject out of hand any patentapplication involving such amachine. Although its mandaterequires it to examine any and allapplications, perpetual motionproposals are specifically exempt.

And so, as stated in the Carnotand Clausius inequalities, the valueof Q/T cannot be reduced; it willalways increase. Some think,incorrectly, that Q/T is entropy,because of its tendency to increase.However, when the system is not inequilibrium, Q/T is not uniquely

defined and can be different invarious locations and at varioustimes. Entropy, on the other hand, iswell defined, namely, for a givensystem it has a unique value.

This discussion can be summedup by saying the increase in thevalue of Q/T expresses nature’stendency to reach equilibrium. Heattends to flow from a hotter object toa colder one because this actionwill lead to an increase in entropy.Thus, any process that increasesentropy will be a spontaneous one,similar to sugar dissolving in wateror cream separating from the milk.Conversely, the fact that the sugar

tends to dissolve in water provesthat the entropy of the system hasincreased. In fact, any change thatoccurs in nature, even a changebrought about through humanintervention, is spontaneous in thesense that if the process occurred,entropy must have increased. And ifentropy increased, it was aspontaneous process.

But if nature is spontaneous, andentropy does in fact alwaysincrease, how is that lake surfacesfreeze in winter, or liquid waterturns to ice in a freezer?* There aretwo ways to explain why waterfreezes:

First, the frozen water is part of alarger system, in which the entropydid in fact increase. That is, whilethe entropy of the water in thefreezer decreased, electricity wasused to power the compressor andfan that cooled the compressed gasand circulated it to make freezerscold, and that electricity wasproduced from some process, at thepower station, that released moreheat to the surroundings than theamount of heat extracted from thewater when it froze, and so on.Overall, therefore, the amount ofentropy in the world has increased.Similarly, the process that causes

lake surfaces to freeze wasprobably due to a cold wind thatwas driven by energy released fromthe sun. The energy released wasgreater than the heat that the lakehas lost. Therefore we must bemore precise and say that theentropy in a closed system is neverreduced. A “closed system” is onethat is defined as thermally isolatedfrom its surroundings.

Another way to address thequestion is to view the entireuniverse as a closed system that isnot in equilibrium. Any subsystemcan reduce its entropy, providedthat entropy increased elsewhere.

Thus, the entropy in the universe isalways on the increase. (Somewould go even further than that andsay that time itself is essentiallyentropy: just like entropy, timeincreases constantly and neverdecreases. But we do not intend toengage in such philosophicaldiscourse.)

Clausius’s definition of entropy,despite its simplicity, does notexplain entropy’s meaning. Clausiusnever stated precisely that entropywas the amount of uncertaintycontained in a system. This wasdone a few years later by two of hiscontemporaries, the Austrian

Boltzmann and the American Gibbs.

Chapter 2

Statistical Entropy

The Boltzmann Entropy

It is unbelievable how simple andstraightforward each resultappears once it has been found,and how difficult, as long as theway which leads to it is unknown.

- LudwigBoltzmann, 189418

Not long after Clausius isolatedentropy in Carnot’s efficiency anddiscovered its unusual properties –nine years, to be precise – adepressive Austrian physicist by thename of Ludwig Boltzmann gaveentropy its modern form, as astatistical quantity. Just as Carnothad realized – long before the three

laws of thermodynamics wereformulated – that energy isconserved (the first law) and thatthere is a temperature of absolutezero at which there is no energy inmatter (the third law), so it wasclear to Boltzmann that matter ismade of atoms and molecules –long before this was accepted bythe entire scientific community. Thisintuitive realization probablyinformed his statistical approach toentropy.

Boltzmann’s expression forentropy is substantially differentfrom Clausius’s: first of all, it is a

statistical expression; and second, itcannot be directly measured.While the Clausius entropy issomething tangible – defined as anactual amount of heat, divided by anactual measure of temperature –Boltzmann’s expression deals withthe uncertainty of a system.

In order to appreciateBoltzmann’s achievement properly,it must be remembered that in the1870s, many leading scientists didnot believe in the existence ofatoms and molecules in the firstplace. Despite an ever growingamount of persuasive, evencompelling evidence, they denied

their existence, even though in thevery same year that Clausiusdefined entropy, 1865, his friendJosef Loschmidt calculatedAvogadro’s number, which haseverything to do with atoms andmolecules.* Nevertheless, manyscientists were still inclined tothink that science should deal withconcrete matters, that is, things thatcould be observed directly. If thereare atoms, then show them to us; ifwe cannot see them, then they donot exist. Incredibly, even DmitriMendeleev, the man who gave usthe periodic table of the elements,subscribed to this view.

This denial of the “atomichypothesis” was ideologicallymotivated. Prolific scientists suchas Wilhelm Ostwald, the father ofphysical chemistry, or Ernst Mach,the founder of fluid mechanics(better known nowadays for theMach unit that is called after him,which is the speed of sound in air),subscribed to a philosophy that waslater dubbed “logical positivism,”and viewed Boltzmann’s work asuseless philosophizing. And beholda paradox: despite (or perhapsbecause of) such an attitude, whichis almost diametrically opposite toour grasp of science today as being

open to new, innovative anddifferent ideas (as long as they arein the realm of accepted scientificmethod), the profound scientificbreakthroughs that occurred at theend of the 19th century and thebeginning of the 20th wereimmensely grander and moresignificant than those of today.

Indeed, the atomic hypothesisbecame acceptable to the scientificcommunity (whatever that means)only after the publication ofEinstein’s 1905 paper in which heexplained Brownian motion influids (like the movement of dust

particles in still air). Brownianmotion is the random movement ofparticles that results from collisionsbetween them. This phenomenonwas first discovered in 1828 byBritish biologist Robert Brown,who observed pollen particles insuspension through a microscopeand noted that they moved to and froin patterns that seemed a result ofcollisions between them. If themomentum of these collidingparticles was greater than the effectof gravity upon them, theirmovement would continue for along time because the particles tendnot to sink to the bottom of the

fluid’s vessel, but rather stay afloatinside it. The movement of particlesin fluids from a place of highconcentration to a place of lowconcentration due to collisions iscalled diffusion – a measurablephysical phenomenon of profoundimportance (i.e., diffusion is theresponsible to the balance ofliquids in our bodies). In aspectacular statistical calculation,Einstein derived the dependence ofthe diffusion coefficient in a systemof colliding particles ontemperature, and provided thenecessary link between measurablevalues from fluid mechanics (such

as the diffusion coefficient andtemperature) and the movement ofparticles. With this, he finally laidto rest all arguments concerning theviability of the atomic hypothesis.

All this, however, had not yetcome to pass when Boltzmannreformulated the concept of entropyand declared that a system’s entropyis directly proportional to thenumber of possible distinguishableways its components can bearranged (or, more precisely, to thelogarithm of this number). A fierceargument broke out betweenBoltzmann with his supporters andthose who opposed his ideas. The

question was whether there wasstill a need for Clausius’sinequality, or perhaps it was just aconsequence of Boltzmann’sdefinition. Boltzmann’s desire toderive Clausius’s inequality fromhis own definition made himinterpret his entropy in acontroversial manner, so that eventhough his mathematical expressionfor entropy was correct, anenormous and eternal intellectualachievement, it dragged him intocontroversies that continue to echoin the scientific world to this veryday. We shall discuss themisunderstanding created by

Boltzmann’s choice of expressionlater.

Ludwig Boltzmann

Ludwig Eduard Boltzmann, 1844-1906

Ludwig Boltzmann was born inVienna, Austria, on February 20,1844, to a middle-class family. Hisfather, a treasury official, earned astable, comfortable living for hisfamily. Boltzmann’s early yearswere spent in northern Austria. Inhis youth, he learned piano withAnton Bruckner, then a thirty-years-old organist in the Linz cathedral,well before gaining his fame as acomposer. Bruckner’s career as apiano teacher to young Boltzmanncame to an abrupt end one day when

he put his wet overcoat on a sofa inthe Boltzmann residence and irkedthe ire of the mistress of the house.Boltzmann, though, continuedplaying the piano and became quiteproficient. Although his family wasnot known for intellectualachievements, Boltzmann himselfwas an unusually brilliant student.When he was fifteen his father died,and a year later his 14 years oldbrother passed away too (both,apparently, from tuberculosis).Boltzmann senior’s death tightenedthe family’s finances somewhat, buthis pension still allowed them tolive comfortably. From then on,

Boltzmann’s mother invested all herenergy and resources in theeducation of her son.

In 1863, Boltzmann went to studyphysics at the University of Vienna.Though he was not aware of this atthe time, Boltzmann found himselfin an institution that, under theleadership of Josef Stefan (thediscoverer of the radiation lawnamed after him), eagerly adoptedthe hypothesis of the atomic theory.Stefan, who was one ofBoltzmann’s teachers, made himone of his protégés. At this point,Boltzmann had not as yet exhibitedany special inclination toward a

specific branch of physics, butimmersed as he was in anenvironment that kept up with allnew developments in this science(such as kinetic theory orelectrodynamics), it no doubtinfluenced his final choice ofdirection. He presented hisdissertation on the kinetic theory ofgases (which combinedthermodynamics with moleculartheory) in 1866. In 1867 he wasappointed lecturer at the university,and also worked as Stefan’sresearch assistant of at the ErdbergPhysical Institute. Stefanencouraged Boltzmann to study

Maxwell’s work onelectrodynamics; discovering thathis student could not read English,he also gave him a book on Englishgrammar. (And indeed, Boltzmannadmired Maxwell and his work,and even described him as the“greatest of theoreticians”.)

While in Vienna, Loschmidt (thephysicist who calculated the valueof Avogadro’s number) befriendedyoung Boltzmann and became tohim something of a father figure. In1869, at the tender age of twenty-five, Boltzmann was appointedProfessor of Mathematical Physicsat the University of Graz in Austria,

and moved to that city. By then hehad already published eightscientific papers. It was animpressive appointment for him, butalso a great benefit for theuniversity, which had not beenparticularly notable till then in thephysical sciences. The budget,however, was rather tight, and thetiny laboratory was freezing inwintertime. (In fact, August Toepler,the head of the physics department,had to lend Boltzmann a Russian furcoat so that he could keep workingon his experiments in the extremecold.) Boltzmann also gave lecturesin elementary physics but not

enthusiastically.19

In 1872 Boltzmann published a100-page long paper whichtransformed him from a fairlytalented physicist into anacknowledged genius. This paper,Further Studies of the ThermalEquilibrium of Gas Molecules,20

dealt with the velocity distributionof gas molecules, and marked thefirst appearance of the expressionknown today as the Maxwell-Boltzmann distribution. At its corewas what Boltzmann called “theminimum theorem,” or what waslater called the H-theorem (see

below).Few were able to understand his

methods, and even fewer managedto read through the lengthy paper inwhich Boltzmann turned his back onthe rigid determinism which was ahallmark of the period. Hiscolleagues found this difficult toaccept. Even Maxwell, who alreadyin 1859 had described thedistribution of the velocity of gasmolecules in thermal equilibrium,wrote to his colleague, PeterGuthrie Tait, in 1873: “By the studyof Boltzmann I have been unableto understand him. He could notunderstand me on account of my

shortness, and his length was andis an equal stumbling block tome.”21 Rumors began to circulatethat Boltzmann had done somethingremarkable, but no one couldunderstand what it was, precisely,and it took some years before a fewscientists engaged in an effort tofathom the depth of the issue.Surprisingly, the importance of hiswork was first recognized inEngland. In 1875, Lord Kelvinmused about Boltzmann’s workwhile riding on a train, and wrote toa colleague: “it is very important...The more I thought of it yesterdayin the train, the surer I felt of its

truth.”22

In May 1873 Boltzmann metHenriette von Aigentler, a nineteen-year-old student at the teachers’training college. A year earlier, thisdetermined young woman haddecided that she wanted to auditscience courses at the university.Even though the administrationinitially refused to allow this (itwas well known that women wereof fickle mind, distractions to themale students and faculty, andincapable of the rational thinkingrequired for the study of chemistry,physics or mathematics), sheeventually obtained the necessary

permissions, not before shepresented testimonials ascertainingher proper, quiet decorum. In 1872she began attending lectures,although was required to presenther letter of permission for eachcourse that she registered for. In1873, after Boltzmann accepted theposition of Associate Professor ofMathematics at the University ofVienna, and returned to that city, thetwo kept up their correspondence.In 1875 he sent her a marriageproposition by letter, and they weremarried in July 1876.

After the publication of theMaxwell-Boltzmann equation,

Boltzmann devoted most of hisenergy to experimental physics. Hepublished many papers – twelve inthree years – but only by the end ofthese three years did he begin todelve deeper into theoreticalphysics. In 1877 he returned toGraz, and this time he remainedthere for fourteen years. In the sameyear, Boltzmann developed astatistical-mechanical explanationfor the second law ofthermodynamics, and in 1877 hepublished it in two paperspresented to the Science Academyof Vienna.

Back in Graz, Boltzmann

attempted, not without success, toreproduce the atmosphere in whichhe had been imbued during his earlyyears as a scholar in Vienna.Among other things, in 1879 hecarried out the research that led himto discover a fundamentalrelationship between radiationtheory and thermodynamics, nowknown as the Stefan-Boltzmann law.His fame has spread by thenthroughout Europe, and studentswere coming from all over to studywith him. Walter Nernst, the 1920Nobel laureate in chemistry, fondlyremembers the open atmospherethat prevailed there and

Boltzmann’s readiness to devotetime to discuss things with his moreadvanced students (but only withthem). To his regret, they were notnumerous.

Highly respected and honored,Boltzmann nevertheless feltscientifically isolated in Graz,distant from other scientific giantsof the period. As a researcher, hebelieved in scientific intuitionregarding the result being aimed for,ignoring hardships along the way. Inaddition, he did not believe in theimportance of scientific elegance.He used to say: “Elegance is for thetailor and the shoemaker.” He

published prolifically during thisperiod. In good days, he was anunforgettable lecturer: enthusiastic,involved and attentive to hisstudents. He devoted a lot of time toteaching, and it was important tohim that his students, even if theywere only medical students, even ifthey did not intend to devote theirfuture careers to research, wouldfully understand what he wasteaching them. But he wrote just ashe spoke, without hesitation,without polish, and without caringabout proper, clear and elegantpresentation – making it difficult forhis readers and listeners to

understand him.In 1885, Boltzmann’s mother

died at the age of 75. Although 41years old, ten years married and afather of four, he sunk into a deepdepression that was not, alas, to behis last one. In that year hepublished only one paper onstatistical mechanics, his responseto a paper by Helmholtz. From thenon, Boltzmann’s life was marked byinstability – frequent moves fromplace to place and sharp moodswings.

In 1887 he was appointedPresident of the University of Graz.However, the administrative duties

forced upon him were not to hisliking, and in 1890, after long,drawn-out negotiations (as was hiswont), Boltzmann, then 46, wasgiven the Chair of TheoreticalPhysics at the University of Munich,in which he did not have to doexperimental physics. The teachingburden was not heavy, nor were hisadministrative duties, and a shorttime after his arrival in Munich,Boltzmann derived the second lawof thermodynamics from theprinciples of statistical mechanicsand published a thick volume,Lectures on Maxwell’s Theory onElectricity and Light.23

Meantime, the University ofVienna was looking for a way toinduce him back to Austria,especially since he had received anhonorary degree in England. Anopportunity presented itself in1893, when Josef Stefan died.Some of the faculty in Viennawanted to see his position given toErnst Mach, then in Prague, whowas considered an excellentlecturer and a multi-facetedresearcher. Again, Boltzmannconducted long and convolutednegotiations before finallyaccepting the offer; it was obviousthat Vienna was willing to agree to

just about anything in order to winhim back. In September 1894, at theage of fifty, he returned to Vienna asa Professor of Theoretical Physicsand the head of the institute wherehe had begun his studies thirty yearsearlier. One year later, in May,1895, Mach too joined the facultyof the University of Vienna, as aprofessor of the history andphilosophy of science.Unfortunately, since he wasBoltzmann’s bitter scientific rival,the relationship between them weredifficult, to say the least, profoundlyaffecting Boltzmann’s state of mind.

The heated debate between

atomists and energists* came to ahead in September 1895, during aconference organized by theGerman Society of Scientists inLübeck. Boltzmann invited WilhelmOstwald from Leipzig toparticipate. These two had first metin 1887, when Ostwald wasstudying in Graz for a few months,and a warm relationship based onmutual respect had developedbetween them. But at the conferenceOstwald showed himself a devoutenergist. In his lecture he declaredthat

The actual irreversibility of natural

phenomena thus proves theexistence of processes that cannotbe described by mechanicalequations, and with this the verdicton scientific materialism settled.

Arnold Sommerfeld, who wouldlater become a famousmathematical physicist, was presentat the meeting and later describedthe argument that ensued betweenOstwald and Boltzmann.

The battle between Boltzmann andOstwald resembled the battle of thebull with the supple fighter.However, this time the bull was

victorious. … The arguments ofBoltzmann carried the day. We, theyoung mathematicians of that time,were all on the side ofBoltzmann.”24

When the convention ended, bothsides were still holding steadfastlyto their respective positions, andthe participants left with a deepsense of discomfort. Ostwald andhis followers felt besieged, andBoltzmann had a sense of failurebecause he did not succeed inconverting any of the energists tohis views. Actually, Ostwald wascorrect in his claim that the

mechanic equations included inBoltzmann’s H-theorem (below)could not explain irreversibility innature. On the other hand,Boltzmann was correct in his claimthat matter is composed of atoms.

Upon his return to Vienna,Boltzmann found himself alone inhis views against the leader of theopposition to atomism, the eminentErnst Mach. Joseph Loschmidt,Boltzmann’s old colleague, died inJuly 1895, the number of Mach’sadmirers was steadily growing, andBoltzmann felt more isolated,unappreciated and miserable thanever. At the same time, Joules Henri

Poincaré published a theoremwhich implied that Boltzmann’sintuitive insight which underlie hiswork on the H-theorem waserroneous, and immediately after,Ernst Zermelo published anotherpaper that questioned thecorrectness of the H-theorem.Boltzmann responded to thesepapers, but his answers wereperceived by others, for goodreason, as somewhat evasive.Planck and Kelvin, too, disagreedwith him. Mach’s supporters beganto view Boltzmann as a relic fromthe past, the last champion ofatomism; indeed, most leading

scientists in Germany and Francewere convinced at the time that thekinetic atomic theory had outlivedits role, because it could notexplain irreversibility in nature.

Maxwell died in 1879, Clausiusin 1888, Stefan in 1893 andLoschmidt in 1895. In 1890 inVienna, Boltzmann found himselfsurrounded by intellectual rivals,with no young physicists to supporthim. Some of his feelings can begleaned in a letter he sent to theeditors of the scientific journalwhich published his comment on apaper by Zermelo:

Since I am, so it appears, now thatMaxwell, Clausius, Helmholtz, etc.,are dead, the last Epigone for theview that nature can be explainedmechanically rather thanenergetically, I would say that in theinterests of science I am duty-boundto take care that at least my voicedoes not go unheard.25

In 1898 he wrote to his assistant,Felix Klein:

Just when I received your dearletter I had another neurasthenicattack, as I so often do in Vienna,although I was spared them

altogether in Munich. With it camethe fear that the whole H-curve wasnonsense.26

This confirms the widespreadsupposition that Boltzmann wasaware of the problematic nature ofhis H-theorem, even though thisdetracts nothing whatsoever fromthe correctness of his expression forentropy. He could not know thatwithin three years, this expressionwould help Planck in hismonumental discovery of theatomistic nature of electromagneticradiation.

Boltzmann remained in Vienna,

his hometown, for another six years,the time when Mach’s influencewas at its peak. Mach preached ascience based on sensoryperceptions and vehementlyopposed any scientific hypothesisabout things that could not be feltnor seen. Meanwhile, Boltzmannwas spending more and more timedefending himself and writingphilosophical essays aiming todiscredit Mach’s outlook. But hisstudents beat a path to Mach’slectures, and few of them didscience for its own sake. Vienna’sdays as an important center forresearch in physics were just about

over. During that period Boltzmannpublished a book called Lectureson the Theory of Gases,27 whichwas included his own research onthe kinetic theory of gases.

The unhappy Boltzmann longedfor a change of place andatmosphere, but only in 1900 couldhe finally move to Leipzig atOstwald’s invitation, barely takingthe time to say goodbye to hiscolleagues and friends. Despite thewarm welcome he received inLeipzig, and despite his goodrelationship with Ostwald, he wasmiserable and deeply troubled bythe rise of energism that Ostwald

was now leading. Academically, hepublished very little, and whateverhe did publish was a kind of blendof physics and philosophy. He wasnot especially interested in thework done by others around him –not even in Planck’s, which wasmaking innovative use of histreatment of entropy. Also,Boltzmann’s eyesight wasdeteriorating and he sufferedviolent mood swings. Six monthshad barely passed, and he wasalready contemplating a return toVienna. And indeed, Vienna –painfully, no longer a bastion of thephysical sciences – was anxious for

him to return. As always,complicated formal negotiationswere required, among them thegetting the Kaiser’s approval,which was given on condition thatBoltzmann would give his word ofhonor never to apply for a positionoutside the Austro-HungarianEmpire. These negotiations went onuntil 1902, while Boltzmann’smental state continued todeteriorate.

In the latter half of 1902,Boltzmann returned to find a pitifulscientific environment. Thestandard of teaching in physics washigh, but there was not one

researcher of his caliber. He gotback his former chair in theoreticalphysics, which had remained emptysince he had left a couple of yearsearlier. During the next few yearsBoltzmann was happier, especiallysince Mach had left the university in1901 due to ill health. In 1903, inaddition to teaching mathematicalphysics, Boltzmann was asked totake over Mach’s course inphilosophy. His philosophy lecturesquickly became a hit, and theauditorium was too small for allthose who wished to attend. YetMach’s health was still goodenough for him to continue his

vehement attacks on atomism and onthe last of its champions, namelyBoltzmann. Despite a temporaryimprovement in Boltzmann’s spirit,his health continued to bother him,and his anxiety attacks never let up.

In 1904 Boltzmann completed thepublication of his last importantbook, Lectures on the Principles ofMechanics.28 He also visited theUnited States, where he had anotherpublic debate with Ostwald onatomism. For the young physicistsattending the event – among themRobert Millikan, later a Nobellaureate – who were alreadyfamiliar with the concept of

atomism, the whole issue must haveseemed archaic and hard to fathom.Regretfully, even thoughradioactivity had already been anestablished field of research for anumber of years, Boltzmann did notunderstand that the new discoverieshe learned about during his visitwere about to prove atomic theoryconclusively, once and for all.Despite this, however, his workcontinued to be criticized, and hefell again into depression. As hesaw it then, his life work was aboutto implode despite all his efforts.

In 1905 Albert Einsteinpublished his three papers that

would change the face of physics(one of them, on the Brownianmovement mentioned above, wouldgive final proof to the atomichypothesis). Astonishingly, eventhough Einstein’s work was aconfirmation of Boltzmann’sdoctrine, Boltzmann did not have aninkling about its very existence. Atthat point he was busy with editinghis lectures on philosophy; inwriting an entry on kinetic theoryfor the Encyclopedia ofMathematics that his friend FelixKlein was editing; and on preparinga volume of his popular articlesfrom past years. Upon completion

of these tasks, he sunk intomelancholy. In May 1906 heresigned his teaching duties.

A month later, in June 1905, hewent with his wife and threedaughters to a seaside resort nearTrieste, Italy, hoping to calm hisnerves before the commencement ofthe new academic year. They stayedthere for three weeks, and at first itlooked as if the vacation was doinghim good. But again he sunk intodepression, so serious that his wifefelt obliged to call for their son,who was then serving in the army.In the evening of September 5,before his son had arrived,

Henriette and their daughters wentdown to the sea. When Boltzmanndid not follow, Henriette sent theiryoung daughter, Elsa, to fetch him.She found Boltzmann hanging fromthe window of their room.

Boltzmann’s work was receivedwith mixed reaction in his lifetime,and to this day there is no consensusabout the merits of his legacy.Boltzmann was viewed by some ofhis contemporaries, and by himself,as the defender of the atomic theoryof matter in an era when leadingscientists in Germany (such asOstwald and Mach) opposed it.

Thus was born the widespread yetfalse impression that hiscontemporaries ignored or opposedhim, and some even relate hissuicide to the awful injustice doneto him. The fact that he diedvirtually on the eve of the victory ofthe atomic hypothesis, thanks to thework of Einstein and Perrin,* addsa dramatic twist to the story.

Actually, Boltzmann’s reputationas a theoretical physicist was well-established and highly regarded.Universities fought for the privilegeof giving him chairs, and he wonmany awards and honors (thoughnever the Nobel Prize). It seems

that his suicide was due more to adeterioration of his physical andmental health than to any academicissue, even though it is possible thathis doubts regarding his H-theoremalso contributed to his mentaldistress. However, Boltzmann’splace in the pantheon of greatscientists is assured thanks to hisstatistical definition of entropy,which he proposed on his own,independently of Josiah WillardGibbs.

In 1933, after years of neglect, atombstone was erected on his gravein the Central Cemetery in Vienna,on which was engraved his

formula:

S = k log W

The H-theorem was derived fromBoltzmann’s wish to show that anideal gas will increase its entropyover time as a result of collisionsbetween its particles, in accordwith Clausius’s inequality. Usingkinetic theory, Boltzmann calculatedentropy as a function of time andgave it – for some unknown reason– the symbol H (in fact, H equals –Q/T). The premise for his formulawas this: a system in a relativelyordered state – for example, a

vessel in which all the gasmolecules are concentrated in onecorner – will reach, in its strivingfor equilibrium, a random state suchthat all molecules are uniformlydispersed throughout the vessel(that is, a state of higher probabilityand less order, compared with astate in which all the molecules arefound in one corner). Thus,supposedly, the system’s entropyhas increased, or equivalently, its Hfunction has decreased. Theproblem with Boltzmann’s premisewas that it was incorrect. Thereremained a probability – albeitquite low – that a random system

undergoing random collisions couldreach a more ordered state. If sucha thing could take place, it wouldcontradict the second law ofthermodynamics, which says thatentropy may only increases orremains the same, but may neverdecrease. This “paradox” can beremoved, however, by introducingthe concept of “microstate” (whichwill be explained shortly). That is,entropy is defined only inequilibrium and is determined byΩ, the number of microstates of asystem. An ordered microstate isequivalent to any other microstateand, in equilibrium, all microstates

have equal probabilities.

In 1877 Boltzmann published apaper in response to JosefLoschmidt’s disagreement with his(Boltzmann’s) statement that H canonly decrease. In fact, Loschmidtclaimed that Boltzmann’s systemswere reversible. In this paper,Boltzmann’s famous equation forentropy, S = k ln Ω, appeared forthe first time. (Actually, this is themodern formulation, slightlydifferent from the one thatBoltzmann wrote himself – the oneengraved on his tombstone.) In thisequation, k is a constant (called

today the Boltzmann constant) andΩ* is the number of differentpossible ways to arrange a system(in terms of the previous example,to place gas particles in differentlocations within the vessel). Thisnumber is variously referred to asthe number of combinations, orconfigurations, or microstates. Inorder to understand the concept of amicrostate, which is often confusedwith other quantities, we shalldefine the concept of macrostate aswell, using the following example:

Let us assume a macrostate of sixlots and three houses. Each lot caneither have a house built on it or

none. This is the system’smacrostate: three lots with houses,three without. For some purposes,this description may suffice; but notwhen it comes to entropy, because itis concerned with microstates.There are several differentpossibilities for how the threehouses can be distributed among thesix lots, and each individualpossibility is called a microstate.

For example, in one microstate,lots 1, 2, and 3 have a house each,and lots 4, 5, and 6 have none.Another microstate would have lots1, 2, and 6 with a house, and 3, 4,and 5 without one, and so forth. In

total (you can see this calculation inAppendix A-2) there are 20different microstates. Therefore, theentropy of these lots, according toBoltzmann, is: S = ln 20 = 2.9957…(Note that in this example we set k= 1, as is its value in informationtheory, which will be discussed inChapter 4.)

Now let us consider what theentropy would be if only one lot outof the six had a house. (In otherwords, the macroscopic systemwould have six lots, but only onewith a house built on it.) Since thereare now only six possiblemicrostates, the entropy would be

smaller: S = ln 6 = 1.7917… Thenumber of states that each lot canhave remains the same (two – withor without a house), but the numberof the microstates has changed.

What would happen if we addedanother empty lot to the first system,so that it will now have amacroscopic system of seven lotswith three houses? The number ofmicrostates will be 35, and entropywill increase accordingly, to S = ln35 = 3.5553… On the other hand, ifwe were to add an empty lot to thesecond system (the one in whichonly one lot has a house), thenumber of the microstates will be 7,

and entropy will grow only slightly,to S = ln 7 = 1.9459…

The above was a simpleexample. It was easy to calculatethe number of microstates, since weassumed that all the houses wereidentical. But even a slight changein the macrostate will substantiallycomplicate the entropy calculation.For instance, if the area of onehouse is 900 square feet, threehouses will have 2,700 square feet.If however the entire informationwe have is that there are six lotswith a total built area of 2,700square feet, what is the number ofmicrostates now? Or, rather, what

would the entropy be now? It wouldbe significantly higher, becausethere are a great many ways to build2,700 square-feet-worth of houseson six lots. To calculate suchproblems, physicists generally use atechnique called Lagrangemultipliers, which enables them tofind a distribution of the buildings’total area among the lots such thatthe number of the microstates is at amaximum (see Appendixes A-4, A-6, B-2, and B-3).

Now we can see the intuitionbehind Boltzmann’s entropy, whichwas really quite simple: the greaterthe number of microstates, Ω, that a

closed system may be found in, thegreater its entropy. In equilibrium,the number of microstates ismaximal, and each microstate hasequal probability.

The way that Boltzmann arrivedat the expression of entropydescribed here was long andconvoluted. Physicist Abraham Paiswrote about it in his biography ofAlbert Einstein, Subtle is the Lord:

Boltzmann’s qualities as anoutstanding lecturer are notreflected in his scientific papers,which are sometimes unduly long,occasionally obscure and often

dense. Their main conclusions aresometimes tucked away amonglengthy calculations.29

By the way, this is the reason wederive the Boltzmann expression inAppendix A-2 the way Planck didit, rather than the way actually usedby Boltzmann.

Boltzmann’s question was this: inwhat way is the heat in a vesseldistributed among the particles ofan ideal gas (an ideal gas is one inwhich no intermolecular forcesoperate between its particles.) Toanswer it, Boltzmann built a modelof a box containing “microscopic

billiard balls” that act upon eachother only through elasticcollisions.

Figure 2: Two possible ways toarrange four balls in 49 places.

In this model we assume thatthere are much fewer balls thanavailable cells (that is, there is aninfinitely small chance of findingtwo balls in the same cell, or state).The actual arrangement of balls inthe box is its microstate. The ballscan be arranged in a very orderedmanner (for example, concentratedin one corner and leaving the rest ofthe area empty, as illustrated on theleft in Fig. 2), or scattered (as onthe right) with no discernible order.There are many more possibledisordered microstates than orderedones, so if we were to shake thebox when it is in a fairly ordered

microstate, we should expect thatthe balls will end up in a different,less ordered microstate.

Returning to the gas in the vessel:if we could take a snapshot of thearrangement of its atoms ormolecules, with exposure time thatis shorter than the time necessaryfor an atom to move from its place,we would be observing onemicrostate. Taking multiplesnapshots over a more extendedperiod would provide us a numberof microstates, since the gasparticles are constantly changingposition due to the unceasingcollisions between them. Taking a

series of snapshots for eternity letsus observe all possible microstates,because the system, theoretically,will change from one microstate toanother, eventually passing throughevery possible one. What isimportant to note is that the entropyin the vessel is not influenced by thespecific arrangement of theparticles at any instant, but rather bythe number of all the possiblemicrostates, both ordered anddisordered. To calculate thisentropy, therefore, one must countall possible microstates.

Boltzmann got his results usingkinetic theory, that is, by analyzing

the dynamics of the collisionsbetween the particles, and not bysimply counting the microstates, andtherefore his intuition provedincorrect. In particular, thefollowing misconceptions must beconsidered:

Shaking a box in which P balls arearranged in one corner willincrease the number of microstates,and therefore will increase entropy.This is a false intuition: the numberof possible microstates in the box isconstant and shaking the box justmoves the balls from one microstateto another. As mentioned, in a vast

majority of cases, shaking the boxwill change the system into a lessordered microstate, because thereare many more disorderedmicrostates than ordered ones.

When all the balls are on one side,the system is not in equilibrium,whereas when the balls arescattered throughout the box thesystem is closer to equilibrium.This too is an erroneous intuition. Infact, non-equilibrium would existonly if specific microstates havelower probabilities than the others.

It was the intuitive yet erroneous

idea that an ordered microstate haslower entropy and a disorderedmicrostate has higher entropy (anintuition based, as noted, on the factthat there is a far greater number ofdisordered microstates than orderedones), that gave entropy its badname, because it gave rise to someother misconceptions. For example:

Order and entropy are opposites.Since the higher the entropy, thehigher the disorder, it stands toreason that there is always atendency for disorder to increase.Lord Kelvin claimed that thesecond law of thermodynamics

would consume the universe bybringing upon it thermal death – astate in which everything would bein chaotic thermal equilibrium (thesame conjecture is attributed toHelmholtz too). In practice, though,entropy is a measure ofuncertainty, not disorder. Theentropy of a system defines thenumber of possible ways (that is,the number of microstates) in whichit may be found if every possiblemicrostate has an equalprobability! The only meaning ofthis definition is that entropy is theuncertainty one has when asked thequestion, what microstate is a

system in? As we shall see inChapter 4, von Neumann andShannon understood this point well.Yet Boltzmann’s error, namelyinterpreting the meaning of entropyas disorder (a mistake stemmingfrom the fact that there are manymore disordered than orderedmicrostates) has continues to hauntus to this day. All the more so since,as we shall see later, there aresituations where an increase inuncertainty actually means anincrease in information.

A spontaneous creation of lifecontradicts the second law, since

living systems seem to be highlyordered and consequently theirentropy is low. But in fact,increasing the complexity of asystem (as in a living system),increases its uncertainty, andtherefore its entropy. Thus, thespontaneous creation of life notonly does not contradict the secondlaw, but, as we shall see later, isdriven by it.

The second law is not necessary.Since entropy is the disordergenerated by random processessuch as rolling dice, we have noneed for any law in order to

understand that the world isstatistical. However, a law ofnature is the point, in a process ofanalysis, where one stops askingquestions and says instead, “That’sthe way it is.” Take for examplegravity, the attraction betweenmasses. Were we to ask an experton general relativity why massesare attracted to each other, we mayget a lengthy and learned answer,but the bottom line will be,“because that’s the way it is.” Andindeed, there is something thatseems almost trivial in the secondlaw. If the second law ofthermodynamics would be

exclusively based on probabilities,then events with low probabilitiescan also occur occasionally,meaning that there is a chance –albeit a low one – to witness someevent that seems to be in violationof the second law. In fact, somescientists occasionally do “showus” some violation of the secondlaw in some system or another. Butthe passage of a system from amicrostate of disorder into one thatis more ordered does not violatethe second law of thermodynamics.A violation would mean that thesystem spontaneously lowered thenumber of its possible microstates.

If that could happen, we shall beable to build air conditioners thatrequire zero energy to run.

In conclusion, the number ofmicrostates – both ordered anddisordered – in a closed system isconstant, and the logarithm of thatnumber is the system’s entropy(which means that the entropy of aclosed [isolated] system remainsconstant). The reason entropyincreases in a non-isolated systemis because there tends to be aspontaneous increase in the numberof microstates that the particles canbe in. If the number of possible

microstates does not increase, thenthere is no spontaneous change inthe system! To decrease the numberof possible microstates for asystem, there must be an input ofenergy (work). For example, workis required to compress gas,because the number of possiblemicrostates has been lowered.*

Boltzmann’s andClausius’s Expressions of

Entropy

The expressions those given byClausius and Boltzmann for entropy

were as different as can be. Whilethe Clausius entropy is the ratiobetween two physical quantities –heat and temperature – Boltzmann’sentropy (as defined above) makesno direct mention of either heat ortemperature. So how is it that thesetwo expressions describe the samequantity? The reason is that thenumber of possible microstates isdependent on the amount of energypossessed by the particles, which isdependent on the overall amount ofenergy possessed by the gas,namely its temperature. Thearrangement of particles illustratedin Fig. 2 above is actually an over-

simplification of a microstate,because in addition to its physicallocation, every particle actuallyalso has velocity – a speed-and-direction vector conventionallydepicted by an arrow. If wedistribute the energy between theparticles in one particular way, thenumber of possible microstates willbe different than when the energy isdistributed in another way. Butentropy must be defined inequilibrium, that is, when thenumber of microstates is at amaximum. Therefore, there must besome point at which the energydistribution gives the maximum

number of microstates. Thisdistribution satisfies Clausius’sinequality. The distribution at whichthere is a maximum number ofmicrostates is generally a functionof temperature: for eachtemperature, there is a differentdistribution.

Appendix A-4, which deals withthe derivation of the bell-shapeddistribution, shows how thedistribution of energy betweenatoms is calculated to yield themaximum number of microstates.Planck used this same principle tocalculate the energy distribution inthe thermal electromagnetic

radiation of matter, and thus madeone of the most importantdiscoveries in the history ofphysics: energy can only beabsorbed or emitted in discretequantities – a discovery that laid thefoundations of quantum theory.

It can said that just as Carnot’stheorem led to the more generalformulation of entropy by Clausius,so Clausius’s inequality led toBoltzmann’s even more generalformulation. Boltzmann’s definitionnow connects a physical concretevalue (the Clausius entropy) withthe uncertainty that exists in asystem. The disadvantage of

Boltzmann’s definition of entropy isthat in many cases its value dependson how the problem is outlined (aswe saw in the lots and housesexample), and one macroscopicsystem may be assigned differentvalues of entropy depending on thespecific statement of the problem.In addition, Boltzmann’s entropyrequires complex calculations todetermine the number ofmicrostates. On the other hand, aswe shall see in the second part ofthis book, the generality ofBoltzmann’s expression does allowus to extend it beyond the physicalsciences to other fields, such as

logic, networking and sociology.

The Gibbs Entropy

“The whole is simpler than thesum of its parts”.”30

- Josiah Gibbs

At approximately the same time thatBoltzmann was working out his

statistical explanation for thesecond law of thermodynamics, in1877, an American scientist namedJosiah Willard Gibbs, working atwhat was in those days abackwoods scientific center – YaleUniversity in Connecticut –published the results of his researchon the same topic. Today, hiscontribution is regarded as one ofthe greatest breakthroughs of 19th

century science, a cornerstone ofphysical chemistry.

As mentioned before, Boltzmannexpressed entropy as a function ofthe number of microstates, a value

that derives from the extensivity ofthe entropies of multiple systems.The Gibbs entropy was derived bymixing these systems. In fact,Gibbs’s expression is not a functionof the number of microstates, butrather a function of theirprobabilities. Of course, the finalresult yielded by both expressionswill give the same value forentropy, but Gibbs’s expression forentropy is more useful for manycases, especially in calculating theentropy of information, which willbe discussed in the second part ofthis book.

Josiah Willard Gibbs

Josiah Willard Gibbs, 1839-1903

Josiah Gibbs was born in NewHaven, Connecticut, on February11, 1839, the only son of a

professor at Yale University (likefive previous generations beforehim). As a boy, he was somewhatwithdrawn socially in school lifeand of rather delicate health, but heproved very diligent student.

When he was fifteen he beganstudying at Yale College, where hewon prizes for excellence in Latinand mathematics. He graduated in1858, and remained at the collegewhere he undertook research inengineering. Most of hisengineering knowledge was self-taught: in college he was taughtmathematics, the classics, and just abit of science. As was common at

the time in many universities, theentire science curriculum was aone-year course called “naturalphilosophy,” which includedchemistry, astronomy, mineralogyand the like, alongside withphysics. But as a graduateengineering student, Gibbs also didsome experimental work in thelaboratory – quite unusual in thosedays, since it was not part of theusual science curriculum.

His dissertation, On the Form ofthe Teeth of Wheels in SpurGearing, in which he usedgeometric techniques to investigategear design, won him the first Ph.

D. degree awarded by Yale inengineering. He received his degreein 1863 and remained to teach inYale for three years: two teachingLatin, and then teaching “naturalphilosophy.” In 1866 he patented adesign for an improved type ofrailroad brake.

Gibbs was never short of means,as his father’s death in 1861 (hismother had died some time before)had left him and his sisters a fairamount of money that allowed themto travel to Europe, and Gibbs tookadvantage of this opportunity tomeet with distinguished scientificcolleagues and learn from them.

Among other places, he spent a yearin Berlin and another one inHeidelberg.

In those days, the Germanacademic institutions were worldleaders in chemistry,thermodynamics and theoreticalnatural sciences. As a rule, in the19th century, the Europeanscientific standards were far higherthan in the United States, in allsciences. (In engineering, though,the U.S.A. did not fall this farbehind.) Gibbs used well thisunique opportunity (after returningto the U.S.A., he never left NewHaven again) and put all his efforts

into acquiring knowledge that wasnot available at the time in hishomeland. Looking at the lecturenotes he took there, it is impossibleto tell which subjects he foundparticularly interesting, but there isno doubt that during his stay inGermany he was influenced byKirchhoff and Helmholtz.

In June 1869 Gibbs returned toNew Haven and resumed his work.Just two years later, when YaleUniversity decided to upgrade itsscience program, Gibbs wasappointed professor ofmathematical physics – before hehad published even one paper. He

received no remuneration for hislectures, which was not unusual inthose days even in importantscientific centers. His studentsfound him an efficient, industriouslecturer, as well as a patient,encouraging and considerate one,and often amusing.

It was not until 1873, whenGibbs was 34 years old, when hepublished his theory ofthermodynamic equilibrium, that thescientific community began torecognize his genius. It was anextremely original and innovativework, published as two papers. Thefirst one, Graphical Methods in the

Thermodynamics of Fluids,31

included his formula for freeenergy. The second one had the titleA Method of GeometricalRepresentation of theThermodynamic Properties ofSubstances by Means ofSurfaces.32

In these papers Gibbs used awide range of novel graphs andillustrations of the kind now calledphase diagrams. These make itpossible to present variouscombinations of quantities, such aspressure, volume, temperature,energy, entropy and the like, in

extremely useful ways.* Forinstance, a graph of entropy vs.volume showed when gas changedto liquid, or liquid to solid. In hissecond paper he presented three-dimensional graphs whose axesrepresent three thermodynamicqualities, i.e. entropy, energy andvolume, at the same time. For eachsubstance, the three-dimensionalspace was divided into areassuitable for gas, liquid or solid, andthe interfaces between themprovided more information (forexample, the temperature andpressure of a phase transition). Insuch a seemingly simple

representation there was a wealthof information that used to beprovided separately in the past, butcould now be presented as a whole.

Maxwell was impressed byGibbs’s work, and in a 1875 lecturespoke about it to the ChemicalSociety in London, in an attempt todraw British scientists’ attention tothis enormous contribution tothermodynamics. He even went sofar as to build a three-dimensionalmodel of Gibbs’s thermodynamicsurface for water. Shortly before hisdeath, Maxwell sent this model toGibbs.33

In 1876 Gibbs published the first

part of his On the Equilibrium ofHeterogeneous Substances; thesecond part appeared in 1878.34

This work added another importantlayer to the study ofthermodynamics, and finally earnedhim the fame he deserved. In thesepapers Gibbs used thermodynamicsto explain some of the physicalphenomena associated withchemical reactions – until then,there was little overlappingbetween these disciplines, and theireffects on each other werepresented merely as a series ofisolated facts – and triggered arevolution in classic

thermodynamics. In fact, Gibbsapplied thermodynamics tochemistry, and since then, everychemistry student solves problemsabout the concentrations ofchemicals in chemical equilibriumusing to Gibbs’s methodology. Inthe third and longest part of hiswork, Gibbs dealt with the problemof thermodynamic stability (namely,equilibrium).

Gibbs’s papers are now regardedas highly important, but a number ofyears were to pass before theirsignificance was widelyacknowledged. It was claimed thatthe reason for the delay in

recognizing the importance ofGibbs’s work was due to his styleof writing, which was difficult tofollow, and also to the fact that hechose to publish his papers in anobscure journal, Transactions ofthe Connecticut Academy ofScience. His brother-in-law, alibrarian, was the editor of thisjournal, and if it had few readers inAmerica, they were even fewer inEurope. Gibbs therefore took painsto send reprints of his papers to theone hundred scientists, more orless, who in his opinion couldunderstand and appreciate his work.Maxwell, Helmholtz and Clausius

were on the list; Boltzmann’s namewas added only by the time the thirdpart was published.

In 1880, the newly establishedJohns Hopkins University inBaltimore, Maryland, offered Gibbsa professorship with a salary of$3,000 a year. Yale responded byoffering him $2,000, but Gibbsaccepted Yale’s offer and remainedin New Haven.

Between 1880 and 1884 Gibbsdeveloped vector analysis bycombining the ideas of WilliamHamilton and Hermann Grassmann.This work is still of immenseimportance in both pure and applied

mathematics. Among other things,he applied his vector methods tochart the path of a comet byobservations from threeobservatories. The method, whichneeded far fewer calculations thanthe previously used Gauss method,described the path of the SwiftComet in 1880. Between 1882 and1889 Gibbs published a series offive papers on the electromagnetictheory of light.

From 1890 and on, Gibbsdevoted most of his time toteaching, and his research duringthose years was of less importancethan his initial contributions to

thermodynamics. Only in 1902 washe was persuaded to compile hisviews and ideas on numeroussubjects in an important volume,Elementary Principles inStatistical Mechanics,35 that wasto provide a general mathematicalframework for both Maxwell’selectromagnetic theory andsubsequently quantum theory. Withthis work, Gibbs has laid theprinciples of statistical mechanicson a firm foundation.

Initially, as noted above, notmany physicists and chemistsrecognized the importance ofGibbs’s contributions. Only after

Wilhelm Ostwald translated hispapers into German (then theleading language in science) in1892, and Henri Louis Le Châtelierinto French in 1899, did his ideasbegin to have an impact onEuropean science. Gibbs’s papers,noted Ostwald in his autobiography,had far-reaching impact on his owndevelopment as a scientist.36

At long last, then, Gibbs’s workwas becoming known, appreciatedand acknowledged by hiscontemporaries, but Gibbs himselfnever made an effort to explain hisideas beyond what he wrote in hisscientific papers, and these – even

to his admirers – were abrupt to apoint that defied full understanding.Preferring to present his ideasmainly as calculations anddiagrams, he was a man of fewwords. To his critics he wouldrespond that on the contrary, hispapers were too long andunnecessarily wasted his readers’time. He was also known to keephis ideas to himself until he hadfully developed them, and neverexplained how he got hisinspiration for doing the researchabout which he wrote.

Apart from those few years inEurope, Gibbs lived all his life in

the home his father had built, a shortdistance from the school in whichhe studied and from the college anduniversity in which he worked allhis life. He died at the age of sixty-four. J. G. Crowther, author of abook about famous Americanscientists, summed up Gibbs’s lifeas follows:

[Gibbs] remained a bachelor, livingin his surviving sister’s household.In his later years he was a tall,dignified gentleman, with a healthystride and ruddy complexion,performing his share of householdchores, approachable and kind (if

unintelligible) to students. Gibbswas highly esteemed by his friends,but U.S. science was toopreoccupied with practicalquestions to make much use of hisprofound theoretical work duringhis lifetime. He lived out his quietlife at Yale, deeply admired by afew able students but making noimmediate impress on U.S. sciencecommensurate with his genius.37

American physicist H. A.Bumstead, Gibbs’s colleague inYale, eulogized him thus:

Unassuming in manner, genial and

kindly in his intercourse with hisfellow-men, never showingimpatience or irritation, devoid ofpersonal ambition of the baser sortor of the slightest desire to exalthimself, he went far towardrealizing the ideal of the unselfish,Christian gentleman. In the minds ofthose who knew him, the greatnessof his intellectual achievementswill never overshadow the beautyand dignity of his life.38

Gibbs’s contribution was onlyfully recognized in 1923, whenLewis and Randall published theirbook Thermodynamics and the

Free Energies of ChemicalSubstances,39 which presentedGibbs’s methods to chemistsworldwide. Since then, chemicalengineering is based on it.

Whereas Boltzmann’s entropy, for asystem with Ω microstates, is theproduct of the logarithm of thatnumber by a constant now calledthe Boltzmann constant, Gibbsdefined the same entropy as the sumof the entropies of the individualmicrostates. Since the entropy ofeach microstate is dependent on itsprobability, Gibbs showed that

entropy can be written as the sum ofthe probabilities (see Appendix A-3). Gibbs’s expression for entropyis

that is, the sum of all themicrostates of pi ln pi, where pi isthe probability of microstate i.Sometimes it is more convenient tocalculate this expression, ratherthan Boltzmann’s. Also, usingGibbs’s equation for entropy wecan divide a system into itscomponents according to our needs.

In fact, the Gibbs entropy hasbecome the most common means forcalculating entropy, now used inthermodynamics, quantum physics(where it is called the von Neumannentropy) and information theory(where it is called Shannon’sentropy).

Gibbs’s expression for entropy isthe one that appears in mosttextbooks on statistical mechanicsor information theory, and it can beinterpreted as a generalization ofBoltzmann’s entropy. Indeed, theGibbs entropy allows us to assign adifferent probability to eachparticular microstate (or to a

number of specific microstates).Returning to the example of the sixlots, three of which have houses, theresult obtained using Boltzmann’sexpression was SB = ln 20. UsingGibbs’s method and the assumptionthat all microstates have the sameprobability, 1/20, the entropy willbe the sum of twenty identical units

This result is indeed the same as theone was obtained usingBoltzmann’s expression (recall thatln = – ln x).

However, what will happen tothe entropy if the owner, who livesin one of the houses, prefers that thelots adjacent to his property on bothsides will remain empty?Previously, there were twentymicrostates, represented as follows(1 represents a lot with a house, 0represents an empty lot):

By adding the constraint that onehouse at least will not have anyneighbors, the number ofmicrostates is reduced to ten:

Let us complicate the problemeven further and assume that theowner will be willing, withreservation, to live in a house thathas one neighbor, the reservationbeing that the house should have aview to the west. We assume thatonly 50% of the lots have a view to

the west. Thus the probability forten of the microstates is 2/3,whereas ten microstates have onlyone half of this probability, that is,1/3. Therefore, every lot amongthose ten without any neighbors hasa probability of p = , and

the ten microstates with oneneighbor each has a probability of p= . Thus, the entropy according to

Gibbs is

This result is smaller than

Boltzmann’s entropy. In general,adding constraints alwaysdecreases entropy, and thus theGibbs entropy will usually belower than Boltzmann’s. So theGibbs entropy is useful whendealing with situations that are notin equilibrium, meaning that often itshould not properly be calledentropy, since entropy is definedonly in equilibrium. Nevertheless,its importance is enormous, as weshall see below.

The example used above, theentropy of lots and houses, seemsutterly useless. It is inconceivablethat any member of any zoning

committee would care about thelogical entropy of lots and houses ina particular area. However, theimplication of such calculations forother areas cannot be overstated.Recall that according to Clausius, S= Q/T, and thus Q = TS, meaningthat the entropy produced by someprocess multiplied by itstemperature gives heat. This is theheat that cannot be used for work,since it goes into increasing entropy– as is necessary to prevent adecrease in total universal entropy(recall that this is the heat thatreduces the efficiency of the Carnotmachine). Therefore, when energy

is released in some process and wewant to know how much of it can beused for other purposes; we mustsubtract the value of TS from theamount of released energy. Thereleased energy minus TS (theenergy which is consumed toincrease entropy) is called freeenergy. Every chemist knows howimportant this value is incalculating the efficiency ofchemical and nuclear reactions,from power stations to batteries. Infact, free energy is Carnot’sefficiency of a chemical reaction,and it was Gibbs who gave sciencethe necessary tool for calculating it,

by defining the entropy ofconstraints.

Putting lots and houses aside, letus review a more practicalexample. Two gases, A and B, areput in a container at a ratio of one toone, namely, there is an identicalnumber of molecules of each gas.Now, assume that we divide thiscontainer into two equal sections,and that some demon has contrivedto put all the A molecules on onepredetermined side of the partition,and all the B molecules on the otherside. This is a system with aconstraint, since there are nomicrostates that have molecules of

A among the molecules of B, orvice versa. Before the demon’sinterference, one section of thecontainer had a 50% probability ofhaving molecule A and a 50%probability of having molecule B.The entropy per molecule is:

due to the fact we have twomicrostates, AB and BA.After the demon’s interference wehave full knowledge about the A’sand B’s molecules location in thesections, namely, we have one

microstate, and zero entropy. Inother words, the entropy of themixed system is greater by ln 2 permolecule. Now, multiplying this bythe Boltzmann constant and thenumber of molecules, we get thatthe entropy of mixing is: S = N kln2.

This means that there is a loss offree energy by an amount of Q = TN k ln2 in favor of the entropy ofmixing. (Generally, if the amountsof A and B are not equal, we woulduse Gibbs’s general equation.)

Applying thermodynamics tochemistry and Gibbs’s phase

diagrams allows chemists to predicthow the boiling or freezing pointsof solutions will change, calculatethe concentrations of reactants andproducts in chemical reactions inequilibrium, design distillationprocesses, calculate the rate ofchemical reactions as a function ofthe temperature, and so forth.Gibbs’s calculations constitute avery important contribution to thethermodynamics of chemicalreactions. But most astonishing isthe fact that his expression ended upbeing the basis for informationtheory, which will be discussed inChapter 3.

The Maxwell-BoltzmannDistribution

The true logic of this world is thecalculus of probabilities.

-James Clerk Maxwell40

Up to this point, it has been shownthat entropy has two faces, aphysical one, described byClausius, and a statistical one,described by Boltzmann and Gibbs.Connecting between these twoexpressions is the distributionfunction of the energy called theMaxwell-Boltzmann distribution(which was independently derived

by both scientists). This distributionis not unique to gases; if fact, itturns up in many other areas, andtherefore merits a specialdiscussion.

James Clerk Maxwell

James Clerk Maxwell, 1831-1879

Maxwell’s biography appears herenot only because of his calculationof the speed distribution ofmolecules in gas (the so-calledMaxwell-Boltzmann distribution),but also, even mainly, inacknowledgment of his contributionto our understanding the nature ofelectromagnetic radiation. As weshall see later, understanding thethermal distribution of radiation iswhat brought Max Planck to theconclusion that energy comes indiscrete, indivisible quantitiescalled quanta (singular, quantum).

The amount of energy in a quantumis proportional to its frequency.This conclusion ushered in a newera in the history of physics, andMaxwell is generally regarded asone of its founding fathers.

Indeed, James Clerk Maxwellwas one of the greatest scientistsever. To his electromagnetic theorywe owe some of the mostsignificant discoveries of modernscience and almost all thetechnology that serves us today.Maxwell also made significantcontributions to mathematics,astronomy and engineering, but firstand foremost, he will always be

remembered– and justifiably so –as the founder of modern physics.Albert Einstein, for example,arrived at his special theory ofrelativity due to his interest in acurious consequence of Maxwell’sequations: that the speed ofelectromagnetic radiationpropagation is independent of thespeed at which its source moves.On this Einstein said: “The specialtheory of relativity owes its originsto Maxwell’s equations of theelectromagnetic field.”41

Much of today’s technologydraws from Maxwell’sunderstanding of the fundamental

principles of the universe. Manydevelopments in the fields ofelectricity and electronics,including radar, radio, televisionand other communication devices,have been based on his discoveries.Rather than synthesize what wasknown before him, Maxwellcompletely changed ourunderstanding of the interactionbetween material substances thatuntil then was based on Newton’sviews. This change deeplyinfluenced modern science andinformed the later stages of theindustrial revolution.

James Clerk Maxwell was born

to John and Frances Clerk on June13, 1831, in a house Edinburgh,Scotland, that his father had built ina new, elegant area of the city. JohnClerk (who later changed hissurname to Clerk Maxwell) was alawyer, and both he and his wifewere quite involved in their city’scultural life. A short time after thebirth of their only son, James, thefamily moved to Glenlair, to agrand house on Middlebee Estatewhich they had inherited. The estatewas in a rather isolated area (atleast a day’s ride by horse andcarriage from the nearest city,Glasgow), and there Maxwell spent

the first ten years of his life. Hismother was responsible for hisearly education until her death whenhe was just eight years old, and thena local 16-year-old boy was hiredto tutor him privately. However,this tutor was unable to get youngMaxwell to apply himself to hisstudies or to discipline him, andthis unsuccessful experiment cameto its end after two years whenMaxwell’s aunt came visiting, sawwhat was happening and put an endto it. In 1841 Maxwell was sent to apublic (namely, in Britain, private)school, the Edinburgh Academy,and stayed with his widowed aunt.

Maxwell’s father and uncle, bothmembers of the Royal Society ofEdinburgh, encouraged his interestin science. At the age of 10, hisuncle took him for a visit at thelaboratory of William Nicol, theinventor of the Nicol polarizingprism, who gave Maxwell two ofhis prisms. This meeting made ahuge impression on the young boy,and later Maxwell described thisvisit as the turning point of his life,when he realized with certainty thathis vocation in life should bescientific research.

Despite this, his first years atschool were quite difficult. His

peculiar accent, strange shoes andthe unusual clothes his father’sdesigned for him singled him outamong his peers, who used to callhim “Dafty.” However, after twomediocre years, Maxwell began toshow his talents. He made somefriends, including Lewis Campbell,later professor of classics at St.Andrew University and hisbiographer, and Peter Guthrie Tait,future Professor of NaturalPhilosophy at Edinburgh University,who became his lifelong friend andpartner in solving mathematicalproblems.

At fourteen, Maxwell met an

artist by the name of D. R. Hay,who was looking for a way to drawoval shapes. The young boygeneralized the definition of anellipse, successfully producingactual ovals identical to thosestudied by René Descartes in the17th century. He then published hisfirst paper, On the Description ofOval Curves, and Those Having aPlurality of Foci.42 Maxwell’sfather showed the method to JamesDavid Forbes, an experimentalphysicist at the University ofEdinburgh. Forbes was impressed,and presented the paper in

Maxwell’s name at a meeting of theRoyal Society of Edinburgh inApril 1846 – without a doubt, anoutstanding accomplishment forsuch a young boy. Of course, hisideas were not new, but it wasremarkable that a young boy of 14could conceive them.

When Maxwell, at the age ofsixteen, completed his studies in1847, he was ranked second in hisclass. Another exemplary student atthe Academy was Peter Tait, oneyear younger than Maxwell, who atthe age of twenty would become asenior wrangler in MathematicalTripos – the youngest in the history

of Cambridge University. Maxwellcontinued his studies at theUniversity of Edinburgh. He choseto study the sciences even thoughhis father encouraged him to pursuelaw, and remained there for threeyears. He also studied philosophywith great intensity. At the age ofseventeen, Maxwell presented apaper to the Royal Society ofEdinburgh on The Theory ofRolling Curves, and then, ateighteen years of age, another paper– On the Equilibrium of ElasticSolids.

In 1850, Maxwell moved toCambridge University. Despite the

great importance accorded to thestudy of the classics in those days,the uppermost target for ambitiousstudents was to win first place inthe Tripos, a series of mathematicsexams: quite remarkably,mathematics dominated the teachingmethods in Cambridge at the time.Here Maxwell received excellenttraining in applied mathematics. Heread a book on analytical statisticsby Belgian mathematician, AdolpheJacques Quetelet, which made himrealize that physics could benefitfrom statistics. In one of his letters,he wrote: “The true logic of thisworld is in the calculus of

probabilities”; and in another:“This branch of Math., which isgenerally thought to favor gambling,dicing, and wagering, and thereforehighly immoral, is the onlyMathematics for Practical Men.”43

(And indeed, in 1859, he appliedthis insight to the explanation that hegave for the stability of Saturn’srings.)

In 1855 Maxwell was made afellow of Trinity College inCambridge, and then published hisfirst study in electromagnetics,which dealt with Faraday’s lines offorce.* This paper linked Faraday’s

field theory with the electricalforce, and showed how magneticinduction can be expressed by adifferential equation. It laterbecame clear that this was the keyto the analysis of electromagneticwaves, and also to field theory ingeneral. Thus Maxwell showed thata few rather simple mathematicalequations can express the electricaland the magnetic fields and theinteractions between them.

Maxwell’s stay at Trinity wasshort-lived. Early in 1856 his fatherfell ill, and Maxwell wanted toreturn to Scotland. He thereforeapplied to Marischal College in

Aberdeen for the fellowship inNatural Philosophy. His applicationwas accompanied by a letter ofintroduction from William Hopkins,a Cambridge scientist and mentor ofits most promising mathematicalstudents of the time (besidesMaxwell, they included alsoWilliam Thompson, the future LordKelvin, George Stokes, one of thefounders of fluid theory, Tait, andothers), which said:

During the last 30 years I have beenintimately acquainted with everyman of mathematical distinctionwho has proceeded from the

University, and I do not hesitate toassert that I have known no onewho, at Mr. Maxwell age, haspossessed the same amount ofaccurate knowledge in the higherdepartments of physical science ashimself… His mind is devoted tothe prosecution of scientific studies,and it is impossible that he shouldnot become (if his life is spared)one of the most distinguished menof science in this or any othercountry.44

Nevertheless, Hopkins believedthat Maxwell, despite his fineaptitude in physics, was somewhat

lacking in mathematical ability, ashis solutions were generallypresented in graphic-geometricmethods, rather than mathematicalanalysis.

In April of that year, at the age of25 years, James Forbes informedMaxwell that a position atAberdeen University had becomeavailable. During the Eastervacation, Maxwell went toEdinburgh, and a short time later,his father died. He returned asplanned to Cambridge, but upon hisarrival, he was offered the Chair ofNatural Philosophy at Aberdeen,which he accepted.

In Aberdeen he met the daughterof the Principal of MarischalCollege, Katherine Mary Dewar,and married her. Not much is knownabout her other than that she was ofgreat help to her husband in hisexperiments, but was not well-likedby his friends. She was, apparently,a hypochondriac, and Maxwellattended to her devoutly.

When St. John’s College inCambridge announced that the topicof their Adams Prize for 1857 was“The Motion of Saturn’s Rings,”Maxwell immediately becameinterested. When still a student atthe Edinburgh Academy in 1847, he

and Tait had considered thisproblem. He now decided tocompete for the prize, and his firsttwo years at Aberdeen weredevoted to the studying of thissubject. Maxwell showed that solidrings could not be stable and thusthey must be composed of small,solid particles. He won the prize,and his explanation was laterproved correct by observations.

In 1860, Marischal College andKing’s College merged to form theUniversity of Aberdeen, andMaxwell, still junior in hisdepartment, found himself – despitehis accomplishments – out of work

and with a pension of only 40pounds. This was a paltry amounteven for those days. However, witha private income of 2,000 pounds ayear from his estate, he was farfrom destitute.

In 1859, after Forbes moved toSt. Andrews College, the Chair ofNatural Philosophy becameavailable at the University ofEdinburgh. Maxwell applied for theposition, but again, his scientificaccomplishments did not help much,and the post went to Tait. Writingabout the announcement, the dailyCourant commented: “ProfessorMaxwell is already acknowledged

to be one of the most remarkablemen known to the scientific world,”and yet: “... there is another qualitywhich is desirable in a Professor ina University like ours and that is thepower of oral expositionproceeding on the supposition ofimperfect knowledge or even totalignorance on the part of pupils.”45

In fact, even though Maxwell’swriting was the epitome of clarity,his lectures and discussions wereincomprehensible and abrupt. Itseemed that his thoughts outpacedhis tongue.

However, this was for the best,since he almost immediately joined

King’s College at LondonUniversity, where he remained until1865. These were the mostproductive years of his career, forthen he began his research inelectromagnetism. In 1864, in alecture on The Dynamical Theory ofthe Electromagnetic Field, given atthe Royal Society of London, hestated: “We can scarcely avoid theconclusion that light consists in thetransverse undulations of the samemedium which is the cause ofelectric and magneticphenomena.”46

In 1865 Maxwell left King’sCollege to oversee the completion

of his house in Glenlair. Thanks tohis private fortune, he could retirefrom the burden of teaching and usehis time to run his estate, travelingthrough Europe, handling anextensive correspondence, andwriting his book, Electricity andMagnetism.47 The set of a fewpartial differential equations,known today as Maxwell’sequations, appeared there for thefirst time in their full form;undoubtedly, this was one of thegreatest scientific accomplishmentsof all time. Apparently, one of thereasons he wrote his book (whichwas published in 1873) was

frustration, since no one could bebothered to compare his andFaraday’s theories, which dealtwith electromagnetic fields, withthe prevailing German theories thatdealt with electric and magneticforces. In the introduction to hisbook he wrote:

Great progress has been made inelectrical science, chiefly inGermany, by cultivators of theory ofaction at a distance… theelectromagnetic speculation carriedout by Weber, Riemann, Neumann,Lorenz etc. is founded on the theoryof action at distance… These

physical hypotheses however, areentirely alien to the way of lookingat things which I adopt… it isexceedingly important that the twomethods should be compared… Ihave therefore taken the part of anadvocate rather than a judge.48

His retirement ended in 1871when he was persuaded to accept,without much enthusiasm, theCavendish Chair of ExperimentalPhysics at Cambridge – though notbefore both Joseph John Thomson,who was to discover the electron25 years later, and Hermann vonHelmholtz had declined the

appointment. He devoted most ofhis energy to designing and buildingthe Cavendish laboratory. Officiallyopened in 1874, it was destined togive the world many importantdiscoveries in physics.

Apart from his talents as atheoretician, Maxwell also had avery practical side. He showedgreat facility in the design ofmechanical instruments todemonstrate various phenomena inphysics. Among others, he built acolored spinning top that showed inan astoundingly simple way how afourth color is obtained from thecorrect mixture of the three primary

colors, and how colors areperceived by the human eye. Healso built, as mentioned earlier, amodel of Gibbs’s thermodynamicsurface for water (a phasediagram). A short while before hedied, Maxwell sent it to Gibbs, andto this day it is on display at YaleUniversity.

On May 17, 1861, at the RoyalSociety of London, Maxwellpresented the world’s first colorphotograph. The photograph wasmade using three filters – red, greenand blue – and the image wasremarkably true to the original.

During his sixth year at

Cambridge, the first symptoms ofstomach cancer began to show, andhe began a long, arduous battleagainst the disease. On November5, 1879, at only 48, Maxwellpassed away. He was buried on hisestate.

In 1859, after reading Clausius’spaper on the collision of gasmolecules, Maxwell began toinvestigate the statistical mechanicsof gases. In 1866, concurrently withBoltzmann yet independently ofhim, he derived what is todaycalled the Maxwell-Boltzmann

distribution. (A diagram of thedistribution of the speeds of theparticles appears in Fig. 3; thecalculations upon which this isbased can be found in Appendix 4-A.) The Maxwell-Boltzmanndistribution describes how energyis distributed among molecules ofgas. Since (in the case of amonatomic gas) the energy of a gasis directly related to the speed of itsatoms, the distribution of energybetween the atoms can be definedby Newton’s laws of motion to givethe distribution of speeds of theatoms in the gas.

Figure 3: This graph depicts therelative number of particles in a systemas a function of their speed. In an idealgas, the average speed rises withtemperature. The number of particleswith the hightest speeds is small, due toexponential decay.

This distribution of energy in

equilibrium has great importance innature, and its far-reachingconsequences are the main issue ofthis book.

According to the second law ofthermodynamics, the function of theenergy distribution in equilibriumwill result in a maximum value forentropy, that is, both the number ofmicrostates, Ω, and the entropy asdefined by Clausius, will be at amaximum. From the derivation ofthis distribution (see Appendix A-4), a bell-like speed distribution,the so-called Maxwell-Boltzmanncurve, is obtained (see Fig. 3). Instatistics, a bell curve is often

referred to as a “normaldistribution.”* In a normaldistribution, the probability that anatom will have a given energydecreases exponentially as theenergy rises.

The bell-like Maxwell-Boltzmann distribution is derivedfrom the exponential decay of thenumber of particles with a givenenergy. That is, the relative numberof particles carrying a particularenergy decreases exponentially as

energy rises according to ,where k is the Boltzmann constant,E is the energy and T is the

temperature. It follows that most ofthe particles possess low energy.Multiplication of the increasedenergy, E, by the exponentiallydecreasing number of particlesyields an energy distribution whichis a rising function at low energiesand a falling function at highenergies. The result is a bell-likedistribution.

The exponential decay is drastic:the number of deviations from theaverage decreases rapidly; most ofthe particles in the distribution haveenergy close to the average. Formany phenomena in nature theaverage has the highest probability;

however, there are also somenatural distributions which includesubstantial probability to theextremes. These distributions, the“long tail distributions,” will bediscussed in detail further on.

One of the astounding things inthermodynamics is the ubiquity ofits consequences. Amazingly, onecan glimpse them not only in theexact sciences such as physics,chemistry or biology, but also inpure mathematics, computerscience, sociological studies, andmore. The bell curve appears inthermodynamics because it wasdeveloped using the gas laws as a

starting point, but – as we shall seebelow, and as we saw earlier in theexamples of lots and houses – thenotion of logical entropy is muchmore general, and thus can beapplied to many other areas. Thesedays, science is making its firststeps towards the understanding ofcomplex systems; even though westill do not understand theparameters that determine theirstatistical distributions, the fact isthat such distributions are similar tothe Maxwell-Boltzmann one. Forexample, if we observe thedistribution of the height of asufficiently large group of people of

the same race and sex, we obtain abell curve. The factors thatdetermine a person’s height aremany and complex: genetic,nutrition, hormonal, etc. Yet thedistribution obtained is identical tothe one obtained from inert gases inthermodynamic equilibrium.

An even more surprising exampleis the distribution of the length ofthe cars in any given country. Thefactors that influence the length of acar are both engineering-related andpsychological. Sometimes, thesignificance of a larger car isprestige: a dimensionless quantitythat cannot interact in any

measurable way with otherquantities. Such quantities arecalled “logical quantities.” As youhave probably guessed, thedistribution of car lengths is alsobell shaped: there are a few shortones, most are of average length,and there are a few long ones.Therefore, a graph of the number ofcars as a function of length is abell-like curve. This distributioncharacterizes many other quantities,too, including intelligence asmeasured by IQ tests. Similarly, ifwe were to ask a group of people toestimate the weight of, let us say, asack of flour, we could predict that

the distribution of their answerswill also produce a bell-shapedcurve around the correct value. Infact, this is the thermodynamic basisfor the concept of “the wisdom ofthe crowd”: our tendency to assumethat the average answer is thecorrect one. Perhaps we naturallysense that a system in equilibrium,in which every microstate has anequal chance, is a “just” system.

The exponentially determined,bell-like curves, appears in naturein many different ways, and whatcharacterizes them is, as stated, thepreference for the average. Forexample, when we examine

mortality tables, according to whichinsurance companies determine thechances that a person of a certainage will die within the next year,we see that the chance of dyingincreases exponentially with age.That is, the older a person is, thegreater are the chances that he willdie within the next year.Correspondingly, most people willdie at or near the average age ofdeath (in Israel, approximately 80years), and fewer will die at an agefar removed from the average, thatis, much older or much younger.

Figure 4: The distribution of death

rate/age resembles, astonishingly, thespeed distribution in gas. In the figure:theoretical death rate/age in Israel.Source: CIA, World Factbook, (est.)2010

Nevertheless, it must be kept inmind that thermodynamicequilibrium does not alwaysprovide an exponentiallydetermined, bell-like curve. Thereis in nature another distribution, noless common, that yields finitedistribution to values far from theaverage. The decay in these cases isnot exponential but a power lawdecay; for example, the distributionof wealth in a given population. In

economics, this distribution isknown as the Pareto Law, or the20:80 rule (sometimes called alsoZipf’s Law, or Benford’s Law).Later on, we shall see that thesedistributions are also consequencesof the second law and its tendencyto increase entropy.

Chapter 3

The Entropy ofElectromagnetic

Radiation

Electromagnetic Radiation

This chapter discusses the entropyof electromagnetic radiation:beginning with the nature of theradiation itself, as explained byMaxwell, going through to MaxPlanck’s discovery that radiationenergy is discrete (quantized), andleading up to the calculation of theenergy distribution of radiation inthermodynamic equilibrium.

As mentioned above, in 1873 –in the golden age of science –Maxwell published his monumentalwork which describes, using a set

of equations subsequently namedafter him, both electric andmagnetic forces and the interactionsbetween them. The unification ofthese two important forces in asingle formalism was a historicalachievement, but furthermore,Maxwell’s equations explainedwhat electromagnetic radiationactually is.

Electromagnetic radiation, itturns out, is the periodic motion ofan electric field and a magneticfield perpendicular to it. Anelectric field is a property of aparticle or object with an electriccharge: in the space surrounding it

there operates a force that attractsto it particles with an oppositecharge, and repels particles with alike charge. A magnetic field isgenerated when there is a change inthe electric field – for example,when an electrically charged objectmoves within it – and influencesobjects that have magneticproperties. The electric andmagnetic fields are properties of thespace. Unlike sound waves, whichare vibrations of medium (such asair), the electromagnetic radiationdoes not require any medium inorder to exist.

One counterintuitive result of

Maxwell’s equations is that thespeed of propagation ofelectromagnetic radiation isindependent of the speed of thesource that emits it. If a persontravelling in a car throws an objectbackward, the object’s speed (asmeasured by an observer by theroadside), will be the sum of theobject’s speed, determined by theforce with which it was thrown, andthe car’s speed relative to theobserver. Yet the speed of the lightemitted from the car’s headlights(as measured by any observer,inside the car or outside) willalways be the same, whether the car

is moving or not. From this peculiarresult Einstein concluded in 1905,in his paper on the special theory ofrelativity, that the speed of light (itbeing a form of electromagneticradiation) is a universal constant.Among the implications of thisastounding result were hisstatements that time is relative, thatmass is a form of energy, and thatthe ratio between energy and massis the speed of light squared.

The periodic motion ofelectromagnetic radiation can bedescribed as an oscillatorcharacterized by its frequency.Perhaps the simplest oscillator

imaginable is a mechanicalpendulum, that is, a weight (bob)suspended from a string of a certainlength that swings back and forth ina periodic motion. Galileo Galileidiscovered in 1602 that the periodtime (the time required for the bobto complete one back-and-forthcycle) is independent of both theenergy of the pendulum and itsmass, and depends only on thestring’s length. This property, calledisochronicity, is used to measuretime in grandfather clocks.49 It isworth noting that all oscillators areisochronous. In old stylemechanical watches, a spring

pendulum with a frequency ofseveral hundred vibrations perminutes is used. Due toisochronicity, the watch does notspeed up or slow down when thespring is tighter or looser; rather, itkeeps at a constant pace.

Most watches today useelectronic oscillators, and only afew still use mechanical ones. Anelectronic oscillator includes aresonant circuit in which an electriccharge moves in periodic motionand a piezoelectric crystal. Theoscillating charge causes a changein pressure on the piezoelectriccrystal – a crystal that applies

mechanical pressure as a result ofan electric field, similar to a springthat applies pressure when amechanical force operates on it. Inelectronic clocks, the piezoelectriccrystal is usually quartz, hence theyare called quartz clocks. Thefrequency of a quartz circuit is afew million oscillations per minute,and thus quartz clocks are muchmore accurate than mechanicalones.

In fact, pendulums are used notonly to measure time, but also tomeasure length. The first standarddefinition of the unit of length, themeter, was proposed in 1668 by

English philosopher John Wilkins,who suggested that the meter is thelength of a pendulum string with aperiod cycle of two seconds (thatis, the time required for the bob tomove from one side to the other isone second). However, because thefrequency of a mechanicalpendulum is not only dependent onthe string’s length, but also on thegravitational force, which differswith the distance from the center ofthe earth, the measurement must beperformed at sea level.

The definition of the standardmeter changes from time to time.One of the most famous definitions

was the one accepted by the FrenchAcademy of Sciences in 1791: onemeter is one ten-millionths thelength of a quarter of the meridianpassing through Paris – from theequator to the North Pole. Based onthis definition, the mythologicalplatinum-iridium meter bar wascast, which is stored in theRepublic Archives in Paris. In1960, the definition of the standardmeter was once again madedependent on an oscillator, this timethe electromagnetic radiationemitted by the isotope 86 of theelement krypton. The InternationalBureau of Weights and Measures

determined that one meter is exactly1,650,763.73 wavelengths of theorange-red light emitted by krypton-86 in vacuum. (A wavelength is thedistance that light passes in onecycle; frequency is defined as thenumber of vibrations of theoscillator in a unit of time, and thusfrequency multiplied by wavelengthyields the speed of light; in otherwords, frequency corresponds tothe inverse of wavelength.) In 1983the meter was redefined once more,as the distance travelled by light in1 / 299,792,458 of a second, againin vacuum. This definition isderived from the fact that the speed

of light, as measured in the mostprecise experiments, is299,792,458 meters per second. Atthe time of writing, this is theaccepted definition.

Some sources of electromagneticradiation emit it in one frequency orin a very narrow range offrequencies (laser, maser). Othersemit it in broader ranges: somestars emit radiation all across theboard, and our sun covers most ofthe whole range. Even a simplelight bulb emits radiation in a rangebroader than the human eye can see.

The various frequencies ofelectromagnetic radiation are useful

to us in many different ways. Atrelatively low frequencies, it isused for radio broadcasts; at higherfrequencies, called microwaves, itis used for short-wave broadcastsand for cooking; at even higherfrequencies, in the range calledinfrared radiation (that is, it isbelow the frequency of the colorred that is visible to the human eye),and its energy is felt as heat. Whenthe frequency increases even more,radiation passes into the visiblespectrum – the colors of rainbowvisible to the human eye, beginningwith red and going on to with blueand violet (when all these

frequencies are combined, weperceive “white” light). At stillhigher frequency, the radiation iscalled ultraviolet; it is damaging toliving cells, and for this reason weare warned to use sunscreens whenat the beach. Next are x-rays, usedin medicine and other applications,and at even higher frequencies aregamma rays, which are emitted inradioactive decay. The highestfrequencies belong to some of thecosmic radiation that comes fromouter space, most of which isscattered by the atmosphere.

The types of interaction betweenelectromagnetic radiation and

matter are numerous, and we usethem for a vast variety ofapplications; however, such detailsare beyond the scope of this book.Here we shall limit ourselves to theabsorption and emission of a lightray of a given frequency. A ray caneither pass through matter withoutany change, or be scattered in it, orbe absorbed by it, or be perfectlyreflected from it (as in the case of aperfect mirror). Theelectromagnetic wave can bedescribed as an oscillator because,as mentioned before, the vibrationsof its electric and magnetic fieldsare synchronized. Oscillators

absorb and emit energy only if itsfrequency is similar to their own.This is familiar to anyone who triedto push a child in a swing at theplayground: we must apply aperiodic force which correspondswith the frequency of the swing. Ifthe swing is pushed forward at aconstant force, it will not moveperiodically; but if the forceapplied is synchronized with theswing’s motion, the swing willabsorb energy from the personpushing it and continue its back-and-forth motion.

Similarly, in order for a ray to beabsorbed by matter, its wavelength

must be much shorter than thethickness of the material slabthrough which the light is passing,so that the electric field will beable to oscillate within that slab anumber of times – at least once (inorder to transfer the energy to theelectrons in the slab). Waves whosewavelength is very long for thewidth of the absorbing slab will notbe able to oscillate within it, andtherefore will not be absorbed. Thisphenomenon is called diffraction.The higher the frequency (a shorterwavelength), the higher theinteraction of the radiation with thematter. Generally, radiation energy

is absorbed by the matter’sparticles (its atoms and molecules)and a short time later is reemitted.If, between the moment ofabsorption and the moment ofreemission, the particle moves, theray will randomly change itsdirection according to the particle’smotion. This is called scattering.

The higher the radiation’sfrequency, the higher is its tendencyto scatter. The reason is that theelectrons surrounding the atom’snucleus responds to changes in theelectric field; a higher frequencymeans a faster change in the field.For this reason, very long

wavelengths (that is, with very lowfrequency) are used for radiobroadcasts. They can travel longdistances, up to thousands ofkilometers, without being absorbedor deflected from their path. Thescattering of visible light is thereason for the sky’s blue color. Bluelight in the sun’s radiation has ahigher frequency than red, and thusit scatters much more in theatmosphere, coloring the sky blue.(The blue reflection from the skycolors the sea blue.) The sun’s light,after the blue has been scattered,turns red, and thus toward evening,the sky appears reddish. This was

first explained by British physicistLord Rayleigh, and therefore thescattering of light due to wavelengthis called Rayleigh’s scattering.*Rayleigh also gave, along with SirJames Jeans, a partial explanationfor black body radiation, whichwill be discussed later on.

Electromagnetic radiation isabsorbed and emitted by matter ininteresting ways. Maxwell’sequations show that an electriccharge vibrating at a givenfrequency will emit electromagneticradiation at the same frequency. Infact, radio transmitters are based onthis principle. Similarly, a static

oscillator with no energy(analogous to a mechanicalpendulum at rest) will absorb anelectromagnetic wave movingtoward it and begin to oscillate,provided the frequency of the waveis the same as the inherentfrequency of that oscillator. Radioreceivers are based on thisprinciple. Most of theelectromagnetic radiation thatsurrounds us is a result of theabsorption and emission of energyby atoms and molecules, andtherefore it is possible to describetheir electrons around the atomicnucleus as microscopic transmitters

and receivers.

Black body Radiation

If both transmitters and receiversare electrical oscillators, whatcauses one to emit radiation and theother to absorb radiation, not theother way around? This is athermodynamic problem, becauseheat is one of the forms of radiation(i.e. the infrared radiation). Weknow that heat is transmitted fromhot bodies to cold ones. Since thestatements of thermodynamics are

universal, energy must flow fromhotter oscillators to colder ones.But what is the temperature of anoscillator? In order to find out, wemust first calculate its entropy.

Already in Boltzmann’s days itwas known that an oscillator’senergy is proportional to itstemperature, that is, the energy isequal to the Boltzmann constanttimes the absolute temperature: E =kT. This conclusion was derived inthe study of the properties of idealgases, as explained in Appendix A-5.

Since entropy according toClausius is energy divided by

temperature, and since energy isproportional to temperature, weobtain . That is, every

oscillator has the constant amountof entropy of one Boltzmannconstant!* This is surprising,because it means that the entropy ofevery single oscillator is a constant,independent of its energy orfrequency. (Recall that anoscillator’s frequency is alsoindependent of its energy). Thisremarkable result implies that theamount of entropy of any oscillatoris the same, even if it is as massiveas an entire star, or as small as one

molecule of an ideal gas. On theother hand, the oscillator’stemperature is proportional to itsenergy. To demonstrate this, we maycalculate the temperature of aswinging bob with energy of onejoule. Since Boltzmann’s constant isapproximately 1.38×10-23 J/K-, thetemperature of such an oscillatorshould be an extremely high7.25×1022 K – greater by fifteenorders of magnitude than thetemperature at the core of the sun,where nuclear fusion takes place.

Can such a temperature be“real”? As a matter of fact, it can.

First of all, experience shows that apendulum tends to stop itsmovement (due to friction amongother reasons), just as a hot objecttends to cool down. Secondly, it iswell known that mechanical motioncan be used to obtain hightemperatures. Prehistoric manwould light a fire, among othermethods, by rapidly spinning ahardwood stick in an indentation ina block of softwood. Nevertheless,it is impossible to measure thependulum with a home thermometer,because the bob’s motion wouldbreak the glass…

What about the oscillator’s

energy? A vibrating mechanicaloscillator is extremely hot incomparison to its surroundings andis very “anxious” to get rid of itsenergy. The reason is obvious – alot of entropy will be generated bythe transfer of the hot oscillator’senergy to the numerous colderoscillators around it. For example,if we take a swinging bob andtransfer its energy to anenvironment that includes Noscillators, we increase entropyfrom k to Nk. In the air, for instance,N is the number of gas moleculesabsorbing this energy.

We can treat electrical

oscillators the same way, thanks tothe universality of thermodynamics.Like mechanical oscillators, anelectrical oscillator will also loseits energy (as an emission ofelectromagnetic radiation), andsince an electric oscillator has bothenergy and entropy, it also hastemperature.

The source of electromagneticradiation is the vibration of anelectrical charge, and so too is itsdemise. The molecules around ushave electrons (which for thepurpose of this discussion can beviewed as minuscule oscillators)that vibrate at various frequencies,

and their energy can be emitted orabsorbed depending on theirtemperature relative to theenvironment. A temperature that ishigher relative to the environmentwill make them emit radiation, anda lower one, to absorb it. Whenradiation encounters an electriccharge that can oscillate at all, itwill induce the charge to vibrateand lose some if its energy to thatcharge.

What happens if both oscillatorand the environment have the sametemperature? The answer is calleda “black body.” In a black body, alloscillators are at the same

temperature, and therefore theprobabilities for an oscillator toeither absorb or emit radiation areequal. By definition, this is a stateof thermodynamic equilibrium.

The abundance of sensationaldiscoveries made during science’sgolden age, at the end of thenineteenth and at the beginning ofthe twentieth centuries, alsoincluded the solution to what wasthen one of the most importantproblems in science: the question ofblack body radiation.

Black body radiation, despite itsname, refers to the electromagneticradiation that is emitted by any

material body (not necessarilyblack). The characteristic of a blackbody is that the frequencies of theradiation it emits are a function ofits temperature only. The energyemitted is completely independentof its composition, structure, or anyproperty other than temperature.Since every object in the universe(including the universe itself)apparently has some temperature,every object in the universe emitsblack body radiation, even if it alsoemits some other type of radiation.*

Black body radiation is similarto the Maxwell-Boltzmanndistribution, in that both are

functions of temperature. But thereis a great difference in the way theywere derived. The atomichypothesis, which was the basis forcalculating the distribution of gasparticles’ speeds, was not acceptedat the time by the majority of thescientific community because “itwas impossible to observe atoms,”and the distribution itself was notmeasured experimentally; rather, itwas derived through a theoreticalcalculation. So in this case,calculation precededexperimentation. In contrast, theblack body radiation emitted by anybody can be observed and easily

measured. These measurementswere the basis for the calculation ofthe black body radiationfrequencies distribution, which wasto lead to the monumentalconclusion that energy is quantized.

As can be seen in Fig. 5, blackbody radiation has a characteristicdistribution at various frequencies(and thus various wavelengths).

Figure 5: The relative intensity ofradiation energy emitted from a blackbody as a function of the radiation’swavelength.

Black body Radiation andBlack Holes

An interesting example thatshows how thermodynamic thinkingcan be useful even without therigorous calculations common inscience comes from a subject thathas recently become very popular –black holes. A black hole is a massthat has been compressed to such adensity that even electromagneticradiation cannot “escape” itsgravitational field. According toEinstein’s general theory ofrelativity, electromagnetic radiationwould be unable to escape from agravitational field whose intensityexceeds a specific value. Yet thesecond law of thermodynamics, as

we shall see immediately, forbidsthe existence of a body that does notemit any radiation.

Einstein’s special theory ofrelativity deals with effects that arebased on the assumption thatelectrical and magnetic fields areproperties of our space. In hisgeneral theory of relativity, Einsteinadded gravitation as a property ofthe space. More precisely, massbends space around it. It is hard toimagine a three-dimensional bentspace. A helpful two-dimensionalanalogy would describe a tightrubber sheet on which we place

iron balls of various weights. Eachball creates a depression of adifferent depth, depending on itsweight, that is, it distorts the rubbersheet in a way that is consistentwith its mass. If we roll a marblealong this sheet, its path will not bea straight line: wherever itapproaches one of the depressions,its path will deviate from thestraight line, in a way that dependson the depression’s depth. Now tryto imagine this in three dimensionsrather than two…

If Einstein was right and hisdescription of a curved space wascorrect, then light – even though it

has no mass – should also deviatefrom its path when it passes near amass. The bending of space by amass is quite miniscule, but it isstill measurable. In order to verifythe validity of Einstein’s assertion,it was suggested to observe a starwhose celestial position isprecisely known, at such a timewhen the line of sight from us to itruns very close to the sun.According to the general theory ofrelativity, the line of sight – the pathon which the light from that startravels toward us – should curve.As a result, the star will appear tous as if its location is different from

its known position. The problem is,of course, that it is impossible toobserve stars when the line of sightruns close to the sun, since sunlightwould make the star’s lightinvisible to our eyes (and to ourinstruments). However, this can bedone during a full eclipse of thesun, because then, for a fewmoments, this problem vanishes.

Einstein published his generaltheory of relativity in 1915. Thenext eclipse of the sun happened in1919, and the renowned Englishastronomer, Arthur Eddington,headed a scientific mission toAfrica, where the eclipse would be

full, to test the theory. Legend has itthat the measurements were notentirely conclusive, but in any case,Eddington announced that the starsobserved did, indeed, seem todeviate from their actual, knownpositions. In his photographicplates, they showed up at thelocations predicted by the generaltheory of relativity – in otherwords, Einstein was right. The nextday all the major world papers, ledby The Times of London, ransensational headlines: NEWTON’SLAWS HAVE COLLAPSED!

Einstein seemed to have been theonly one who was not unperturbed.

He was invited to the conference inwhich Eddington was to announcethe results of his observations, butEinstein said that he was busy.When the announcement wastelegraphed to the University ofBerlin, where Einstein worked atthe time, a spontaneous celebrationerupted, and one lady said toEinstein, “You probably didn’tsleep last night.” Einstein answeredthat actually he had slept very well.“But what would have happened,”she insisted, “if Eddington wouldhave proclaimed that you werewrong? “If that happened,”responded Einstein, “I would have

had to pity our dear Lord. Thetheory is correct all the same.”50

Despite his nonchalance, Einsteinbecame overnight a celebrity of astature equaled today only by rockstars. No other scientist ever, evenin the golden age of science, hasreceived anywhere near thepublicity and honor that Einsteindid.* No scientist in human memoryhas gained the acclaim and respectaccorded Einstein. The nameEinstein is now a synonym for awise person, and his face, with theemblematic hair and moustache, haspractically turned him into a brand-

name.Modern cosmology is based

entirely on the general theory ofrelativity, and one of its mostfascinating topics is the study ofblack holes. A black hole is amassive celestial body with agravitational field so powerful, thatnot only does it force light todeviate from its path, it does noteven allow light to escape its grasp.A black hole can be visualized as apit with smooth walls that devourseverything and emits nothing, noteven electromagnetic radiation.Although black holes researchbegan more as logical speculation

rather than as solid science basedon observation, it ignited thepopular imagination, and thescientists’ response was not long incoming. In the scientific searchengine, Google Scholars, there areover half a million links toscientific papers on black holes(although only about fifty actuallyclaim to relate them to actualobservations). The high priest ofblack holes study is StephenHawking, a fascinating andinspiring scientist, who, in spite ofbeing completely paralyzed in allfour limbs, can be said to be aliveand kicking. We may be assumed

that anyone reading this book isfamiliar with his image. His fameas a scientist today, among thegeneral public, is second only toEinstein himself.

One of the most peculiar thingsabout black holes is the minuteamount of information they provide:a regular physical system has plentyof properties, but a black hole hasonly four at most: mass, electricalcharge (if it has one), angularmomentum (ditto), and the diameterof is “event horizon.” (An eventhorizon is a sphere surrounding ablack hole, such that anything thatpasses through it is swallowed by

the black hole and cannot escape.)That is all. And if we omit the eventhorizon, because it is a function ofthe mass, we are left with a merethree properties. Even a single atomhas more properties.

With respect to thermodynamics,the black hole presents a paradox:if a black hole is a “point,” it musthave no entropy at all. Hence, ablack hole that swallows up somematerial object decreases theuniverse’s entropy, but then, it iswell known that the entropy of aclosed system can only remainunchanged or increase – andobviously, the universe is a closed

system.The answer to this paradox was

provided by a young postdoctoralfellow at Austin University, Texas,by the name of Jacob Bekenstein(now a professor at the HebrewUniversity, Jerusalem). Bekensteinknew that theoretical calculationsshowed that when two black holesmerge, the area of the event horizonof the resulting black hole is thesum of the horizon areas of the twooriginal ones. This is exactly theextensivity property we used inorder to derive Boltzmann’sentropy. On the other hand, nophysical mechanism was known

which was capable of making thearea of the event horizon smaller.This analogy between the area ofthe event horizon and entropyintrigued Bekenstein, and hecontemplated this problem until hefinally determined that black holesdo indeed have entropy, which isproportional to their event horizons.

But if a black hole has entropy,then according to Clausius, it musthave temperature. Hence a blackhole must emit black body radiation– yet by definition, a black hole isnot supposed to emit any radiationwhatsoever. Bekenstein’s claimirritated Hawking, who lashed out

at Bekenstein quite nastily. Yet lateron Hawking accepted the notion(though not without a final snipe atBekenstein: “[I]t turned out in theend that he was basically correct,though in a manner he had certainlynot expected”51), and evencharacterized the radiation that ablack hole must emit in view of itsfinite entropy. The explanation forthis radiation is complicated andrequires rather a long digressioninto quantum mechanics, which isbeyond the scope of this book.However, the bottom line is that theevent horizon of a black hole mustemit radiation in the form of energy-

bearing elementary particles, andthus black holes do not existforever. Every black hole willeventually (over tens of billions ofyears) evaporate through thisemission and disappear. Today, thisradiation is called Hawkingradiation, although many call itBekenstein-Hawking radiation.

This is how the second law ofthermodynamics led to theconclusion that a black hole, eventhough (according to its initialdefinition) it was incapable ofemitting radiation, must emit someradiation after all, just like anyother object. The story illustrates

the phenomenal change that hasoccurred in science during the lastfew decades. In the golden age ofscience, in the 19th century, therewere only about one thousandphysicists in the whole world –people who believed (sometimestoo firmly) in experimentation andobservation according to theprinciples of logical positivism. Itseems that today, with about half amillion links in Google Scholarpertaining only to black holes (andof those, about fifty thousand ontheir entropy), and despite the factthat no one has actually managed to“see” even one of them, the

pendulum has swung the other way.

The Distribution ofEnergy in Oscillators –

Planck’s Equation

“A new scientific truth does nottriumph by convincing its opponentsand making them see the light, butrather because its opponentseventually die, and a new generationgrows up that is familiar with it.”

- Max Planck52

The man who explained thefrequency distribution of blackbody radiation was a Germanscientist, a cultured man and a

broad-minded intellectual namedMax Planck. Planck, who at the endof his career saw (rather passively,some say) the collapse of Germanscience with Hitler’s rise to power,is mainly known for the explanationhe gave for black body radiation,based on his revolutionarydetermination that the energy ofelectromagnetic radiation isquantized, that is to say, comes indiscrete, indivisible “packets”.This hypothesis eventually led tothe development of quantum theory,an area of utmost importance inphysics today.

Max Karl Ernst LudwigPlanck

Max Karl Ernst Ludwig Planck,1858-1947

Max Planck was born in Kiel,

Germany, on April 23, 1858. Hewas the sixth child in the family.His family was highly educated,and his father was a law professor.When Max was nine years old, hisfather was appointed to a chair atthe University of Munich. His sonattended the MaximilliansGymnasium there, where heemerged as a gifted and diligentstudent. He excelled in alldisciplines, but stood out in musicand philosophy particularly. Hewas a gifted pianist with a perfectpitch, played the organ and cello aswell, and even composed liederand operas. He completed his

studies in the Gymnasium at the ageof 17 with distinction, and began toconsider which subject he shouldpursue at the university. As is oftenthe case, an acquaintance with acharismatic math teacher, HermannMüller, who discussed with him avariety of subjects – including theprinciples of conservation of energy– led him to decide finally onphysics. He was not dissuaded bythe words of Philip von Jolly, aphysics professor in Munich, whoadvised him against this subjectsince “in physics they havediscovered just about everything,and all that is left is to fill a couple

of holes.”53 To his lecturer at theuniversity he said that his goal wasnot to discover new things but tounderstand the known fundamentalsof the field.

Planck began his studies at theUniversity of Munich in 1874. Histeachers there were good, but nonewere scientists of note, and hefound refuge in self-study. In 1877he transferred to Berlin for a year,where he was fortunate enough tostudy under leading physicistsHermann Von Helmholtz and GustavKirchhoff, whose names arecelebrated to this day. Young Planckwas not very enthusiastic about

their lectures (he said thatHelmholtz was never quiteprepared for his lectures, spokeslowly, made numerousmiscalculations, and bored hislisteners; whereas Kirchhoff gavecarefully prepared lectures thatwere dry and monotonous).Nevertheless, studying in Berlin,then a major scientific center,broadened his horizons immensely.In particular, Clausius’s papers onthermodynamics kindled his interestand imagination, and this remaineda topic that would play a major rolein his scientific career. In 1879,having returned to the University of

Munich, he presented hisdissertation, Über den zweitenHauptsatz der mechanischenWärmetheorie (On the secondfundamental theorem of themechanical theory of heat).

One year later, at 22 years of ageand after completing his academicrequirements at Munich, he wasaccepted as a Privatdozent –lecturer without pay. As can beexpected, this position did notimprove his financial situation, andhe continued to live with his parents– much to his chagrin, since he feltthat he was a burden to theirhousehold. He continued to work on

the theory of heat, andindependently discovered thethermodynamic formalism thatGibbs had developed. Clausius’sideas about entropy greatlyinfluenced his thinking, and hebelieved that the second law ofthermodynamics and entropy play amajor role in physics.

He remained in Munich for fiveyears, and then, in 1885, his father’sconnections helped secure him anappointment as an associateprofessor of theoretical physics atthe University of Kiel. He remainedthere for four years, continuing towork on entropy. Now that his

annual income was 2,000 marks, hefinally felt, at the age of 27, that hecould afford a home of his own. In1887 he married his childhoodfriend, Marie Merck.

Planck was a family man – warmand gregarious – and his homebecame an active social centerfrequented by well-knownmusicians and scientists. In theevenings they would often playmusic – with Planck at the pianoand famous violinist and composerJoseph Joachim on the violin. Onchoral evenings, his students wouldjoin in. In later years, his guestsincluded Albert Einstein (with his

violin), as well as Otto Hahn andLise Meitner, who subsequentlybecame pioneers in the study ofnuclear fission. In addition tomusic, Planck loved hiking andmountain climbing, an activity hewas to pursue well into his senioryears (at the age of 80 he was stillfit enough to climb 3,000-meterpeaks in the Alps).In 1889, following the death ofGustav Kirchhoff, the first and onlychair of Theoretical Physics at theUniversity of Berlin becameavailable, and Planck received it.He remained there as an associateprofessor, but was also appointed

director of the Institute forTheoretical Physics (where he wasthe only professor). The Universityof Berlin was then the mostimportant center of physical andscientific research in Germany, anda world leader in theoreticalphysics. This appointment indicatedthat his outstanding talents wererecognized even before he had donehis monumental work. In 1892 hewas appointed full professor. Helectured on all branches oftheoretical physics and was theadvisor to a total of nine doctoralfellows. He was considered a goodlecturer; Lise Meitner noted that he

was “dry, somewhat impersonal,using no notes, never makingmistakes, never faltering; the bestlecturer I ever heard.”54

Planck decided to be atheoretical physicist at a time whentheoretical physics was not yet afield in its own right. About thisperiod he later wrote:

In those days I was essentially theonly theoretical physicist there,whence things were not so easy forme, because I started mentioningentropy, but this was not quitefashionable, since it was regardedas a mathematical spook.55

In 1894 Planck became interestedin black body radiation, and in1901 he presented his revolutionarysolution to a problem that had beenoccupying the world of physics foryears (one of those “holes” that stillneeded filling, according to vonJolly), as described below. In 1905Albert Einstein used Planck’s ideasto explain the photoelectric effect(and for this, not for the theory ofrelativity, Einstein received the1921 Nobel prize). Plank was oneof the first major physicists to adoptEinstein’s theory of relativityimmediately upon its publication in1905, and supported it against its

numerous opponents. In 1918Planck received the Nobel prize forhis achievements.

In 1909 after twenty-two years ofhappy marriage, Marie Planck died,apparently of tuberculosis, and lefthim with two sons and twindaughters. Following his secondmarriage in 1910, he had anotherson, but tragedy kept stalking him.His eldest son was killed in theBattle of Verdun in the First WorldWar (1916), his daughter Grete diedin childbirth one year later, and in1919, his second daughter, Emma,also met the same fate.

In 1914 Planck succeeded (in

collaboration with physical chemistWalter Herman Nernst) to bringEinstein to the Kaiser-WilhelmInstitute for Physics in Berlin. Afterthe World War I he also brought toBerlin his favorite student, Max vonLaue, a pioneer in the research of x-ray diffraction. Planck continued hisactivities at the University of Berlinfor over half a century, until 1928.Upon Planck’s retirement, ErwinSchrödinger, one of the fathers ofquantum mechanics (whichoriginated, as mentioned above,from Planck’s research on blackbody radiation) was appointed tofill his place. For a time, Berlin

continued to play a role as a majorcenter of theoretical physics – untilJanuary 1933, when Adolf Hitlerrose to power.

In addition to his great academicachievements, Planck was involvedin the science community as aleader and educator. He nurturedpromising young students, foundcreative solutions for problems inthe organization of scientific work,and filled a number ofadministrative posts from 1930 to1937 in the German PhysicalSociety, the Kaiser WilhelmSociety* (known today as the MaxPlanck Society), and the respected

scientific journal, Annalen DerPhysik. These were very influentialpositions.

Even though Planck’s activitiesremained within the boundaries ofscience, he was a prominent publicfigure due to his scientificreputation and his exceptionalachievements. His political viewswere typical of German Empire –conservative and patriotic. Hefound it hard to stomach theliberalism of the Weimar Republic,yet accepted the National-SocialistParty’s policies without any publicprotest. Nevertheless, he spared noeffort to aid his colleagues who lost

their jobs because of Nazipersecution, and attempted to workbehind the scenes for the freedom ofscience. There is no doubt that theThird Reich took advantage of hiscompromising attitude for itspolitical purpose. When Jewishscientists were being expelled fromGermany and some of their non-Jewish colleagues were seeking toemigrate, Planck urged them to stayput in order to preserve andpromote German science. OttoHahn, later a Nobel laureate, askedPlanck to gather well-knownprofessors for a publicdemonstration against the regime’s

treatment of Jewish professors.Planck’s answer was:

If you are able to gather today 30such gentlemen, then tomorrow 150others will come and speak againstit, because they are eager to takeover the positions of the others.56

During those years, Planck andothers struggled to preserve sciencein Germany. At Planck’s direction,the Kaiser-Wilhelm Societyavoided outright dissension with theNazi regime, yet he had no qualmsabout praising Einstein, at a timewhen the Nazis’ fiercely condemned

him and his work. He even met withHitler in an attempt to stop thepersecution of Jewish scientists andto discuss the fate of Fritz Haber,but to no avail.*

About Planck’s behavior duringthis period Einstein said in 1934:

He tried to moderate where hecould and never compromisedanything by his deeds or words ...And yet ... even as a goy, I wouldnot have remained president of theAcademy and the Kaiser-WilhelmSociety under such conditions.57

Einstein had good reasons for

saying this. Planck believed thatEinstein should have supported theregime no matter the circumstances,and since he failed to do so, Planckrecommended his expulsion himfrom the Academy. Ironically,Haber supported Planck on thisissue; von Laue was the only onewho opposed him. When Einsteinwas later asked if he wanted tosend his regards to anyone inGermany, he answered: “Only toLaue, and to no one else.”58

At the end of 1938, The PrussianAcademy of Science lost itsindependence and was taken overcompletely by the Nazis. Planck

resigned its presidency in protest(some say that the Nazi regime putpressure on him not to seek anotherterm in office). The politicalclimate was becoming more andmore hostile, and the Nazi JohannesStark, a member of the GermanAcademy of Physics and a 1919Nobel prize laureate, attackedPlank, Arnold Sommerfeld andWerner Heisenberg, whom helabeled “white Jews,” because theycontinued to teach Einstein’s theoryof relativity. The Nazi governmentoffice for science investigatedPlanck’s ancestry, but all they couldfind was that he was only 1/16th

Jewish.World War II deeply affected

Planck’s final years, a time of deeppersonal suffering and severeblows. He lost almost all hisproperty and all his notebooks andscientific diaries in an Alliedbombing raid on Berlin in 1944.His son Erwin, with whom he had aunique relationship, was cruellyexecuted by the Gestapo followingthe unsuccessful attempt on Hitler’slife in the same year. During thosewar years, Planck lived nearMagdeburg, in the country estate ofan industrialist friend. At the war’send he was 87 years old.

In his last few years, Planckconcentrated on philosophical andideological issues in modernscience, and worked tirelessly tobring his findings to the public’sattention and to instill his deep faithin the contribution of science tohumanity. He was often interviewedby the press and on radio, wrotearticles for daily newspapers andpopular science magazines, andlectured all over Germany and theworld. He himself explained themotivation behind this work asfollows:

At my age of 89 I cannot be

productive in science anymore:what remains for me is thepossibility of reaching out to peoplein their search for truth and insight,above all young people.59

His final years were spentmodestly in a relative’s house inGöttingen. He was sick and weak,but active nonetheless. He eventraveled to England to take part inthe Royal Society celebration ofNewton’s tricentennial in 1946(which had been postponed forobvious reasons; actually, Newtonwas born in 1642). In 1947, atalmost ninety years old, Plank died

in Göttingen.

In 1894, Max Planck becameinterested in black body radiation.Electric equipment manufacturerscommissioned him to find a way toproduce light bulbs (invented in1878) that emit the maximumamount of light for a minimumamount of energy. The problem ofblack body radiation had alreadybeen introduced in 1859 byKirchhoff: How does the intensityof the electromagnetic radiationemitted by a black body (a perfectabsorber, also known as a cavity

radiator) depend on the radiation’sfrequency? The problem wasexplored experimentally, becausethere was no consensus at thetheoretical level. A law discoveredin 1896 by Planck’s colleagueWilhelm Wien, linking the maximumfrequency to temperature, intriguedhim greatly, and he decided toderive Wien’s law from the secondlaw of thermodynamics. The resultwas a work whose scientificimportance cannot be overstressed,and here are its main principles:

Given a material object thatcontains within it all possible

oscillators (also called “radiationmodes”) – a dense object, eithersolid or liquid, is preferable forthis purpose, since unlike gas,which is mostly vacuum, electronscan move in solids and liquids inmany more orbits – the number ofpossible radiation modes isrepresented by N(v) (the Greekletter v, nu, denotes frequency).How would the expected energyemission look like as a function offrequency at any given temperature?

If you have carefully read the textto this point, your answer should beN(v)kT. Since the energy of eachoscillator is kT, the energy of each

oscillator times the number ofpossible frequencies N(v), willproduce the spectrum illustrated inFig. 5. This idea was introduced byLord Rayleigh and Sir James Jeansin 1897.

This elegant solution has oneconsiderable disadvantage – it isnot consistent with experimentalresults. The number of radiationmodes N(v) in a given volumeincreases as the frequency increases(since the wavelength becomesshorter, and therefore more of themcan exist within this given volume),and becomes astronomical at veryhigh frequencies. According to

Rayleigh and Jeans, if everyradiation mode emits kT of energyat high frequencies, the energyemission will become infinite –which both contradicts the first lawof thermodynamics (conservation ofenergy) and is inconsistent withexperimental results.

Planck first guessed the correctsolution already in 1900, butproved it using the second law onlya year later, and then published hispaper “On the Theory of theEnergy Distribution Law of theNormal Spectrum” in Annalen derPhysik. 60 To this day, there arethose who claim that Planck’s result

was merely a conjecture, but thetruth is that like other lions ofscience, he guessed first and provedlater. Planck’s solution included asignificant, far-reachingassumption: the idea that energy isquantized, that is, rather thanexisting in any arbitrary amount, itonly comes in quantized amountswhich he named “quanta” (plural of“quantum”, Latin for “how much”).The idea is that an oscillatorvibrating at frequency v can emit orabsorb energy only in multiples ofhv. The constant h, nowadays called“Planck’s constant,” was calculatedempirically from black body

radiation (as described in AppendixA-6). Remember that at that time,prominent scientists were stillarguing whether matter was discreteor continuous, and all of a suddenthere appeared the idea that evenenergy was discrete. The quantumof electromagnetic radiation iscalled a photon.

Mysterious are the ways ofscience. Newton had claimed, morethan two hundred years beforePlanck, that light was composed ofparticles, but this was perhaps hisonly idea rejected already by hiscontemporaries, because it waswave theory, rather than the theory

of the mechanical motion ofparticles (which he hadformulated), that gave a betterexplanation for optical phenomenasuch as interference and diffraction.On the other hand, light alsoshowed particle-like properties,and for many years scientists havestruggled with the problem, whethermatter was particulate or awavelike. Putting the question thisway is misleading, since it impliesthat matter must be either this or that– particulate or wavelike – wherein fact, matter shows properties ofboth particles and waves. Thewavelike properties of matter were

demonstrated in experimentsinvolving interference of massiveparticles, such as electrons orprotons. And now Planckdemonstrated, in his studies ofblack body radiation, thatelectromagnetic radiation too,which has no mass, neverthelesshas both particulate and wavelikeproperties.

Planck used Boltzmann’s entropyto calculate the distribution of Pparticles in N states (radiationmodes), such that entropy would bemaximized. He was the firstscientist to use Boltzmann’sequation for a calculation of

thermodynamic distribution.* In hisderivation, he made twoassumptions: first, that radiation isquantized; and second, that theamount of energy embodied in aquantum of electromagneticradiation depends on its frequency.That is, the higher the energy of aphoton, the higher its frequency. Noless important, he proved that anoscillator’s energy is proportionalto its temperature (that is, E = kT)only when the number of photons ineach radiation mode is very high,that is, at very low frequencies.When the frequencies are very high,the number of photons in one

radiation mode decreasesexponentially and thus the energyemitted is lowered dramatically.Thus Planck solved the problem ofinfinite radiation at high frequenciesthat arose from the Rayleigh andJeans model. The calculation ofPlanck’s equation is shown inAppendix A-6.

Planck’s monumental work hasmany ramifications, not the least ofwhich is quantum mechanics, butanother one of its majorconsequences was in explaining theentropy of electromagneticradiation and its quantization. Fouryears later, Einstein used the

quantization idea as an explanationfor the photoelectric effect, andspared Planck the Via Dolorosatraveled by Boltzmann with theatomic hypothesis. The term thatPlanck coined, “quantum,” becamepart of physics with thedevelopment of quantum mechanics,which unites the wavelike andmaterial properties of particles inone formal theory. Yet Planckhimself cannot be properlyregarded as one of the pioneers ofquantum mechanics. On thecontrary, he disapproved of it fromthe start. Quantum mechanics wasdeveloped, beginning twenty years

after Planck’s great discovery, byscientists such as Niels Bohr,Werner Heisenberg, Paul Dirac,Erwin Schrödinger, and otherswhose fascinating story exceeds thescope of this book.

Physical Entropy – ASummary

To sum up our discussion up to thispoint, entropy can be defined in twocomplementary ways:

The first is Clausius’smacroscopic entropy which is

involved in the transfer of energyfrom a hotter object to a colder one.When energy flows from hot tocold, entropy spontaneouslyincreases. This flow of energy isperfectly analogous to the flow ofwater from a higher place to alower one. This is the second lawof thermodynamics.

Clausius’s entropy, which isbased on our understanding of theconcept of temperature, uses asystem’s temperature of as ameasurable physical quantity, in thesame way as its mass or length, andis defined only in equilibrium. In anutshell, the change in the system’s

entropy, according to Clausius, isits heat divided by its temperature –in equilibrium.

The second definition of entropyis Boltzmann’s statistical entropy,or Gibbs’s entropy of mixing.Boltzmann’s entropy is thelogarithm of the number of asystem’s microstates times aconstant today called the Boltzmannconstant. Neither temperature norenergy appear explicitly inBoltzmann’s expression for entropy,although the number of microstatesis usually dependent on them.

To calculate Boltzmann’sentropy, the number of microstates

must be counted. In equilibrium,every microstate has an equalprobability, so that a system in anyparticular microstate has nostatistical incentive to shift toanother microstate.

Furthermore, the second lawrequires that entropy be calculatedin equilibrium, since in this state,the number of microstates is at amaximum. When the number ofmicrostates is not dependent onenergy and temperature (as with theexample of inert particles in a box),the number of microstates isconstant and calculating thesystem’s entropy is straightforward.

In contrast to Clausius’smacroscopic entropy, whichrequires the use of a calorimeterand a thermometer for its directmeasurement, Boltzmann’sstatistical entropy – which cannotbe measured directly – is related tothe distribution of energy amongparticles, or the radiation modes.The energy distribution can bemeasured, as we saw in the case ofa black body, and entropy can becalculated from it either bycalculating the number of possibleconfigurations (according toBoltzmann), or by calculating theirprobabilities (according to Gibbs).

Both Clausius’s andBoltzmann/Gibbs’s methods mustyield the same value for the entropy.Let us now review the assumptionswhich led to each of the twodifferent energy distributions,Maxwell-Boltzmann’s and Planck’s.

Maxwell-Boltzmann: TheMaxwell-Boltzmann energydistribution is calculated bymaximizing the number ofmicrostates and is based on theassumption that the number ofparticles is constant and muchsmaller than the number of possiblestates. This distribution is

exponential, and is called thecanonic distribution. Thedistribution of velocities derivedfrom the canonic distributionproduces the normal (bell-like)curve.

Planck: The energy distributionof a black body, as calculated byPlanck, assumes that the number ofparticles can be greater than thenumber of states (radiation modes).

Since it was not known at thetime that light has particulateproperties, Planck had to makeanother assumption, namely, thatenergy can be absorbed by orreemitted from an oscillator only in

multiples of a quantum, which is anindivisible unit of energy. The valueof the energy in a quantum isdirectly proportional to theoscillator’s frequency. Thisassumption was applied to thenumber of the microstates, and thetemperature was calculated fromClausius’s inequality. In theapproximation where the quantum’senergy was low compared to kTand energy could be removed froman oscillator (almost) continuously,Planck’s equation yielded theenergy of the radiation mode as E =kT, which is the case of thevibration of a diatomic molecule in

an ideal gas, or a mechanicaloscillator. When the amount ofenergy that could be removed wasgreater than the oscillator’s energykT, Planck’s distribution gave theMaxwell-Boltzmann distribution.

The quantum approximation,which yielded the canonicdistribution, was a surprising result.In a classical system, it isimpossible to obtain from anoscillator (or from any other object,for that matter) more energy than itcontained, in accordance to the lawof conservation of energy. Yet in thequantum approximation, it ispossible for a photon emitted from

an oscillator to have more energythan the oscillator itself!

What is the meaning of thisstrange result? The explanation isthat a number of oscillators collecttheir energy and emit it via oneoscillator chosen at random, thesame way that a lottery winnergains much more money than he orshe invested; of course, this is themoney contributed by many otherticket-buying participants. Most ofthe energy in black body radiation –and as we shall soon see, in othersystems in equilibrium as well – isfound in classical quanta, whoseenergy is lower than the average

energy. On the other hand, there arevery few quanta with higher energy,much greater than the average. Theelectromagnetic quantum energydistribution greatly resembles thedistribution of wealth: many quantaare energy poor, and a few are quiteenergy rich. We shall discuss thissimilarity between the numbers of“poor” photons and poor peoplethoroughly later on, when we’lldiscuss the economic aspects ofentropy.

Part II

Logical Entropy

Chapter 4

The Entropy ofInformation

Shannon’s entropy

“My greatest concern was whatto call it. I thought of calling it‘information,’ but the word wasoverly used, so I decided to call it‘uncertainty.’ When I discussed itwith John von Neumann, he had abetter idea. Von Neumann told me,‘You should call it entropy, for tworeasons. In the first place youruncertainty function has been usedin statistical mechanics under thatname, so it already has a name. Inthe second place, and moreimportant, no one really knowswhat entropy really is, so in a

debate you will always have theadvantage.”

-Claude Shannon61

So far we have discussed entropy inthe context of physical and chemicalprocesses. However, it does appearin other areas as well. Americanengineer Claude Shannon was thefirst to point out the important roleentropy has in communications.Before discussing thethermodynamics ofcommunications, it is veryimportant to define the concepts we

used – communication,information and content. Intechnical contexts, these terms havequite specific meaning, differentthan their everyday use. Forexample, when we say somethinglike, “the communication betweenBob and Alice is terrible becausethey tend to yell at each other,” theword communication refers to thequality of their personalrelationship. However, in thecontext of our discussion,communication means the processof transferring content; therefore,communication by yelling is notalways a “bad thing”: it may

actually prove quite efficient.Similarly, the concept of contenthas nothing to do with the amount of“wisdom” being transferred incommunication, but simply refers toa sequence of transmitted signals,such as letters and digits(commonly referred to as characterstrings).

The word information as we useit daily is not well defined.However, Shannon definedinformation as the number ofsignals, e.g. in the sequence of thebinary characters zero (0) and one(1), that can be transferred over a

given communication channel. Thatis to say, information is a series ofchoices between “yes” and “no”(represented by “one” and “zero”);or more concretely, Shannondefined information as the logarithmof the number of possible differentcontents that can be transmittedalong a channel.

In the technological sense, then,communication is the transfer ofinformation from a transmitter to areceiver (or multiple receivers).Content transfer happens only whenthe sender and the receiver ofinformation can interpret theinformation signals exactly the same

way. For example, if your newlypurchased electrical appliance isaccompanied by a user’s manual inChinese, and assuming that youcannot read Chinese, you havereceived quite a large amount ofinformation, but no contentwhatsoever.

People communicate amongthemselves through a wide varietyof means. The most common arespeaking, reading, writing, and evenbody language. In a face-to-faceconversation, each person takesalternately (or occasionallysimultaneously…) the roles of both“transmitter” and “receiver.” The

transmitter can be our vocal chords,bodily gestures, or even releasedchemical substances, and thereceiver will then be our ears, eyesor noses. Many technologicaldevelopments, including thetelegraph, radio, television,landline and mobile phones, fax,and computer, have steadilyenriched our means ofcommunication and have allowedus to communicate over practicallyany distance. While listening to theradio, the communication comesfrom the broadcasting station,which transmits to a large numberof receivers at different locations.

Two users (ignoring, for themoment, conference calls) cancommunicate using messaging,telephone or fax, no matter wherethey are and how distant from eachother they may be.

Until recently, information wastransmitted through telephone, fax,radio, and television using analogmethods. Even today, many publicradio broadcasts are analog. Tounderstand analog transmission, letus look at the radio. Here, atransmitter emits electromagneticwaves at a given frequency inwhich the sounds of speech ormusic have been incorporated in a

process called modulation, and areceiver receives the modulatedwaves, extracts the voiceinformation and transform it intovibrations of sound. There are twomain modulation methods: One ofthem, known as AM (amplitudemodulation), changes the amount ofenergy carried by the wave at anypoint in time, according to theintensity of the voice. In otherwords, the wave’s amplitude ismodulated. When the sound wavereaches the receiver, the amount ofenergy absorbed rises and fallsaccording to the wave’s amplitude,continuously changing the

characteristics of the sound emittedfrom the receiver. The secondmethod, FM (frequencymodulation), changes the wave’sfrequency according to thecharacteristics of the sound itcarries. The amplitude of the wavestays the same over time, but itsfrequency fluctuates. The higher thedeviation of the carrier’s resonancefrequency from that of the receiver,the lower the intensity of the soundemitting from the receiver.

Analog broadcasts have greatadvantages because they are lesssensitive to noise (extraneoussignals) during transmission, and

this is the reason that they havedominated the field until recentyears, when the quality oftransmission improved to a levelthat allows efficient digital datatransfer. The reason that analogtransmission is less sensitive tonoise is the data redundancy. Forexample, when a color picture istransmitted by analog modulation,the transmitted image is similar tothe original picture that is formed inour eye’s retina. If duringtransmission, noise is added to thepicture, such as optic distortion or ablur, the quality of the image formedin our retina will be reduced, but

not totally destroyed. An analogpicture of a blue sky presents a bluepixel for every point in the sky’sarea, and corrupted data for thispixel would create only a blank ordifferently colored spot in the sky.With a digital transmission,however, the blue area of thepicture is encoded by a few bits(explained below) defining theentire area of the sky’s picture andits color: one incorrect bit wouldchange the color of the whole sky,and “noisy” bits will, in manycases, completely corrupt the wholepicture. All the same, there aregreat advantages to digital

transmission as well: If we fax adocument (a fax is an analogdevice), the receiver never obtainsa document as sharp as the original,either because of the finiteresolution of the electro-opticscanner of the fax machine, or as aresult of noise on the telephone line,or both. If we forward this receivedfax further on, the qualitydeteriorates with each sending. Infact, repeating this process manytimes will eventually corrupt thedocument entirely, due to theaccumulated noise. In contrast,digital information sent by e-mailretains the same quality, no matter

how many times it was forwarded.There is in information theory a

“sampling theorem” that states thatit is possible to express any analoginformation by a digital file, andvice versa. Therefore, the relevanceof the thermodynamic analysis thatwill be done hereafter for digitalfiles is applicable for analog filesas well. However, a discussion ofthe sampling theorem would bebeyond the scope of this book.

Computers enable simultaneousdigital communication betweenmany users. That is, they transferinformation files – sequences of“ones” and “zeros” – as short,

electrical or electromagneticpulses. The pulses travel from onecomputer to another via a devicecalled a “modem” (modulator-demodulator). When a receivedpulse reaches a computer’s modem,the modem orders the computer’scentral processing unit to write a“one” in its memory; when there isan interval between pulses, thecomputer is programmed to write a“zero” in its memory. When thecomputer sends a sequence ofpulses (a file) via the modem, theyare processed successively: Whenthe processor reads “one,” itinstructs the modem to send a pulse,

when it reads “zero,” the modemdoes nothing. The content of the fileitself is thus expressed as a seriesof “ones” and “zeroes,” and isindependent of the intensity of thepulse: the value of any pulse is“one.” This is in contrast, forexample, to AM transmission,where intensity is the content.

A file made up of “ones” and“zeroes” is called a binary file. Afile that consists of logicalcharacters (not just “one” and“zero,” but also other characters) iscalled a digital file. All binary filesare therefore digital. By the sametoken, a file that consists of decimal

digits – the multiplication table, forinstance – is digital and decimal.

During the last few decades, theanalog transmissions that had oncedominated electroniccommunication media (telephone,radio, television, fax, and so on)have been largely replaced bydigital transmissions; hence theimpression that the analog methodof communication is the “simpler”one. In fact, the opposite is true:technologically, it is much simplerto transfer information in digitalform than in analog. Digitalcommunication, so common today,is naturally associated in our mind

with modern computer technology,but in fact it predates the computerera. The first to use this technologywas the American painter andinventor Samuel Morse, with hisinvention of the telegraph in 1836 –forty years exactly beforeAlexander Graham Bell inventedthe telephone (an analog device).

Morse (1791-1872) was aprofessor of art and painting at NewYork University and an importantartist in his own right. His interestin communication was kindled in1832 when, on the deck of a ship onits way from Europe to America, hemet and had a casual conversation

on that subject with an expert onelectromagnetism. The meetinginspired him with the idea of atelegraph, and he was so carriedaway that he abandoned his art,which had provided him with ahandsome income, and devoted allhis time and energy, with a greatpersonal and economic sacrifice, tothe development of the telegraph.The result was a method oftransferring text based on strong andweak “taps.” (Today, aftersubsequent modifications, they arecalled “dashes” and “dots,”respectively.) These taps wereproduced using a key, today named

after Morse, to close an electriccircuit which includes this key aswell as the receiver, a source ofelectric power (such as a battery)and an electric wire of suitablelength connecting the key to thereceiver. The main principle behindthe receiver is the existence of asmall electromagnet that produces aweak or strong tap, depending onthe strength of the signal receivedover the wire. Most importantly,Morse developed a code, alsonamed after him, which convertedthe entire alphabet – that is, letters,numbers and punctuation marks – tocombinations of the binary

characters – that is, “dots” and“dashes.” With such a system, it ispossible to transmit text in anylanguage, each language having itsown Morse code. Expert operatorscan listen to the tapping andunderstand the sequences as if itwere human language.

In 1836, the year in which hedemonstrated his telegraph tocolleagues at New York University,Morse also ran in (and lost) in theelections for mayor of New York.His repeated requests to thegovernments of the United States,France and Britain to allow him toestablish telegraph lines were

rejected. Only in 1843 did hefinally succeed and, after mucheffort, received a budget of $30,000to establish the first telegraph linebetween Capitol Hill inWashington, D.C., and Baltimore,Maryland; the rest, as they say, ishistory.

During the nearly one hundredyears that passed between theinvention of the telegraph and thearrival of Claude Shannon, the firstperson to quantify the entropyassociated with the transfer ofcontent, telegraph went throughsignificant developments – mainlydue to its enormous military,

economic and social value. Overthe years, the wired telegraph wasjoined by a wireless telegraph (justas today phones can be eitherlandline or mobile) and a devicewas added to register the receivedsignals (as dots and dashes) – aforerunner of today’s fax. It is worthnoting that Lord Kelvin – a scientistof great achievements and no littlearrogance, after whom, we mayrecall, the absolute temperaturescale is now named, and whoenthusiastically claimed that noobject heavier than air will ever fly(said about ten years before theAmericans Wright brothers flew,

but quite a few millions of yearsafter the first uneducated, heavier-than-air, creatures were gentlygliding through the skies) – wasmade Peer of the Realm not for hisachievements in pure science, butfor his contribution to the laying ofa submarine telegraph cablebetween England and France in1857. To this day, the telegraph isthought to be one of the most usefuland successful inventions ever, andeven though the computer hasreplaced it in general use, radiobuffs still enjoy communicatingamong themselves using the Morsecode.

Two significant technologicaldevelopments allowed the presenttremendous progress in the world ofcommunication. First, with theinvention of computers, was theability to process data quickly;second, but no less important, wasthe ability to transmit and receiveshort pulses with extreme accuracyof duration. Between them, theyrendered the Morse code outdatedand led to the replacement of itsdots and dashes with the “zeroes”and “ones” of binary code. It is nowonder that at Bell Labs (the highlyacclaimed research laboratories ofthe great telephony company, AT&T

in New Jersey), engineers beganinvestigating more efficient ways totransfer information overcommunication channels.

To understand the essence ofcommunication, let us examine thefollowing example:

Bob promises Alice that if hewill be free to meet her for dinner,he will call her by seven o’clockp.m. If he does not call, it is a signthat he won’t be free. As it turnsout, Bob could not come, so he didnot call. The message had been“delivered,” and Alice, who knewnow that she should not expect Bobto meet her, went for a cards game

with her friends.It appears that Bob, by not taking

any action, nevertheless conveyedinformation to Alice. However, thisis not so simple, as it is obviousthat every time Bob does not callAlice does not mean that he is busyin the evening. A better explanationis that information is adaptive. Inother words, it depends on thecircumstances. The reason that theabsence of a call before seveno’clock could be consideredinformation was that Bobpreviously gave Alice specificinstructions: she should expectanother message from him before

seven. If he sends another message,it is a sign that he is free to come, ifhe does not, he is not. In otherwords, Bob laid the groundworkand both he and Alice agreed that acall is a positive message and theabsence of a call is a negativemessage.

In a way, this is the essence ofcommunication and information. Interms of computer communication,Bob sent Alice a message of onebit: “1” = Bob is coming, “0” =Bob is not.

It was Claude Shannon whocoined the term “bit” (aportmanteau of “binary” and

“digit”) and defined communicationin terms of calculable probabilisticquantities. In order to calculate theamount of information, he had todefine its elusive meaning. Hedefined information as the numberof bits, “0” or “1,” that can betransferred over a channel, that is,as a series of choices between“yes” and “no” (“1” and “0”),expressed in binary code. Eachsuch possible choice is called a bit.He defined the amount ofinformation that can be transferredover a communication channel asthe uncertainty within it, or, as thesum total of the different possible

choices. The logic behind hisdefinition is this: the higher theuncertainty in a communicationchannel, the higher the content thatcan be transferred.

To illustrate Shannon’sdefinition, let us look at anotherexample: Let us say that Bob wantsto send Alice his telephone number,which consists of a three-digit areacode followed by seven digits.Altogether, Bob wants to sendAlice a number made of ten decimaldigits. It is clear that at the end ofhis transmission, Alice will havereceived one specific number out of9,999,999,999 (approximately

1010) possible numbers, assumingthat each one of these numbers hasan equal probability. Any one ofthose 1010 numbers can be viewedas a microstate. Thus, as we saw inthe discussion of Boltzmann’sentropy, the entropy of Bob’smessage to Alice will be thelogarithm of the number ofmicrostates. Therefore, the entropy(or as Shannon called it at first, theuncertainty) that is associated withBob’s telephone number (whichAlice does not know) is N = ln1010

= 10 ln10.What will be the number of

microstates if Bob transfers thatten-digit number using binary code(that is, sends it in bits)? Since eachbit can only be “1” or “0,” we shallneed to solve the equation 2N =1010 (since each additional bitdoubles the number of microstates).Therefore, N = . The value

N, which was obtained from the logof the number of possibilities, is thenumber of bits required to transferthe phone number, and thus it givesShannon’s entropy for a ten-digitdecimal number. (Actually,engineers use the logarithm of base2, since in this particular case the

number of bits is exactly equal toShannon’s entropy, as we shall seelater on.) In general, Shannonshowed that the number of bitsrequired to transfer a file with agiven content is equal to its entropy.Shannon’s definition for the numberof possible choices on acommunication channel, which iscalled either “Shannon entropy” or“Shannon information,” differs fromthe concept of “information” whichis used in everyday language andwhich in this book, as mentionedearlier, is called “content.

Claude ElwoodShannon

Claude Elwood Shannon, 1916-2001

Claude Shannon was born in 1916in Petoskey, Michigan. His fatherwas a businessman and his mother ateacher. The first sixteen years ofhis life were spent in the town ofGaylord, Michigan. While still achild, he already showed a talentfor mathematics and mechanics, andbuilt model airplanes, some of themradio-controlled. He once built amodel telegraph that connected himto a friend’s house more than half amile away. His hero was ThomasA. Edison, who, as he found outlater in life, was actually a distantrelative. Like other American

culture heroes, in his youth Shannonworked as a paper delivery boy andrepaired radios at the localdepartment store. (Perhaps being adelivery boy is a necessarycondition for becoming great inAmerica!)

In 1932 he began studyingelectrical engineering andmathematics at the University ofMichigan. He received his BSc in1936, and then began working as aresearch assistant in the Departmentof Electrical Engineering at theMassachusetts Institute ofTechnology (MIT), at the same timepursuing graduate studies. His

master’s thesis demonstrated howBoolean algebra could be used toanalyze and synthesize relayswitching circuits and computers.His first published paper on thissubject (1938) aroused greatinterest. One should bear in mindthat the first digital computersappeared only in the mid-1940s. In1940 he received the Alfred NobleAmerican Institute of AmericanEngineers Award – an annual prizegranted to a person under thirty fora paper published in one of themember companies’ journals. (Thisis not the well-known Nobel prize,which he was never to win). A

quarter of a century later, computerhistorian H. H. Goldstine describedthis work as follows:

This surely must be one of the mostimportant master’s theses everwritten... The paper was alandmark in that it helped tochange digital circuit design froman art to a science.62

In 1938 Shannon moved to theDepartment of Mathematics at MIT,and while working as a teachingassistant there, he began writing adoctoral thesis entitled An Algebrafor Theoretical Genetics, in which

he used algebra to organize geneticknowledge, similar to his previouswork with switches.

At that time, Shannon began tocontemplate various ideasconcerning computers andcommunication systems. By thespring of 1940, he had fulfilled allthe formal requirements for hismaster’s degree, other than foreignlanguage – his Achilles’ heel.Desperate, he hired private tutors inFrench and German. Finally (afterfailing German once), he passed hisexams and in April 1940 wasawarded his MSc in electricalengineering for his paper on

Boolean algebra. Immediately afterthat he received his PhD for hisdissertation on the use of algebra ingenetics.

In 1940 and 1941 he worked as aresearch fellow at the Institute forAdvanced Study at Princeton, NewJersey, under the direction ofHermann Weyl, a Germanmathematician and a renownedexpert on symmetry, logic andnumbers theory. There Shannonbegan seriously to develop hisideas about information theory andefficient communication systems.

In 1941 Shannon joined AT&T’sBell Labs in New Jersey, and

worked there for 15 years indiverse areas, most famously in“information theory,” a field whichhe founded by publishing hisfamous paper, A MathematicalTheory of Communication, in1948.63 This study is widelyregarded as Shannon’s greatestcontribution to science, since itfundamentally changed all aspectsof communications, in theory aswell as in practice. Shannon themathematician was well-versed instatistical mechanics, and in hispaper he demonstrated that allsources of information – telegraphkeys, personal conversations,

television cameras, and so on –have a “source rate” measured inbits per second, and that all thevarious communication channelshave a capacity that again can bemeasured in the same units. It ispossible to transfer informationover a channel only if the sourcerate does not exceed the channel’scapacity, said Shannon. Hecalculated the maximum informationthat can be transferred over achannel if it can transfer N pulsesper second. Shannon believed thathis theory could also be applied tobiological systems, since theprinciples that operate mechanical

and living systems are similar.Shannon’s work laid the

theoretical foundation for a highlyimportant field, file compression,which will be described below.However, it should be noted thatwhen Shannon published his ideas,the vacuum-tube circuits whichwere then used in computers couldnot run the complicated codesneeded for efficient filecompression. The theoretical limitthat a file can be compressed to iscalled the Shannon limit. As weshall show below, the Shannon limitcorresponds to the thermodynamicequilibrium of the files undergoing

ideal transmission over a channelwithout noise.

It was only in the early 1970s,with the development of fasterintegrated circuits, that engineerscould reap the benefits of hiscontributions. Today, Shannon’sinsight is part of the design ofvirtually all storage systems thatdigitally process and transferinformation, from flash memorieson through computer and telephonecommunication, to space vehicles.

In 1949 Shannon married MarieElizabeth Moore, a computer expertwho was a pioneer in thedevelopment of a computerized

loom in the 1960s. The couple hadthree children. Shannon and hiswife enjoyed spending theoccasional weekend in Las Vegaswith Ed Thorp, an MITmathematician, and John Kelly, Jr.,a physicist who worked at BellLabs. These three developed agambling method combininginformation theory with game theoryand won vast sums at roulette andblackjack tables. Shannon andThorp also applied their theory,later called the Kelly criterion, tothe stock market, with even morebeneficial results.

Besides his interest in gadgets,

Shannon loved music and poetryand played the clarinet. Heoccasionally wrote ditties, joggedregularly and enjoyed juggling. OneChristmas, his wife, aware of hispeculiar interests, gave him aunicycle. Within a few days,Shannon was riding it around thehouse; a few weeks later he wasjuggling three balls astride it. A fewmonths later he had alreadyinvented a special unicycle wherethe rider bobs up and down whilepedaling forward. He also designeda gadget for solving Rubik’s Cube.

In 1956 Shannon was invited as avisiting professor to MIT, and from

1957 to 1958 he held a fellowshipat the Center for the Study ofBehavioral Sciences in Palo Alto,California. One year later hebecame a Donner Professor ofScience at MIT, continuing hisresearch there in various areas ofcommunication theory. He retainedhis affiliation with Bell Labs until1972.

Shannon’s papers were translatedinto many languages, but mostthorough in this endeavor were hisRussian translators, who haddisplayed great interest in his workfrom early on. In 1963 he waspresented with three 830-page

copies of a collection of hisscientific writings in Russian. Yearslater, while on a visit to the SovietUnion, he learned that he hadearned some thousands of rubles inroyalties from the sale of his books,but much to his regret, he could nottake the money out of the country.With no choice but to spend themoney there, he bought a collectionof eight musical instruments.

At some point, Shannon began tosense that the communicationrevolution has gone too far. Hewrote: “It has perhaps beenballooned to an importance beyondits actual accomplishments.”64 His

personal feelings notwithstanding,the digital communicationrevolution was as important as theindustrial revolution two hundredyears earlier. Obviously, the laborsof many people are involved in anyrevolution. But if the landmarks ofthe industrial revolution were theimprovement of the steam engine byJames Watt and the development ofthermodynamics by Carnot,Clausius and others, then theinformation revolution can beattributed to the development of thethermodynamic theory ofinformation by Shannon, as well asthe inventions of microprocessors

and integrated circuits by others. Ifthe flow of information in copperwires or optical fibers can becompared to the flow of liquids in apipe, then Shannon demonstratedhow the flow of information can bequantified in a manner similar tomeasuring the flow of a liquid.What is amazing is that such a basicquantity in nature took so long to bedefined. And perhaps no lessamazing is the fact that the NobelPrize Committee failed toappreciate the import of Shannon’sdiscovery. The quantified definitionof information is an importantcornerstone in information

technology and computer science,and has changed not only thetechnical aspects of IT (InformationTechnology), but also, for better orfor worse, the way in which wethink about information – or, morespecifically, the way we think abouteverything.

Claude Shannon died in 2001 atthe age of 84 after a long strugglewith Alzheimer’s disease. In hisfinal days, he was not even awareof the wonders of the digitalrevolution to which he hadcontributed so much.

In order to understand Shannon’sentropy, let us return to Bob andAlice and assume that they have acommunication channel that iscapable of transferring one pulse byseven p. m. What Shannon did wasto attempt to quantify the amount ofinformation that Bob transfers toAlice. At first blush, it seems thatthe amount of information is the sumof probabilities. In other words, ifthe probability that “Bob is busy”(0) is p, then the probability that“Bob is free” (1), is 1 – p, meaningthat the total probability is one unit(one bit), no matter what the valueof p actually is.

But Shannon gave a differentsolution, namely:

I = –[p ln p + (1 – p) ln (1 –p)],

That is to say, the amount ofinformation, I, is a function of theprobability of the bit being “0”times the log of this probability,plus the probability of the bit being“1”, times the log of thisprobability. For this solution,Shannon deserved a place in thepantheon of great scientists.

First we have to understand whyI = p+(1 – p), that is, one bit, is not

a good answer. It is possible that onthe day Bob made his arrangementwith Alice, he had already plannedanother meeting and that the onewith Alice (who was aware of this)was a secondary option. If wesuppose that the chance for thisoriginal meeting to be cancelled, sothat Bob would meet with Alice, isten percent, then the bit with a valueof “0” carries less information toAlice (0.1 bits), while the “1” bitcarries more information (0.9 bits).If we add these probabilities, theanswer is always “1” regardless ofthe relative probabilities (as knownto Alice) of the bit being “0” or

“1.” If Alice had not been awarethat the probability was 90:10, shemight have assumed that theprobability was still 50:50, and thebit with the “0” value would carrythe same value of information as the“1”. Therefore adding probabilitiesoverlooks the a priori knowledgethat is so important incommunications.

In the expression that Shannongave, on the other hand – and hewas aware of the fact that thisexpression was the same as that ofthe Gibbs entropy – summing up theprobabilities does not disregard the90% chance of the bit being “0” and

the 10% chance of it being “1” (orany other value that Alice wouldassign to her chances of meetingwith Bob).

If Alice evaluates the probabilitythat Bob will be free that evening as50%, the amount of information thatBob sends her is at a maximum,because Alice will be equallysurprised by either value of the bit.Formally, in this case bothpossibilities have the sameuncertainty, and the information thatthe bit carries is:

That is, the maximum amount ofinformation that a bit can carry isequal to one half the natural log of 2(50% chance that Bob will come),plus one half of the natural log of 2(50% chance Bob will not).* Sinceengineers prefer to calculate base 2logarithms rather than use the“natural” base, e, the maximumamount of information that a bitcarries is 1.

If the probabilities that Bob willor will not show up are not equal,the bit that he sends to Alice carriesless information than one unit; ifthey are equal, the information

carried in the bit is equal to oneunit. If Alice knows that the chanceof seeing Bob is just 10%, theamount of entropy according toShannon will be:

That is, the bit carries informationthat is just 0.32/ln2 = 0.46 of themaximum value.

Shannon’s definition is superiorto the one that is independent of theprobability. If we send N bits, themaximum information is N bits. Thereason for this is that Shannon’s

expression acquires its maximumvalue of one for each and every bitwhen “0” and “1” have equalprobabilities; but when theirprobabilities are not equal,Shannon’s information is less thanone. And this was Shannon’s greatinsight: for an N-bit file, there is achance that its content could betransferred via digitalcommunication using less than Nbits. In principle, then, it is possibleto compress files without losingcontent, and of course, this is veryimportant for the speed, cost, andease of transferring information.

Can Shannon’sInformation Be Regarded

as Entropy?

Before Shannon published hisseminal work in 1949, he ponderedthe question of an appropriate namefor the quantity he was defining. Inan interview he gave to ScientificAmerican, Shannon related thatJohn von Neumann had advised himto call the quantity “entropy,” sincethe concept of entropy was vagueenough to give him an advantage indebates. As we shall see

immediately, von Neumann – one ofthe most brilliant mathematicianswho ever lived – gave Shannonsome very sound advice.

This advice notwithstanding,Shannon’s theory is today called“information theory,” althoughShannon himself actually called hispaper A Mathematical Theory ofCommunication [emphasis added].There is nothing wrong with callingShannon’s theory “informationtheory,” since it does concern thetransfer of information, but it shouldbe borne in mind that Shannon’s“information” is actually entropy.

In previous chapters we saw thatentropy is a calculated value basedon the number of microstates. Atany given moment, every statisticalsystem, even a dynamic one, is inone and only one microstate. This isespecially true for a data file,whose entire epistemologicalmeaning is a single microstate,which is its content!

The confusion betweenShannon’s entropy and information– in the sense of content – isderived from our tendency toconsider a stored computer file anda transmitted file as one and thesame. In other words, for us,

“information” usually refers to thecontent that we have received,while Shannon treated informationas the uncertainty that accompaniesthe transmission of the content.

If a file is not transmitted fromthe computer’s memory, the issue isjust academic. Suppose we have inour computer memory a file with Nbits and H Shannon information.The file has content, exactly in thesame way as gas molecules in avessel are at every instant in asingle microstate (although, incontrast to gas, the microstate of thefile in our computer is static, this isimmaterial for our present

discussion, which focuses on therebeing just a single microstate at anygiven time.)

What is the content of the file?There are three possibilities: (a) thecontent is unknown to us, and wewant to know it; (b) we alreadyknow it; and (c), we do not knowbut we do not care either. If no onereads the file, we can leave thequestion of the content and itssignificance to the epistemologists.On the other hand, if someone doesread the file, the problem becomesa thermodynamic one. In fact, inorder to read a text, our eyes mustscan the letters, whose forms are

reflected by light onto our retinasand then interpreted by our brain.The transmission of content fromthe page to the eye involves energyexpenditure, and is thereforesubject to the laws ofthermodynamics. All the more sowith regard to a file in a computer’smemory, when it is transmitted tosome device that will change it tocharacters (or sounds, or pictures)through any number of processes.

It starts with the computer’smagnetic head reading the bits ofthe file. If the value is “1”, it sendsout a pulse (of electric current orelectromagnetic radiation) that is

physically an oscillator. Normally,the electromagnetic or electricalvibration undergoes amplification.This amplification of a classicaloscillator is comparable to theCarnot heat engine as we saw inChapter Three, and the temperatureof a classical oscillator is itsenergy divided by the Boltzmannconstant. Thus, according to thesecond law of thermodynamics, thetransmission of a file is essentiallya flow of energy from the hottransmitter to the cold receiver.Since the transfer of energyinvolved in the transmission of thefile is subject to the second law,

and since Shannon’s entropy, aswill be shown, is thermodynamicentropy, it becomes evident that thetransmission of the file increasesentropy. Since entropy isproportional to the number ofcontents (microstates), it means thatthe number of contents tend toincreases.

Now is the time to demonstratehow physical entropy, as describedabove, is connected to information,as Shannon defined it. Physicalentropy, it will be recalled, hascertain properties andcharacteristics: the flow of energyfrom hot to cold, uncertainty,

extensivity, and a tendency toincrease (as expressed byClausius’s inequality). And sinceShannon’s information possessesthese very properties, this is enoughto justify calling it “entropy.”

Energy flow: In Shannoninformation, energetic bits aretransferred from a “hot” transmitterto a “cold” receiver, similar to theway Clausius entropy is generatedby transfer of energy from a hotterobject to a colder one.

Uncertainty: When Alice iswaiting for Bob to send his bit ofinformation, she has no idea of itsvalue. If she knew the value (“0” or

“1”), Bob would not have to bothersending it. Therefore, the essence ofthe bit is the uncertainty inherent init. Information that passes through achannel is the log of the number ofpossible contents – similar to thenumber of possible microstates. (Asmentioned above, Shannon wasaware that his expression was thesame as the Gibbs entropy.)

Extensivity: Given two files, Aand B, the amount of informationtransferred if both files aretransmitted will be the sum of theinformation in files A and B addedtogether. This is analogous to theadditive property of both the

Boltzmann and the Gibbs entropies.Clausius’s inequality: As

mentioned already, Shannon’sexpression for information wasidentical to the Gibbs entropy sincehe assumes that the probabilities ofall possible contents are notnecessarily uniform. Yet in a filewhere all probabilities for thecontents are the same, the amount ofentropy, I, is maximum. If wedenote as H the value obtained fromShannon’s expression when theprobabilities are not uniform, wecan write that H ≤ I. This isidentical with Clausius’s inequality.

Thus, entropy as defined by

Shannon is equivalent to entropy asdefined by Clausius, Boltzmann andGibbs.

If we have a file (a series of“1’s” and “0’s”) N bits in length,and assume that each bit carriesmaximum entropy, that is ln2 nat,the file will carry I = N ln2 nat. Thenumber of possible ways that N bitscan be arranged, where each bit hasan equal probability of being either“1” or “0”, is Ω = 2N. This showsthat Shannon’s expression is noneother than the logarithm for thenumber of different possibilities forarranging a file, Ω, or in otherwords, the number of possible files

(contents), that is, I = lnΩ. Thisexpression is identical to theBoltzmann entropy, except that itlacks the Boltzmann constant.

The Boltzmann constant does notappear in the Shannon informationbecause Shannon referred to bits aslogical conceptual quantities. Froma historical thermodynamicperspective, this is a pity. Had heconsidered the “1” bit to be aclassical energetic oscillator (thatis, a classical electrical oscillator)whose entropy, as we recall, is k,then information as he defined it,and entropy as Gibbs defined it,would have been identical. As we

saw in the previous chapter, theentropy of a classical oscillator isone Boltzmann constant and isindependent of its energy, in thesame way that Shannon’s entropy isindependent of the energy of a “1”bit.

Accordingly, there are threeanalogues and one identity betweenShannon entropy and the statisticalentropy of thermodynamics:

An analogue between the numberof different contents and the numberof microstates;

an analogue between a state inwhich there are unequal

probabilities for the “0” and “1”bits, and between the Gibbs entropyand Clausius’s inequality;

an analogue between equalprobabilities of the bits being “0”and “1,” and between the equalprobabilities of microstates inBoltzmann’s entropy;

and an identity regardingextensivity for both the Boltzmannand Gibbs entropies and theShannon’s entropy.

To review: “Shannon’sinformation,” as it is defined, is theGibbs entropy without theBoltzmann constant. The Boltzmannconstant must be added when the

information-carrying bits areoscillators, such as classicalelectromagnetic pulses, say, whereeach has an entropy of oneBoltzmann constant. That is, theShannon information is thelogarithm of the number of possiblecontents of a file, which expressesthe uncertainty of its content. Thegreater the uncertainty of a file, thegreater the amount of content. Thus,one may extrapolate and say that thesecond law of thermodynamicstends to increase the amount ofcontent. This point will beexplored in more detail in chapterfive.

Equilibrium and FileCompression

The concept of equilibrium isintegral to the definition of entropy,but we have yet to deal with it withrespect to Shannon’s entropy. Nowwe shall demonstrate that a state ofthermodynamic equilibrium alsoexists in communications.

One way to formulate of thesecond law of thermodynamics is tostate that every system tendstowards equilibrium. The reasonfor this is that in equilibrium, the

value for entropy is at a maximum.Therefore, with respect toinformation, one can expressequilibrium as a state in whichevery one of the possible differentcontents has equal probability. Italso means that each bit has anequal probability of being either “0or 1” – as if each one of the bits istaking part in some sort of unbiasedcoin tossing for its value. (Ofcourse, there is a chance, albeitminiscule, that an N-bit file inequilibrium – even though createdusing a fair, 50:50 coin tossing –may be, for instance, comprisedonly of “0”s.)

In a file in which theprobabilities for a bit to be “0” or“1” are not equal, the uncertaintyvalue is smaller. Therefore, eventhough the length of such a file maybe equal to that of another one withequal probabilities, it may haveless content. This leads to theconclusion that a file in equilibriumis the shortest possible file that canbe sent for any given content. Foreconomic and engineering reasons,engineers are obviously interestedin sending the shortest filenecessary to carry a given content.For this purpose they run somecompression software, which

compress the file before itstransmission or storage. Forexample, if a file contains onemillion “1” bits, a compressed filewill contain some short formulawhose meaning is “a million 1’s”rather than a string of one million“1’s”. Such a compression changesthe probability of any bit to beeither “1” or “0” to about 50%.Thus, an ideal compressionmaximizes the uncertainty of a file,making it reach thermodynamicequilibrium – that is, the shortestpossible file that can carry thegiven amount of content. A file thatwas compressed to its maximum is

said to have reached the ShannonLimit, that is to say, thermodynamicequilibrium.

Usually, communication channelstransfer not only content, but also“noise.” For example, inelectromagnetic broadcasting, thetransmission channels are exposedto electromagnetic radiation frommany sources, e.g. black bodyradiation emitted from objects intheir environment, radiation atvarious frequencies from all sortsof devices in use in thesurroundings, cosmic radiation, andradioactive radiation. Sinceelectromagnetic channels are

affected by this random radiation,the content that Bob sends to Alicemight get corrupted: the radiationmay alter the values of some of thebits in the file, and Alice mayreceive a file different from the onethat Bob sent. In fact, sometimes thefile that Alice receives may losevirtually all its meaning because ofthis extraneous noise.Thermodynamically speaking, overa noisy channel, Alice will receivea microstate different from the onethat Bob sent. Nevertheless, thisnoise actually increases the amountof information in the file (accordingto Shannon’s definition). This

paradox can be understood if weremember that even though thesubjective content has beenreduced, the uncertainty, i.e. theentropy – the Shannon information –actually increased. This processoccurs spontaneously and reflectsthe tendency of systems to increasea file’s entropy according to thesecond law of thermodynamics. InAppendix B-1, we shall discuss thismore extensively.

Indeed, increasing entropy isoften done purposefully in anattempt to correct errors due torandom noise. To protect theintegrity of the content that Alice is

to receive, specific content – calledcontrol – is usually added to thetransmitted files explicitly forchecking whether any errors wereintroduced during transmission and,if there were any, to correct them.

The Distribution of Digits– Benford’s Law

The effects of the second law ofthermodynamics occur in each andevery spontaneous process innature; however, sometimes theseeffects are overlooked. When we

consider why water flows from ahigh place to a lower one, or why apendulum gradually slows itsmotion until it stops, we think of theforce of gravitation for the formercase and friction for the latter one.Few will explain these phenomenain terms of the second law ofthermodynamics, since our naturaltendency is to give explanationsbased on deterministic forces,overlooking the inherentirreversibility and uncertainty ofnature. But when we discussdistributions, the forces-basedexplanations become extremelycomplicated, if not impossible. We

saw how the second law ofthermodynamics explains thedistribution of the velocities of gasparticles – a cornerstone of gastheory, which contributed to theacceptance of atomic hypothesis.Similarly, we saw how the secondlaw explains the frequencydistribution of a black body’selectromagnetic radiation of a giventemperature – an explanation thatled to the discovery of the quantumnature of radiation. Does Shannon’sentropy yield similar logicaldistributions?

Indeed, what is a logical

distribution, and how is itexpressed? As we are going to see,logical thermodynamic distributionsaffect our lives no less thanphysical distributions.

We start with the equilibriumdistributions of signals in files. Abinary file in equilibrium iscompressed. That is, if we sample alarge number of files, we shall seethat the number of “1” bits is equalto the number of “0” bits in them.There is no need to be familiar withthermodynamics to understand this.However, the distribution ofdecimal digits in random numbers,contrary to intuition, is not uniform.

Experience has shown that high-value digits will appear fewertimes than low-value digits. Thisphenomenon, which has beenobserved in numerous random files,is called Benford’s Law. It hadraised many problems for themathematicians, until it becameclear that this is a directconsequence of maximizingShannon’s entropy. In order tounderstand Benford’s law, let uscalculate the ratios among of thefrequencies of digits in a decimalfile in Shannon limit.

Digits (and the numberscomposed of them) are different

from letters (and the words made ofthem). The logical meaning of anumber is clear and well defined.Whereas combinations of wordsmay excite or annoy or bore, or, inmost cases, lack any meaning,combinations of digits are always anumber with some significance. Therelationship between digits andnumbers is defined by the laws ofarithmetic, as opposed, of course,to letters and words.

It is possible to simulate a digit,n, as a box containing n balls. Thus,the ten decimal digits are analogousto ten boxes, in which the number ofballs they contain is equal to their

value, as shown in the followingtable:

A possible way of coding the

boxes-and-balls model for thetransmission of a decimal digit is toform a sequence of 10 bits for eachdigit (as shown above), such thatthe number of “1” bits in eachsequence is equal to n. Thedistribution of the n “1” bits in thesequence has no significance. Forexample, a number with 20 digitswill be coded by twentyconsecutive sequences of ten bitseach, and the number of “1” bits ineach sequence will be equal to thevalue of corresponding digit in thenumber.

To restore the number that Bobtransmitted, Alice has to split the

binary file she received into ten-bitsequences. Now she should countthe number of “1” bits in eachsequence and write down theappropriate decimal digit; finally,she has a sequence of digits thatmatches the number that Bob senther. What will the distribution ofthe digits in the sequences be? If weimagine each “1” bit as a ball, andeach sequence of ten bits as a box,the calculation of the distribution of“1” bits in the sequences will beidentical with calculating thedistribution of balls in each boxsuch that Shannon’s entropy ismaximized. Calculating the

distribution of P balls in N boxes issomewhat similar to thecalculations that Planck did whenhe figured the distribution ofphotons in the radiation modes of ablack body. Obviously there aredifferences: for example, with adecimal file, there cannot be morethan nine balls in a box. Using adifferent method of counting, forexample the hexadecimal system(based on 16), the number of ballscannot exceed fifteen. However, thenumber of photons which can befound in a radiation mode can be aslarge as P. In Appendix B-2, thestandard method of Lagrange

multipliers is used to calculate thedistribution of digits that will yieldthe maximum number ofmicrostates, and the obtainedrelative frequency of the digit n,ρ(n), is given by the function:

This expression deserves someattention. Its significance is that itshows that in a random decimalfile, the relative normalizedfrequency of the digits is given by afunction that is the logarithm of theratio between a digit plus one andthe digit itself. Now it is possible to

calculate the frequency distributionof the various digits in a file ofrandom data, as shown in Fig 6.

Figure 6: Benford’s law – the relativefrequency of a digit in a file of randomnumbers in not uniform. The

frequency of the digit “1” is 6.5 timesgreater than that of the digit “9”.

These results arecounterintuitive, since we naturallyexpect a uniform distribution ofdigits. Yet the “1” digit appears 6.5times more frequently than the “9”digit! The reason is that inequilibrium, each microstate has anequal probability, and thus everyball will have an equal probabilityof “drawing” any box. That is, inorder to divide fairly P balls into Nboxes, P drawings for N boxes mustbe performed. At each drawing, onebox will “win” one ball. Obviously,

the probability that in the end of ourgame one particular box will havewon many balls is lower than theprobability that it will have wononly a few balls. Put simply, it iseasier to win one ball than to winnine balls. Life is tough!

Does this imply that the secondlaw of thermodynamics is alsoapplicable to mathematics?Surprisingly, this very distribution –with the very same mathematicalexpression – was discoveredempirically in 1881 by Canadian-American astronomer SimonNewcomb while he was looking ata booklet of logarithmic tables.

(Before the era of the calculator,these tables were used formultiplication, division, finding theexponents of numbers, etc.)Newcomb noticed that the pagesbeginning with the digit “1” weremuch more tattered than the others.He concluded that if one randomlychose a digit from any string ofnumbers, the chance of finding the“1” digit is greater than finding anyother digit. He examined the chartsbeginning with the digit “2” and soon, and came up with – as aconjecture – the very same formulathat is derived in Appendix B-2.

In 1931, an employee of the

General Electric Company,physicist and electrical engineerFrank Benford, demonstrated thatNewcomb’s Law applied not onlyto logarithmic tables, but also to theatomic weights of thousands ofchemical compounds, as well as tohouse numbers, lengths of rivers,and even the heat capacity ofmaterials. Because Benford showedthat this distribution is of a generalnature, it is called “Benford’sLaw.” Since then, researchers haveshown that this law also applies toprime numbers, company balancebooks, stock indices, and numerousother instances.

It should be noted, however, thatif we were to allocate P ballsamong N boxes not by “drawing thelucky box” one ball at a time, butrather by using, for example, a topwith ten faces bearing the digits “0”to “9”, and allowing each spin ofthe top to determine one digit in anon-biased way, the distribution ofthe digits among the boxes will beuniform. Why? It is clear that thedigit symbol marked on any face ofthe top does not affect theprobability of it falling on thatparticular side. This is why thevarious lottery games use suchtechniques for drawing numbers as

give a uniform distribution, so thatgamblers familiar with Benford’slaw will not have an advantage.

At first glance, Benford’sdistribution seems counterintuitive,because we tend to regard digits assymbols rather than physicalentities. The fact that most peopleare not aware of Benford’sdistribution is used by taxauthorities in some countries todetect tax fraud by companies orindividuals. If the distribution ofdigits in a balance sheet does notmatch Benford’s law, the taxauthorities suspect that the numbershave been fixed, and may launch a

more thorough inspection.To illustrate briefly the ideas

presented more fully in AppendixB-2, let us examine a model of threeballs and three boxes. The numberof possible microstates (orconfigurations), according toPlanck, is ten, as follows:

In equilibrium, eachconfiguration has an equalprobability. Thus we count thenumber of occurrences for eachdigit in all configurations. “1”occurs nine times, “2” occurs six

times, and “3” only three. This isthe distribution expected from thecalculations in Appendix B-3 forthe more general case (Planck-Benford), as applied to our case.

We therefore conclude that:

The distribution of digits in adecimal file that is compressed toShannon’s limit obeys Benford’slaw;

Benford’s law is the result of thesecond law of thermodynamics.

Mathematical operations have aphysical significance.

In the next chapter we shall see

that this distribution, whenexpanded from decimal coding toany other numerical coding (as wehave just seen in the case of threeballs and three boxes), can explainmany phenomena commonly presentin human existence and culture, suchas social networks, distribution ofwealth (Pareto’s principle), or theresults of surveys and polls.

Chapter 5

Entropy in SocialSystems

The Distribution of Linksin Social Networks –

Zipf’s Law

So far, we have reviewed theeffects of entropy on the flow anddistribution of energy, thetransmission of information, and thedistribution of digits in data files.During this journey, we came torealize that there is a significantdifference between entropy andmost the other physical quantities innature.

Looking at quantities such as

mass, time, length, electricalcurrent, electrical charge,temperature, energy, speed, lightintensity, and so forth, we noticethat they all describe “something”measurable: each one (and manyothers) can be measured by someinstrument or another. For instance,the instruments that measure thequantities listed above are,respectively, scales, clock, ruler,amperemeter, electrometer,thermometer, calorimeter,speedometer and photometer. Butthere is no “entropymeter.” The factthat entropy cannot be measureddirectly is of great help for

scientists in the field: they can writeabout entropy almost anything thatcomes to mind. At the same time, itdemonstrates the fact that entropy isdifferent from all these otherphysical quantities.

As we saw, entropy is calculatedfor equilibrium. When we observegas in a vessel or read a transmittedfile, we do not learn from theamount of entropy any of thesystem’s details. On the contrary,entropy is a measure of our lack ofknowledge about it. The uncertainty(lack of knowledge) about thesystem’s configuration is the totalnumber of its possible microstates,

when we do not know in whichparticular microstate it is at a givenmoment — even though, at anygiven moment, the system must be inone and only one of these possiblemicrostates. When we say that thesystem is in equilibrium, it meansthat all the possible microstates ofthe system have equal probability.Therefore, it is arguable thatentropy is a property of a system,and not a physical quantity like theones mentioned above.

The second law ofthermodynamics states that everysystem tends to change in order toincrease its entropy (specifically,

the logarithm of the number ofmicrostates), that is to say, it willtend to reach equilibrium, in whichthe number of microstates is at amaximum. The number ofmicrostates in logical entropy is theamount of content possible, andtherefore, as the number ofmicrostates increases, so will theamount of content. Hence, theoperation of second law ofthermodynamics tends to increasecontent.

However, we should not confusecontent and entropy: entropy is theuncertainty that accompaniescontent. If we transmit a file with N

bits M times, we shall increase theentropy M times – exactly the sameas it if we had sent M different filesof N bits each. Nature does not careabout the content, but about theuncertainty accompanying it. Thereis an old story about a man whocrossed the U.S.-Mexican borderevery day with a bale of hay on hisbicycle. The suspicious customsofficers rummaged through thosebales to no avail, and it took a longtime before it dawned on them thathe made his living by smugglingbicycles. The content that weconsume is, in a matter of speaking,the straw through which we

rummage with passionate curiosity,and the bicycles are the entropy inwhich nature is interested.

Is life itself a result of entropy’stendency to increase? Is the human“will” an expression of entropy’s“will” to increase? It seems that theanswers to both questions are“Yes”. Changes in which entropy isincreased are spontaneous andirreversible, just like biologicalactivity and its accompanyinghuman activity. Imagine ahypothetical observer stationed inspace who has been looking downon earth for many millions of years.At the start he would have seen

creatures appearing, plants andanimals multiplying, and, in duecourse, also roads being paved,skyscrapers rising and electricallights flashing; and flying throughthe air are not only birds, but alsoairplanes. Will this observer givecredit to human ingenuity for someof these developments? It is morelikely for it to think that it has allappeared spontaneously, accordingto the laws of nature. Humans arepart of nature, therefore the human“will” is also part of nature, andhence, human actions and desiresshould also obey the second law ofthermodynamics.

It seems that the basiccharacteristic of life is thereplication of information. Britishbiologist Richard Dawkins65

claims that the purpose of naturalselection is not necessarily theimprovement of a species, butrather the replication of “selfish”genes. A living creature is a kind ofsmart coalition of cells, whosepurpose is to replicate the geneswithin it. Obviously, biologicalreproduction is “desirable”according to the second law ofthermodynamics, and thus it followsthat our desire to reproduce isrooted in the second law. Dawkins

even points out a connectionbetween the replication of genesand the replication of information àla Shannon. In analogy to “gene”, hecoined the term “meme,” which canbe an idea, a song, or even acomputer virus. Memes exist in theworld of logical entropy ofcommunication networks; theyundergo spontaneous reproductionand instead of eating food, they arenourished by electricity.

Therefore, one may assume thatour “will” drives us to increaseentropy. Early in the nineteenthcentury, philosopher ArthurSchopenhauer claimed that the

essence of all beings is its Will. Inaddition, he claimed that a man cando what he wills, but cannot willwhat he wills.66 If we push furtherin this direction, we must concludethat it is not in our power to restrainour will to increase entropy. Inother words, entropy, with itstendency to increase, determineswhat we want, and motivates us todo things that increase entropy. Anybusinessman will tell you that anystep he makes is aimed to increasethe number of options available tohim. In other words, he will choosea course of action that will increasethe number of alternatives he could

choose from in the future. Inthermodynamic terms, he isessentially saying: “I want toincrease the entropy of the optionsat my disposal.”

This tendency of entropy toincrease is, if you will, thepowerful driving force of allchanges in nature. In general, thesefall into four categories:

Physical changes, such as thetendency of water to flow down;

chemical changes, such as thetendency of iron to rust in damp air;

biological changes, such as thetendency for creatures to multiply;

andsocial changes, which are related

to us, humans, and our culture.

Is it possible to characterize asocial system using the same toolsthat serve us when we analyze aphysical system? And if so, what isthe social analog to a physicalsystem? What is the analog to theflow of energy? And what is theanalog to the distribution of energyin equilibrium? In other words, is itpossible to analyze changes in asocial system in terms of the secondlaw of thermodynamics asformulated by Clausius, Boltzmann,

Gibbs, Planck, and Shannon?To answer these questions, we

must first see what a social systemis. Examination of the fabric of anysocial system, taking into accountboth human and environmentalfactors, immediately brings to minda network. Looking at the definitionof “network” in some dictionarieswe find various kinds of relations:physical (railways, roads, etc.),human (family, support groups,work groups), or communication(the phone network, computernetwork, and of course, THEnetwork, the internet). Any suchnetwork comprises nodes, that can

be represented by dots or boxes,and links that connect the nodes, areusually represented by lines.

Is it possible to learn anythingabout the structure of a networkfrom the second law ofthermodynamics? Does entropy’stendency to increase influence theconnections between the networks’nodes? To answer these questions,let us use the air transport networkas an example.

In the air transport network, anairport is a node and the flightsbetween one airport and another arethe links. If one wants to fly fromTel Aviv to Recife, Brazil, one has

to go through several airports(nodes), since there are no directflights from Tel Aviv to Recife. Thetravel agent may suggest, forinstance, flying from Tel Aviv toParis, then from Paris to Rio deJaneiro, and finally from Rio toRecife.

Non-direct flights throughseveral nodes are common becausethe airlines want to fill their planes.In our case, the number of travelersgoing from Tel Aviv to Recife onany given day is not enough to fill aplane. On the other hand, every daythere are enough people to fill aplane to Paris. For travelers from

Tel Aviv, Paris is a much morepopular destination than Rio deJaneiro, let alone Recife. Had therebeen enough people to fill a planefrom Tel Aviv to Rio, there wouldbe direct flights, which save money,fuel and time. Since this is not thecase, it is more efficient to createnodes with different numbers oflinks. For example, in a smallcountry with a small volume of airtraffic there is only one airport,from which one can reach a numberof busier airports in other countries,and from any of them it is possibleto reach many more internationaldestinations. Nodes from which it is

possible to go to numerousdestinations are termed “busynodes” – transfer points for thosearriving from less busy nodes. Atbusy nodes, passengers changeflights, so that they get to theirdesired destination, and the flightsare filled, ideally to capacity.

What makes an airport a busynode? The answer is economic. Inorder for a node to be busy it musthave certain characteristics, such asproximity to a large, dynamiceconomic center; a suitablegeographical location; anappropriate distance from otherbusy nodes; the entrepreneurial

aptitude of the landowners; and, ofcourse, a heaping measure of luck.

Economically, a busy node isadvantageous since it promotes thedevelopment of logistic-basedservices, such as shops andindustry, which provideemployment. The economicadvantages must outweigh negativeoutcomes such as noise andpollution.

In a certain sense, socialnetworks are similar totransportation networks (eventhough there is a significantdifference, which will be explainedlater on). A person can be

perceived as a node connected to afinite number of people (family,friends, neighbors, acquaintances,colleagues, and so forth). Thenumber of links to other peoplevaries from person to person. Somepeople have numerous links andother only a few. A person withmany acquaintances is a busy node:the more links, the more likely thathe or she is influential, that is, hasthe ability to exchange informationwith many people. Moreover, themore access a person has to otherpeople who are themselves busynodes, the more that person willbenefit. The reason for this is that in

times of need, he or she can reach –via a relatively small number oflinks – more people who are in aposition to help, advise, or endorsethem, cooperate with them invarious areas, and so forth.

In 1976, social psychologistStanley Milgram proposed theSmall World Theory, according towhich every person in the UnitedStates is connected to every otherperson in the United States through,at the most, six connections.67 Thistheory, also known as “six degreesof separation” through six contacts,or “six acquaintances,” has beenexamined in various ways, and

there is no consensus amongscientists on whether or not it iscorrect. Nevertheless, it is similarto our story about the traveler whowishes to go from Tel Aviv toRecife, Brazil. At first, it mightseem that the number of connectionsthat Milgram proposed is too small,yet one can argue that everybodyknows at least one person who is abusier node than themselves. Thisbusy node is similarly connected toan even busier node, and so on.Taking this line of thinking, theMilgram number does not seemsurprising. In fact, by adding justone or two nodes, the American

network can be extendedworldwide.

As an illustration, consider thequestion: assuming that Mr. Levifrom the Israeli town of PardesHanna and Mr. Smith from Oshkosh,Wisconsin, are complete strangers;how many people does Mr. Levihave to go through in order to reachMr. Smith? As a resident of PardesHanna, Mr. Levi probably knowssomeone who is acquainted with thetown’s mayor. This worthynecessarily knows the IsraeliMinister of the Interior, whoprobably knows his counterpart inthe United States, who knows the

senator from Wisconsin, whoknows the mayor of Oshkosh, whoknows one of his employees whoknows Mr. Smith: eight degrees ofseparation, or fewer if, for instance,Mr. Levi knows his mayorpersonally.

Hereafter, we must make adistinction between two kinds ofnetworks: symmetrical andasymmetrical.

Symmetrical networks are thosethat have bilateral equalconnections between any twonodes, as for example, railroadsand transportation networks inwhich it is equally possible to

travel from node A to node B andfrom node B to node A along thesame route. Telephone networks arealso symmetrical: Bob speaks toAlice over the same channel thatAlice speaks to Bob.

In asymmetric networks, thelinks between nodes are notbilateral, but rather hierarchic.Asymmetric networks are morecommon in general, and especiallyin social networks, as will beexplained below. In an asymmetricnetwork, Bob the TV newscaster isable to talk to Alice the viewer, butthis does not necessarily imply thatAlice has the ability to talk to Bob.

The link resembles a one-waystreet. A simple example of anasymmetric network is the military.In this network, a number ofsoldiers are linked to their sergeant,a number of sergeants are linked toa junior officer, a number of juniorofficers are linked to a higherranking officer, and so on up to thetop. In such a network, a brigade’scommanding officer, for example,can address a private soldier, but aprivate cannot immediately addressa brigade CO. Generally, only thoseof the same rank can address eachother directly. Yet if a sergeant fromone brigade wants to communicate

officially with a sergeant fromanother brigade, he or she willnormally have to turn to their CO,who will call the CO in the otherbrigade, who will then make theconnection with the other sergeant.Officers are “busier” nodes thansergeants, in the sense that the rulesof hierarchy allow them tocommunicate with all privates andsergeants under their command andalso with their counterparts in otherunits, with whom sergeants are notallowed to communicate officially.If the officer whom the sergeantapproached is not authorized tocontact his or her counterpart in the

other brigade, one may expect thatthis officer will turn to one ofhigher rank, perhaps even as far asthe brigade CO. In this case, thebrigade CO will contact the otherbrigade CO, plowing a similar yetopposite route back down to thesergeant in question.

In general, if you consider Mr.Big to be your friend, it does notnecessarily mean that Mr. Bigconsiders you to be his friend. Itmay be that Mr. Big considers youmerely an acquaintance – unlessyour name happens to be Mr.Bigger. It is obvious that thecommanding officer of a brigade

can freely address any of hissubordinates (unless this is beneathhis dignity), while the privateusually has no access to the brigadeCO. The basic idea behind ahierarchical network is the unequaldistribution of links between nodes,which is derived from asymmetry inthe connection between the nodes.

This links-and-nodesphenomenon is much more commonthan what may seem at first blush.As an example, most people in theworld know the name of thePresident of the United States. Onthe other hand, it is conceivable thatthe President of the United States

does not even remember the namesof all the other nations’ presidents,which are probably whispered onhis ear when necessary. Mostpeople cannot just phone thePresident of the United States, yetthe American President hasunlimited one-way access to almostanyone (in other words, he is a busyperson of many links). Wealthypeople enjoy a similar privilege. Itcan be assumed that you would notrefuse to meet with Bill Gates, evenif you are smart enough to know thatMr. Gates will not offer you, duringthat meeting, to share his wealthwith you in one way or another. On

the other hand, unless your name isMr. Bigger, you will probablyconsider it an honor if your requestto meet Mr. Gates will be politelyrejected by none other than hispersonal assistant herself. It may bethat the easiest way to estimate thewealth or status of someone is tocount the people with whom he orshe can make direct contact.

The human network is basedupon an unequal distribution oflinks among nodes, becauseinformation transmission in a non-uniform network is more efficient.A non-uniform distribution of linksto nodes is necessary, since a

uniform distribution is problematic.To illustrate this, let us examine agroup of people with a uniformdistribution of links, where eachmember knows one tenth of thenetwork’s population. In this case,there is a high probability thatclosed secondary groups in whichall the links begin and end withinthat group (a bubble) will beformed. This phenomenon was quitecommon in the past, whencommunication between variouscommunities was not welldeveloped. This is how such a widevariety of cultures and languageshave evolved, whose existence has

made communication betweenpeople from different countriesproblematic until very recently.Even now there are such bubbles insome remote corners of the world,as well as groups that choose tolive in isolation, such as someclosed religious sects. Surprisingly,the bubble phenomenon exists inacademia too, where internationalgroups of scientists conductresearch that no one from outsidethe bubble is interested in. Theexistence of such groups limits theflow of information.

In Medieval Europe, a merchantwho came to another country to

establish business ties would findhimself in difficulty – a strangelanguage, a different culture, anatural suspicion of strangers, scantlegal protection, etc. Yet, if he wasJewish, he could go into asynagogue and find people who,while denizens of the other country,still shared a language (Yiddish)and a culture with him, and werewilling to trust him because he was,as they were, subject to rabbinicallaw and could therefore, if thingscame to that, be sued in a rabbinicalcourt in his country of residence,under threat of excommunication. Inother words, Jews had a unique

ability to form “inter-bubble”connections, and thus fulfilled animportant role in the developmentof trade and banking in Europe.

And so, the hierarchal system isanalogous in its working mechanismto the connection between Mr. Levifrom Pardes Hanna and Mr. Smithin Oshkosh. Since a hierarchalsystem, for instance a military one,prevents the formation of bubblessuch as those that appears in anegalitarian system, it has beenadopted by successful organizationsno less important than the military –companies, corporations andinstitutions.

In a hierarchal system bubbleformation is not possible, becausethe busier nodes are connected tomany less busy nodes. Thehierarchic distribution enables theconnection of all nodes into onenetwork with a minimum number oflinks. For this reason, there aremany nodes with few links, and afew nodes with many. All this is themanifestation of entropy, which, inits tendency to grow, leads to thedevelopment of mechanisms whichprevent bubble formation.

How can links be optimallydistributed between nodes? Sincewe do not know with certainty how

to define an ideal network, this isnot a simple question. Moreover,systems – usually, and perhapssurprisingly – are not planned bysome intelligent master planner butrather evolve spontaneously. Forexample, the internet uses thetelephony network (landlines,cordless and satellite), and thecables infrastructure as well. Theseinfrastructures were put in place bymany individuals and firms,developed independently inseparate locations before beinglinked together in one way oranother. Obviously, whoeverplanned the telephone system in

Pardes Hanna, Israel, never gave athought to the cable network inOshkosh, Wisconsin, and still thereare direct telephone and internetservices between Pardes Hanna andOshkosh, using these combinedinfrastructures. As mentionedabove, telephone communication isan example of a symmetricalnetwork. When Mr. Levi talks withMr. Smith, they each have the sameability to receive and transmitinformation. On the other hand,when we listen to the radio, we canonly receive information. In internetand cable television, we usuallyreceive much more information than

we send. Social networks are afairly novel exception in this sense,and they too are formedspontaneously. Even in an idealworld, where we could choose ouracquaintances at will, we stillwould not be able, unfortunately, tochoose our acquaintances’ friends.In fact, no person ever controls anentire network.

If networks are indeed createdspontaneously, this seems towarrant that they would tend toreach a state of equilibrium. Butwhat is “equilibrium” in a network?If we try to relate it to Planck’sformula, we can conclude that in

equilibrium, the distribution of thelinks among the nodes will give thelargest number of different ways toconstruct the network.

Paradoxically, the asymmetry ofthe links simplifies themathematical calculations requiredto work out the entropy of networks.Let us say that we have a networkof N persons, with P asymmetricallinks. The essence of a one-waylink is that one person’s connectionto another is independent of thenumber of links that these peoplehave to him or her. In other words,one may say that a unilateral link isassociated to a specific person in

the same way that a particle belongsto a box. To calculate thedistribution of links in equilibrium,we have to find the distribution ofthe links among the nodes thatmaximizes Shannon’s entropy. Theresult thus obtained is in a way ageneralization of Benford’s Law, asdiscussed in the previous chapter.

The maximum entropydistribution of P links among Nnodes (for the derivation, seeAppendix B-3) is given by

Here, n can be any integer,provided that it does not exceed thetotal number of links, P (i.e., n ≤ P).It can be seen that the relativedensity of the links, ρ(n), is adecreasing function of n. nexpresses the ratio between thenumber of links (or particles) andthe number of nodes (or boxes). Weintuitively call this number a“rank.” Nodes with a rank of 1 arethe poorer nodes, because they haveonly one link. There are more nodeswith a few links than with many,because as n increases, ρ(n)decreases. This is not surprising,since we saw that Mr. Levi had to

made contact with Mr. Smiththrough several busy nodes(including two ministers and onesenator) and several less busy ones.A small number of busy nodes and alarge number of poorly-linkednodes enable efficientcommunication. The intuitiveexplanation for this is that a remotenode still offers at least one link toa busier node, from whichconnection to several busierdestinations is possible, as in theairport example. Aircraft flying atcapacity are analogous to well-exploited communication lines. Thedistribution of links in nodes is

identical to the distribution offrequencies in the radiation mode ofa black body at the classical limit.

It seems that this distribution isidentical for many phenomenacommon in human society. Forexample, in a social network, aperson is analogous to a node, andthe number of people he or she hasdirect access to is analogous to thelinks. In an airline network, a nodeis an airport and the links are thedirect-fight destinations. As weshall see below, a node may also bean internet site, or even a givenbook. In these cases, the surfers or

readers are the links. Many otherdistributions follow this sameequation.

This distribution, when plottedon a log-log scale, gives a straightline with a slope of -1 as can beseen on the left hand side of Figure7. Therefore, this is called a powerlaw distribution.* This distributionis also known as Zipf’s Law, namedafter George Kingsley Zipf, anAmerican linguist and philologist,who discovered this phenomenon inthe frequency of words in texts.What he found was that insufficiently large texts, the mostcommon word appears twice as

often as the second most commonword, the second most commonword appears twice as often as thefourth most common word, and soon. The Zipf distribution also givesa straight line of -1 slope on alogarithmic scale.

Figure 7: The logarithm of the relativefrequency of nodes having n links

(horizontal axis) plotted against thelogarithm of the number of links, n,(vertical axis) produces a straight linewith a slope of -1 for any integer n.This distribution is called a power lawdistribution. To the right of the verticalaxis, the number of links is smaller thanthe number of nodes (n < 1), leading toexponential decay and a curved line.

In Appendix B-3, it is shown thatZipf’s law is obtained for themaximum entropy distribution of Plinks (particles) in N nodes (boxes)when P ≫ N and n = 1,2 ….., P. Inthis case nφ(n) = C. That is, theproduct of the relative frequency ofa node with n links by n is constant

– exactly what Zipf foundempirically with regard to therelative frequency of words in texts.(Zipf’s law is analogue to theclassical limit of Planck’s equation,namely, E = nhv = kT = 1/β, ornv(n) = = C.) The distribution ofword frequencies in texts is anexample of a poll in which theauthors vote for words according totheir popularity. Poll distributionsin general are discussed later in thischapter.

To sum up, we assumed that aspontaneously-evolves network(without a master plan), like the

networks that are formed in nature,is structured according to thesecond law of thermodynamics andtherefore tends to reach equilibrium– the state in which the distributionof links among the nodes providesthe maximum number of possibleconfigurations.

We calculated the distribution oflinks between nodes in networkswhen the number of microstates isat a maximum, and demonstratedthat there are more nodes withfewer links than nodes with many.This distribution is also called the“long-tailed distribution” (seeFigure 8). The distribution of links

among the nodes is identical withthe distribution of frequencies inradiation mode in a black body atthe classical limit, as suggested byRayleigh and Jeans and calculatedby Planck (figure 7 above).

If indeed an ideal thermodynamicnetwork exists, a theoreticalimplication is that any link addedmay actually reduce the efficiencyof the network! This apparentparadox can be expressed evenmore paradoxically: can theaddition of a road to atransportation system increasetraffic congestion? Is it possiblethat adding a road to the system

would result in a reduction of theaverage road speed? Or the otherway round, could shutting a roadimprove the flow of traffic? In fact,this is exactly what was observedin a number of cities, includingStuttgart, Germany, and Seoul,South Korea, where blocking amain road actually led to animprovement in the traffic flow.68

Even more astonishingly, Germanmathematician Dietrich Braesspredicted this effect in 1969.69 Theexplanation given to thisphenomenon – the so-calledBraess’s paradox – is that it

represents a deviation from theNash equilibrium in gamestheory,70 which is based on optimalstrategies that individuals choosebasing on what they know at thetime about the strategies of otherplayers. In the case in point, manydrivers who choose, in view oftheir experience on the road, anoptimal route to bring them to theirdestination, would change theirchoice upon receiving theinformation that a certain road wasblocked. As a result, the efficiencyof the road system may improve.However, a full discussion of theNash equilibrium exceeds the scope

of this book.This second law explanation has

two advantages: it is quantitative,and it is purely statistical – that is,not influenced by non-measurablefactors.

The Distribution of Wealth– The Pareto Law

Sultan Kösen soars to a height of2.51 meters and is considered today(2013) to be the tallest man in theworld. By comparison, ChandraBahadur Dangi, only 0.55 meter

tall, is regarded by the GuinnessBook of World Records as theshortest adult in the world. (Theratio of their heights is about 1:4.5.)This means that the heights of theapproximately 4.5 billion adults inthe world range between 55 and251 centimeters. The average ofthese two heights is 1.53 meters. Noone knows precisely what theaverage height of an adult man is,but it can be assumed that theaverage of Kösen’s and Dangi’sheights is a pretty good estimate.

The statistical distributions andthe laws that govern them were

examined extensively because ofthe enormous importance of theirfluctuation to our lives. Think, forexample, about the distribution(congestion) of traffic on the roads,or the distribution of calls made toa telephone switch as a function ofthe time of day, or the distributionof electrical consumption as itrelates to the weather, or any othersimilar example. All these issueshave vast economic importance,since they allow one to evaluate theimpact of various changes on therespective networks, which we alluse on a daily basis. Should 30% ofa bank’s customers happen to

withdraw instantaneously theirsavings, each for his or herindividual reasons, the bank wouldcollapse. Computer hackers toppleinternet sites by accessing the sitenumerous times simultaneously.While every single act by everysingle person is taken for somespecific, valid deterministic reason– perhaps known only to themselves– the behavior of a crowd is muchmore predictable. As Gibbs said,the whole is simpler than the sum ofits parts. Let us assume that oneclear morning; you wake up with athrobbing toothache. Your regulardentist, who lives on Main Street

near your home, is on vacation(financed by you, needless to say).Therefore, you have to drive toanother clinic, using the highway.Your decision-making ability iscompromised because of thattoothache, so you get on thehighway at 8:30 a.m. on a weekday.For you, getting on the highway atthis time of the day is an extremelyuncommon thing to do, a result ofboth bad luck and bad judgment.Nevertheless, you find yourselfcaught in the usual eight-thirty-on-a-weekday-morning traffic jam. Andalthough you see the other drivers inthe traffic jam as criminal, and think

that everyone would be better off ifthey all were behind bars, the factis that you too are now taking justas much of a part as they are increating this traffic jam, whichnormally has nothing to do withyou. Traffic congestion isspontaneous and unplanned, yetcommon. It is formed by thenature’s tendency to create non-uniform distributions. Thesephenomena are the driving force ofeconomic activity.

Of all the distributions examinedtill now – energy in atoms,frequencies in electromagneticradiation, digits in numerical data,

and links in nodes – the mostintriguing one is perhaps theequilibrium distribution of wealth.

Financial capital is notdistributed among people the waybody heights are distributedbetween Sultan Kösen and ChandraBahadur Dangi. The majority ofpeople know of someone who ismany times richer than themselves.The richest person in the world ismillions of times richer than mosteverybody. Our natural inclinationis to think that being rich is neitherfair nor just, although this belief isprobably more common amongpoorer people than richer ones.

Communism, the ideology thatpretentiously aspired to createeconomic equality between allpeople, was never all thatsuccessful, and where it wasenforced, it led to the poverty ofnations and to extreme inequalitywithin each one of them. The richtend to think that their wealth is aresult of their efforts, skill andvision. Economists sometimes tendto think that money attracts money –that is, it is easier to earn thesecond million than the first. Thisbelief is also supported by thetendency of the rich to marry offtheir offspring to those similarly

endowed. Yet this theory has manyserious flaws, and it is notuncommon to see an enormousfamily fortune fade away within afew generations. As they say, theeasiest way to make a small fortuneis to start with a huge one.

Is the distribution of wealthinfluenced by the second law ofthermodynamics? In order toexamine this we shall look into ahypothetical country called Utopia,with a population of N residents.From time to time, money entersthis country in the form of silvercoins. Suppose that in Utopia,people do not have to work for a

living because there is enough food,drink and accommodation for all.The Department of Justice andWelfare is entrusted with dividingup the coins between the residents.For this purpose it hires theservices of the President’s cousin(who gets a decent honorarium forhis trouble), charging him with thedistribution of these coins to all thepeople equally. In Utopia, it shouldbe pointed out, everybody (exceptfor the President’s cousin) getsequal pay.

One day, however, this idyllicsituation went awry. A group ofresidents, led by Mr. Envious, had

noticed that the president’s cousinhas many more friends than any ofthem (probably because of thefrequent parties he holds at hishouse). They gathered, all thesebitter residents, for a veryexhaustive discussion, and arrivedat the logical conclusion that thereason for the cousin’s many friendsmust be his having more silvercoins. In order to rectify thisinequality, they suggested buying amachine that will divide up thecoins fairly. The reasoning behindtheir suggestion is that at the one-time cost of the machine, the smugsmile will be wiped forever off the

favorite’s face – and this alonewould be worth every penny!Envious, in person, guided themanufacturers in the construction ofa machine that will draw, withequal probability, each coin thatenters to Utopia among all theresidents. The machine arrived, andthe task of distributing the coinswent from the favorite to themachine. However, when Enviousthe Idealist later saw the results ofthe draw, he was horrified: Byintroducing his so-called“unbiased” lottery machine, hehimself has unintentionallyintroduced capitalism into Utopia.

Why should Envious be sohorrified? How were the coinsactually distributed between theresidents? We have already seen inour discussion of Benford’s lawthat if we draw P particles among Nboxes, such that each particle has anequal probability of being in anybox, the Plank-Benford expressionis obtained. According to thesecond law of thermodynamics, thefavorable distribution is the one thatwill yield a maximum number ofmicrostates.

Theoretically, there is somechance that the coins will beequally distributed among the

residents. However, as we sawwhen we were dealing withinformation, the probability of thisparticular microstate (which is thesame as the probability of any othergiven microstate) is infinitesimallysmall. Conversely, the probabilityof an unequal distribution of somekind is almost a certainty. The mostprobable microstate is theequilibrium distribution. Obviously,an equal distribution has relativelylower entropy, because if weswitch coins between residents Aand B, the system does not change.Therefore, it is clear that in orderfor entropy to be at a maximum,

every resident in Utopia must havea different number of coins.

The thermodynamic “justice” isthe justice of configuration, and notthat of the individual. It was forgood reason that Gibbs claimed thatthe whole is simpler than the sum ofits parts. The distribution of coins(that is, the distribution of wealth)is concerned with efficient systemcommunication and has nothing todo with equality among individuals.Such an equality is foreign tonature. On the other hand, the ideaof “equal opportunity” which isaccepted throughout the civilizedworld (at least in theory) increases

entropy by giving equal probabilityto every microstate and state. Equalopportunity removes the constraintsinherent in a class-based societyand provides an equal chance toeach individual, even if,unfortunately, the chances for mostpeople are to be poor.

The thermodynamic distributionof money has already beencalculated in the section dealingwith networks, which means thatwe already have a formula withwhich we can describe thedistribution of coins in equilibrium.To illustrate the injustice that iscreated by an unbiased lottery

machine, let us assume that thereare 1,000,000 people in Utopia.What portion of the coins will therichest person get? From Zipf’s lawwe may conclude that the richestperson in Utopia will receive amillion times more coins than thepoorest person.

Looking at the table below, wecan see that those at the lowerdecile by income, 289,000 men andwoman, have one tenth of thewealth enjoyed by the upper incomedecile, 40,000 people!

As a result of the outrageousinjustice perpetrated by presumablyfair means, the citizens of Utopia

changed the name of their country toRealia and started working in orderto improve their luck. The richestperson, for his or her part, willhave to work even harder tomaintain their fortune.

Figure 8: The thermodynamicdistribution of wealth in a country withone million residents.

Entropy cannot predict who willget rich and who will be poor. Thedesire of each and every one of usto increase our links and ournumber of coins in order to increaseentropy is the source of thedynamics and activity thatcharacterizes Realia.

One of the interesting features ofthe Planck-Benford distribution isits self-similarity – that is, itsinsensitivity to the order ofmagnitude of the numbers (this is a

property of any function thatproduces a straight line on a log-logscale.). The Planck-Benforddistribution is insensitive to eitherthe actual number of persons or theactual amount of wealth. That is tosay, the distribution of wealthbetween the deciles in countries inequilibrium will be identical,regardless the size or relativewealth of the country.

Figure 9: Full Logarithmic graph of thedistribution of wealth gives the typicalstraight line of a power law function.

We can calculate the relativedistribution of wealth from the

Planck-Benford formula,

When relative wealth ranges from 1(lowest) to 10, we get therespective percentage of thepopulation with the relative wealthas follows:

That is, 4% of the population willhave 10 times the wealth of the28.9% poorest in the population(rank 1). We see that the strongestgroup, ranking 6 through 10,comprise 25% of the populationand has 72% of the wealth.

This phenomenon is known asPareto’s law, named after Italianeconomist Vilfredo Pareto (1848-1923), a contemporary of MaxPlanck, who empirically suggestedthe approximate rule to be 80:20.Pareto’s rule, as we could expectfrom any thermodynamic law, isalso valid for other socialphenomena. For example, it iscommon to think that 20% of thedrivers are responsible for 80% ofthe traffic accidents; or that 20% ofthe customers are responsible for80% of any business’s revenues.Pareto himself suggested hisempirical rule when he discovered

that 80% of Italy’s property washeld by 20% of the population.Obviously, his 80:20 ratio was notquite precise. As we saw, thethermodynamic ratio is more like75:25, and it might be more fittingto call Pareto’s rule the “75:25 law.

Vilfredo Pareto

Vilfredo Pareto, 1848-1923

Vilfredo Pareto, an Italianindustrialist, sociologist, economistand philosopher, was born in Parisin 1848. His mother was French;his father, an Italian marquis, wasan engineer who, like other Italiannationalists during that period of

foreign rule, had left Italy forFrance in 1835.

In 1858 the family returned toItaly. Young Vilfredo pursued hisacademic studies in the classics andengineering at the PolytechnicInstitute of Turin. His doctoralthesis in engineering was entitledThe Fundamental Principles ofEquilibrium in Solid Objects. Uponcompletion of his studies in 1870,at the top of his class, he receivedhis first position as manager ofRome’s railway company. Fouryears later he was appointedmanager of an iron and steelcorporation in Florence.

Exasperated with governmentbureaucrats, and having observedthat economic legislation wasmotivated solely by protectionismand political advantage, he turnedto political activity. At the time, hewas a liberal-democrat whoembraced the ideas of free trade,free competition and socialwelfare. He believed inmeritocracy and detested thecorrupt collusion between thenobility and the government. In1882 he ran for the Italianparliament as a representative ofthe opposition party in the Pistoiaregion, but failed to get elected.

Upon the death of his parents, in1889, Pareto inherited the title ofmarquis, but never used it. Hemarried a penniless Russian womanwhom he had met in Venice, andthey moved to a villa in Piazolla.Here he began to lecture and writearticles critical of the government.He was quickly became the subjectof police investigation, and as aresult was unable to obtain anyuniversity post. Nonetheless, hispapers attracted the attention of theleading neoclassical economist inItaly at that time, MaffeoPantaleoni, who taught Pareto thebasics of economic theory. Using

the mathematics he had acquired inhis engineering studies, Paretobegan to publish sophisticatedpapers on economics. In 1893, onPantaleoni’s recommendation, hewas appointed head of theDepartment of Political Economy atthe University of Lausanne,Switzerland. Living in Switzerlanddid not stop his ongoingcondemnations of the Italiangovernment in the newspapers. Heassisted escaping socialists andradicals threatened by thegovernment, and even gave themshelter. During the Dreyfus affair inFrance, he denounced the anti-

Semitic French authorities. At thesame time, he published threevolumes of his lectures (1896-1897).71 He examined thedistribution of income inpopulations and was the first todiscover that in every country andduring every era, incomes aredistributed according to the law thattoday bears his name – the law thatreflects the long-tailed distributionexplained above. Some say that hisdoctorate thesis in engineering,which dealt with equilibrium,motivated him to find equilibrium ineconomy too. Later, Joseph Juranextended Pareto’s distribution to

include other areas of humanactivity besides income, and calledit “Pareto’s Law.”

In 1900, in a famous paper calledRevista,72 Pareto made an about-face and spoke out againstdemocracy. The political instabilityof the 1890s in Europe led him tothe conclusion that radicalmovements do not truly seekequality and social justice, butrather take advantage of the massesin order to substitute one elite foranother. Disillusioned, he realizedthat all ideologies, of whateverbrand, were just smokescreens forthe central motive – domination and

power, as well as the desire toenjoy the benefits of power.

Pareto’s social observationswere the basis of his later ideas.One of them was that economicplanning and economic forecasts,despite the logic inherent in them,could not pass the test of reality,because most human actions aregoverned by illogical, rather thanlogical considerations. Therefore,all forecasts based on rationaleconomic reasoning wereempirically doomed to failure.

In 1907 Pareto resigned the chairat Lausanne and moved to a placenear Geneva. He suffered from

heart problems, and nursed themsurrounded by a dozen cats, anastounding library, a cellar full ofexcellent wines, and a cabinetstocked with fine liqueurs. His wifehad left him in 1901, and since thenhe was living with a Frenchwomanwhom he married only in 1923,after he was able to resolve theproblem of divorcing his first wife(not a minor issue in Catholic Italy).During this period he wrote hisbook, Trattato di sociologiagenerale (Treatise on GeneralSociology),73 published in 1916, inthe midst of the First World War. Inthis book he claimed that ideologies

are rationalizations that peoplemake up for irrational actions. Andindeed, today economists acceptthat irrationally-motivated actionsare an important driving force ineconomics. In 2002, DanielKahneman received the Nobel Prizein economics on his research (incollaboration with Amos Tversky,who died before the Prize wasawarded), on irrational decisionmaking in economics.

Pareto’s theory on irrationalmotives attracted many fascists.When Mussolini and the fascistsgrabbed power in Italy, Pareto, inhis sickbed, muttered sadly, “I told

you so.” He was unhappy aboutthis, but his disapproval did notprevent the fascists from bestowinghonors upon him; they evenappointing him as senator. Herejected most of the honors, butspoke favorably about some of thereforms that were made.Nonetheless, he warned againsttyranny, censorship and economicincorporation. When the fascistsdecreed against freedom of speechat the universities, he wrote a letterof protest.

In Mussolini’s reign Pareto saw atransition period at the end ofwhich, he believed, “pure” market

forces would be released, since atfirst, Mussolini indeed hadreplaced public management withprivate initiatives, eliminatedvarious taxes, gave preference toprivate-sector industrialdevelopment, and so on. Paretodied less than a year afterMussolini had come to power,before the more dreadful aspects ofhis rule became apparent. Thefascists continued to use hiswritings as scientific justificationsfor their methods. Pareto’s theoriesdid not make an impression in histime, but gained prominence yearslater – mainly for what is known

today as Pareto’s law, or the 80:20law (rule).

Polls and Economics –The Planck-Benford

Distribution

We have observed a remarkablephenomenon: not only does thephysical world behave according tothermodynamical statistics, butsociology too does so. In fact, thePlanck-Benford distribution workedout above yields Benford’s Law,which pertains to the distribution of

digits, and also Zipf’s law, whichgoverns the distribution of links innetwork, words in texts, and evenwealth among people (Pareto’slaw). The Planck-Benforddistribution is a generalization ofthe Maxwell-Boltzmann distribution(the normal distribution) forsystems where the number ofparticles can be greater than thenumber of available boxes.

We can extend thisthermodynamic line of thinking evenfurtherer: We intend to argue thateven our thoughts (or morespecifically, our opinions) are the

result of the second law ofthermodynamics. Since “thinking”is not a well-defined action, weshall reduce the concept of“thinking” into “the expression ofan opinion,” which will be furtherreduced to choosing among variousanswers to a specific question.

For the sake of simplicity, let usdistinguish between two kinds ofquestions. The first is the kind ofquestion that has an absoluteanswer. For example, it is acceptedthat the correct answer to how muchis one plus one is two. If youdisagree, people will probably saythat you are either very stupid or

very smart.The second kind of questions is

where the answer is a matter of apersonal opinion. For example,what was the best movie producedlast year? Obviously, not everyonewill pick the same movie.Questions with non-definitiveanswers are also called polls.

Contrary to lottery drawings,which are random by definition, ouropinions are the results of logicaloperations. We do not understandthe considerations that lead anyindividual to make one decision oranother, and usually we cannot becertain whether the decision finally

made was correct or not. Moreover,it is difficult to apply the Planck-Benford statistics to the decisionmaking processes of a single personin a way that is empiricallyverifiable, since a single personusually decides only once on anyparticular matter.

Therefore, the way to test ourtheory is by using polls, in whichmany participants choose theirfavorite answer out of a range ofoptions. There are two reasons forthis: (a) in polls, the number ofchoices, N (the boxes or nodes orwords) is an integer; and (b) thenumber of participants, P (the

particles or links or authors) is alsoan integer and much larger that thenumber of choices.

Polls are highly important in ourlives, not only because so manycountries in the world choose theirleaders by popular vote which is ineffect a poll, but also, and perhapsmainly, because in a free economy,goods, assets, and services arepriced basing on the opinion ofbuyers as to their value: if morebuyers are interested in certaingoods, services or assets, the pricegets higher. Take, for example, abook. A copy of a specific book(after it has been written and

prepared for publication) is acommodity that can be printed ondemand. The more people that wantto read this book, the more copiesare sold. Bookshops can thencharge a higher price, which wouldincrease their revenues as well asthe publisher’s, and hopefully theauthor’s, too.

On the face of it, we wouldexpect that the number of copies ofa specific title sold would directlyreflect its quality. In other words, iftitle A is a thousand times betterthan title B, it should sell onethousand times more copies. Yetthis is an unrealistic expectation,

for a number of reasons: first, wedo not know for certain that title Ais regarded as better than title B ineveryone’s opinion; and second,even if we knew that it is better,there is yet no criterion forevaluating exactly how much betterit is. Nevertheless, there is onequantitative measure for evaluatinga title, which is simply the numberof books sold – the bestsellers’ list.Effectively, the number of copiessold is a kind of “vote” based onthe considered opinion that eachpotential buyer uses in making apurchase. If nature worked this way,one would have expected the

distribution of the number of bookssold among the various titles to berandom. We know, of course, thatpeople do not buy booksnecessarily according to theirquality, whatever “quality” meansin this context. In fact, we buy abook before we read it! Thus, wecan conclude that the number ofcopies of a book sold is a reflectionnot of our opinion, but rather theopinion of those who bought thebook before us. However, thosewho bought the book before us alsohad not read it before buying, andso their opinion, too, is notreflected in their decision to buy.

How then do we decide whichtitle to buy? Which politician tovote for? Where to go on vacation?Or what to watch on television?

It is not clear that this questionhas a clear-cut answer, and eventhough it is intriguing, it is not thefocus of our discussion. However,polls, as we shall soon see, tend toobey the Planck-Benford statistics.Obviously, the Planck-Benfordstatistics do not tell us which title(or product, or political party) willbe the hit that will enrich itscopyright owners or sway theelectorate. But it does tell us whatthe relative success of the second

title will be compared with the firstone – or more generally, it tells uswhat will be the distribution ofbooks sold among of the varioustitles.

Predicting bestselling titles orpredicting the next fashion trend, oras it is called in the industry, “thenext big thing,” is the holy grail ofany entrepreneur. Is it possible?According to the second law ofthermodynamics, it is impossible.The quantity that tends to grow isnot the microstate itself, but ratherthe amount of uncertainty thataccompanies it. The Planck-Benford distribution was derived

under the assumption that theuncertainty of the system is at amaximum.

A dynamic financial system ischaracterized by inherentuncertainty. There is no economicswithout uncertainty. It is impossibleto make money from the knowledgethat the sun rises in the morning andsets in the evening; however, thestability of the earth under our feet,which is not certain, does providean opportunity for making money:Insurance companies are doingexactly that. Think what wouldhappen should NASA announce thatin one year’s time Earth will be

struck by a massive, unstoppablemeteorite: It is conceivable that inthe course of the year betweenannouncement and impact,civilization will exterminate itselfwithout the help of that meteorite.The very existence of civilization isbased on both the expectation ofcontinuity and on uncertainty. It isreasonable to assume that we wantto know the future only in terms ofprobability. Most people would notwant to live out their life countingdown toward a known date ofdeath.

Economics means business, andbusiness means risk management

under uncertainty, or in otherwords, entropy. A best-selling titlemay have been a success because itwas good, but it also had to tackle anumber of hurdles put up by chanceand luck. The first one is a reviewby an editor at the publishing house.Would the greatest Hebrew poetshave made it? Not necessarily. “OBird,” Chaim Nachman Bialik’sfirst published poem (1892), isconsidered a masterpiece ofHebrew poetry. Googling “O Bird”+ Bialik (in Hebrew) brings upmore than one hundred thousandlinks. In the Hebrew Wikipediathere is an entry dedicated just to

this one poem. Every schoolchildknows by heart its first stanza –“Welcome back, O lovely bird,returning from warm climes to mywindowsill...” (it was written whenthe poet still lived in Russia) – andpossibly the next one. Problem is,there are 18 stanzas in all, whichhardly anyone would recognizebeyond the first and second ones.According to an apocryphal butfunny story, in the 1990s an Israelijournalist sent out the last five (andvirtually unknown, except to a fewscholars) stanzas of this immortalmasterpiece, under a pseudonym, tothe literary supplements of all the

daily newspapers in Israel. Eachand every editor summarily rejectedit, except for one, who happened tobe familiar with the poem frombeginning to end (even theunexpected does, sometimes,occur). This editor’s answer was:“Welcome back, O Chaim NachmanBialik, returning from the dead.”

In 1988, British mountaineer JoeSimpson wrote a book calledTouching the Void, about anincident in which he was almostkilled while climbing the Andes inPeru. The book was not animpressive success. Ten years later,Random House published John

Krakauer’s book, Into Thin Air,which also tells of a climbingtragedy. This book became asensational bestseller. And then,surprisingly, the sales of Touchingthe Void began to increase too.Random House, recognizing thetrend, printed a new edition andpromoted this book alongsideKrakauer’s. Sales grew even more.The new edition was on the NewYork Times’ Bestsellers’ list forfourteen weeks. At the same time, adocudrama of Simpson’s book wasreleased on television, and the salesof Touching the Void now doubledthose of Krakauer’s. In retrospect, it

is possible to explain the dynamicsthat led to the sales of these twotitles, but it is obvious that it couldnot have been foreseen.

In the city of big business, NewYork, millions of dollars areinvested in Broadway productions.The night after a production’s debut,the reviews come out – and in manycases, the cast is summarilydismissed. Is it possible that a tinyheartburn pill taken in advance bythe reviewer of the New York Timeswould have saved the investment?

The most prestigious award inthe world is the Nobel Prize. It ishard not to admire the Royal

Swedish Academy of Sciences(RSAS) for doing a wonderfulbranding job over the years for theirprize, and there is no doubt thatmost of its prestige is due to theoutstanding achievements of thosewho have received it: the crème dela crème. And yet, Boltzmann neverwon the Nobel, even though Planck(who was a recipient) highlyrecommended him. Neither Gibbsnor Shannon were honored in thisparticular way. We shall not try tolist the names of scientists who didwin the prize, but whose work, asseen today, had been marginal atbest, and sometimes even

erroneous. It is not our aim toderide the work of the RSAS, justto point out that chance and luck areinevitable, even in the best-informed decisions made withinsystems operating at the uppermostprofessional level.

These were just a few examplesof the first stages on the way tosuccess, namely passing through the“filter of luck.” Is it possible toestimate with certainty the value ofa work of art? It can be presumedthat the value of a painting by avantgarde artist Jackson Pollock (1912-1956) climbed dramatically aftersomeone in Hollywood decided to

make a movie about him. Would thatsomeone have made a movie aboutPollock had he not been soeccentric? Was the moviesuccessful only because the qualityof Pollock’s art? It is impossible totell.

It is reasonable to assume that ifyou had written something good andthen jumped off the top of the EiffelTower, this act will benefit thesales of the book (and enrich thebeneficiaries of your will). Even ifthe book was not very good, but youwent on to commit some cruel andunusual crime, you may beremembered sufficiently to increase

its sales significantly. On the July21st, 356 BC (entirely bycoincidence, the date of Alexanderthe Great’s birth), a Greek calledErostratos set fire to the Temple ofDiana in Ephesus, one of the sevenwonders of the ancient world, andburned it to the ground. Erostratosnever tried to hide his crime, on thecontrary, he declared publicly that itwas he who had burnt the temple –in order that his name beremembered by history. TheEphesians had him executed, andeven proclaimed that anyone whouttered his name would be punishedby death, but this did not help.

Erostratos achieved his goal, andhistory does indeed remember hisname. About 2,300 years later,Mark David Chapmann gave asimilar reason for his assassinationof John Lennon, and his name, too,is remembered by many. Brutuswon a place in history for themurder of Julius Caesar, and JohnWilkes Boothe for the murder ofPresident Lincoln, and so forth. Wedo not need to check the Internet toknow who assassinated JFK. Hisname is well-remembered, betterthan for instance the guy whoinvented Teflon, for example –what’s his name? (Oh, yes, Roy J.

Plunkett.) It is not surprising that itis in the United States, where famecan be cashed in at the bestexchange rate, that the mostoutstanding crimes are committed.

It is thus clear that our decision-making on subjects which are notabsolutely defined is a process inwhich chance plays a major part.Presumably, the probability that anIraqi will think Mohammad a moreimportant prophet than Jesus is veryhigh, and equally, an Italian willthink that the opposite is true. Beingborn in a country where almosteverybody believes in Mohammad,as compared to being born in a

country where almost everybodybelieves in Jesus is a purelyprobabilistic process. And if thesecond law of thermodynamicsstates that the number of microstatestends to increase to a maximum, weshould therefore be able tocalculate the most stabledistribution between prophets, eventhough we shall not be able to tellwhich one will be the morepopular. Let us now examine thePlanck-Benford distribution using aconcrete example. A recent poll inIsrael asked respondents tocomplete the sentence “The greatestincentive for me is…”, and got the

following distribution:cash, here and now – 59% of the

respondents;promotion, credit and

responsibility – 23%;personal development and

creativity – 18%.The results we would expect

from a distribution based on thesecond law of thermodynamicswould be:

In other words, in equilibriumwe should expect that 50% of therespondents will choose answer

(1), 29% will pick answer (2), andthe remaining 21% will giveanswer (3) – a result which is fairlyclose to the actual poll cited above.

Obviously, the theoreticaldistribution has nothing to do withthe meaning of the questions asked,or the answers given. As we cansee, that the answers in thisparticular case deviate somewhatfrom the theoretical expectation,giving a greater emphasis to thepreference for money; this can beexplained as a reflection of ourmaterialistic society. However, ifwe took numerous polls withdifferent questions, we may expect

the average to be in very closeagreement with the Planck-Benforddistribution, since the effects ofculturally biased or any otherdeviation, in any individual poll,will tend to cancel each other out.The following table and illustrationpresent the average of eight three-choices polls, each answered by1,500 respondents each. Andindeed, we can see that theagreement is much better.

Figure 10. The combined results ofeight polls on various subjects,answered by about 1,500 respondentseach, show that the average distributionof answers is in good agreement withthe theoretical distribution.

Figure 11. A graphic presentation ofFigure 10’s results. The solid lineshows the actual distribution, and thedashed line, the theoretical distribution

obtained by maximizing Shannon’sentropy.

In conclusion, let us revisitbubbles. A common phenomenon,bubble formation actually reflects atendency to disequilibrium, and thisis also significant to the presentdiscussion.

In equilibrium, the number ofmicrostates is at a maximum and istherefore characterized by a non-uniform distribution. Thethermodynamic distribution inequilibrium may sometimesresemble a bell-like curve, as wesaw in the case of ideal gases, and

sometimes it has a long tail. A polldistribution in which mostrespondents think alike is analogousto a network in which all nodes areconnected to a single central node(a star-shaped network). This typeof network is not in equilibrium,because the entropy is not at amaximum. Similarly, a network inwhich every node has an equalnumber of links is not inequilibrium. A network indisequilibrium is an unstable one,and hence strives to reachequilibrium. That is to say, thesecond law of thermodynamics“dislikes” uniformity, as in polls

which show that everyone thinksalike. By the same token, it does notfavor situations where each ideahas an equal number of supporters.We can pursue this analogy one stepfurther, and say that while thesecond law of thermodynamicsdislikes dictatorship, it alsodislikes anarchy.

Bubble formation is an essentialstage in the development of largenetworks, since bubbles tend tocombine eventually and become alarger network. Occasionally,however, a natural desire forcommunication may lead to theformation of suicidal bubbles. An

astounding example of a suicidalbubble is a school of herringsattacked by a killer whale. Contraryto any logic, as an outside observermay see it, the school convergesinto a sort of virtual ball, in whicheach fish swims at amazing speed ina path wondrously synchronizedwith all other herrings. But then thewhale opens its mouth and “poof!”– about thirty percent of the herringsare swallowed at once. Obviously,a herring that distanced itself fromthe school at such a time wouldgreatly increase its chances ofsurvival. After all, even a not toohungry person, armed with just a

roll and a hard-boiled egg, wouldrather chase several herrings ratherthan a solitary one.

Are humans smarter thanherrings? It seems that occasionallypeople do behave like a school ofherrings. One of the more appallingexamples history provides is theGerman Third Reich. Germany, acradle of rational culture, thebirthplace of many of the ideasdiscussed in this book, entered inthe third decade of the twentiethcentury into a tailspin that ended upin a way not dissimilar to a schoolof herrings which forms a ball whena whale approaches it.

Astonishingly (from the standpointof the tendency for pluralism innature according to the second lawof thermodynamics) an entire nationunited, democratically and almostunanimously, around the disastrousideas of Adolf Hitler – just likethose herrings. The usual range ofideas that existed when it was aplural society disappeared almostat once. This disappearance ofpluralism – a result of equilibriumderived from the maximization ofthe number of microstates – draggedthe human race into inconceivablehorrors and slaughter. Germany’sdeparture from internal equilibrium,

as a result of developments whichtook place after the First World War(the harsh terms imposed by thepeace agreement, the world-wideeconomic depression, and the riseof communism in Europe), whichcan be depicted as analogous to thewhale that stirred the school ofherring out if its equilibrium.

In economics, “bubbliness” is amotive force. Wherever aconsensus develops amonginvestors about the desirability onething or the other – say, a newtechnology, or a wondrousinvention – a bubble is likely toform. The most well-known bubble

in economics is perhaps the goldrushes that took place in the US,Canada, Australia, South Africa andelsewhere in the 19thcentury. Anenormous wave of immigrationdeveloped following the spread ofrumors about rich gold deposits inremote areas. For many of thesegold-diggers the result, sopoignantly depicted in CharlieChaplin’s film The Gold Rush, waseating their scruffy shoes.

We all recall the Internet bubble.Without a doubt, the importance ofthe invention of the World WideWeb was second only to theinvention of the printing press

(which is considered by many to bethe greatest invention of the secondmillennium). Around 1999, itbecame widely believed that mostof the trade will soon move into theInternet, that Internet services willbe free, and that the main source ofincome for Internet servicesproviders will come fromadvertising. Therefore, the waytelevision ratings determine profits,the number of hits on a site willdetermine the cost of advertisement,and consequently the site’sprofitability. The obviousconclusion drawn was that creatinga “sexy” Internet site, to attract

numerous hits, was the path towealth. The excitement about thenew network technology wasjustified in itself, and led to aconsensus concerning its gloriousfuture – but at the same time, manyexcited people tended to ignore thefact that a business model based onselling advertisement is inherentlyconstrained (for instance, by theamount of money people are readyto spend for content, as well as bythe low technological barrier inwhich competitors could enter). Noless important was this simplelogical truth: it is impossible foreverybody to get rich from the same

resource.In spite of all this, anything that

had something to do with theInternet was assessed atastronomical prices. High techshares skyrocketed. In a sense itlooked like a pyramid scheme,where no actual profits arenecessary in order to get rich, aslong as enough people are joiningin, pouring ever more money intothe pot; of course, there comes amoment comes when the number ofpeople wanting to realize theirprofits is greater than the number ofnew participants (such a moment isinevitable in any finite-sized

system). Then prices plummet, andexisting investors rush in to realizewhatever profits left to be had, andfinally make every effort to cut theirlosses. And indeed, high-tech stockquotations started spiraling downduring 2000, creating a sort ofbubble within bubble – a bubblethat came into being from the newaversion to high-tech stocks.Finally, in 2001, the share prices oftechnological companies andassociated sectors collapsed in away the market could not recoverfrom for a whole decade. Thecollapse was not the result ofdisappointing technology. On the

contrary, Internet technologycontinued to be overwhelminglyimportant. Even the aforementionedproblematic business model wasnot the cause of the collapse. Thebubble burst was a result ofbubbliness itself, that is to say, ofthe pyramid mechanism thatproduced profits from the very rushon high tech shares, caused by aconsensus that did not maximize theentropy.

Take another example: sometimeshouse prices go up, sometimesdown. An increase in house pricesreflects one and only one fact:demand is greater than supply.

Accordingly, a drop in pricesreflects the fact that more peoplewant to sell than buy. Thus,fluctuations in the housing marketindicate that the numbers ofpotential buyers and sellers vary intime: there are times when it isconsidered worthwhile to buy ahouse (and lo and behold, the pricesgo up) and there are times whenbuying a house is considered a badidea (and sure enough, the pricesdrop). In fact, price increases anddecreases are both bubbles. Inequilibrium, the price of a housereflects the potential profit it mayyield or the costs of building it, and

price fluctuations should benegligible. In periods ofequilibrium, house price dynamicswill only reflect local changes insupply demand as certain areasbecome more or less desirable,rather than an overall rise or fall in,such as sometimes happens.

Sharp changes in prices based onconsensus are bubbles, whereas inequilibrium, opinions aredistributed so that the number ofpossibilities is at a maximum.

The problematic nature ofeconomic prediction is rooted inbubbles. Anyone who can predictprice fluctuations have his or her

future assured, since all they haveto do in order to get rich is to applythe age-old formula: buy low, sellhigh. And in fact, there are somewho succeed in doing this. As wesaw in the discussion of thedistribution of wealth, thedistribution of success is alsodetermined by the second law ofthermodynamics.

Conclusion

We have traveled a long way – fromCarnot to Shannon – toward theunderstanding of entropy and the

second law of thermodynamics,which states that entropy can neverdecrease, and tends only toincrease. Our journey began withCarnot, who derived the expressionfor the maximum efficiency of atheoretical machine that producesmechanical work from thespontaneous flow of energy from ahotter object to a colder one. Next,Clausius identified entropy inCarnot’s efficiency equation, andgave it its definition: heat dividedby temperature. Clausius alsoformulated the second law ofthermodynamics, which says that allsystems tend to reach equilibrium, a

point where entropy is at amaximum.

A huge breakthrough in theunderstanding of entropy wasindependently made by Boltzmannand Gibbs, who showed that theentropy of a statistical system isproportional to the logarithm of thenumber of distinguishablearrangements in which it may befound. Each possible arrangementof a system is called a microstate,and since every system, at any givenmoment, can only be in onemicrostate, the entropy thusacquires the meaning of uncertainty.

This is because if a system is in oneparticular but unknown to usmicrostate, the more microstatespossible, the greater the uncertainty.Thus the second law ofthermodynamics obtained a newmeaning: Every system tends tomaximize the number of itsmicrostates and its uncertainty.

How does nature maximize thenumber of microstates? If weconsider an ideal gas as anexample, its energetic particles aredistributed among the possiblelocations (states) within a system ina way that maximizes the number ofmicrostates. The distribution in

which the number of microstates isat a maximum is called theequilibrium distribution.

We examined two importantdistributions that are common innature: one kind, an exponentialone, occurs in systems where thenumber of particles is smaller thanthe number of states – for example,the distribution of the energy ofparticles of gases, or thedistribution of heights amongpeoples. These distributions(graphically depicted by bell-likecurves) favor the average. Anotherkind includes power-lawdistributions (graphically, these are

long-tailed curves), which occur insystems where the number ofparticles is greater than the numberof states. These distributions allowmuch greater divergence from theaverage and are observed, forexample, in the classical limit to theenergy distribution of black bodyradiation and in the distribution ofwealth.

Up to that point, we havereviewed the contributions to theunderstanding of entropy made byCarnot, Clausius, Boltzmann,Gibbs, Maxwell and Planck. Theirworks were carried out between theearly 19th century and the early

20th century, during the golden ageof the physical sciences.

Later on it was also shown thatthe uncertainty associated with datatransmission, such as receiving acomputer file, is in fact the file’sentropy. This entropy is named afterShannon, who was the first tocalculate it. How is Shannon’sentropy different from the entropyof, for example, a container full ofgas? The difference is due to thefact that a digital file is transmittedby electric or electromagneticharmonic oscillators. Theoscillators used to transfer files aremuch hotter than their surroundings

– that is, they have a lot of energy.As we saw, when an oscillator isvery hot, its entropy is independentof its energy and has a value of oneBoltzmann constant. In such cases,as in digital transmissions, wherethe entropy of a file is not a functionof its energy, we call it logicalentropy. By way of analogy, oneplus one is always two, regardlessof the amount of energy each digitone may have. This contrasts withoscillators (such as the molecularoscillators in a gas, for example)whose temperature is closer to theenvironment’s, such that a change inthe oscillator’s energy leads to a

change in its entropy. It thus followsthat if oscillators whosetemperature is closer to theenvironment’s were used to transferfiles, any change in the energy of thefile during transmission wouldaffect its entropy. Since it isimpossible to transfer files withoutlosing energy during transmission,content transfer using oscillatorswith a temperature similar to theirsurroundings is inefficient.

If the laws of thermodynamicsalso apply to logic, they should beobserved in logical distributions.And indeed, based on thisassumption, we showed how the

non-uniform distribution of digits inrandom numerical files, such asbalance sheets, logarithmic tablesand so forth (Benford Law), does,indeed, obey the second law ofthermodynamics. This distributionwe call the Planck-Benforddistribution, after Planck who firstcalculated it and Benford whodiscovered this distribution innumerous databases.

We next examined the effects ofthe Planck-Benford distribution insocial systems, and we found it inmany networks, such as social,transportation and communicationnetworks. All these networks are

generated by a spontaneous linkingof nodes. The invisible hand thatdistributes the links among nodes isentropy, namely, the tendency ofspontaneous networks to increasethe number of microstates. Callingthe relative number of links a rank,we saw that texts obey Zipf’s law,which states that the product of therank and the frequency of any wordin a long enough text is constant. Inother words, nodes with many linksappear less frequently, and nodeswith few links appear morefrequently.

We explored the distribution ofwealth and saw a surprising result:

When particles (coins) aredistributed among states (people),such that every microstate has anequal probability, a highly non-uniform distribution is obtained(Perato’s 80:20 rule), which isreflected in the distribution ofwealth in free economies. In otherwords, there is a high probabilitythat a small number of randomlychosen individuals will have hugeamounts of money, whereas mostindividuals will have to make dowith much less – a counterintuitiveresult in view of our equalprobability starting point, yet all toofamiliar in real life. No less

surprising, we saw that surveys andpolls tend to obey the same Planck-Benford statistics.

Is our thinking also influenced bythe second law of thermodynamics?Let us rephrase the question: do wehumans aspire to increase thenumber of possibilities(microstates) at our disposal?Business people will certainlyagree without hesitation, and evengo so far as to add that anyone whodoes not act in this way is a sucker.To reinforce this point, let us tell ananecdote we heard on Israeli radioby author and journalist AmnonDankner.

Dankner was good friend ofanother author, Dan Ben-Amotz.Ben-Amotz’s personal philosophywas based on his firm belief thatnobody should give anybodysomething for nothing. Thisphilosophical school is also knownas “penny-pinching.” One day,while Ben-Amotz was on anextended stay in New York,Dankner paid him a visit, and thetwo went for breakfast in aninexpensive eatery. When Danknersaw the paltry bill, he offered topick up the tab. To his surprise,Ben-Amotz frowned. Now, hadBen-Amotz been your typical

cheapskate, his reluctance whenoffered a free meal would havebeen peculiar indeed. But, asmentioned, Ben-Amotz wassomething of a philosopher, and hisexplanation for rejecting the offerwas this: the next time they woulddine together, the place wouldprobably be classier, hence the billwould be larger, and since it willbe his turn to pick up the tab, thismorning’s “free meal” wouldeventually cost him more.

Unconvinced, Dankner did someback of envelope calculations andshowed Ben-Amotz that even if he(Ben-Amotz) would always end up

paying for the more expensivemeals, it would not cost him morethan fifty dollars over a whole year.So he asked him: “Why does thisbother you so much?” (Ben-Amotzwas a man of means.) Ben-Amotzanswered: “I hate being a playedfor sucker.” And then Danknerasked him a really profoundquestion: “Why is it so important toyou not to be played for a sucker?”

The rest of the story has nothingto do with the topic at hand, that is,why do we so hate to feel that weare suckers? But we shall continuewith it anyway, so as to not leave itunfinished.

Ben-Amotz’s answer, whichcame after a long reflection, was:“Let me think about it.” Danknerhimself, so he said, totally forgotabout it as soon as the meal wasover. But four years later, Ben-Amotz, on his deathbed at home,called for Dankner. When Danknerarrived, Ben-Amotz, his remainingstrength almost spent, whispered:“Do you remember that you askedme why I hate so much being asucker?” Dankner nodded. AndBen-Amotz told him: “I don’tknow.”

Any one of us sometimes doesthings which are plainly irrational

because of this aversion to being asucker. Kahneman and Tversky74

discussed the illogical economicdecisions that people make justbecause of loss aversion. Butbefore turning to loss aversion, wemust deal with a more fundamentalquestion: Why do we want to getricher and richer? This is aninteresting question, because wealways seem to want more moneythan we have, no matter how muchwe actually have. It seems that noamount of money that can satisfy ourhunger for even more money – asopposed to food: everybodyunderstands that there is no need to

have infinite amounts of it.If we interpret a microstate as an

option, we can understand thatmoney increases the number ofoptions open to us. The greater thebudget, the greater the selections ofproducts we can choose from asbest suits us. Therefore, our desireto increase the number of optionsopen to us is an outcome ofentropy’s tendency to increase.

Let us return now to Ben-Amotzand to his grumpiness at Dankner’soffer to pay for his meal. Accordingto Ben-Amotz’s reasoning, whatDankner did was reduce the numberof options open to Ben-Amotz in the

future: declining the offer, it wouldbe up to him whether or not toinvite Dankner for a meal later on;accepting it, only one of theseoptions would be left. In general,receiving an expensive gift maymake us feel distressed because webecame indebted to the giver.Suppose you are celebrating youronly son’s Bar-Mitzvah. You inviteyour good friend, who happens tohave two sons, ages eleven andtwelve, and he gives the boy apresent worth one thousand dollars.Chances are you may feel somewhatlike Ben-Amotz because now yourallegedly good friend has reduced

the number of options open to youupon receiving successiveinvitations to his sons’ Bar-Mitzvahs. His present actually costyou one thousand dollars! What adreadful man.

Reducing the number of optionsavailable to us thus means a netloss. Every system in nature“wants” to increase its entropy, thatis, the number of options availableto it. But reducing the entropy of asystem is possible only by applyingwork (power) from the outside.

Freedom, then, is the ability tochoose at will from the greatestnumber of options available to us.

In other words, entropy is freedom;and the equal opportunity (ratherthan equality per se) that maximizesthe number of options available isthe second law of thermodynamics.When the number of optionsavailable to us is infinite, choicebecomes random and the microstatein which we exist is our fate that isdetermined by God’s game of dice.

Appendices

Appendix A-1

Clausius’s Derivation ofEntropy from Carnot’s

Efficiency

Clausius realized that it is possibleto rewrite Carnot’s efficiency,

in a slightly different way, namely,

where Q is the heat removed from

the hotter object (let us designate itQH) and Q – W is the heattransferred to the colder object(designated QL). Now Carnot’sinequality can be written as

The meaning of this inequality isthat the energy absorbed by thecolder object, divided by thetemperature of the colder object, isalways higher than the energyremoved from the hotter objectdivided by the temperature of thehotter object. In the case of

reversible action, the two ratios areequal. Namely, the entropy that isgenerated in the colder object isalways greater than the entropy thatis reduced in the hotter object,except in the case of reversibleaction, where the system is inequilibrium and the two entropiesare equal, namely:

Clausius called the quantity inequilibrium “entropy,” anddesignated it S. Strictly speaking,since Q is energy in transit (like

work, heat is energy that is addedor removed), S designates thechange of entropy. Thus, in general,

Appendix A-2

Planck’s Derivation ofBoltzmann’s Entropy

Working under the assumption thatthe “atomic hypothesis” wascorrect, Boltzmann wanted to findout the statistical meaning ofentropy. He built a model of anideal gas consisting of “billiardball” particles involved in elastic

collisions, in which no kineticenergy is lost. To find such asystem’s entropy, we have first tocalculate the number of differentstates in which the system can befound. In figure 12 we have aschematic vessel, with N states andP particles. Although in the figurebelow we show only the spatiallocation, the word “state” refers toa particle its velocity (its speed anddirection) as well as location. Weassume that the number of particlesis much smaller than the number ofstates, namely, P ≪ N. Therefore,we may neglect the probability oftwo particles occupying the same

state. The number of different waysto arrange the particles in the statesin this case is

Figure 12: Two possible ways forarranging 4 particles in 49 boxes.

The first particle can be placedin any of the 49 boxes. The secondparticle can be placed in one ofonly 48 boxes; therefore it ispossible to place two particles in49 × 48 ways. However, since theparticles are identical, the numberof different configurations must bedivided by 2. The third particle canbe in any of 47 different boxes, andthe number of identicalconfiguration is 2 × 3, so thenumber of different ways will be

, and so on. In mathematics,

we define n factorial as n! ≡ n × (n– 1) × … × 2 × 1. Therefore, thenumber of different ways to put 4particles into 49 boxes is

. Eachdifferent way to distribute Pparticles over N states is called aconfiguration. In statisticalthermodynamics it is called amicrostate.

A microstate is one arrangementof a system that can bedistinguished from any otherarrangement of that system. Thetotal number of the microstates isdesignated Ω, and Boltzmann

assumed that each microstate in asystem in equilibrium has an equalprobability.

Let us now calculate the relationbetween entropy and the number ofmicrostates. We use the extensivity(additive) property of entropy, andassume that the system’s entropy, Sis related to the number ofmicrostates, Ω, such that S = f(Ω).We want to find the function f. Todo so, we take a vessel, A, similarto the one described in figure 12.Now, we put another vessel, B, nextto vessel A. The particles cannottransfer between the vessels. Totalentropy will be

S = SA + SB = f(ΩA) + f(ΩB).

Since each microstate of vessel Acan coexist with all the microstatesof vessel B, the total number of themicrostates is Ω = ΩA × ΩB.Therefore,

f(ΩA × ΩB) = f(ΩA) + f(ΩB).

This equation is a simplemathematical problem. The functionthat solves it is the logarithmicfunction multiplied by a constant k:

S = k lnΩ.The value of constant k, known

today as Boltzmann’s constant, wascalculated empirically using the gaslaws, and is one of the importantconstants of nature. This expressionof entropy is an essential tool incalculating the distributions ofstatistical systems (such as idealgases, see Appendix A-4.)

Appendix A-3

Gibbs’s Entropy

As we saw in the appendix A-2, theBoltzmann entropy is based on twoassumptions: first, that the entropyof two adjacent vessels is the sumof the entropies of both vessels, andsecond, that the total number of themicrostates of the two vessels is theproduct of the number ofmicrostates of each of them.

We obtained entropy as S = k

lnΩ, were k is Boltzmann’s constantand Ω is the number of microstates.Gibbs’s question was, what will bethe entropy of a single microstate?The entropy value of a singlemicrostate depends on the numberof microstates. If a system has Ωmicrostates, the probability of thesystem, in equilibrium, to be in asingle microstate will be

If the number of microstates isgreater than 1, there is someuncertainty, which increases theentropy. Since the entropy of a

microstate is proportional to thelogarithm of the probability, p, theentropy of a microstate isproportional to lnp ≡ .

The sum of the entropies of allthe microstates should yieldBoltzmann’s entropy. Assuming thatthe entropy of a single microstate isΦ ln p, Φ is an unknown functionthat we seek to find. If all themicrostates have equalprobabilities, and since = lnΩby definition, then

This means that or

the entropy of each microstate is –kp lnp. If we allow each microstatehave a different probability, pi, thanwe obtain the expression known asGibbs’s entropy:

The main advantage of Gibbs’sentropy is our ability to assign adifferent probability to eachmicrostate, and therefore todescribe systems not in equilibrium.Perhaps it would be better to callthis quantity – as Boltzmann andShannon did – H; however, this

expression is today called Gibbs’sentropy. We discussed this point inthe chapter on Shannon entropy andfile compression.

Appendix A-4

Exponential Energy Decayand the Maxwell-

Boltzmann Distribution

How is energy distributed amongparticles? All microstates, bydefinition, have equal energy. Also,in equilibrium, all microstates havethe same probabilities and entropyis at a maximum. What we want isto calculate the distribution ofenergy among particles such thatentropy will be at a maximum. First

we express the entropy in terms ofthe probabilities of the particles inthe various states. In a case wherethe number of particles is muchsmaller than the number of possiblestates, the number of microstates is

We can use Stirling’sapproximation, InN! ≅ NlnN – N.Let the probability of a particlebeing in a given state be .Then, Boltzmann’s entropy is

S = kln Ω ≅ –Nk{plnp + (1 –p)ln(1 – p)}.

One should not confuse theprobability p in this equation,which is the probability of a state,with the microstate probability ofGibbs’s entropy. To avoidconfusion, let us use the index j forstates:

Normally, when we want to findthe minimum or maximum of afunction, we determine where theslope with respect to a variablevanishes, namely, where

The reason is because the slope of afunction at its minimum or maximumis zero.

However, here we have toconsider an additional point: wehave a fixed amount of energy, Q, todistribute among the particles.Therefore we have to find themaximum S, taking into account that

where εj is the energy of the state jor,

The last equation is called aconstraint. We want to find themaximum of S under the constraintthat the total energy in all the statesis Q. Problems like this are usuallysolved using a technique calledLagrange multipliers, namely, bydefining a function as follows:

f(pj) = S(pj) + β(Q – pjεi).

This function is identical to theentropy since the second term is

null. However, now the derivationcontains another variable, β, calledthe Lagrange multiplier, which isrelated to the energy of the system.*

The derivation .

When pj ≪ 1, we obtain the so-called “normal distribution,”namely,

Since all the microstates have thesame energy, it is easy to see fromthe gas laws that .

Usually, we are interested in the

relative energy of the state ρ(εj).Therefore, we shall divide theprobability of the energy εj by thetotal sum of states .

The function Z, is called thepartition function. Therefore,

We see that as the energy of astate rises, its probability decreasesexponentially. This is a typicalbehavior that is seen in manydistributions in nature. In this case,the distribution of speeds of the gas

particles is “Gaussian” (bell-like).The reason for this is that the speedof a particle (we are dealing with“billiard balls” particles) is relatedto its energy according to ε =½mv2. When we add energy to thegas, it tends to flow to the colderparticles. Only a very few particlescan hold a lot of energy. The resultis that exponential decay prefersaverage-energy particles, with justa few energy-rich ones. As we shallsee later, this is not the onlycommon distribution in nature.

Appendix A-5

The Temperature of anOscillator

The molecules of an ideal gasexchange energy among them onlyby elastic collisions. The energycan be stored in the particles astranslational energy, which changesthe spatial location of the particles.It can also be stored in therotational motion of the particlesand in the vibrations of moleculesalong chemical bonds. Any

movement that is independent of anyother movement is called “a degreeof freedom.” For example, aparticle in space has threetranslational degrees of freedom: upand down, left and right, andforwards and backwards. A degreeof freedom is a microstate;therefore, in equilibrium, eachmicrostate will have the sameamount of energy as all othermicrostates.

If we take a single-atom ideal gassuch as helium or neon, we shallsee that in order to raise theaverage temperature by one kelvin,

it is necessary to heat it by joulesper atom. If the molecule of a gashas two atoms, such as oxygen,nitrogen, or hydrogen, it isnecessary to heat each molecule by

joules in order to raise thetemperature by one kelvin. Theconclusion is that each degree offreedom holds Joules.

Why? A single-atom gas hasthree degrees of freedoms;therefore, its energy is joules perkelvin, or joules in total. In adiatomic molecule there are, inaddition, two rotational degrees of

freedom. The third one, along theaxis of the chemical bond, isnegligible, as the atom’s angularmomentum is infinitely smallbecause it is a point mass. Diatomicmolecules have, in addition,vibration, and this vibration (thiscan be checked using polyatomicmolecules) has kT energy. This isbecause the vibration contains bothpotential and kinetic energy, andeach one of them contributes kT/2energy. We shall soon see that thisresult has importantthermodynamical consequences.Moreover, it is also a result ofPlanck’s black body equation.

Since thermodynamicconsiderations are universal, theresult for a harmonic oscillator, E =kT, must always hold true. Todemonstrate this, we can show thata harmonic oscillator – for examplea mechanical pendulum – is aCarnot machine.

Figure 13: A bob hanging by a rodmoves from the point of maximumenergy, EH,, to the point of minimum

energy allowed by the rod, EL,converting the energy difference, EH –EL, into mechanical movement (work)over and over. This is a periodicCarnot machine.

Pushing the bob by applyingwork, W, it will move up to itsmaximum height, which is

where g is the gravitational forceper unit mass that the earth appliesat the altitude where the pendulumis placed. The bob also carriesenergy when it is at rest. This is apurely potential energy, due to the

gravitational force. Part of theenergy can be extracted byincreasing the length of the rod: anoperation equivalent to reducing thetemperature of the colder reservoirin a classic Carnot machine.

The energy balance of the bob isEH = EL + W. It is possible toconvert the movement of the bobinto work by letting it collide with amechanical gear or a wheel. Thework obtained is ≤ EH – EL. Sincein any harmonic oscillator E = kT,we obtain

which is Carnot’s efficiency. Thisderivation explains the analogybetween the production of work bya waterfall, which interestedCarnot’s father, and Carnot’s heatengine.

Figure 14: It is possible to producework from a waterfall in an amount

equal to the difference in the water’senergy between the higher point, hH,and the lower point, hL. This analogywas known to Carnot and was pointedout in his book.

Appendix A-6

Energy Distribution inOscillators

Planck was well versed inthermodynamics. He was the first touse both Boltzmann’s entropy andClausius’s entropy in his solutionfor one of the most importantproblems in science, concerningblack body radiation, namely, thedistribution of the radiation energyamong the radiation modes. Planckfirst assumed that the energy of the

radiation is quantized, and then heasked how many microstates arepossible if we have P quanta (todaycalled photons) in N states(radiation modes). The answer is:

Figure 15: Planck’s distribution.

Contrary to the Maxwell-Boltzmanngas statistics, in which two atomscannot be in the same state,

Planck’s statistics allows forseveral particles (photons) in thesame state (radiation mode).Therefore, the number ofpermutations is the number of statesplus the number of particles minus 1factorial and not like in Maxwell-Boltzmann case the number of statesfactorial. Why? In the Maxwell-Boltzmann statistics, the firstparticle can be in N states, thesecond in N – 1 states, etc. InPlanck’s statistics, on the otherhand, every particle can be in all Nstates. Therefore, every state can beidentified also by the number ofparticles in it. That means that we

can have N + P differentmicrostates, less one. This can beillustrated by an example: Supposewe have 1 state and 9 particles. Ifwe do not subtract the 1, we obtain10 microstates, instead of 1 whichis the correct answer (there isobviously only one possible way toput 9 particles in one state).

For simplicity’s sake, we shall notfollow Planck’s originalcalculations, but rather derivePlanck’s distribution using theLagrange multipliers technique, aswe did for the Maxwell-Boltzmannenergy distribution derivation. We

denote

Here, n is the number of photons ina state (radiation mode). In theMaxwell-Boltzmann distribution,we called the same ratio p, from theword probability (as it was alwayssmaller than 1). However, here itcan be as large as P. UsingStirling’s approximation, as before,we get

Again, we can write thisexpression as a sum of the radiationmodes:

and again use the constraintregarding the fixed amount ofenergy,

We define the function

f(n) = k{(n + 1)ln(n + 1) –nlnn } + β(Q – nε),

find the maximum,

and obtain

Planck then assumed that theenergy of a particle ofelectromagnetic radiation isproportional to its frequency, ν,namely ε = hv, where h is a constant

today called Planck’s constant.Recall that Therefore, wecan write this highly importantresult:

Thus, the equation for thespectral radiation energy emissionof a black body is given by

Q(v, T) = n(v, T)Nhv.

From this equation, and based onWien’s displacement radiation law,Planck derived the value of h, and

quantum mechanics took off.When n is very large, the amount

of energy that the photon carries ismuch smaller than the averageenergy of the radiation mode, kT,and one obtains

Q(v, T) = NkT,

which is the Rayleigh-Jeans law.When n is very small, one obtainsthe Maxwell-Boltzmann law,

This calculation that Planck madewas one of the most important

calculations of science, so itdeserves a verbal explanation too.Planck made two separateassumptions: (a) that energy isquantized and therefore it ispossible to calculate entropy bymaximizing the number ofmicrostates; and (b) that the amountof energy in a quantum isproportional to its frequency. As weshall see later, a similar calculationcan explain many social phenomenain our life.

Appendix B-1

File Compression

Let us examine a few examples offile compression.

Suppose Bob sends Alice a textfile. Texts files are not randomcollections of words. In everylanguage there are words thatappear more frequently than others(as described by Zipf’s law,discussed in Chapter 5 and inAppendix B-3). For example, inmost texts, the probability of finding

the word “if” is much larger thanthe probability of finding the word“epistemology.” Bob knows thecontent of the file that he wants tosend, and therefore he can calculatethe minimum amount of bitsrequired to transmit this content bycomposing a lookup table tocompress the file. For example, ifBob wants to convey to Alice theday of his visit, he can construct atable in which Sunday=000,Monday=001, Thursday=011, etc.and send her a three-bit file.

Practically, file compression isnot done this way. In order forAlice to decompress any file she

receives, she needs some kind ofuniversal tool. Moreover, forobvious practical reasons, it shouldbe done transparently (without userintervention). Bob too does notwish to prepare and transmit an adhoc lookup table every time hewants to send a message to Alice.This is the reason why most peopleuse a multipurpose compressionalgorithm that works automaticallyon files of the same kind. Forexample, we use a ZIP algorithm tocompress a text file and a JPGalgorithm to compress picture files.JPG compression carries lesscontent than that of the original file

because our brain does not need allthe details that appear in a rawimage taken by a camera tocomprehend a picture. In texttransmission we have to compressthe file in such a way that all itscontent will be fully preserved.

Not everyone is aware of the factthat when we talk over thetelephone our voice is digitized andcompressed in the handset; it is thantransmitted to the listener,uncompressed in his handset, andthe membrane in the handsetvibrates using the decompresseddata in a way that mimics thevibrations of our original voice. All

these operations are donetransparently and automatically.File compression is very importantin engineering because it savesmoney, and many engineers workingin information technology aredealing with it. However, furtherdiscussion of this area is beyond thescope of this book.

There are three stages in anycompression algorithms:

Bob goes through the file lookingfor redundant structures.

The file is rewritten as a randomfile such that the redundantstructures will be much shorter than

their original length. For example,if we have a file whose content isone million “1” digits, the file willbe written as “1 × 1,000,000,” andthen sent to Alice.

Alice receives the compressedfile, decompresses it using her tableand thus reconstructs the originalfile.

Obviously, Bob and Alice musthave the appropriate matchingsoftware. If Bob and Alice areconventional users, they can buycommercial software. Or Bob andAlice may prefer to program thesoftware themselves in order to

achieve higher compressionefficiency for specific data or forthe transmission of classifiedinformation.

When Alice receives an N-bitfile, she obtains N bits ofinformation, even if the file can becompressed into M bits. The reasonis that she has no a prioriknowledge about the file’s content.Indeed, that is the connectionbetween information theory andentropy: entropy is a quantitativeexpression of the lack of knowledge(uncertainty) that an observer of aphysical system has, orequivalently, it is the lack of

information that the receiver of acommunication has. Bob, on theother hand, knows the content of thefile and, if he wishes, he cancompress it and send Alice an M-bit file.

For example, let us assume thatBob has a file that contains P“1”bits and N – P “0” bits. Thenumber of possible different filesis:

By how much can this file becompressed into a random file, M

bits in length, in which Ω = 2M?We can find M from this

equation:

Using the Stirling formula, as wedid in previous appendices, weobtain:

where is the fraction of the1’s in the file. If we use harmonicoscillators for transmission, weobtain that the entropy of the

compressed file is S = kM ln2,where k is the Boltzmann constant.

Obviously, if p is not ½ then M <N, where M is the Shannon limit.

So, by taking an uncompressedfile of length N and compressing itto the Shannon limit, we received ashorter file of length M, which is inthermodynamic equilibrium. WhenAlice receives an uncompressedfile, all she knows is that itsentropy, S, is kNln2. However, infact the file carries only kMln2entropy. Since in general N ≥ M, wecan write,

S ≥ kM ln2,

which is Clausius’s inequality.

Let us examine a numericalexample: Alice is in the USAstudying global warming. Bob, inIsrael, is asked to send Alice thedeviations from the average of themaximum daily temperatures in TelAviv in August. Since Bob andAlice know that the average dailymaximum temperature in Tel Avivin August is 30°C, they agree thatBob will send “1” when thetemperature is higher than or equalto 30°C, or “0” if it is lower. Since

there are 31 days in August, Bobtransmits to Alice a 31-bit file with31 bits of information.

What happens if Alice asks Bobto send her a file in which “1”represents the hottest day in August,and “0” for any other day in thismonth? Alice will again receive a31-bit file that contains 31 bits ofinformation. However, if weassume that there is only one“hottest” day in August, it is clearthat Bob and Alice know that thefile has less information, andtherefore it can be compressed. Thecompression can be done in manyways. For instance, they can agree

that Bob will send only the date ofthe hottest day. How many bits arerequired for sending the date? Thiscan be found from 2M = 31, whichmeans that they need 5 bits. Thetheoretical limit can be found in theexpression for M that appearsabove, namely,

Practically, it is impossible tosend fractions of bits, so Bob mustsend a 5-bit file, which is theintuitive result we got earlier.

This example demonstrates the

confusion that the concept ofentropy can cause. In the first case,Bob sent a 31-bit file. There was noa priori knowledge whether themaximum temperature in a certainday was above or below 30°C.Therefore, the file was in a state ofequilibrium even if only one day (orno day!) had a temperature higherthan 30°C.

In the second case, we did have apriori knowledge that only one daycould have a maximum temperature.Therefore, the amount of uncertaintywas smaller. The file was not inequilibrium and could becompressed to a 5-bit file in which

every bit has an equal probabilityof being either 1 or 0.

Appendix B-2

Distribution of Digits inDecimal Files – Benford’s

Law

Let us calculate the distribution ofdigits in a decimal file thatmaximizes Shannon’s entropy (i.e.,the number of microstates). Weshall calculate the distribution of Poscillators in N sequences, where asequence is a digit, and the numberof oscillators in it is the value of thedigit. According to Planck, the

number of microstates is,

Shannon’s entropy is S = lnΩ.The number of oscillators in asequence is , where n can beany integer from 1 to 9 (we are notinterested in empty sequences).From the Stirling formula we obtainthat

This equation is similar to

Planck’s equation. We designateφ(n) as the number of digits havingn oscillators (which is the numberof digits, n, in the decimal file). Wehave a constraint on the totalnumber of oscillators in the file,namely,

Therefore, the Lagrange equationwill be

The extremum with respect to nis ∂f (n) / ∂n = 0, or

In order to find the relativedistribution of digits regardless ofthe length N of a file, we divideφ(n) by the total number of digits:

In Appendix A-4 we divided theprobability pj of the particles

having energy εj that we got fromthe Lagrange equation by thepartition function Z = Σjpj to obtainthe normalized distribution,

The normalization

here is analogous to the Maxwell-Boltzmann distribution, .

In the present case, β, which is soimportant in classicalthermodynamics disappears, and we obtain

This is Benford’s law.

Appendix B-3

Planck-BenfordDistribution – Zipf’s law

According to Planck, the number ofpossible ways to distribute P linksamong N nodes is

Shannon’s entropy is S = lnΩ.Since the numbers of links in a nodeis given as an integer, we

can write the Lagrange equation asin Appendix B-2, namely,

where φ(n) is the number of nodeshaving n links, and the total numberof links in the net is

The net includes only nodes thathave at least one link (otherwisethey would not be in it). Theextremum of the Lagrange equation

with respect to n is identical to theone obtained in Appendix B-2,namely,

We shall count the total numberof nodes as a function of n and β,namely N =

and obtain the normalizeddistribution

This extremely importantexpression is worth somediscussion. It means that in anoptimal network, the number ofnodes with many links is lower thanthe number of nodes with fewerlinks. Or in other words, only a fewnodes will have many links,whereas many nodes will have tomake do with a few links. This isthe equation responsible for allinequality in life: wealth, socialnetworks, and hierarchicalorganizations such as armies,

corporations, and companies, toname a few.

This derivation yields Zipf’slaw, which states that in manylanguages the most frequent wordappears twice as often as thesecond most frequent one, and thesecond most frequent one willappear twice as often the fourthmost frequent one, etc. Theexpression φ(n) may be rewritten as

In the classical limit n≫ 1, βφ(n)≪ 1. In this case the equation

above can be approximated as

which means that the product of thenumber of links and the number ofnodes (or in general, the number ofparticles multiplied by the numberof boxes where there are manymore particles than boxes) obeysthe rule

This rule is known as Zipf’s law.

Biographical Sketches

Frank Albert Benford (1887–1948)

American electrical engineer andphysicist. Graduate of the

University of Michigan. Benfordworked his entire life at GeneralElectric; his main expertise was inoptic metrology. In 1937 hedeveloped an instrument to measurethe refractive index of glass. Hewrote more than 100 papers andhad about 20 patents to his credit.Benford is mainly known for thelaw that bears his name, concerningthe unequal distribution of digits inrandom lists of numerical data,which was first discovered byNewcomb by observing the wearand tear of logarithmic tables.Benford tried, unsuccessfully, toderive the law that now bears his

name.

Dietrich Braess (1938 –)

German mathematician, professor atRuhr University in Bochum,Germany. In 1986 he observed thatin an optimal user-network, theaddition of a link changes itsequilibrium and reduces theefficiency of the network. A famousexample: when a section of 42nd

Street in New York City wasclosed, instead of the trafficcongestion predicted by experts, the

traffic flow actually improved.Similarly, when a new road wasbuilt in Stuttgart, Germany, trafficconditions worsened, and improvedonly after the road was closed totraffic. This phenomenon is nowcalled “Braess’s Paradox.”

Anton Josef Bruckner(1824–1896)

Austrian musician, considered oneof the most important post-romanticcomposers. He worked for a fewyears as an assistant teacher, gaveprivate music lessons (Boltzmannwas one of his students), and in theevenings, played the violin invillage dances. He studied withSimon Schechter and Otto Kitzleruntil the age of forty. Only in his

mid-forties was his talentrecognized, and he achieved fameonly in his mid-sixties. Hecomposed symphonies and sacredchoral works.

Henry Andrews Bumstead(1870-1920)

American physicist who worked inthe field of electromagnetism. Astudent of Gibbs, he penned Gibbs’seulogy based on his personalacquaintance with him. He was aprofessor of physics at YaleUniversity, and the director of itsSloane Physics Laboratory.

Robert Bunsen (1811–1899)

German chemist who worked at theUniversity of Göttingen andconducted research into theemission spectra of heatedelements. Along with Gustav

Kirchhoff, he used spectroscopicmethods to discover the elementscesium and rubidium. He developedanalytic methods for studying gases,and was one of the pioneers ofphotochemistry. With technicianPeter Desaga, he developed theBunsen burner that is used inlaboratories all over the world tothis very today. The Bunsen burnerproduced hotter, clearer flame thanany previous gas burner. Bunsenalso developed the most effectiveantidote against arsenic poisoning.This research almost led to his owndeath from arsenic poisoning anddid lost him the use of one of his

eyes.

Lazare Hippolyte Carnot(1801–1888)

French educator and politician,Sadi Carnot’s younger brother.From 1815 (Napoleon’s final

defeat) until 1823, he lived with hisfather in exile in Magdeburg,Germany. His early interests lay inliterature and philosophy, until heentered politics. In 1839, 1842, andagain in 1846, he was elected to theChamber of Deputies as deputy forParis, representing the radical leftwing. In 1848 he was appointedMinister of Education, and set outto organize the system of primaryeducation. He proposed a billcalling for free compulsoryeducation, but was forced to resigna short time later on a point ofprinciple – he refused to hold anyposition which required taking an

oath allegiance to Emperor LouisNapoleon. From 1864 to 1869 hewas a member of the republicanopposition. He died three monthsafter his son, Marie François SadiCarnot, was elected to thepresidency of the Republic. Amonghis writings was a two-volumememoire about his father, LazareNicolas Marguerite Carnot.

Anders Celsius (1701–1744)

Swedish astronomer, professor atUppsala University. His father andgrandfather had both beenprofessors of astronomy andmathematics. Celsius studied atUppsala University, where hisfather taught; in 1730 he too beganteaching there, and remained theretill his death from tuberculosis in1744, at the age of only forty two.

Celsius is widely known for themetric temperature scale used inmost of the world (with the notableexception of the United States,where the Fahrenheit scale is stillused). His outstanding contributionwas assigning the freezing andboiling points of water as thebenchmark points of the scale (0° Cand 100° C, respectively), after aseries of meticulous experimentsdemonstrated that the freezing pointof water is independent of latitudeor atmospheric pressure, whereasthe boiling point was dependent onatmospheric pressure. The resultshe obtained were amazingly

accurate, even to modern daystandards. In addition, Celsiusformulated a rule for determiningthe boiling point of water when thebarometric pressure deviated fromthe standard of one atmosphere.Actually, when Celsius proposedhis temperature scale in 1742, itwas upside down to today’s scale :0° was the boiling point of water,and 100° the freezing point. In1745, one year after his death, hisscale was reversed and became thecurrent Celsius scale. It is worthnoting that in 1730, Réaumursuggested another scale, based onthe freezing and boiling points of

water and divided into eightydegrees. There is evidence that in1740 the Réaumur scale wasdivided into 100 degrees and wascalled centigrade, as it is calledtoday; nevertheless, Réaumur’sname was largely forgotten. In1948, the General Conference onWeights and Measures adopted theCelsius scale as the standard, yet in1956, it changed the standard to theKelvin scale, in which 0° K is theabsolute zero, but each unit (calledkelvin) is still equivalent to adegree of the Celsius scale.

Carlo Cercignani (1939-2010)

Italian professor of theoreticalmechanics at the PolytechnicInstitute of Milan. A member of theItalian Academy (Accademia deiLincei) and a foreign member of theFrench Academy of Sciences inParis. The chairman of a number ofinternational companies andcommittees in the field ofmathematics, mechanics and spaceresearch. Won two medals for hisachievements in mathematics, andan honorary doctorate from the

Université Pierre et Marie Curie.He wrote and edited scientific texts,among them a book on Boltzmannentitled The Man Who TrustedAtoms.

Benoît Paul ÉmileClapeyron (1799–1864)

French road and bridge engineerand an expert in the design of steamengines, who initiated and built thefirst railway line between Paris andSaint-Germaine. His work led himto take interest in the efficiency ofthe steam engines and Carnot’swork. In his papers, he presentedCarnot’s work in a straightforward,graphic manner, while furtherdeveloping the reversibilityprinciple. His main contribution tothermodynamics was introducingCarnot’s work, and thus giving ahuge push to the development of thefield. He also served as a professorin the Académie des Sciences in

Paris.

Gabriel Daniel Fahrenheit(1686–1736)

Dutch inventor. Born in Poland to awell-to-do a merchant, but livedmost of his life in the DutchRepublic. At the age of fifteen, afterboth his parents had died, he

became an apprentice to a merchantin Amsterdam. Four years later hebegan working as a glassblower,and by 1714 had invented the firstscientific thermometer, whichcontained alcohol. At some point,he decided to replace the alcoholwith mercury. The scale that waslater named after him was definedonly ten years later, spanning 0 to212 degrees. The 0° degree was thetemperature of a brine solution, 32°was the temperature of a mixture ofice and water, and 100° was thehuman body’s temperature,measured under the arm. He died,still a bachelor, in The Hague,

Netherlands.

James David Forbes (1809-1868)

Scottish physicist and glaciologistthat worked in the areas of heatconduction, seismology and the

study of glaciers. He graduatedfrom the University of Edinburgh,eventually becoming a professorthere from 1833 to 1859, when hebecame president of the UnitedCollege of St. Andrews. Already atthe age of nineteen he had become afellow of the Royal Society ofEdinburgh, and four years later waselected to the Royal Society ofLondon. His research on glaciersbegan in 1840, and after manyobservations in Switzerland andNorway, he surmised that a glacieris an imperfect liquid or a viscousbody, that is pushed along theslopes as a result of the mutual

pressure of its parts. During hisexpeditions, he also measured theboiling point of water at variousaltitudes. His data, published in1857, is known in statistics as “theForbes data.”

Günther HermannGrassmann (1809–1877)

German mathematician andphysicist, known at the time mainlyas a linguist, but today moreappreciated as a mathematician. Apaper on the theory of tides that hepresented in 1840 included the firstmention ever of what is knowntoday as linear algebra, as well asthe concept of vector space. In 1844

he published his masterpiece, abook on the foundations of linearalgebra. In 1853 he published histheory on color mixing – the law ofthree colors – that is today namedafter him. In 1861 he presented thefirst axiomatic formalism ofarithmetics using induction.Because his mathematical work didnot earn any recognition in hislifetime, the disappointedGrassmann turned to linguistics. Hepublished a dictionary, a collectionof folk songs, translations, andmore, and indeed was renown inthis area. Only after his death werehis mathematic methods slowly

adopted, among others, by Gibbs.

Fritz Haber (1868–1934)

Jewish-German chemist, the 1918Nobel laureate in Chemistry for hiswork on the synthesis of ammoniafrom its components. Theimportance of his work to the

chemical fertilizer industry and toagriculture on the one hand, and toexplosives and poison gases on theother, made him one of the greatestchemists of his time. From 1886 to1891 he was a student of Bunsen’s.Before Hitler’s rise to power,Haber, a staunch German patriot,was at the forefront of his nation’sscience.He was called “the father ofchemical warfare” for his work onthe development of chlorine andother poison gasses during WorldWar I. His wife, Clara Immerwahr,also a Ph.D. in chemistry, could notreconcile herself to his obsession

for chemical warfare andcommitted suicide in the garden oftheir home the day Haber left for theEastern Front to oversee the use ofgas against the Russians. For theservices to his country, he wasgreatly honored. In the 1920s,scientists at his institution haddeveloped the insecticide Zyklon Awhich was later upgraded to ZyklonB, the gas used by the Nazis in theirextermination camps. With Hitler’srise to power, Haber was strippedof all his rights and forced to leaveGermany. He worked in Cambridge,England, for a short time and wasconsidering a position at the

Weizmann Institute in Israel, butcould not find his place at either.One year after leaving Germany, in1934, Haber died in exile from aheart attack. He did not live longenough to witness the use to whichhis invention was put to exterminatehis own people, among themmembers of his immediate family.

Otto Hahn (1879–1968)

German chemist, the 1944 Nobellaureate in chemistry for his workon the fission of the atomic nucleus.He worked in close collaborationwith Lise Meitner, a convertedJewess. Their professionalcollaboration began in 1907 andwent on for thirty years; and their

friendship lasted a lifetime. Duringthe First World War, Hahn serveduntil 1916 in Haber’s special unitfor chemical warfare, whichdeveloped, tested and producedpoison gas for military purposes. In1921 Hahn discovered a materialthat he named “uranium z” (theelement protactinium) whosesignificance in nuclear physicsbecame clear later. In 1924 Hahnwas elected a full member of thePrussian Academy of Sciences. In1938 he and Fritz Strassmanpublished results of radiochemicalexperiments that indicated thepresence of lighter elements among

the products of the bombardment ofuranium nuclei by slow neutrons.He sent these baffling results toLise Meitner, who had escaped toSweden from Nazi persecution, andshe correctly interpreted them asnuclear fission. Otto Hahn opposedthe Nazi dictatorship and assistedmany others of his colleagues toflee Germany. He triedunsuccessfully to persuade Planckto enlist well-known professors toprotest publicly the attitude of theNazi regime towards Jewishscientists. As early as 1934, heresigned from the University ofBerlin in protest over the dismissal

of his Jewish colleagues LiseMeitner, Fritz Haber, and JamesFranck, yet remained on as thedirector of the Kaiser-WilhelmInstitute for Chemistry in Berlin(where he served from 1928 until1946). In 1944 he was captured bythe British, who suspected him ofcollaborating on the Germannuclear bomb project, even thoughhe had had no part in it. InNovember 1945, while he was stilldetained in England, he read in anewspaper that he had won the1944 Nobel Prize in Chemistry.Hahn later returned to the then WestGermany, where he again directed

the Max Planck Institute forChemistry (as the Kaiser-WilhelmSociety had been renamed) from1948 to 1960. After World War IIhe became a relentless protester theproliferation of nuclear weapons.

Sir William Rowan Hamilton(1805–1865)

Irish-English physicist, astronomer,and mathematician, who made manycontributions to classicalmechanics, optics and algebra. Hisgreatest contribution was areformulation of Newtonianmechanics, known today asHamiltonian mechanics, which iscentral to electromagnetism andquantum mechanics. In mathematics,

he is best known for discoveringquaternions. Dublin-born Hamiltonshowed a rare aptitude forlanguages and at the age of thirteenalready read Hebrew, classical andmodern European languages, aswell as Persian, Arabic,Hindustani, Sanskrit and Malay.When he was sixteen, he readEuclid’s Elements and Newton’sPrincipia. At the age of seventeenhe began studying at TrinityCollege, Dublin, specializing inmathematics. There, in 1827 – whenhe was barely 22 – and had not yetgraduated, he was appointedProfessor of Astronomy. That same

year he presented a theory of asingle function, known as a“Hamiltonian” that unified andcombined components ofmechanics, optics and mathematicsand is the basis of wave theory. Inmathematics, he developed amaximization technique that,ironically, is now called Lagrangemultipliers, extensively used in theappendices to this book. In 1837 hewas knighted. He presented hisquaternion (an expansion ofcomplex numbers into fourdimensions) in 1852. Manyopposed his idea, but it wasadopted by Tait, and Gibbs also

made use of it. Today, quaternionsare used in computer graphics,cybernetics and signal processing.

Werner Karl Heisenberg(1901–1976)

German physicist, the 1932 Nobellaureate in physics for hiscontribution to quantum mechanics.

Alongside Niels Bohr, Paul Diracand Erwin Schrödinger, he isconsidered one of the foundingfathers of that field. He isespecially known for his uncertaintyprinciple in quantum theory. Inaddition, he contributed to nuclearphysics, the quantum field theoryand particle physics. With the riseof the Nazis to power, he wasinterrogated by the SS followingcriticism from members of theGerman physicists community, buthis name was later cleared. He wasone of the directors of the Germanatomic bomb project, whichfortunately was never realized, and

no one took seriously his claimafter the war that he haddeliberately sabotaged the project.He was arrested at the end of theSecond World War and detained inEngland from 1945 to 1946. Uponhis return to Germany, he becamedirector of the Kaiser-WilhelmInstitute for Physics, which hadbeen renamed later the Max PlanckInstitute for Physics andAstrophysics. Heisenberg directedthe institute until 1970. He died ofcancer in 1976.

Herman von Helmholtz

(1821–1894)

German physician and physicistwho made significant contributionsto the fields of physiology andphysics. In physiology, hecalculated the mathematics of theeye and formulated theories onvision, on visual spatial perceptionand on color vision. In physics, he

is well known for his law of theconservation of energy, and for hiscontributions to bothelectrodynamics and chemicalthermodynamics. His work onconservation of energy in 1847stemmed from his observations ofthe muscle’s metabolism: he soughtto show that energy was not lostduring the movement of a muscle.Based upon the early works of SadiCarnot, Émile Clapeyron and JamesJoule, he calculated therelationships between work, heat,light energy, electrical energy andmagnetic energy, and treated themall uniformly. He became famous in

1851 for his invention of theophthalmoscope, an instrument forexamining the eye’s interior. Healso wrote about auditoryperception and invented theHelmholtz resonator, whichmeasures the intensity of varioussounds. As a philosopher, heexplored the philosophy of science,writing about the relation betweenthe laws of perception and the lawsof nature, on aesthetics, and on thecivilizing power of science. In1858 he was appointed Professor ofAnatomy and Physiology at theUniversity of Heidelberg. Hecontinued, over the years, to probe

deeper into physics, and in 1871 heleft Heidelberg to receive the chairof physics at the University ofBerlin. In 1888 he was appointedthe first director of the Physico-Technical Institute in Berlin, wherehe stayed until his death.

Sir James Hopwood Jeans(1877–1945)

British mathematician, physicist andastronomer, one of the fathers ofstatistical mechanics and radiationtheory. He was a partner in

formulating the Rayleigh-Jeans Lawwhich is the classic approximationof black body radiation. Despitethis, when Planck published his fullsolution to black body radiation,Jeans did not accept it. One of hismost important discoveries iscalled the Jeans length – the criticalradius of an interstellar cloud,which depends on the temperatureand density of the cloud and themass of its particles. A cloudsmaller than the Jeans length willnot have sufficient gravity toovercome the pressure of the gasand will not collapse into a star,whereas a cloud that is larger than

the Jeans length will collapse.Apart from his scientific books,upon his retirement he also wrotepopular science books.

James Prescott Joule(1818–1889)

English physicist, the son of a

brewer. He studied arithmetic andgeometry at the Manchester Literaryand Philosophical Society, wherehe became interested in electricity.He managed the brewery that heinherited from his father for anumber of years until he sold it in1854. During this time he hadregarded his dabbling in science asa hobby, and in 1838 had alreadypublished his first paper onelectricity. In 1840 he discoveredthe law that bears his name today,which calculates the amount of heatproduced by an electrical currentpassing through a resistor. Despitehis achievements, it was difficult

for him to find a place for himselfin the scientific community, wherehe was perceived as a parochialdilettante. In 1843 he published theresults of his research on heat,including his findings on theequivalency of heat and mechanicalwork. This led him to the idea ofconservation of energy, which thenled him on to the first law ofthermodynamics. A short time later,in Germany, Helmholtzacknowledged Joule’s work.Helmholtz’s conclusive statement in1847 of the law of conservation ofenergy brought about a widespreadrecognition of Joule’s contribution.

In 1847, at Oxford University, Joulepresented his theory to MichaelFaraday and Lord Kelvin, yet theyremained skeptic. Some time haspassed before Kelvin acceptedJoule’s authority and collaboratedwith him, among other things, on thedevelopment of a scale of absolutetemperature. His collaboration withKelvin eventually led to thediscovery of what is known now asthe Joule-Thomson effect, amodification in the behavior of anideal gas due to inter-molecularforces in gases. The publication ofthese results contributed to theacceptance of Joule’s achievements

as well as the establishment of thekinetic theory of gases. The metricunit of energy was named after him.

Joseph M. Juran (1904–2008)

Romanian management consultant,whose main contribution was in thearea of quality control; indeed, hewas known as “the father ofquality.” In 1937 Juran extrapolatedthe Pareto Principle (20:80 rule) insuch a way that allowed managersto distinguish between “the vital

few” and “the useful many” in theiroperations. He was also the first tocall this principle “Pareto’s Rule.”From 1951 until his death in 2008,he kept busy with improving theperformance and quality of goodsand services, and wrote extensivelyabout it. He is mainly rememberedas the one who added the humanelement to quality control. In 1979he founded the Juran Institute for thepurpose of doing research andproviding practical solutions toquality control issues in industrialorganizations.

Daniel Kahneman (1934–)

Israeli-born American psychologist,the 2002 Nobel laureate ineconomics. Born in Tel Aviv, helived in France, mainly on the runduring World War II, and thenemigrated to Israel. He studied forhis PhD in psychology at theUniversity of California inBerkeley. In 1961 he joined thefaculty of the psychologydepartment at the HebrewUniversity in Jerusalem. Hisresearch with Amos Tversky onjudgment and decision-makingunder uncertainty earned him the

Nobel Prize (Tversky had diedearlier, but the Nobel Committeesaw fit to note his contribution). Intheir research, they demonstratedthat – contrary to the most basicassumption of economic analysis,namely, that decision-making is arational process – adisproportionately large amount ofweight is attached to small risks, incomparison with the weight givento potential profits; that thepresence or absence of alternativescan change priorities; and that theway in which alternatives aresemantically and mathematicallyframed has an irrational influence

on decision makers. In 1978 he tooka position at the University ofBritish Columbia, Canada; in 1986,at the University of California,Berkeley; and in 1993, at PrincetonUniversity, where he works atpresent. Since 2000, he has alsobeen a member of the Center for theStudy of Rationality at the HebrewUniversity in Jerusalem.

William Thomson, LordKelvin (1824–1907)

British mathematical physicist andengineer. Born in Belfast, he movedto Glasgow when his father wasappointed to the Department ofMathematics at the University ofGlasgow. He received hiseducation in mathematics from hisfather. He began his studies at theUniversity of Glasgow, and laterstudied at Cambridge. His first

paper, dealing with Fourier’sanalysis, was published in 1841. In1846, at the age of only twenty-two,he was appointed Professor ofNatural Philosophy at theUniversity of Glasgow, where heworked on the mathematicalanalysis of electricity and onthermodynamics. In 1848 heproposed an absolute temperaturescale based on Carnot’s theory,used today as the standard scientifictemperature scale. Between 1849and 1852 he published threeinfluential papers on heat theory;however it appears that heoverlooked the connection between

Joule’s findings and Carnot’sprinciple, whereas Clausius didincorporate it in his studies on thesame subject. In 1852 he readJoule’s work, but it took him manyyears to accept it. In 1856 hepublished a paper on electricity andmagnetism that helped leadMaxwell to his monumental theoryof electromagnetism. Surprisingly,Kelvin did not support Maxwell’sideas, and his search for a unifyingtheory only served to distance himfrom Maxwell’s ideas. Some saythat in the first half of his career,Kelvin could not be wrong, and inthe second half, he seemed

incapable of being right! Amongothers, he dismissed atomic theory,opposed Darwin’s theory ofevolution (due to an erroneouscalculation concerning the age ofthe Earth), and rejectedRutherford’s ideas aboutradioactivity. In 1860 he coauthoreda book with Peter Guthrie Tait,Treatise on Natural Philosophy.Thomson made importantcontributions to the laying of thefirst transatlantic telegraph cablebetween Ireland andNewfoundland, Canada, for whichhe was awarded knighthood fromQueen Victoria in 1866, becoming

Sir William Thomson, and in 1892he was dubbed Baron Kelvin ofLargs. His patents for measuringtransmitted signals over cables (themirror galvanometer) andconsultancy services to variouscompanies made him wealthy. Hepublished 600 papers, was electedto the Royal Society in 1851, andwas its president three times. Theunit of the absolute temperaturescale was named after him; thekelvin is equivalent to 1°Centigrade.

Gustav Kirchhoff (1824–

1887)

German physicist, one of the fathersof electrical circuit theory as wellas spectroscopy, and one of the firstscientists to investigate black bodyradiation (in fact, it was he who in1862 coined the term “black body

radiation”). Four different sets ofphysical laws are named after him,in the fields of electric circuittheory, thermal emission,spectroscopy, and thermochemistry.He formulated his circuit laws as aseminar exercise presented in 1845when still an undergraduate student;this later developed into his PhDdissertation. In 1859 he proposedthe law of thermal radiation, and in1861 he proved it. Beginning in1854 he worked at the University ofHeidelberg, in collaboration withRobert Bunsen, with whom hedeveloped spectroscopy anddiscovered the elements cesium and

rubidium. He was part of theacademic and social circle thatsurrounded Helmholtz. At the sametime, Kirchhoff discovered(simultaneously with butindependently of Wilhelm Weber)that the speed of an electric currentin a wire conductor is independentof the substance of the wire, and isalmost identical to the speed oflight. They both interpreted thissimilarity as a coincidence, insteadof taking the next step, as Maxwelldid five years later, when hededuced that light was anelectromagnetic phenomenon. In1875 he received the chair of

theoretical physics at Heidelberg.Kirchhoff was physically disabled,and most of his life used crutches ora wheelchair. Even though hereceived many offers from a numberof universities, he rejected thembecause he was content atHeidelberg. Nevertheless, as theyears progressed, his experimentalwork, from which he derived muchsatisfaction, became wearisome dueto his disability, and therefore heaccepted in 1875 an offer from theDepartment of MathematicalPhysics at Berlin.

Leo Koenigsberger (1837–1921)

German mathematician and ahistorian of science. Known mainlyfor his extensive biography ofHermann von Helmholtz, which hewrote from 1902-1903. He was

born in Posen (today Poznań,Poland), the son of an affluentmerchant. Studied at the Universityof Berlin, and taught in Greifswald,Heidelberg, Dresden and Viennabefore settling back in Dresden,where he taught from 1884 to 1914.Koeningsberger’s main interests layin elliptical functions anddifferential equations.

Gottfried Wilhelm Leibniz(1646–1716)

German mathematician andphilosopher. Despite his enormouscontributions to mathematics,philosophy and many fields ofscience, he was never part of any

academic institution, perhapsbecause his attempts to combinedifferent disciplines were ahead ofhis time. While in Paris in the1670s, he developed the basis ofcalculus. After publishing a numberof papers on the subject, IsaacNewton claimed that Leibniz stolefrom him the basic ideas of thisimportant branch of mathematics,and a bitter quarrel eruptedbetween them. The basic facts arethat Newton had indeed developedcalculus before Leibniz did, butpublished his findings only decadeslater. Today it is accepted that thetwo worked entirely independently

on the development of calculus;nevertheless, the mathematicalnotation used today are Leibniz’s,not Newton’s. Leibniz traveledwidely in Europe and was made amember of both the FrenchAcademy of Science and in theBritish Royal Society. But as aprotégée of the Duke of Brunswickin Hanover, he had to return there,serving as court Counselor for manyyears. His intellectual achievementsare too many to relate here, and it ispossible that a great many of themare still not known (presumablyburied in the cellars of the RoyalLibrary in Hanover). In

mathematics, aside from his workon calculus, Leibniz contributed tothe theory of probability andimproved mechanical calculators.In physics, he studied the subject ofkinetic energy, potential energy andmomentum. In theology, hepublished a book called Théodicée,in which he grappled with thequestion of why the world, thecreation of a perfect God, is notperfect. He also contributed tobiology, geology, medicine,psychology, linguistics and thesocial sciences, which were then intheir infancy. Among all those otherthings, he was also the first

European to take an interest in theteachings of Confucius. Leibniztook an active part in the struggle ofthe Duke of Hanover to assume theEnglish crown, but when the Dukebecame King George I, he showedno gratitude (actually, he joinedNewton’s supporters in the bitterpriority dispute over calculus), andLeibniz was forgotten by all by thetime he died. Not one of thedistinguished people with whom hehad been associated during his lifetook the trouble to attend his funeralin 1716. The Berlin Academy,which he founded and of which wasthe first president, the Royal

Society of London and the FrenchAcademy of Science all ignored hisdeath. Today he is considered oneof the greatest thinkers of all times.

Hendrik Antoon Lorentz(1853–1928)

Dutch physicist, the 1902 Nobellaureate in Physics, together withPieter Zeeman, for the discoveryand theoretical explanation of theZeeman Effect (the splitting of aspectral line in the presence of amagnetic field). Zeeman, Lorentz’sstudent, discovered the effect in1896, and Lorentz provided thetheoretical explanation. Lorentz ismainly remembered for hismathematical transformation thatappears in the special theory ofrelativity. Indeed, Einstein andPoincaré called the equations ofspecial relativity “the LorentzTransformations,” and the special

theory of relativity was initiallycalled the Lorentz-Einstein Theory.

Josef Loschmidt (1821–1895)

Austrian-Bohemian physicist andchemist, son of farmer. Thanks to apriest who persuaded his parents to

send him to high school, he went onto study at Prague’s Karl FerdinandUniversity. His first paper, in 1861,received little attention from thechemistry community. At the age of44, while still a high schoolteacher, he calculate the size of amolecule, solving one of the mostimportant problems of his time. Todo so he used the kinetic theory ofgases innovatively to derive a valuefor the diameter of a molecule(which then made possible thecalculation of Avogadro’s number).His work was done during a periodwhen the kinetic theory and theexistence of molecules were still

controversial. The calculation ofAvogadro’s number (or, as it wascalled in German-speakingcountries, Loschmidt’s number)finally won him recognition. In1866, thanks to Josef Stefan,Loschmidt was given a post at theUniversity of Vienna, and in 1872he became a full professor. JamesClerk Maxwell used Loschmidt’sdata to calculate the diameters ofvarious gas molecules. Even thoughhe was a close friend of Boltzmann,Loschmidt questioned hiscolleague’s attempt to derive thesecond law of thermodynamicsfrom kinetic theory, that eventually

led to Boltzmann’s statisticalexpression of entropy. The leadingchemists of his time rejected orignored his pioneering work onchemical structure, but Stefan,Boltzmann, Maxwell and otherleading physicists accepted hisideas and used them. In Boltzmann’sopinion, Loschmidt’s excessivemodesty was the reason that he didnot receive the appreciation andhonor he deserved.

Ernst Mach (1838–1916)

Austrian-Bohemian physicist andphilosopher. One of the fathers offluid mechanics, whose name isfamiliar today mainly because theunit for the speed of sound wasnamed after him. His father, agraduate of the University ofPrague, was a high school teacher.Until age 15, Mach was taught by

his parents, then he went to agymnasium, and subsequently to theUniversity of Vienna, where hestudied mathematics, physics andphilosophy. In 1860 he received hisPhD, and in 1864 he was appointedProfessor of Mathematics at theUniversity of Graz. In 1867 he wasappointed professor ofexperimental physics at KarlFerdinand University in Prague,where he remained for 28 years. In1901 he retired from the universitybecause of his ill health. His workfocused mainly on interferometry,refraction, polarization andreflection of light (the Mach-

Zehnder interferometer is still inuse today in various fields). Thisled him to studies on supersonicvelocities, and he published a paperon this subject in 1877. He alsodeveloped a highly influentialbranch of philosophy of science,which was known later as logicalpositivism.

Lise Meitner (1878–1968)

Austrian physicist. Born into aJewish family, she converted toLutheranism as an adult. Shespecialized in nuclear physics andradioactivity and worked for thirty

years with Otto Hahn, the chemistand 1944 Nobel laureate, onresearch that led to nuclear fission.Even though she was alongstanding, active partner inHahn’s research and contributed thetheoretical calculations proving thatthe nucleus of the atom indeed splitsunder certain conditions, the NobelCommittee decided not to awardher the prize together with Hahn.Until 1934 Meitner held a seniorposition on the faculty of sciencesin the University of Berlin, then shewas dismissed because of herJewish origin; yet as an Austriancitizen, she was allowed to

continue her work with Hahn at theKaiser-Wilhelm Society. But whenAustria was annexed to Germanyshe was forced to escape andmoved to Stockholm. Hahnmaintained his correspondence withher, and while in Sweden, she wasthe first to interpret the data he senther as evidence of nuclear fission.In 1949 she received Swedishcitizenship, and later moved to theUnited States, but had no successfinding a suitable academic positionand retired from research. In 1997,element 109, meitnerium, wasnamed in her honor.

Stanley Milgram (1933–1984)

American social psychologist. Inthe 1950s he published the “sixdegrees of separation” study, whichdeals with the necessary number oflinks required to connect twopersons who do not know eachother. He also earned publicity for

his psychological studies that testedthe readiness of individuals toabuse others while following theorders of an authoritative figure (themotivation for these experimentswas his desire understand thebehavior of individual Nazis duringthe Holocaust). Milgram studiedpolitical science at Queens College,New York, and then went on tostudy for his Ph.D. at theDepartment of Social Psychology atHarvard University. He receivedhis doctorate in 1960. Hiscontroversial studies raised manyethical issues, in part because hisexperiments generated highly

stressful conditions that weresometimes turned out to have beentraumatic for his subjects. At thesame time, there is no dispute aboutthe importance of his findings.Between 1963 and 1967 he was alecturer at Harvard, but neverreceived tenure because of thecontroversy surrounding hisresearch. From 1967 to 1984 hetaught at New York University. Healso conducted research onviolence in television, urbanpsychology and more. Milgram diedof a heart attack at the age of 51.

Walther Nernst (1864–1941)

German physicist and chemist, the1920 Nobel laureate in chemistryfor his pre-WWI work onthermodynamics andphotochemistry, in which heformulated the third law ofthermodynamics. He was one of the

first scientists involved in chemicalwarfare research during World WarI. He joined Ostwald in Leipzig,where he did important work. In1894 he decided to accept an offerfrom Göttingen, where he foundedand served as director of theInstitute of Physical Chemistry andElectrochemistry. In 1905 he wasappointed Professor of Chemistry,and later of physics, at theUniversity of Berlin. He advocatedfor the military advantage to begained from chemical weaponryand suggested introducing certaincompounds, for example tear gas,into bullets in ordinary use. The

experiment was a failure, and heleft chemical weaponry andreturned to his laboratory in Berlin.In 1924 he was appointed directorof the Institute for Physics andChemistry in Berlin and remainedthere until his retirement in 1933.Both his sons were killed in theFirst World War. He spoke warmlyof the open atmosphere thatBoltzmann created in Graz, and thetime he was willing to spend withoutstanding students.

Simon Newcomb (1835–1909)

Canadian-American astronomer andmathematician, who contributedmuch to economics, statistics, andeven to science fiction literature.

As a child he received no formaleducation, and learned mainly fromhis father, who was a teacher. Whenhe moved to the United States in1854, he worked for a few years asa teacher, and in his spare timestudied a wide range of topics, inparticular mathematics andastronomy. In 1857 he went to theLawrence Scientific School atHarvard University, graduating witha B.Sc. in 1858. In 1861 he wasappointed Professor of Mathematicsand Astronomy at the United StatesNaval Observatory, WashingtonD.C. Following that, he took thepost of professor of mathematics

and astronomy at Johns HopkinsUniversity. His astronomicalmeasurements from that time cameto be international standards.Between 1878 and 1883 he workedon measuring the speed of light(part of the time with AbrahamMichelson, who was Helmholtz’sstudent, and who received theNobel Prize for his work in thefield in 1907). In 1881 hediscovered the statistical law of theuneven distribution of digits knowntoday as Benford’s law. He died ofbladder cancer.

William Nicol (1768–1851)

Scottish physicist and geologistwho, in 1828, invented the firstdevice to polarize light, todaycalled the Nicol Prism. Not much isknown about him, other than the facthe lectured in natural philosophy atthe University of Edinburgh andlater retired and lived in isolationin Edinburgh. He conductednumerous studies on fluidinclusions in crystals and on themicroscopic structure of fossilwood, but until 1826, he had notpublished anything. The prism heinvented allowed him to extend his

research on the refraction andpolarization of light, and was laterused to study the optical activity oforganic compounds, but again, thelate publication of his discoveriesdelayed their use by about fortyyears. In 1815 he developed amethod for preparing especiallythin sections of rocks and crystals,that allowed for the first time theobservation of the internal structureof the minerals using transmittedlight, instead of reflected light. TheDorsum Nicol ridge on the moonwas named after him.

Wilhelm Ostwald (1853–1932)

German chemist, born in Riga, the1909 Nobel laureate in Chemistryfor his work on the catalysis,reactions’ rate and chemicalequilibrium. He is considered thefather of physical chemistry, and

among other achievements, hecoined around 1900 the term“mole” (the molecular weight of asubstance expressed in grams).Along with Ernst Mach and others,he opposed atomic theory, althoughironically, the concept of “mole”was directly related to it (a mole ofany substance has exactly the samenumber of molecules). In additionto chemistry, he engaged inphilosophy and painting, and wasvery interested in the theory ofcolors. He was also an activemember in the World EsperantoAssociation. Between 1875 and1881 he worked at the University of

Tartu, Estonia, and from 1881 to1887 at the Riga Polytechnicum. In1877 he moved to Leipzig, wherehe worked until his death.

Jules Henri Poincaré (1854–1912)

French mathematician andtheoretical physicist who madesubstantial and fundamentalcontributions to pure and appliedmathematics, including topology,and mathematical physics. One ofthe fathers of topology, heformulated one of the most famousproblems in this field, on thecollapse of spherical universes –the Poincaré conjecture, that wasproven in 2002 by the Russian-Jewish mathematician GrigoriPerelman. Poincaré was the first todescribe a chaotic deterministicsystem, and thus laid the foundationfor modern chaos theory. He

presented the modern principle ofrelativity, and was the first toformulate the Lorentztransformations in their modernsymmetrical form. He discoveredvelocity transformations pertainingto the theory of relativity which arestill in use. The Poincaré group,used in mathematics and physics,was named after him. Poincaréstudied mathematics at the ÉcolePolytechnique (founded by LazareCarnot), then went on to studymathematics and mine engineeringat École des Mines, after which hebecame a mining inspector and alecturer in mathematics at Caen

University, where he taught for twoyears, but did not earn any praisefor his teaching style. His PhDthesis was in the field ofdifferential equations. In 1881 hejoined the faculty of the Sorbonne,where he stayed for the rest of hiscareer, holding the chairs ofPhysical and ExperimentalMechanics, Mathematical Physicsand Theory of Probability, andCelestial Mechanics andAstronomy. In 1887, at the age of32, Poincaré was elected to theFrench Academy of Sciences,whose president he became in1906. Already in 1888 he won a

mathematical competition held bythe Oscar II, King of Sweden, for apartial solution to the problem ofhow stable is the solar system (thefamous three-body problem). Henever left the realm of mining,becoming the chief engineer ofFrench mines in 1893, and later onnational chief inspector. Other thanhis numerous contributions to manyand varied subjects, he wrotepopular books on mathematics andphysics. In contrast with BertrandRussell, who was convinced thatmathematics was a branch of logics,Poincaré believed that intuition wasat the heart of mathematics. He

would solve problems in his head,and only later write the solutionsdown on paper. In 1912, at the ageof 58, he died after prostate surgery.

John William Strutt, LordRayleigh (1842–1919)

English physicist, the 1904 Nobellaureate in physics for hisdiscovery of the element argon. Hisinitial work was concerned withoscillators and optics. Later on heworked in many field in physics:sound, wave theory, colorperception, electrodynamics,electromagnetics, the scattering oflight, flow, hydrodynamics, gasdensity, viscosity, capillary action,elasticity and photography. Hisaccurate experimental data led himto determine standards forelectrical resistance, electriccurrents and electromotive forces.His final efforts focused on

electrical and magnetic problems.He explained the phenomenon oflight scattering as a function ofwavelength, now called theRayleigh scattering. A mathematicalmethod for calculating frequenciesis also name after him, as are twocraters, one on the moon and one onMars. He inherited the title of baronin 1873 upon the death of his father.Between 1879 and 1884 he was theCavendish Professor of Physics atCambridge, taking up the post afterMaxwell vacated it. In 1884 he leftCambridge and continued hisexperimental work at his estate inEssex. From 1887 to 1905 he was

Professor of Natural Philosophy atthe Royal Institution of GreatBritain, as Tyndall’s successor.Between 1905 and 1908 he servedas president of the Royal Society.

Georg Friedrich BernhardRiemann (1826–1866)

German mathematician who mademany important contributions toanalytic and differential geometry,some of which paved the way forthe development of the generaltheory of relativity. His father was aLutheran pastor. From a very youngage Riemann showed great interestin mathematics, and while in highschool he read a huge volume byLegendre (900 pages) in six days.In 1846, when he was 19, he beganstudying philology and theology atGöttingen University, with an aim tohelp his family’s finances, butalready in that first year he obtainedhis father’s approval to study

mathematics. He studied elementarymathematics courses with Gauss,and later did his PhD under him. Hewas also much influenced by Weberduring the year and a half heworked as his teaching assistant. Healso spent two years in Berlin,where he studied under Jacobi andDirichlet. As part of his obligationsas a lecturer, he had to give threelectures, which went on to becomethe foundations for moderngeometry – Riemannian geometry –which was to provide, as mentionedabove, the basis for the generaltheory of relativity. In 1859 hebecame chairman of the

mathematics department inGöttingen. Riemann was the first topropose the use of more than threeor four dimensions to describephysical reality. Throughout his lifehe spent time on efforts tried toprove mathematically the truth ofthe Scriptures. He died in 1862from tuberculosis, survived by hiswife and daughter. Thehousekeeper, upon hearing about hisdeath, threw away all his papers; itis possible that she thus destroyedthe proof for what is today calledthe Riemann hypothesis on thesource of prime numbers. To thisday, nobody has been able to prove

it, and it is considered one of themost important unsolved problemsin mathematics.

Arthur Schopenhauer(1788–1860)

German philosopher, known for hispessimism, pacifism andcosmopolitan outlook. The son ofan affluent merchant family, he

became financially independent atage 21, having inherited his father’sfortune. In 1809 he went to studymedicine at Göttingen University,but after two years left to studyphilosophy in Berlin. He receivedhis Ph.D. from Jena University. In1820 he became professor atUniversity of Berlin. He adamantlyopposed Hegel’s philosophy, anddeliberately scheduled his lecturesto coincide with Hegel’s. He didnot have much of an audience. Henever married, and had no contactwith his mother. He was prone toanxiety, and some claim that thisinfluenced his pessimistic outlook.

Schopenhauer claimed that the Willis the essence of things, coined theterm “unconscious” and claimedthat the sex drive was the basicexpression of the Will. His ideasinfluenced Freud. He admiredgenius, because he saw in it arelease from involvement in theworld, from passion and frominterests. He believed that Naturewas the highest aristocrat, and thatthe differences in social status thatnature proscribes are much greaterthan that of the society of any nationbased on birthright, title, means orstatus.

Erwin Schrödinger (1887–1961)

Austrian theoretical physicist, the1933 Nobel Prize laureate inphysics for the equation named afterhim, which is considered one of thegreatest achievements of 20th

century science. One of the fathersof quantum mechanics (whose basic

concept, the quantum, was a resultof black body radiation research).Schrödinger’s equationrevolutionized quantum mechanics,and consequently chemistry andphysics. In 1927 Schrödingersucceeded Planck at the Universityof Berlin, but in 1933, disgustedwith the Nazi regime, he decided toleave Germany. He became aFellow of Magdalene College atOxford University, and a shortwhile later won the Nobel prize(along with Paul Dirac). He did notfit in at Oxford, partly because ofhis unconventional lifestyle (helived in a ménage à troi). In 1934

he visited Princeton University, andalthough offered a permanentposition, he did not accept it, again,perhaps, because of his lifestyle.He finally accepted an offer in 1936from the University of Graz inAustria. After the Anschluss in1938, Schrödinger published astatement recanting his previousletter of opposition to the Naziregime. He later regretted this, andeven personally apologized toEinstein. Nevertheless, he waseventually fired from the universityfor his “political unreliability” andleft the Third Reich. In 1940 heaccepted the position of Director of

the School for Theoretical Physicsin Dublin, Ireland, where heremained for 17 years. Hecontinued his wild lifestyle,including involvement with femalestudents, and fathered two childrento two Irishwomen. In 1944 hewrote one of the first books on theorigin of life, What is Life?, whichincluded a discussion of negativeentropy and greatly influenced thediscoverers of the structure ofDNA, Francis Crick and JamesWatson. Schrödinger retired in1955, returned to Vienna in 1956,and died there from tuberculosis in1961.

Arnold Sommerfeld (1868–1951)

German theoretical physicist, one ofthe pioneers of atomic and quantumphysics. He studied mathematicsand physics in Königsberg, East

Prussia, and remained there afterreceiving his PhD in 1891. FelixKlein, his doctoral advisor,persuaded him to concentrate onapplied mathematics. In 1897 hetook over Wilhelm Wien’s post atthe Clausthal-Zellerfeld Academy.At Klein’s request, he undertook toedit the fifth volume of hisEncyklopädie der mathematischenWissenschaften mit Einschluss ihreAnwendungen, an effort that wouldconsume some 28 years. From 1906on, he was professor of physics andDirector of the Theoretical PhysicsInstitute at the University ofMunich. He had a special ability to

discover potential talents, and manyof his students –Wolfgang Pauli,Werner Heisenberg, Hans Bethe,Peter Debye, Linus Pauling, andIsidor I. Rabi – became Nobellaureates. He had warm, informalrelationships with his students, andencouraged open, productivescientific discussions.Sommerfeld’s mathematicalcontribution to Einstein’s specialtheory of relativity helped it winover skeptics. One of his best-known accomplishments was amodel of a stable atom that hedeveloped with Niels Bohr. Despitebeing nominated for the Nobel Prize

a grand total of 81 times (anachievement in itself), he neverwon one. Sommerfeld died in 1951in Munich from injuries suffered ina traffic accident while walkingwith his grandchildren.

Johannes Stark (1874–1957)

German physicist, the 1919 Nobellaureate in physics for hisdiscovery of the Stark effect – thesplitting of spectral lines in electricfields (similar to the Zeeman effect,which was explained by LordRayleigh). He studied physics,mathematics, chemistry andcrystallography at the University of

Munich and finished his PhDstudies in 1897. He worked in anumber of universities (Hanover,Aachen, and Greifswald). Hefocused on three main areas:electric currents in gases, analyticspectroscopy, and chemicalvalance. From 1933 until hisretirement in 1939, he was thepresident of the Physical-TechnicalSociety of Berlin. He was amember of the “German Physics”movement – a Nazi-inspired groupof scientists that utterly rejectedtheoretical physics in general andthe “Jewish” theory of relativity inparticular – even though previously,

in 1907, as an editor of theRadioactive and ElectronicAnnual, he had written a glowingreview of Einstein’s (barely knownthen) paper on the principle ofrelativity. Under the Nazi regime,his goal was to become the Führerof German physics. He came outstrongly against Einstein and otherJewish scientists, and even againstWerner Heisenberg – who was nota Jew, and participated in Hitler’satomic program, but had “sinned” inhis defense of Einstein’s theory ofrelativity, and was thus regarded asa “white Jew.” Stark alsothreatened Max von Laue, and

demanded that he toe the party line“or else.” As he saw things, onlypure-blooded Arians could holdscientific positions in Germany,because Jewish scientists lackedthe attitude and the creativitynecessary for work in the naturalsciences. After the war he didresearch in his private laboratory inhis country home in Upper Bavaria.

Joseph Stefan (1835–1893)

Austrian physicist, mathematicianand poet. Both his parents wereSlovenians of the lower middleclass: his father was a miller andbaker, and his mother amaidservant; both were illiterate,

and married only when he was 11years old so that their son would beallowed to study at the gymnasium(illegitimate children could notattend school). His talents wereapparent already as a boy, and histeachers recommended he continuehis studies. He finished first in hisclass, and considered joining theBenedictine order, but his interestin mathematics and physicsprevailed. In 1857 he graduated inmathematics and physics from theUniversity of Vienna. During thattime he also published a number ofpoems in Slovene, but abandonedthat endeavor after his work was

harshly criticized. He taught physicsat the University of Vienna, becamedirector of the Physical Institute in1866, and was elected Vice-President of the Vienna Academy ofSciences. He is mainly rememberedfor discovering the relationbetween the intensity of black bodyradiation and its temperature. Stefanderived his law from experimentalmeasurements made by Irishphysicist John Tyndall. In 1884Boltzmann, one of Stefan’s students,derived the law theoretically. Thisis known today as the Stefan-Boltzmann law. Using his law,Stefan was the first to calculate the

temperature of the sun’s surface.Stefan devoted his life to science,often sleeping in his laboratory, andeven though it did not leave himmuch time for a social life, he wasmuch admired by his students,regarded as a scientist who waseasy to talk to, pleasant andencouraging. He married when hewas 56, but died of a heart attackone year later.

Sir George Gabriel Stokes(1819–1903)

Irish mathematician and physicistwho made many importantcontributions to fluid dynamics (theNavier-Stokes equations), opticsand mathematical physics (Stokes’stheorem). He was secretary, andlater president, of the RoyalSociety. Stokes was born in Irelandand pursued his academic studies at

Cambridge. He graduated in 1837,and began publishing significantwork already in 1840. Stokes had arare combination of richmathematical knowledge andexperimental skill, both of which hemade much use of. Most of his workwas concerned with waves and thetransformations imposed on themduring passage through differentmedia. He investigatedfluorescence, among otherphenomena, and in one of hisexperiments discovered thatultraviolet light can pass throughquartz (although not through glass).In other experiments, he helped

determine the composition ofchlorophyll. His broad outlookallowed him to understand andappreciate contributions of otherphysicists, such as Joules, beforethe scientific community hadaccepted their theories. It may benoted that much of his research, forexample in spectroscopy, was notpublished in his lifetime. Due to hisfame he was asked to investigatethe notorious train accident knownas the Dee bridge disaster in 1847.He discovered that the reason forthe accident was the use of castlead for the supporting beam, whichcollapsed under pressure. As a

result of this discovery, manybridges in Britain were demolishedand then rebuilt. For his work andpublic activity he was awardedmany honors, including the title ofbaronet.

Peter Guthrie Tait (1831–1901)

Scottish physicist; classmate andgood friend of James ClerkMaxwell. He is known mainly forthe physics book he wrote with

Kelvin in 1860. He was one of thepioneers of thermodynamics, and in1865 wrote a book on the history ofthis science.

Sir Joseph John Thomson(1856–1940)

British physicist, the 1906 Nobellaureate in physics for discoveringthe electron and his work on theconduction of electricity in gases.Born in Scotland, he studied

engineering at Manchester and in1876 moved to Trinity College,Cambridge. By 1884 he had alreadyreplaced Lord Rayleigh as theCavendish Professor of Physics. Inthe same year he was elected to theRoyal Society and later served asits director (1916–1920). He was agifted instructor and seven of hisassistants, and also his son, werethemselves Nobel laureates.Thomson’s earthshaking discoveryof the electron was published in1897. He was knighted in 1908, andlater became the Master of TrinityCollege until his death in 1940. In1913 he discovered the existence of

isotopes of stable elements andpaved the way for the developmentof the field of mass spectroscopy.He is buried in Westminster Abbey,the resting place of many a greatBriton, including Sir Isaac Newton.

Robert Henry Thurston(1839–1903)

American engineer, inventor andlecturer. One of the first graduatesof Brown University’s School ofEngineering. In 1885, when he was46, he took over as director ofSibley College of mechanicalengineering at Cornell University,turning it into the best engineeringcollege in the United States. Today

he is mainly known for histranslation of Carnot’s paper on themotive power of heat.

August Toepler (1836–1912)

German physicist, known for hisexperiments in electrostatics.Graduated in chemistry in 1860 andlater turned to experimentalphysics. After teaching at thePoppelsdorf Academy and at the

Polytechnic Institute of Riga, hewas appointed a professor at theUniversity of Graz in 1868, wherehe established a new physicsinstitute. In 1876, when he left Grazto become director of the PhysicsInstitute at the Dresden TechnicalUniversity, his former position wasgiven to Boltzmann, who acceptedit eagerly, since he could now teachphysics instead of mathematics.Toepler remained in Dresden untilhis retirement in 1900. In 1865Toepler applied Foucault’s knife-edge test, initially developed toexamine lenses, to telescopemirrors and to the examination of

the flow of liquids or gases. Themethod is called today schlierenphotography, or the schlierentechnique. Using this technique, hewas the first successfully to makeacoustic waves “visible.” Thismethod is important in high-speedcinematography and in windtunnels, and is used to even today.Toepler invented other instruments,including a mercury pump that bearshis name.

Amos Tversky (1937–1996)

Israel-born American psychologist,

one of the pioneers of research inthe field of cognitive bias, incollaboration with DanielKahneman. His father was aveterinarian and his mother was amember of the Israeli parliament.During his service in the Israelarmy, he was decorated for savingthe life of a comrade. Afterreceiving his doctorate from theUniversity of Michigan, he returnedto Israel in 1965, taking a positionat Hebrew University in Jerusalem.In 1978 he moved to StanfordUniversity in California, where hemarried a professor of psychologyat the same university. After

Tversky’s death, Kahneman, alongwith Vernon Smith, was awardedthe Nobel Prize in Economics in2002, to a large extent for theresearch that had been done withTversky. Tversky did not receivethe prize, as it is not awardedposthumously. Yet exceptionally, hisname was mentioned by the NobelCommittee as Kahneman’s partnerin the prize-winning work onjudgment and decision-makingunder uncertainty. Their paper, “TheFraming of Decisions and thePsychology of Choice,” was amajor breakthrough. In 1982 theypublished a book (along with Paul

Slovik), Judgment UnderUncertainty. Tversky made majorcontributions in other areas ofpsychology. He died of a metastaticmelanoma.

Max von Laue (1879–1960)

German physicist and the 1914Nobel laureate in physics for thediscovery of the diffraction of x-rays in crystals. He also contributedto the fields of opticscrystallography, quantum theory,superconductivity, and the theory ofrelativity. He was instrumental inreestablishing science in Germanyafter World War II. Von Lauestudied under Planck and wasfriends with Einstein. In Berlin, heworked on entropy in radiation andon the thermodynamic significanceof the coherence of light waves.During Hitler’s reign, he secretlyhelped Jewish scientists emigrate

from Germany, and openly opposedthe regime’s policies. At a physicsconvention in 1933 he compareddubbing the theory of relativity as“Jewish physics” to the persecutionof Galileo; he blocked themembership of Nazi physicistJohannes Stark (the 1919 Nobellaureate in physics) to the PrussianAcademy of Science; he comparedFritz Haber’s exile to Themistocles’expulsion from Athens; and more.When the Nazis invaded Denmark,Hungarian chemist George deHevesy dissolved von Laue’s andJames Franck’s Nobel Prize goldmedals to prevent the Nazis from

stealing them. At the end of the war,the Nobel Society recast the medalsfrom the original gold.

Julius Robert von Mayer(1878–1814)

German physicist and physician,

one of the fathers ofthermodynamics. In 1841 heformulated the law of conservationof energy (known today as the firstlaw of thermodynamics: energy canneither be created nor destroyed.)Upon the publication of Joule’spaper on the conversion ofmechanical energy into heat in1843, von Mayer claimed to havediscovered this phenomenon first.And indeed, wrote a paper on theconversion of mechanical work intoheat in 1842 and sent it to Annalende Physik, which was turned down.He rewrote the paper (adding,among other things, claims that

plants convert light into chemicalenergy) and published it in adifferent journal – in the field ofchemistry and pharmacology. Thescientific establishment, includingHelmholtz and Joules, were wary ofhis discoveries. When Joulesclaimed priority, and after two ofhis children had died one after theother in 1848, Mayer threw himselfout of his third-floor window in1850 and was hospitalized for atime in an asylum for the insane.Only in 1862 did he finally win therecognition he deserved, whenphysicist John Tyndall presented hiswork at the Royal London

Institution. In 1867, he wasawarded personal nobility inGermany, adding the prefix “von” tohis name. He died of tuberculosis in1878.

John von Neumann (1903–1957)

One of the greatest mathematiciansof the twentieth century, the son of aJewish family in Hungary, where he

received his education in chemistryat the University of Budapest. In1930, upon the death of his father,he immigrated to the United States.He made significant contributions tomany areas of science andengineering. Besides originating theidea of computer memory, he andOscar Morgenstern introducedgame theory. In addition, he was thefather of the operator theory inquantum mechanics, and alsocontributed to the development ofthe nuclear bomb; unlike many ofhis Manhattan Project colleagues,he went on to work on “super” – thethermonuclear bomb. Already at the

age six he could divide an eight-digit numbers in his head. Aftercompleting his studies at theUniversity of Budapest, he got adegree in chemical engineering inZurich and in Berlin. He wrote hisPhD dissertation in mathematics in1928. By the age of 25, he hadalready published ten importantpapers. In 1930 he was invited tovisit Princeton, and in 1933 he wasone of the first four members, withEinstein and Gödel, at the Instituteof Advanced Study there, where heremained until his death. Between1936 and 1938 Alan Turing was astudent at the Department of

Mathematics in Princeton; vonNeumann invited him to stay on ashis research assistant, but Turingchose to return to Cambridge.During the war, his vast knowledgein hydrodynamics, ballistics,metallurgy, game theory, andstatistics contributed to thedevelopment of mechanical devicesfor computation. Throughout thewar years, he worked as aconsultant to a number of nationalcommittees. Afterwards he focusedon developing the computer at theInstitute of Advanced Study. Hecontinued working with the LosAlamos Group, and further

developed computer capabilitiesfor the solution of the problemsinvolved in building the hydrogenbomb. Since he was dissatisfiedwith the computers of his time, hereorganized the logical structure ofthe computer and added to itinternal memory. In 1955 hedeveloped either bone or pancreaticcancer, probably due to exposure toradiation from the atomic bomb testduring the war, and died a year andhalf later. On his deathbed, to theastonishment of his friends, heasked for a catholic priest.

Wilhelm Eduard Weber(1804–1891)

German physicist who madeimportant contributions in the fieldsof magnetism and electricity. HisPhD dissertation, presented to theUniversity of Halle in 1822, dealt

with the acoustic theory of reedorgan pipes. He became a lecturerat Halle and remained there until1831, when, upon therecommendation of Gauss, he wasappointed professor of physics atthe University of Göttingen. In hiswork with Gauss, he preparedsensitive magnetometers and otherinstruments to measure both directand alternate current. In 1833 hepreceded Samuel Morse in theconstruction of an electricaltelegraph 3,000 meters long thatconnected his physics laboratory toGauss’s observatory; this was theworld’s first working telegraph. He

invented the electrometer andprecisely defined the unit ofelectrical resistance. Two of hisbrothers, Ernst Heinrich (whoformulated Weber’s Law inpsychology that deals withphysiological sensitivity toperceived changes in stimuli) andEduard Frederic, were well-knownscientists in their own right in thefields of anatomy and physiology. In1837, with six of his colleagues, hewas dismissed from GöttingenUniversity for signing a petitionprotesting the abolition of theliberal constitution by the Duke ofHanover. He continued to do

research at Göttingen, without aformal position, but then he wasappointed professor of physics atLeipzig in 1843. In 1849, hereturned to his former position inGöttingen. In 1871 he suggested thatatoms are comprised of positivecharges surrounded by negativecharges, and that the application ofan electric potential on a conductorwill cause the negatively chargedparticles to move from one atom tothe next. He gave similarexplanations for thermalconductivity and thermo-electricity.At the time, most scientists did notbelieve in the existence of atoms.

The unit of magnetic flux, the weber(Wb), was named after him.

Hermann Weyl (1885–1955)

German mathematician who madehis mark in the fields of symmetry,logic and numbers theory, one of themost influential mathematicians ofthe 20th century. Among other

achievements, he was one of thefirst who connected the ideas ofgeneral relativity with Maxwell’selectromagnetic laws. He got hisPhD at the University of Göttingenunder the supervision of DavidHilbert. He was in Zürich whenEinstein began working on hisgeneral theory of relativity, andunder Einstein’s influence, Weylbecame enthralled by mathematicalphysics. There he also metSchrödinger, and they became closefriends. In 1930 he left Zürich tosucceed Hilbert at Göttingen, butwith the rise of Nazism in 1933, heleft (his wife was Jewish) for the

Institute for Advanced Study inPrinceton. He worked there until hisretirement in 1951. At Princeton, hebegan to develop his ideas oninformation theory and efficientinformation systems.

Wilhelm Wien (1864–1928)

German physicist, the 1911 Nobellaureate in physics for hiscontribution to the understanding ofblack body radiation. He worked inHelmholtz’s laboratory from 1883to 1885, and later moved to theUniversity of Berlin and workedwith Max Planck. In 1893 hediscovered that as the temperature

of an emitting body rises, so doesthe maximum energy frequency (inother words, he identified therelationship between the maximumemission frequency of a black bodyto its temperature). In 1896 heformulated this discovery as theemission law bears his name, onwhich Planck based his quantumtheory in 1901. Additionally, Wienlaid the foundations of massspectroscopy.

Ernst Zermelo (1871–1953)

German mathematician whose workhad major implications for thefoundations of mathematics andphilosophy. He studiedmathematics, physics andphilosophy in Berlin. After hisgraduation in 1889, he remained inBerlin as research assistant toPlanck, and under his guidance

began studying hydrodynamics.From 1897 to 1910 he was inGöttingen, at that time the leadingcenter for mathematical research inthe world. From 1910 to 1916 heworked at Zurich University. In1926 he was appointed to anhonorary chair at the University ofFreiberg, but his abhorrenceHitler’s regime made him resign in1935. After the war he wasreinstated, at his request, to hisposition. In 1904 he wrote a paperin which he proved that any finitenumerical set could be well-ordered. The paper was a responseto the challenge proposed by David

Hilbert to the participants of theInternational Congress ofMathematicians in 1900: to find thesolution to 23 unsolved fundamentalquestions during the coming century.His publication earned him fameand the position of Professor inGöttingen in 1905. Among otherthings, he wrote a paper questioningthe correctness of Boltzmann’s H-theorem.

George Kingsley Zipf (1902–1950)

American linguist who studied thestatistical distributions of words invarious languages, searching foruniversal relationships revealed intexts regardless of the languageitself. He worked at HarvardUniversity, specializing in Chineselanguages. The law named after him

states that people use a few wordsfrequently and many words seldom.The most-used word in any givenlanguage will appear in anysufficiently long text twice as oftenas the second-most-used word,which will appear twice as often asthe fourth-most-used word, and soon. Zipf also showed that in manylanguages, such as English, there isan inverse relation between thefrequency of a word and its length.That is, the most common words arealso the shortest. He described thisas an aspect of the principle of leasteffort. His obsession with this idea,along with some of his political

views regarding Hitler’s Germany,gave him a reputation of beingsomewhat eccentric, if notoutrightly insane. Zipf discoveredthat his law could also be appliedto the distribution of urbanpopulations or other communities,and added that the distribution ofincome as a function of socialstanding also behaved according tohis law (this observation was madeby Pareto, before him).

Bibliography

“Anders Celsius,” Wikipedia,http://en.wikipedia.org/wiki/Anders_Celsius

Angrist, S. W. and Helper, L. G., Order andChaos – Laws of Energy andEntropy (New York: Basic Books,1967)

Bader, A. and Parker, L., “JosephLoschmidt, Physicist and Chemist,”Physics Today Vol. 54(3) (March2001): 45

Barabashi, A.L., Links, the New Science ofNetworks (Tel-Aviv: YediotAhronot and Hemed Books, 2004)

Beaudry, P., “Lazare Carnot: Organizer ofVictory – How the ‘Calculus of

Enthusiasm’ Saved France,”American Almanac (July 21, 1997)

Bellis, M., “Outline of Railroad History,”http://inventors.about.com/library/inventors/

blrailroad.htm——, “What is Thermometer,”

http://inventors.about.com/b/2004/11/16/the-history-behind-the-thermometer.htm

Benford, F., “The law of anomalousnumbers,” Proceedings of theAmerican Mathematical Society,78 (1938): 551-572

Bogomolny, A., “Benford’s Law and Zipf’sLaw,” http://www.cut-the-knot.org/do_you_know/zipfLaw.shtml

Boltzmann, L., Vorlesungen überGastheorie (2 vols., Leipzig: J. A.

Barth, 1896-1898)Vorlesungen über Maxwells Theorie

der Elektrizität und des Lichtes (2vols., Leipzig: J. A. Barth, 1891,1893)

Vorlesungen über Die Prinzipe DerMechanik (2 vols., Leipzig: J. A.Barth, 1897, 1904)

Boltzmann, L., “Weitere Studien über dasWärmegleichgewicht unterGasmolekülen”, SitzungsberichteAkademie der Wissenschaften 66(1872): 275-370, in idem.,Wissenschaftliche Abhandlungen,Vol. 1, 1909, pp. 316-402.

Braess, D. von, “Über ein Paradoxon ausder Verkehrsplanung,”

Unternehmensforschung (1969),12: 258–268

Braess’s Paradox (Counter Intuitive Theorybeing used by World’s MajorCities),http://expertvoices.nsdl.org/cornell-info204/2010/04/13/braess-paradox-counter-intuitive-theory-being-used-by-worlds-major-cities/

Bumstead, H. A., “Josiah Willard Gibbs,”American Journal of Science, Ser.4, Vol. XVI (September 1903): 187-202

Campbell, L. and Garnett, W., The Life ofJames Clerk Maxwell (London:Macmilian and Co., 1882)

Carnot, S., Réflexions sur la puissance

motrice du feu et sur les machinespropres à développer cettepuissance (Paris: Bachelier, 1824);Trans., ed., intr. R. Fox, SadiCarnot – Reflections on the MotivePower of Heat and on MachinesFitted to Develop that Power(Manchester: ManchesterUniversity Press, 1986); also idem,Reflections on the Motive Power ofHeat, Trans., ed., intr., Thurston, R.H. (New York: John Wiley & Sons,1897)

Cercignani, C., Ludwig Boltzmann: TheMan Who Trusted Atoms (NewYork: Oxford University Press,1998)

“Claude Elwood Shannon,” Wikipedia,http://en.wikipedia.org/wiki/Claude_Shannon

Clausius, R., Abhandlungen über diemechanische Wärmetheorie,Zweite Abteilung ( Braunschweig:Friedrich Vieweg und Sohn,1867);Trans. W. R. Browne, TheMechanical Theory of Heat(London: John van Voorst, 1879)

——, “Über verschiedene für dieAnwendung bequeme Formen derHauptgleichungen der mechanischenWärmetheorie,” Annalen derPhysik und Chemie, 125 (1865):353; trans., excrp. in WilliamFrancis Magie, A Source Book inPhysics (New York: McGraw-Hill,

1935)Memoires: Mechanical Theory of

Heat, 1850-1865 (London, May1867); inhttp://www.humanthermodynamics.com/Clausius.html

Cohen, E. G. D., “Boltzmann and StatisticalMechanics in Boltzmann’s Legacy150 years After His Birth,” Atti deiConvengni Lincei (AccademiaNationale dei Lincei, Roma) 131(1996): 9-23; inhttp://xxx.lanl.gov/abs/cond-mat/9608054

——, “Entropy, Probability and Dynamics,”http://arxiv.org/PS_cache/arxiv/pdf/0807/0807.1268v2.pdf(10 July 2008)

Cohen, D. and Talbot, K., “Prime numbers

and the first digit phenomenon,” J.Number Theory 18 (1984): 261-268.

Crease, R. P., “Science as Drama,”physicsworld.com/cws/article/print/25726(Sept. 1, 2006)

Crowther, J. G., Famous American Men ofScience (New York: Freeport,1969)

Dawkins, R., The Selfish Gene (New York:Oxford University Press, 1989)

Drake, S., Galileo at Work: His scientificbiography. (Mineola, NY: CourierDover, 2003)

Dyson, F. J., 11-16 Nov. 2007. “Why isMaxwell Theory so hard tounderstand,” The Second European

Conference on Antennas andPropagation, 2007. EuCAP 2007; inhttp://www.clerkmaxwellfoundation.org/DysonFreemanArticle.pdf

Everitt, F., “James Clerk Maxwell: a forcefor physics,” Physics World (Dec.1, 2006): 1-12

Erlichson, H., “Sadi Carnot, Founder of theSecond Law of Thermodynamics,”European Journal of Physics 20(1999): 183-192

“Ernst Mach,” Wikipedia,http://en.wikipedia.org/wiki/Ernst_Mach

Flamm, D., “Ludwig Boltzmann – A pioneerof Modern Physics” inhttp://arxiv.org/abs/physics/9710007(7 Oct. 1997)

——, “Boltzmann: A disordered genius,” in

physicsworld.com/cws/article/print/2481Ferris, T., Coming of Age in the Milky Way,

(new York: New York, Perennial,2003)

Forfar, D. O., “Origins of the Clerk(Maxwell) Genuis,” Bulletin of theInstitution of Mathematics and itsApplication, Vol. 28, No. 1/2(January-February 1992): 4-16

Forfar, D. O., “James Clerk Maxwell: hisqualities of mind and personality asjudged by his contemporaries,”Mathematics Today, Vol 38, No. 3,(June 2002): 83

“Gabriel Daniel Farenheit,”http://www.answers.com/topic/fahrenheit

Garber, E. S. G. et al. (eds.), Maxwell on

Heat and Statistical Mechanics(Bethlehem: Lehigh UniversityPress, 1995)

Gibbs, J. W., Elementary Principles inStatistical Mechanics with SpecialReference to Thermodynamics(New York: Dover, 1902)

The Scientific Papers of J. WillardGibbs, Bumstead, H. A., and VanName, R. G., eds. (New York:Dover, 1961)

Goldstine, H. H., The Computer fromPascal to Von Neumann (Princeton:Princeton University Press, 1980)

Harman, P. M. (ed.), The Scientific Lettersand Papers of James Clerk

Maxwell 1846-1862 Vol. 1, 197(Cambridge: Cambridge UniversityPress, 1990)

Hawking, S., A Brief History of Time, (NewYork: New York, Bantam, 1966)

Heilbron, J. L., The Dilemmas of an UprightMan: Max Planck and the Fortunesof German Science (Cambridge:Harvard University Press, 2000)

Helmholtz, H. von, Über die Erhaltung derKraft: eine physikalischeAbhandlung (Berlin: Reimer,1847).

Hutchinson, I., “James Clerk Maxwell andthe Christian Proposition,” MITIAP Seminar: The Faith of GreatScientists (Jan. 1998, 2006): 3-20

“James Clerk Maxwell,”Wikipediahttp://en.wikipedia.org/wiki/James_Clerk_Maxwell”

“James Prescott Joule (1818-1889),http://www.corrosion-doctors.org/Biographies/JouleBio.htm

“Josiah Willard Gibbs (1839-1903),”http://www.corrosion-doctors.org/Biographies/GibbsBio.htm

“Josiah Willard Gibbs 1839-1903,”http://www.aip.org/history/gap/Gibbs/Gibbs.html

“Josiah Willard Gibbs,”http://www.answers.com/topic/willard-gibbs (2006)

Jurgan, J. M. (ed), Quality ControlHandbook (New York: McGraw-Hill, 1951)

Kahneman D. and co-authors (2005)Rationality, Fairness, Happiness.Selected Writings, Edited by MayaBar-Hillel (Hebrew).

Kossovsky, A. E., “Towards a betterunderstanding of the leading digitsphenomena,” arXiv:math/0612627

Lightman, A. P. (2005). “The discoveries:great breakthroughs in twentieth-century science, including theoriginal papers”, (Toronto: AlfredA. Knopf Canada, 2005)

Lewis, Gilbert N. and Merle Randall,Thermodynamics and the FreeEnergies of Chemical Substances(New York: McGraw-Hill, 1923)

Lienhard, J. H., “Lazare & Sadi Carnot,” in

http://www.uh.edu/engines/epi1958.htmLindley, D., Boltzmann’s Atom – The Great

Debate That Launched a Revolutionin Physics (New York: The FreePress, 2001)

Longair, M., “James Clerk Maxwell:Scotland’s Greatest Physicist,” TheScotman (15 April 2006): 4-16

“Ludwig Boltzmann (1844-1906),”http://corrosion-doctors.org/Biographies/BoltzmannBio.htm

“Ludwig_Edward Boltzmann,”http://www.answers.com/topic/ludwig-boltzmann

McCartney, M., “William Thomson: king ofVictorian physics,”http://www.physicsworld.com/cws/article/print/16484

Dec 1, 2002Magie, W. F., A Source Book in Physics

(New York: McGraw-Hill, 1935).“Max Planck”, Britannica Concise

Encyclopedia (2006)“Max Planck,” Wikipedia,

http://en.wikipedia.org/wiki/Max_PlanckMax Planck Institute, “Max Planck - His

Life, Work, and Personality,”http://www.max-planck.mpg.de/framset_e.htm

Maxwell Foundation, “The Impact ofMaxwell’s Work,”http://www.clerkmaxwellfoundation.org/html/further_maxwell’simpact.html

Maxwell Foundation, “Who Was JamesClerk Maxwell?”http://www.clerkmaxwellfoundation.org/html/who_was_maxwell_.html

Maxwell, J. C., “On the description of ovalcurves, and those having a pluralityof foci,” Proceedings of the RoyalSociety of Edinburgh, Vol. II(1846)

Maxwell, J. C., “A Dynamical Theory of theElectromagnetic Field,”Philosophical Transactions of theRoyal Society of London 155(1865): 459-512

Maxwell, J. C., A Treatise on Electricityand Magnetism (Oxford:Clarendon Press, 1873)

Milgram, S., “The Small World Problem”,Psychology Today, 1967, Vol. 2,60-67

Miller, G. A. and Newman, E. B., “Tests of a

statistical explanation of the rank-frequency relation for words inwritten English,” American Journalof Psychology, 71 (1958): 209-218.

Moffatt, K. Homage to Clerk Maxwell,http://www.clerkmaxwellfoundation.org/html/further_documents.html

Nash, J., “Non-Cooperative Games,” TheAnnals of Mathematics 54(2),1951: 286-295

Newman, M. E., “Power-law, ParetoDistribution and Zipf’s law,”Contemporary Physics Vol. 46, No.5 (29 May 2006): 323-351.

Newcomb, S., “Note on the Frequency ofUse of the Different Digits inNatural Numbers,” AmeicanJournal of Mathematics 4 (1881):

39-40“Nicolas Leonard Sadi Carnot,” Wikipedia,

http://en.wikipedia.org/wiki/Nicolas_L%C3%A9onard_Sadi_CarnotNigrini, M., “A Taxpayer Compliance

Application of Benford’s Law,”Journal of the American TaxationAssociation 18 (1996): 72-91

Nobel Web, “Max Planck, The Noble Prizein Chemestry 1909,”http://www.nobelprize.org/nobel_prizes/chemistry/laureates/1909/planck-bio.html (2008)

Nobel Web, “Wilhelm Ostwald, The NobelPrize in Chemistry 1909,”http://www.nobelprize.org/nobel_prizes/chemistry/laureates/1909/ostwald-bio.htm

O’Connor, J. J. and Robertson, E. F.,“Claude Elwood Shannon,” in

http://www-history.mcs.stand.ac.uk/Mathematicians/Shannon.html(October 2003)

——, “Historic topics: A visit to JamesClerk Maxwell’s house,” inhttp://www-history.mcs.st-and.ac.uk/HistTopics/Maxwell_House.html(November 1997)

——, “James Clerk Maxwell”, inhttp://www-history.mcs.st-and.ac.uk/Mathematicians/Maxwell.html(November 1997)

——, “John William Strutt (Lord Rayleigh)”in http://www-history.mcs.st-and.ac.uk/Mathematicians/Rayleigh.html(October 2003)

——, “Josiah Willard Gibbs,” http://www-

history.mcs.st-and.ac.uk/Mathematicians/Gibbs.html(February 1997)

——, “Lazare Nicolas Marguérite Carnot,”in http://www-history.mcs.st-and.ac.uk/Biographies/Carnot.html(December 1996)

——, “Ludwig Boltzmann,” in http://www-history.mcs.st-and.ac.uk/Mathematicians/Boltzmann.html(September 1998)

——, “Max Karl Ernst Ludwig Planck”.http://www-history.mcs.st-and.ac.uk/Mathematicians/Planck.html(October 2003)

——, “Nicolas Léonard Sadi Carnot,” inhttp://www-history.mcs.st-

and.ac.uk/Mathematicians/Carnot_Sadi.html(October 1998)

——, “Rudolf Juluis Emmanuel Clausius,”in http://www-history.mcs.st-and.ac.uk/Mathematicians/Clausius.html(December 2000)

——, “William Thomson (Lord Kelvin),” inhttp://www-history.mcs.st-and.ac.uk/Mathematicians/Thomson.html(October 2003)

Pais, A., Subtle is the Lord... (Oxford:Oxford University press, 1982)

“Pareto Principle – The 80-20 Rule –Complete Information,”http://www.gassner.co.il/pareto

Pareto, V. Cours d’économie politique(Lausanne et Paris: Rouge, 1897)

Pareto, V., “Un applicazione di teoriesociologiche,” Revista Italiana disociologia, 1901, pp. 402-456

Pareto, V., Trattato Di Sociologia Generale(4 vols.), (Firenze: Barbèra, 1916)

Pareto V., The History of Economic ThoughtWebsite, “Vilfredo Pareto, 1848-1923,”http://cepa.newschool.edu/het/profiles/pareto.htm

“Peter Guthrie Tait,” Wikipedia,http://en.wikipedia.org/wiki/Peter_Guthrie_Tait

Planck, M., Scientific Autobiography andOther Papers, trans. F. Gaynor(New York: Philosophical Library,1949)

M. Planck, “Über das Gesetz derEnergieverteilung im

Normalspectrum,” Annalen derPhysik 4: 553 (1901).

“Rail Transport,” Wikipedia,http://en.wikipedia.org/wiki/Rail_transport2008

Rautio, J. C., “Maxwell’s Legacy,” IEEEmicrowave magazine (June 2005):46-53

“Revolution and the growth of industrialsociety, 1789–1914, EncyclopediaBritannica,http://www.britannica.com/EBchecked/topic/195896/history-of-Europe/58403/Revolution-and-the-growth-of-industrial-society-1789-1914

“Rudolf Clausius,” Wikipedia,http://en.wikipedia.org/wiki/Rudolf_Clausius

Schopenhauer, A., Über die Freiheit desmenschlichen Willens, 1839;Trans. K. Kolenda, On theFreedom of the Will (Oxford: BasilBlackwell, 1985)

“Shannon explaining why he named hisuncertainty function ‘entropy’,”Scientific American Vol. 225, No 3(1971): 180

Shannon, C. E., “A mathematical theory ofcommunication,” Bell SystemTechnical Journal Vol. 27 (July andOctober, 1948): 379-423, 23-656respectively

Shannon, C. E. et al. (eds.)., Claude ElwoodShannon: Collected Papers (NewYork: IEEE Press, 1993)

Slepian, D. (ed.), Key Papers in theDevelopment of Information Theory(New York: IEEE Press, 1974)

Srinivasan, J., “Sadi Carnot and the SecondLaw of Thermodynamics,”Resonance (Nov. 2001): 42-48

Stanford Encyclopedia of Philosophy(2004), “Boltzmann’s work inStatistical Physics”

Stearns, P. N. (ed), The Encyclopedia ofWorld History, Vol. V, The ModernPeriod, 1789-1914 (New York:Houghton Mifflin, 2001)

Tversky, A. and Kahenman D., “Lossaversion in riskless choice: Areference-dependent model”,Quarterly Journal of Economics

106 (1991): 1039-1061University of Cambridge, Biographical

Information, James Clerk Maxwell,1831-1879, Professor ofExperimental Physicshttp://www.nahste.ac.uk/isaar/GB_0237_NAHSTE_P1314.html

“Vilfredo Pareto,” Wikipedia,http://en.wikipedia.org/wiki/Vilfredo_Pareto

Weaver W., and Shannon C. E., TheMathematical Theory ofCommunication (Urbana, Illinois:University of Illinois Press, 1949)

“Wilhelm Wein,” Wikipedia,http://en.wikipedia.org/wiki/Wilhelm_Wien

“William Thomson, 1st Baron Kelvin,”Wikipedia,http://en.wikipedia.org/wiki/William_Thomson,_

1st_Baron_Kelvin”Zipf, G. K., Human Behavior and the

Principle of Least-Effort (NewYork: Addison-Wesley,1949)

Endnotes

Prologue

Angrist and Helper (1967), p. 215.

Chapter 1

Stearns (2001), pp. 424-427.

Bellis, M.,http://inventors.about.com/library/inventors/blrailroad.htm.

Carnot (1824), ed. Thurston (1897), pp. 37-41.

Ibid., p. 48.

Beaudry (1997); see also Lienhard, inhttp://www.uh.edu/engines/epi1958.htm

Carnot (1824), ed. Thurston (1897), p. 28.

Ibid., pp. 35-36.

Crease (2006), Act III, Scene 2,physicsworld.com/cws/article/print/25726

Helmholtz: (1847),

Carnot (1824), ed. Thurston (1897),Introduction, p. 6.

Clausius (1867), p. 365.

E. E. Daub, cited in O’Connor andRobertson (Dec. 2000), http://www-history.mcs.st-and.ac.uk/Mathematicians/Clausius.html

Clausius (1850), in Magie (1935), pp. 79,

368-97, 500-24.

Clausius (1865).

Ibid., p. 125.

http://en.wikipedia.org/wiki/History_of_entropy#cite_ref-Clausius_6-0.

Chapter 2

Quoted in D. O. Forfar, “James ClerkMaxwell” (1992).

Lindley (2001), p. 47.

Boltzmann (1872).

Flamm, D., April 9, 1999,physicsworld.com/cws/article/print/2481

Lindley (2001), p. 56.

Boltzmann (1891, 1893).

Quoted in O’Connor and Robertson,“Ludwig Boltzmann”, http://www-history.mcs.st-and.ac.uk/Mathematicians/.html.

Lindley (2001), p. 146.

Boltzmann (1896, 1898).

Boltzmann (1897, 1904).

Pais (1982), p. 65.

Columbia Encyclopedia, “Gibbs, JosiahWillard”.

“Graphical Methods in the Thermodynamicsof Fluids”, in Gibbs (1961).

“A Method of Geometrical Representationof the Thermodynamic Properties ofSubstances by Means of Surfaces,” inibid.

O’Connor and Robertson, http://www-history.mcs.st-and.ac.uk/Mathematicians/Gibbs.html.

“On the Equilibrium of HeterogeneousSubstances,” in Gibbs (1961).

Gibbs (1902).

Lindley (2001), p. 155.

Crowther (1969), “Gibbs”.

Bumstead (1903).

Lewis and Randall (1923).

Harman (1990), Vol. 1, p. 197.

Maxwell Foundation: Who was James ClerkMaxwell?http://www.clerkmaxwellfoundation.org/html/who_was_maxwell_.html

Maxwell (1846).

Campbell and Garnett (1882), p. 80.

Harman (1990), Vol. I, p. 197.

Maxwell Foundation, “Who was JamesClerk Maxwell?”

Maxwell (1865); Quoted in: MaxwellFoundation, “Who Was James ClerkMaxwell?”

Maxwell (1873) Vol. II.

Ibid., p. xi.

Chapter 3

Drake (2003), pp. 20–21.

Ferris (2003), p. 204.

Hawking (1996), p. 10.

Planck (1950), p. 33.

Lightman (2005), p. 8.

Max Planck;http://en.wikipedia.org/wiki/Max_Planck

Heilbron (2000), p. 150.

Ibid., p. 151.

Max Planck Institute; Worldview,www.max-planck.mpg.de/framset_e.htm.

M. Planck (1901), 4: 553

Chapter 4

Scientific American (1971), Vol. 225, p.180.

Goldstine (1980), p. 119.

Shannon (1948), pp. 379-423 and 623-656.

Weaver and Shannon (1949), p. 121.

Chapter 5

Dawkins (1989).

Schopenhauer (1985).

Milgram (1967), Vol. 2, pp. 60-67.

http://expertvoices.nsdl.org/cornell-

info204/2010/04/13.

Braess (1969).

Nash (1951), pp. 286-295.

Pareto (1897).

Pareto (1901), pp. 402-456.

Pareto (1916).

Conclusion

Tversky and Kahneman (1991), pp.1039-1061.

Chapter 1

In those days it was accepted,following Aristotle, that meteors werean atmospheric phenomenon.

Originally formulated by Gay-Lussac,Carnot’s teacher, and John Dalton ofEngland.

Lord Rayleigh, whom we will meetfurther on in connection with thethermodynamics of light, later showedthat the real reason was the scatteringof light according to its wavelength, aphenomenon called today the Rayleighscattering.

See Appendix 1-A. It will be explained in the next chapter,

which deals with the connectionbetween entropy and the possible waysin which a system can be ordered, theamount of entropy in ice is smaller thanthat of the same amount of liquidwater.

Chapter 2

Avogadro’s number is the number ofmolecules of a substance whoseweight, in grams, is equal to themolecular weight of that substance.

The energists were scientists whobelieved in logical positivism, insistedthat natural phenomena can beexplained in terms of energy andrefused to accept the “atomichypothesis”.

Jean Perrin (1870–1942) was a Frenchscientist who, in 1908, conductedexperiments that proved the validity ofEinstein’s explanation for the Brownianmotion under various conditions.

Boltzmann used the Latin letter W;nowadays the Greek letter omega isused.

In 1895, Jules Henri Poincarépublished a theorem that showed thatany closed mechanical system must atsome point return to its initial state.That is, it will pass from one microstateto another, until it returns at one timeor another to its initial microstate. Thistheorem seems to prove thatBoltzmann’s intuitive idea, on which hebased his H-theorem, and according towhich the random scattering of

particles increases the number ofmicrostates (and thus increasingentropy), was incorrect. However, it isimportant to realize that Boltzmann’sentropy deals with a closed system andis solely dependent on the number ofpossible microstates, which is constant,and not at all dependent on thedynamics of the changes from onemicrostate to another, which was whatPoincaré was studying. Simply put,collisions between atoms do notincrease the number of microstates.

The cost of printing Gibbs’s paperswas exceptionally high because of theirlength and the abundance ofmathematical formulas and graphics,and in order to have them printed, itwas necessary to raise donations from

the journal’s subscribers as well asNew Haven businessmen, many ofwhom could not make head or tail ofthese papers, but neverthelesssubsidized them.

Michael Faraday, 1791–1867, was oneof the pioneers in this area.

It is worth noting that the normaldistribution, without the long tail to theright, is the distribution that isdescribed in most textbook and is anapproximation of the Maxwell-Boltzmann distribution that is shown inthe figure.

Chapter 3

Recall that this was the topic ofClausius’s dissertation, but his

explanation, which was based on theabsorption and emission of light, waserroneous.

In principle, a single oscillator is asingle microstate, and therefore wewould expect it to have zero entropy.Yet the uncertainty associated with it asa result of its vibration gives it finiteentropy of one Boltzmann constant.This result, which is empiricallyconsistent with the gas laws and thestudy of blackbody radiation (as weshall see below), is a law of nature.

In 1893, Wilhelm Wien discovered thatas the temperature of an emitting bodyincreases, the maximum frequency ofthe radiation (the frequency with themost energy), vMax, increases

according to the equation vMax (2T) =2vMax (T). That is, if the temperatureis doubled, the maximum frequencyalso doubles. Josef Stefan (who wasBoltzmann’s dissertation advisor)discovered that the intensity of theradiation of a blackbody is proportionalto the fourth power of its temperature.That is, I = σT4, where I is theintensity of the radiation (intensity isenergy per unit time per unit area). Theconstant σ is called the Stefan-Boltzmann constant. This means thatthe hotter the body, the higher theintensity and the frequencies of theradiation it emits.

In 1952, Israel’s prime minister DavidBen Gurion, offered Einstein the

presidency of the State of Israel (thathad become vacant after ChaimWeizmann’s death), explaining that hehad to make the offer first of all to thegreatest Jew alive then. Einsteinpolitely declined.

The Kaiser-Wilhelm Society, foundedin 1911, was an umbrella organizationfor a large number of scientificresearch facilities in Germany. In 1959the Society and all its institutes wererenamed after Max Planck.

Haber, a Jew and a wholeheartedGerman patriot, was one of Germany’sgreatest chemists, the first to synthesizeammonia from its components. Theimportance of this work to thechemical fertilizers industry andagriculture on the one hand, and to

explosives and other materials of waron the other hand, was immense. Hewas also the father of Germany’schemical warfare in Wolrd War I.Before the Hitler’s rise to power,Haber was at the forefront of sciencein Germany, but the Nazi regimestripped him of his position and rightsand he was forced to leave Germany.One year later, in 1934, Haber died inexile, before he could see how hisinvention, Zyklon B, was used toslaughter his people, includingmembers of his own family.

In Appendix A-6 we derived theMaxwell-Boltzmann distribution usingPlanck’s method. Both Maxwell’s andBoltzmann’s methods were different.In this book, we unified all the

derivations and used the Lagrangemultipliers technique.

Chapter 4

In communication theory, thenumerical value ln 2 ≅ 0.693 is called a“nat”.

The name derives from a polynomialdistribution which, on a logarithmicscale, gives a straight line whose slopeis the power of the polynom. A powerlaw polynomial decays less steeply thanan exponential decay. Therefore, theprobability of a node to have manymore “links” than the average is muchhigher than that in exponentialdistributions decay. For example, ourchances of living much longer than the

average is much smaller than ourchances of being much richer thanaverage, since life expectancy tends todecrease exponentially with age.

Appendix A-4

In fact, one can also subtract theconstraint. However the result thusobtained is not physically sound.