The law of seriesSoMaChi Symposium, November, 2007

Tomasz DownarowiczInstitute of Mathematics and Computer Science

Wroclaw University of TechnologyWybrzeze Wyspiańskiego 27

50-370 Wroclaw, Poland

What is the law of series?

In the common sense, a series is noted when a random event considered extremely rare happens several times in a relatively short period of time. The name "law" suggests that such series are observed often enough to indicate an unexplained physical force or statistical rule provoking them.

Examples

For 250 years English bell ringers have tried to answer the question whether it is possible to ring all permutations of seven bells following a certain "method" called common bob Stedman triples. On January 22, 1995 a team at Saint John's Church in London, has finally succeeded. Within few days it was revealed that two other teams, working independently and not knowing about each-other, had also solved this century-old bell-ringing (and in fact mathematical) mystery.

In 1891 an Englishman Charles Wells, during one night, twelve times broke the bank at one of Monte Carlo casinos winning a million francs. At one stage he won 23 times out of 30 successive spins of the wheel. Wells returned to Monte Carlo in November of that year and won again. During this session he made another million francs in three days, including successful bets on the number five for five consecutive turns.

In Lower Silesia, Poland, two major floods occurred in 1997 and 1998, after many years of silence.

Clusters

In many random processes in reality we observe this phenomenon, often called clustering. This applies for example to climate anomalies, some other natural cataclysms, power shortages, certain types of occurrences in the stock market, etc.

Understanding whether clusters indeed appear (or are they just a random fluctuation), and why, could be an important step toward a having a better control over these processes.

Paul Kammerer

An Austrian biologist dr. Paul Kammerer (1880-1926) was the first scientist to study the law of series (law of seriality, in some translations). His book Das Gesetz der Serie contains many examples from his and his nears' lives.

Kammerer observed (or received reports from his relatives and friends) of ”series” of encounters of the same number (in various situations), meeting people with the same last name, and the like. He spent hours sitting in a park and noting ”series” of people wearing glasses. Or he simply observed the times when clients enter a shop and ”discovered” that the average number per minute actually occurs very rarely, yielding to periods of absence or of high occupancy.

Kammerer’s ”series”

The law of series is a particular case of the theory of synchronocity postulated by a Swiss professor of philosophy Karl Gustav Jung (1875-1961 )

Synchronicity

Wolfgang Pauli

and a Nobel prize winner in physics, Austrian Wolfgang Pauli (1900-1958)

The opposition

Other scientists believe that synchronicity and law of series are just manifestations of ordinary random fluctuations.

American mathematician, Warren Weaver (1894-1978)(collaborator of Calude Shannon)argues that such events have positive probability, hence MUST occur from time to time.

Attracting and repellingIn order to give the law of series a definite meaning we define attracting as a deviation of a signal process toward clustering stronger than in the Poisson process . Similarly, repelling is defined as clustering weaker than in the Poisson process.

There is no doubt that the postulates of Pauli and Jung concern so defined attracting. As to Kammerer, apparently less familiar with probability theory, it seems that in most of his experiments he merely "discovered" the ordinary clustering of the Poisson process.

• In many real processes attracting is perfectly understandable as a result of strong physical dependence. A good example here are series of ill-fallings due to a contagious disease.

• The dispute on the law of series clearly concerns only events for which there are no obvious attracting mechanisms, and we expect them to appear completely independently, governed by pure chance.

Our voice in the debate

Jointly with Yves Lacroix we have obtained two results in ergodic theory which support Pauli and Jung. Roughly speaking, we have proved that with regard to "elementary" events (cylinder sets of small probability):

1. In any process of positive entropy attracting is possible while repelling is not;

2. In the majority of processes many elementary events occur with very strong attracting.

Rigorous definitions

Because attracting and repelling must not depend on the intensity of the signals we change the time unit so that = 1 We call this normalization. In the normalized Poisson process, the waiting time for the first signal has exponential distribution:

FP(t) = 1– e–t (t 0).

We call such behavior unbiased.

• We will say that a normalized stationary signal process reveals attracting with intensity from a distance t, if the distribution function F of the waiting time for the first signal satisfies the inequality

F(t) < 1– e–t – • Generally, we will say that a process reveals

attracting only, if at each t holds, F(t) 1– e–t , 

without the two functions being equal (i.e., strict inequality holds at some point). Analogously, inverted inequalities (with +) define repelling. 

InterpretationIn an interval of time of length t the expected number of signals is t, i.e., t. The value F(t) is the probability, that in such interval there will be at least one signal. The ratio t/F(t) hence represents the conditional expectation of the number of signals in these time intervals of length t in which at least one signal is observed. If  F(t) < 1– e–t, this conditional expectation is larger than in the Poisson process. In other words, if we observe the process for time t, there are two possibilities: either we detect nothing, or, once the first signal occurs, we expect a larger global number of observed signals than if we were dealing with the Poisson process. The first signal "attracts" further repetitions.

Ergodic theory setup

• We will consider a dynamical system consisting of a probability space (X,Σ,μ) and a measure-preserving transformation T: X→X. If P is a finite measurable partition of X, it generates a process (X,Σ,μ,P) the elements of which are sequences over the alphabet P and the transformation is the shift T(xn) = (xn+1).

• A cylinder set B of length n is a set

{x: x[0,n-1] = B} (BєPn)

The main theorems• I. Any stationary ergodic finite-states process with

positive entropy has the following property: For every >0 the joint measure of all cylinders of length n, revealing repelling with intensity (from any distance t) converges to zero as n tends to infinity. 

• II. In every non-periodic dynamical system there exists a residual set of partitions P with the following property: There exists a subset of natural numbers of upper density 1, such that every P-cylinder of a length from this set reveals attracting with intensity close to 1.

A comment on the theorem• It concerns exclusively small sets, i.e., rare events.

It does not apply to  the observations of single numbers in a roulette, umbrellas or glasses on passing pedestrians (one of Kammerer's favorite experiment subjects). More adequate is the event of breaking the roulette bank.

• It concerns cylinder sets, that is exact repetitions of a specific configuration of large events. Breaking a roulette bank is a union of several cylinders rather than one cylinder.

• The best example is the occurrence of a specific long configuration in a computer program or in a genetic code.

Sketch of the proof of Theorem 1The proof consists of one technical trick and two major observations. The trick is to consider the repetitions of a concatenation BA whose left part B is much longer than the right part A. For a moment we can think that A is the last letter of the considered block. Then we observe the process of repetitions of the block B and the process of symbols directly following these repetitions. On the figure below, a realization of such "induced" process is the sequence ...

A-1A0A1A2...

We prove that for a typical long block B, such induced process is almost independent.

Crush course on entropy

• If P = {C1,C2,…,Cr} is a finite partition of X into sets of measures μ(Ci) = pi (i=1,2,…,r), then its (Shannon or static) entropy equals H(μ,P) = -Σ pi logpi.

• Given two finite partitions P and Q their join is defined as PvQ = {C∩D: CєP, DєQ}.

• Conditional entropy:

H(μ,P|Q) = ΣDєQ μ(D)H(μD,P) = H(μ,PvQ) – H(μ,Q) Always H(μ,P|Q) ≤ H(μ,P) and H(μ,P|Q) = H(μ,P) if and only if the partitions are independent.

• Partitions P and Q are almost independent if H(μ,P|Q) > H(μ,P) - .

Entropy of a process

The Kolmogorov-Sinai or dynamical entropy of a process (X,Σ,μ,P) is defined as

h(μ,T,P) = H(μ,P|P-) = lim H(μ,P|P-n),

where P-n = PvT(P)v…vTn-1(P).

Clearly h(μ,T,P) ≤ H(μ,P), and equality holds only for an independent process (the partitions Tn(P) are independent). The process is called

- independent if

h(μ,T,P) > H(μ,P) -

Proof of -independence of the induced process

This implies that except for cylinders B of joint measure at most the induced process is -independent

• Much more difficult is to prove that this process is also -independent of the positioning of the repetitions of B.

• The second key observation is much easier. Assume for simplicity, that the independencies are strict. Then it is not hard to prove that the strongest repelling for the block BA occurs, when B is distributed with the maximal possible repelling, i.e., periodically. But with such distribution of the B's, and with the assumed independence, the occurrences of BA are the same as in an independent process with discrete time (with unit equal to the period of the B's).

The proof of the second result (that in a typical process cylinders of certain lengths reveal strong attracting) is completely different and does not use advanced ergodic theory. Instead we construct explicitly a dense family of partitions which satisfy an open condition.

Conclusion

Suppose we observe a process which we believe is independent (for example tossing a coin). In reality such process is never perfectly independent; the generating partition is always slightly perturbed from independent. Then we know that such perturbation may only result in attracting, never in repelling. Moreover, typically it will lead to strong attracting for cylinders of certain large lengths. For example, if we are tossing a 0-1 coin and we are interested in seeing a specific sequence (e.g. the first 1000 digits of the binary expansion of π) then it is very likely that we will observe this sequence appearing in strong clusters (relatively to its probability).But remember! This implies that the waiting time for the first occurrence usually is much longer than the inverse of the probability!

THANK YOU