50
1 Cluster Monte Carlo Cluster Monte Carlo Algorithms: Algorithms: Jian-Sheng Wang Jian-Sheng Wang National University of Singapore National University of Singapore

1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

Embed Size (px)

Citation preview

Page 1: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

1

Cluster Monte Carlo Cluster Monte Carlo Algorithms Algorithms

Jian-Sheng WangJian-Sheng WangNational University of SingaporeNational University of Singapore

Cluster Monte Carlo Cluster Monte Carlo Algorithms Algorithms

Jian-Sheng WangJian-Sheng WangNational University of SingaporeNational University of Singapore

2

Outlinebull Introduction to Monte Carlo and

statistical mechanical modelsbull Cluster algorithmsbull Replica Monte Carlo

3

1 Introduction to MC 1 Introduction to MC and Statistical and Statistical

Mechanical ModelsMechanical Models

1 Introduction to MC 1 Introduction to MC and Statistical and Statistical

Mechanical ModelsMechanical Models

4

Stanislaw Ulam (1909-1984)

S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling

5

Nicholas Metropolis (1915-1999)

The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century

6

The Name of the Game

Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino

Monte-Carlo Monaco

7

Use of Monte Carlo Methods

bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling

bull Using random numbers in an essential way

bull Simulation of stochastic processes

8

Markov Chain Monte Carlo

bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)

bull Move X by the transition probability W(X -gt Xrsquo)

bull Starting from arbitrary P0(X) we have

Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin

9

bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo

bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)

Necessary and sufficient conditions for convergence

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 2: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

2

Outlinebull Introduction to Monte Carlo and

statistical mechanical modelsbull Cluster algorithmsbull Replica Monte Carlo

3

1 Introduction to MC 1 Introduction to MC and Statistical and Statistical

Mechanical ModelsMechanical Models

1 Introduction to MC 1 Introduction to MC and Statistical and Statistical

Mechanical ModelsMechanical Models

4

Stanislaw Ulam (1909-1984)

S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling

5

Nicholas Metropolis (1915-1999)

The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century

6

The Name of the Game

Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino

Monte-Carlo Monaco

7

Use of Monte Carlo Methods

bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling

bull Using random numbers in an essential way

bull Simulation of stochastic processes

8

Markov Chain Monte Carlo

bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)

bull Move X by the transition probability W(X -gt Xrsquo)

bull Starting from arbitrary P0(X) we have

Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin

9

bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo

bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)

Necessary and sufficient conditions for convergence

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 3: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

3

1 Introduction to MC 1 Introduction to MC and Statistical and Statistical

Mechanical ModelsMechanical Models

1 Introduction to MC 1 Introduction to MC and Statistical and Statistical

Mechanical ModelsMechanical Models

4

Stanislaw Ulam (1909-1984)

S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling

5

Nicholas Metropolis (1915-1999)

The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century

6

The Name of the Game

Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino

Monte-Carlo Monaco

7

Use of Monte Carlo Methods

bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling

bull Using random numbers in an essential way

bull Simulation of stochastic processes

8

Markov Chain Monte Carlo

bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)

bull Move X by the transition probability W(X -gt Xrsquo)

bull Starting from arbitrary P0(X) we have

Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin

9

bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo

bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)

Necessary and sufficient conditions for convergence

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 4: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

4

Stanislaw Ulam (1909-1984)

S Ulam is credited as the inventor of Monte Carlo method in 1940s which solves mathematical problems using statistical sampling

5

Nicholas Metropolis (1915-1999)

The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century

6

The Name of the Game

Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino

Monte-Carlo Monaco

7

Use of Monte Carlo Methods

bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling

bull Using random numbers in an essential way

bull Simulation of stochastic processes

8

Markov Chain Monte Carlo

bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)

bull Move X by the transition probability W(X -gt Xrsquo)

bull Starting from arbitrary P0(X) we have

Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin

9

bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo

bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)

Necessary and sufficient conditions for convergence

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 5: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

5

Nicholas Metropolis (1915-1999)

The algorithm by Metropolis (and A Rosenbluth M Rosenbluth A Teller and E Teller 1953) has been cited as among the top 10 algorithms having the greatest influence on the development and practice of science and engineering in the 20th century

6

The Name of the Game

Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino

Monte-Carlo Monaco

7

Use of Monte Carlo Methods

bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling

bull Using random numbers in an essential way

bull Simulation of stochastic processes

8

Markov Chain Monte Carlo

bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)

bull Move X by the transition probability W(X -gt Xrsquo)

bull Starting from arbitrary P0(X) we have

Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin

9

bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo

bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)

Necessary and sufficient conditions for convergence

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 6: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

6

The Name of the Game

Metropolis coined the name ldquoMonte Carlordquo from its gambling Casino

Monte-Carlo Monaco

7

Use of Monte Carlo Methods

bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling

bull Using random numbers in an essential way

bull Simulation of stochastic processes

8

Markov Chain Monte Carlo

bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)

bull Move X by the transition probability W(X -gt Xrsquo)

bull Starting from arbitrary P0(X) we have

Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin

9

bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo

bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)

Necessary and sufficient conditions for convergence

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 7: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

7

Use of Monte Carlo Methods

bull Solving mathematical problems (numerical integration numerical partial differential equation integral equation etc) by random sampling

bull Using random numbers in an essential way

bull Simulation of stochastic processes

8

Markov Chain Monte Carlo

bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)

bull Move X by the transition probability W(X -gt Xrsquo)

bull Starting from arbitrary P0(X) we have

Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin

9

bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo

bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)

Necessary and sufficient conditions for convergence

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 8: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

8

Markov Chain Monte Carlo

bull Generate a sequence of states X0 X1 hellip Xn such that the limiting distribution is given P(X)

bull Move X by the transition probability W(X -gt Xrsquo)

bull Starting from arbitrary P0(X) we have

Pn+1(X) = sumXrsquo Pn(Xrsquo) W(Xrsquo -gt X)bull Pn(X) approaches P(X) as n go to infin

9

bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo

bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)

Necessary and sufficient conditions for convergence

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 9: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

9

bull Ergodicity[Wn](X - gt Xrsquo) gt 0For all n gt nmax all X and Xrsquo

bull Detailed BalanceP(X) W(X -gt Xrsquo) = P(Xrsquo) W(Xrsquo -gt X)

Necessary and sufficient conditions for convergence

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 10: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

10

Taking Statisticsbull After equilibration we estimate

1

1( ) ( )P( )d ( )

N

ii

Q X Q X X X Q XN

It is necessary that we take data for each sample or at uniform interval It is an error to omit samples (condition on things)

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 11: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

11

Choice of Transition Matrix W

bull The choice of W determines a algorithm The equation P = PW or P(X)W(X-gtXrsquo)=P(Xrsquo)W(Xrsquo-gtX)has (infinitely) many solutions given PAny one of them can be used for Monte Carlo simulation

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 12: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

12

Metropolis Algorithm (1953)

bull Metropolis algorithm takes

W(X-gtXrsquo) = T(X-gtXrsquo) min(1

P(Xrsquo)P(X))where X ne Xrsquo and T is a symmetric stochastic matrixT(X -gt Xrsquo) = T(Xrsquo -gt X)

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 13: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

13

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 14: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

14

Model GasFluidA collection of molecules interact through some potential (hard core is treated) compute the equation of state pressure p as function of particle density ρ=NV

(Note the ideal gas law) PV = N kBT

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 15: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

15

The Statistical Mechanics of Classical Gas(complex) FluidsSolids

Compute multi-dimensional integral

where potential energy

( 1 1)

1 1 2 2 1 1

( 1 1)

1 1

( )e

e

B

B

E x yk T

N N

E x yk T

N N

Q x y x y dx dy dx dyQ

dx dy dx dy

1( ) ( )N

iji j

E x V d

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 16: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

16

The Ising Model

- +

+

+

+

++

+

++

++

+

++

+

+-

---

-- -

- --

- ----

---- The energy of

configuration σ is

E(σ) = - J sumltijgt σi σj

where i and j run over a lattice ltijgt denotes nearest neighbors σ = plusmn1

σ = σ1 σ2 hellip σi hellip

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 17: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

17

The Potts Model2

1

3

1

2

3

2

2

2

1

2

2

13

2

2

2

3

32

1

2 2

1 3

3

3 32

2

1

111

1The energy of configuration σ is

E(σ) = - J sumltijgt δ(σiσj)

σi = 12hellipq

1

See F Y Wu Rev Mod Phys 54 (1982) 238 for a review

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 18: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

18

Metropolis Algorithm Applied to Ising Model

(Single-Spin Flip)

1 Pick a site I at random2 Compute E=E(rsquo)-E() where rsquo

is a new configuration with the spin at site I flipped rsquoI=-

3 Perform the move if lt exp(-EkT) 0ltlt1 is a random number

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 19: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

19

Boltzmann Distribution

bull In statistical mechanics thermal dynamic results are obtained by expectation value (average) over the Boltzmann (Gibbs) distribution

( ) ( )

( ) ( )

( )e( )

e

E kT

E kT

QQ

Z

Z

Z is called partition function

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 20: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

20

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

2 Swendsen-Wang 2 Swendsen-Wang algorithmalgorithm

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 21: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

21

Percolation ModelEach pair of nearest neighbor sites is occupied by a bond with probability p The probability of the configuration X is

pb (1-p)M-b

b is number of occupied bonds M is total number of bonds

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 22: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

22

Fortuin-Kasteleyn Mapping (1969)

( 1)

1 0

(1 )

1

i jij

i j ij ij

c

K

n nn ij

M b Nb

X

Z e

p p

p p q

where K = J(kBT) p =1-e-K and q is number of Potts states Nc is number of clusters

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 23: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

23

Sweeny Algorithm (1983)

Heat-bath rates

w( -gt1) = p

w( -gt ) = 1 ndash p

w( -gt 1β) = p( (1-p)q +p )

w( -gt β) = (1-p)q( (1-p)q + p )

P(X) ( p(1-p) )b qNc

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 24: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

24

Swendsen-Wang Algorithm (1987)

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

An arbitrary Ising configuration according to

( )i j

ij

K

P e

K = J(kT)

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 25: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

25

Swendsen-Wang Algorithm

++

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

Put a bond with probability p = 1-e-K if σi = σj

1 0( ) (1 )i j ij ijn n

ij

P n p p

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 26: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

26

Swendsen-Wang Algorithm

Erase the spins

1 0

( ) (1 )

(1 )

i j ij ij

c

n nij

Nb M b

P n p p

p p q

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 27: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

27

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Assign new spin for each cluster at random Isolated single site is considered a cluster

Go back to P(σn) again

---

- -+

+

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 28: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

28

Swendsen-Wang Algorithm

++

++

+

++

+

+

+

+

+

+

++ ++

+

+

+

+

-

-

-

---

-

- --

-- - - -- -

--

- -

Erase bonds to finish one sweep

Go back to P(σ) again

---

- -+

+

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 29: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

29

Identifying the Clustersbull Hoshen-Kompelman algorithm

(1976) can be used bull Each sweep takes O(N)

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 30: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

30

Measuring Error

bull Let Qt be some quantity of interest at time step t then sample average is

QN = (1N) sumt Qt

bull We treat QN as a random variable By central limit theorem QN is normal distributed with a mean ltQNgt=ltQgt and variance σN

2 = ltQN2gt-ltQNgt2

lthellipgt standards for average over the exact distribution

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 31: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

31

Estimating Variance

22

1 1

int

1

var( ) ( ) 1

var( )

N

N t s t st s

N

t N

Q Q Q QN

tQf t

N N

QN

H Műller-Krumbhaar and K Binder J Stat Phys 8 (1973) 1

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 32: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

32

Error Formulabull The above derivation gives the well-

known error estimate in Monte Carlo as

where var(Q) = ltQ2gt-ltQgt2 can be estimated by sample variance of Qt

intvar( ) 1Error N

QN N

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 33: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

33

Time-Dependent Correlation Function and

Integrated Correlation Time

bull We define

and

22( ) s s t s s t

s s

Q Q Q Qf t

Q Q

int0 1 2 1

( ) 1 2 ( )t t

f t f t

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 34: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

34

Critical Slowing Down

Tc T

The correlation time becomes large near Tc For a finite system (Tc) Lz with dynamical critical exponent z asymp 2 for local moves

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 35: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

35

Much Reduced Critical Slowing Down

Comparison of exponential correlation times of Swendsen-Wang with single-spin flip Metropolis at Tc for 2D Ising model

From R H Swendsen and J S Wang Phys Rev Lett 58 (1987) 86

Lz

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 36: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

36

Comparison of integrated autocorrelation times at Tc for 2D Ising model

J-S Wang O Kozan and R H Swendsen Phys Rev E 66 (2002) 057101

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 37: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

37

Wolff Single-Cluster Algorithm

void flip(int i int s0) int j nn[Z] s[i] = - s0 neighbor(inn) for(j = 0 j lt Z ++j) if(s0 == s[nn[j]] ampamp drand48() lt p) flip(nn[j] s0)

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 38: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

38

Replica Monte CarloReplica Monte CarloReplica Monte CarloReplica Monte Carlo

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 39: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

39

Slowing Down at First-Order Phase Transition

bull At first-order phase transition the longest time scale is controlled by the interface barrier

where β=1(kBT) σ is interface free energy d is dimension L is linear size

12 dLe

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 40: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

40

Spin Glass Model+

+

++

+

+

+

+

+

+ +

+++

+

+

+

+

++

+

+ ++

-

-

- -- -

-- -

- -- -

-- - - -- - -

- - - -

A random interaction Ising model - two types of random but fixed coupling constants (ferro Jij gt 0) and (anti-ferro Jij lt 0)

( ) ij i jij

E J

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 41: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

41

Replica Monte Carlobull A collection of M systems at

different temperatures is simulated in parallel allowing exchange of information among the systems

β1 β2 β3 βM

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 42: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

42

Moves between Replicas

bull Consider two neighboring systems σ1 and σ2 the joint distribution is

P(σ1σ2) exp[-β1E(σ1) ndashβ2E(σ2)] = exp[-Hpair(σ1 σ2)]

bull Any valid Monte Carlo move should preserve this distribution

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 43: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

43

Pair Hamiltonian in Replica Monte Carlo

bull We define i=σi1σi

2 then Hpair can be rewritten as

1 1pair

1 2

where

( )

ij i jij

ij i j ij

H K

K J

The Hpair again is a spin glass If β1asympβ2 and two systems have consistent signs the interaction is twice as strong if they have opposite sign the interaction is 0

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 44: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

44

Cluster Flip in Replica Monte Carlo

= +1 = -

1

Clusters are defined by the values of i of same sign The effective Hamiltonian for clusters is

Hcl = - Σ kbc sbsc

Where kbc is the interaction strength between cluster b and c kbc= sum over boundary of cluster b and c of Kij

bc

Metropolis algorithm is used to flip the clusters ie σi

1 -gt -σi1 σi

2 -gt -σi2 fixing

for all i in a given cluster

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 45: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

45

Comparing Correlation Times

Correlation times as a function of inverse temperature β on 2D plusmnJ Ising spin glass of 32x32 lattice

From R H Swendsen and J S Wang Phys Rev Lett 57 (1986) 2607

Replica MC

Single spin flip

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 46: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

46

2D Spin Glass Susceptibility

2D +-J spin glass susceptibility on 128x128 lattice 18x104 MC steps

From J S Wang and R H Swendsen PRB 38 (1988) 4840

K511 was concluded

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 47: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

47

Heat Capacity at Low T

c T 2exp(-2JT)

This result is confirmed recently by Lukic et al PRL 92 (2004) 117202slope = -

2

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 48: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

48

Monte Carlo Renormalization Group

YH defined by

with RG iterations for difference sizes in 2D

From J S Wang and R H Swendsen PRB 37 (1988) 7745

H

( ) ( 1)

( ) ( )

( ) ( )

[ ]2

[ ]

n nJy

n nJ

n ni

i

q q

q q

q

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 49: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

49

MCRG in 3D3D result of YH

MCS is 104 to 105 with 23 samples for L= 8 8 samples for L= 12 and 5 samples for L= 16

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems

Page 50: 1 Cluster Monte Carlo Algorithms: Jian-Sheng Wang National University of Singapore

50

Conclusionbull Monte Carlo methods have broad

applicationsbull Cluster algorithms eliminate the

difficulty of critical slowing downbull Replica Monte Carlo works on

frustrated and disordered systems