A Simple Model for Sequences of Relational State Descriptions

EC

ML

- 08

Ingo Thon, Niels Landwehr, Luc De Raedt

Thursday 18. September 2008

EC

ML

- 08

Outline

Introduce a rarely investigated setting

Sequences where states are relational interpretations (simplified: graphs)

Define a simple model allowing for efficient inference techniques

Experimental evaluation shows1.able to capture real worlds domain dynamics

2.scalability

EC

ML

- 08

Sequences of Relational State Descriptions

Interactions of (human/artificial) agentsBusiness applications

Computer games

EC

ML

- 08

Sequences of Relational State Descriptions

Interactions of (human/artificial) agentsBusiness applications

Computer games

Computer network modeling

EC

ML

- 08

Sequences of Relational State Descriptions

Interactions of (human/artificial) agentsBusiness applications

Computer games

Computer network modeling

Music

EC

ML

- 08

Sequences of Relational State Descriptions

Interactions of (human/artificial) agentsBusiness applications

Computer games

Computer network modeling

Music

Traffic Modeling

EC

ML

- 08

Sequences of Relational State Descriptions

Interactions of (human/artificial) agentsBusiness applications

Computer games

Computer network modeling

Music

Traffic Modeling

....

EC

ML

- 08

Sequences of Relational State Descriptions

Interactions of (human/artificial) agentsBusiness applications

Computer games

Computer network modeling

Music

Traffic Modeling

....

EC

ML

- 08

Sequences of Relational State Descriptions

Computer games

EC

ML

- 08

Welcome to Travian

Travian: A massively multiplayer real-time strategy browser gameCommercial game run by TravianGames GmbH

~3.000.000 players spread over different “worlds”

~25.000 players in one world

Classical strategy gamePlayed on a grid map

Cities, resources, buildings, technology, armies

Attacks/Conquests, trading

Alliances: Boxes/Color-coded

Players: Diamonds

Cities: Circles

Fat arrows Conquests

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

EC

ML

- 08

City/Player/Alliance Graph

Alliances: Boxes/Color-coded

Players: Diamonds

Cities: Circles

Fat arrows Conquests

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

~10 alliances~200 players~600 cities

border

border

border

border

Alliance 1

Alliance 2

Alliance 3

Alliance 4

Alliance 5

Alliance 6

Alliance 7

Alliance 9

Alliance 10

Alliance 11

P 1

742

885

809

888

971

753

647

650

460

706

P 2

1010

994

978

P 3

1029

846

1009

991645

P 5

900

1002

647

955

968

1036

P 6

1025

701

647

P 7

984

867

831

931

930

937

828

997

789

845

697

862

883

735

630

631

P 8

971

949

970

977

P 12

1002

1003

854

P 13

626

606

P 14

798

709

P 15

885

969

721

953

P 16

994

745886

P 17

922

816

P 18

961

937

932942

959

1004

869

920

896

P 19

1051

1023

972

569

759

852

859

P 21

750

776

918

948

P 22

833

813

885

663

P 24

958 912

659

892840

609

854

P 26

767

P 28

915

P 30

941

872

P 33

872762

P 34

1030

825

P 36

904

1003

1028 1036

972 976

982

920

823

783

717

P 37

1033

919

900

766

P 38

732

856

P 40

707

657

P 41

951

984

811

765574

P 43

815

901

P 45

735

P 46

1005

863

P 47

991

826

638

P 50

902

877

887

P 51

994

1031

760

896

606

P 55

672

629

P 56

1046

1017

802

986

P 57

1008

946

940

941

1038

838

951

1020

922

10041015

900

861

P 59

667

903 559

P 61

1021

736

845

672

860

P 63

965

978

808

P 64

979 948

881

902

P 66

903

864947

891

811

710

643

P 67

526

435

P 68

657

636

592

P 70

946845

943

818

P 72

802

644

697

P 74

811

795

1007

839

931

784709

P 75

1080

976

813

1008

P 76

999

802

P 77

9171005

667

P 78

950

549

P 81

1010

984

961

839

1054

1044

905

1048

934

768

930

842

P 82

799

P 83

761

P 85

552

667

586

P 86

763

725

784

P 88

874

830769 746

P 89

862

601

P 91

1049

863

P 93

P 95

749

972

937

964

P 97

944

P 98

1006

P 99

919

P 100

851 1021

815

P 101

859818

772

P 105

922

872

776

856

826

P 106

762

P 107

750

P 111

992

1014

858

472

772

648 517

P 116

1002

986

592

P 120

922

976

968

916

799

732

777

661

P 121

1059

1029

980

998

962

769

P 122

1089

933

1001

P 123

799

P 124

10151063

957

954

P 125

826

869

920

634

P 126

1190

640

859

912P 127

810

806

924

849

880

625

731

P 128

898

P 129

648

1001

675

882

P 132

782

593

480

P 134

740

892

707

P 136

725

757

732

869

965

759

844

P 137

668

681

754

765

677

P 138

687

982

737

716

789

904P 139

806

982

615

P 140

948

964

770

P 141

876

832

885

909585

P 143

1033

700

P 144

895

P 146

1106

1068

P 150

1008

811

832

P 153

919

969

783

905

991

954

946

910

888804

596

P 154

926 884

P 156

760

947

625 611

P 158

922

P 159

965

P 167

873

858

762

847

833

785697

P 172

872

632

641

742

994

P 174

533582

639

P 176

605

P 177

649

746 731

P 179

742

887861

574

P 180

967

P 181

1036

1009

966

1039 1046

1020

683

668

1026

1030

P 183

916

863

716

P 185

965

968

994

844

854

803

P 187

934

P 189

1005

751

883

989

796

1008

1022

825

810

P 191

924

856

P 193

756

P 195

772

576

P 199

975

959

P 200

850

P 201

736

612

P 202

911

956

870

960

879

867

710

809

798

902

692

836

809

P 203

786

P 205

917

945

582

946

915

732

673

P 212

929

523698

701

566 585

P 213

623

P 217

1000

980

1071

978

1015

1066

940

P 220

948

943

792

P 221

806

794

704

625

587

P 222

899

813

790

P 223

795

P 224

901

791

477

P 228

709619

677

P 230

852

P 231

808

886

833

725

P 233

611

P 234

754

P 235

982

495

908

804

P 236

1230

784

591

617P 238

556

P 239

957

903

1046

1036

989

925

1055

1063

828

1046

878

P 240

1015

940

860

P 242

1069

909

956

1059 898

929

1035

820

892

1059

1039

950

997

853 797

929

965

757

916

P 244

745

724

808

718

P 246

781

787

776

939

851

EC

ML

- 08

City/Player/Alliance Graph

Alliances: Boxes/Color-coded

Players: Diamonds

Cities: Circles

Fat arrows Conquests

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

~10 alliances~200 players~600 cities

border

border

border

border

Alliance 1

Alliance 2

Alliance 3

Alliance 4

Alliance 5

Alliance 6

Alliance 7

Alliance 9

Alliance 10

Alliance 11

P 1

742

885

809

888

971

753

647

650

460

706

P 2

1010

994

978

P 3

1029

846

1009

991645

P 5

900

1002

647

955

968

1036

P 6

1025

701

647

P 7

984

867

831

931

930

937

828

997

789

845

697

862

883

735

630

631

P 8

971

949

970

977

P 12

1002

1003

854

P 13

626

606

P 14

798

709

P 15

885

969

721

953

P 16

994

745886

P 17

922

816

P 18

961

937

932942

959

1004

869

920

896

P 19

1051

1023

972

569

759

852

859

P 21

750

776

918

948

P 22

833

813

885

663

P 24

958 912

659

892840

609

854

P 26

767

P 28

915

P 30

941

872

P 33

872762

P 34

1030

825

P 36

904

1003

1028 1036

972 976

982

920

823

783

717

P 37

1033

919

900

766

P 38

732

856

P 40

707

657

P 41

951

984

811

765574

P 43

815

901

P 45

735

P 46

1005

863

P 47

991

826

638

P 50

902

877

887

P 51

994

1031

760

896

606

P 55

672

629

P 56

1046

1017

802

986

P 57

1008

946

940

941

1038

838

951

1020

922

10041015

900

861

P 59

667

903 559

P 61

1021

736

845

672

860

P 63

965

978

808

P 64

979 948

881

902

P 66

903

864947

891

811

710

643

P 67

526

435

P 68

657

636

592

P 70

946845

943

818

P 72

802

644

697

P 74

811

795

1007

839

931

784709

P 75

1080

976

813

1008

P 76

999

802

P 77

9171005

667

P 78

950

549

P 81

1010

984

961

839

1054

1044

905

1048

934

768

930

842

P 82

799

P 83

761

P 85

552

667

586

P 86

763

725

784

P 88

874

830769 746

P 89

862

601

P 91

1049

863

P 93

P 95

749

972

937

964

P 97

944

P 98

1006

P 99

919

P 100

851 1021

815

P 101

859818

772

P 105

922

872

776

856

826

P 106

762

P 107

750

P 111

992

1014

858

472

772

648 517

P 116

1002

986

592

P 120

922

976

968

916

799

732

777

661

P 121

1059

1029

980

998

962

769

P 122

1089

933

1001

P 123

799

P 124

10151063

957

954

P 125

826

869

920

634

P 126

1190

640

859

912P 127

810

806

924

849

880

625

731

P 128

898

P 129

648

1001

675

882

P 132

782

593

480

P 134

740

892

707

P 136

725

757

732

869

965

759

844

P 137

668

681

754

765

677

P 138

687

982

737

716

789

904P 139

806

982

615

P 140

948

964

770

P 141

876

832

885

909585

P 143

1033

700

P 144

895

P 146

1106

1068

P 150

1008

811

832

P 153

919

969

783

905

991

954

946

910

888804

596

P 154

926 884

P 156

760

947

625 611

P 158

922

P 159

965

P 167

873

858

762

847

833

785697

P 172

872

632

641

742

994

P 174

533582

639

P 176

605

P 177

649

746 731

P 179

742

887861

574

P 180

967

P 181

1036

1009

966

1039 1046

1020

683

668

1026

1030

P 183

916

863

716

P 185

965

968

994

844

854

803

P 187

934

P 189

1005

751

883

989

796

1008

1022

825

810

P 191

924

856

P 193

756

P 195

772

576

P 199

975

959

P 200

850

P 201

736

612

P 202

911

956

870

960

879

867

710

809

798

902

692

836

809

P 203

786

P 205

917

945

582

946

915

732

673

P 212

929

523698

701

566 585

P 213

623

P 217

1000

980

1071

978

1015

1066

940

P 220

948

943

792

P 221

806

794

704

625

587

P 222

899

813

790

P 223

795

P 224

901

791

477

P 228

709619

677

P 230

852

P 231

808

886

833

725

P 233

611

P 234

754

P 235

982

495

908

804

P 236

1230

784

591

617P 238

556

P 239

957

903

1046

1036

989

925

1055

1063

828

1046

878

P 240

1015

940

860

P 242

1069

909

956

1059 898

929

1035

820

892

1059

1039

950

997

853 797

929

965

757

916

P 244

745

724

808

718

P 246

781

787

776

939

851

Can we model world dynamics?

How do players act, and why?

Can we predict how the world will evolve?

EC

ML

- 08

City/Player/Alliance Graph

EC

ML

- 08

World Dynamics

Can players’ actions be explained from “social context”?

Typical patterns conquer a city which is close

conquer a city of a player that has been attacked by alliance member

tendency of alliance to conquer whole areas on the map (clusters)

retaliate against attacks at an alliance level

change alliance if you are under attack

border

border

border

border

Alliance 2

Alliance 4

Alliance 6

P 2

9181090

931

779

977

835

781

9581087

808

701

P 3

838

947

1026

1081

833

1002987

827

994

663

P 5

1032

1026

1024

1049

905

926

P 6

986

712

985

920

877

807

P 7

895

959

P 10

824

EC

ML

- 08

World Dynamics

Can players’ actions be explained from “social context”?

Typical patterns conquer a city which is close

conquer a city of a player that has been attacked by alliance member

tendency of alliance to conquer whole areas on the map (clusters)

retaliate against attacks at an alliance level

change alliance if you are under attack

border

border

border

border

Alliance 2

Alliance 4

P 2

948

951

786

990

856

795

980

828

730

P 3898

803

860

964

1037

1085

925

689

10051007

899

1005

760

P 5

1051

1051

1040

860

774

1061

886

944

844

945

713

P 10

839

796838

EC

ML

- 08

World Dynamics

Can players’ actions be explained from “social context”?

Typical patterns conquer a city which is close

conquer a city of a player that has been attacked by alliance member

tendency of alliance to conquer whole areas on the map (clusters)

retaliate against attacks at an alliance level

change alliance if you are under attack

border

border

border

border

Alliance 2

Alliance 4

P 2

948

951

786

990

856

795

980

828

730

P 3898

803

860

964

1037

1085

925

689

10051007

899

1005

760

P 5

1051

1051

1040

860

774

1061

886

944

844

945

713

P 10

839

796838

Relations Important

➙Statistical Relational Learning

EC

ML

- 08

CPT-L

EC

ML

- 08

CPT-L

Standard SRL:Very general, expressive models

Not specifically tailored to sequences (of interpretations)

➡ No guarantee for tractability

EC

ML

- 08

CPT-L

Standard SRL:Very general, expressive models

Not specifically tailored to sequences (of interpretations)

➡ No guarantee for tractability

Generative probabilistic model for sequences of interpretations

Based on CP-Logic (Causal Probabilistic Logic)

Efficiency byFocussing on sequences

Assuming data fully observable

Use of efficient data structures (BDD) for inference

CPT-L:

Idea: Model is Markov Chain

EC

ML

- 08

CPT-L

P (I0, . . . IT |T ) = P (I0)!

t=1...T

P (It|It!1, T )

Idea: Model is Markov Chain

EC

ML

- 08

CPT-L

P (I0, . . . IT |T ) = P (I0)!

t=1...T

P (It|It!1, T )

Close

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

C

C2

Owner Attacker

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

P (It|It!1, T )

conquer a city which is close

EC

ML

- 08

CPT-Rules

border

border

border

border

Alliance 2

Alliance 4

Alliance 6

P 2

9231090

941

784

983

844

786

9661087

815

711

P 3

864

986

842

1032

1083

868

712

10021000

858

996

696

P 5

1039

1037

1030

1053

826

933

P 6

985

807

P 7

894

963

P 10

829

781828

b1, . . . bn ! h1 : p1 " . . . " hm : pm

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

conquer a city which is closecity(C, Owner), city(C2, Attacker), close(C, C2)

EC

ML

- 08

CPT-Rules

border

border

border

border

Alliance 2

Alliance 4

Alliance 6

P 2

9231090

941

784

983

844

786

9661087

815

711

P 3

864

986

842

1032

1083

868

712

10021000

858

996

696

P 5

1039

1037

1030

1053

826

933

P 6

985

807

P 7

894

963

P 10

829

781828

cause (past)b1, . . . bn ! h1 : p1 " . . . " hm : pm

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

Close

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

C

C2

Owner Attacker

conquer a city which is close

EC

ML

- 08

CPT-Rules

border

border

border

border

Alliance 2

Alliance 4

Alliance 6

P 2

9231090

941

784

983

844

786

9661087

815

711

P 3

864

986

842

1032

1083

868

712

10021000

858

996

696

P 5

1039

1037

1030

1053

826

933

P 6

985

807

P 7

894

963

P 10

829

781828

cause (past) effect (future)

b1, . . . bn ! h1 : p1 " . . . " hm : pm

city(C, Owner), city(C2, Attacker), close(C, C2)!conquest(Attacker, C) : p " nil : (1# p)

Close

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

C

C2

Owner Attacker

GreeksArchilleus

Menelaos

Paris

Troys

Troy

Sparta

Mykena

Pelion

EC

ML

- 08

Tasks

EC

ML

- 08

Tasks

Sampling: Random sequence of states given a theory

Inference: Probability of sequence of states

Parameter Estimation: Given a set of sequences of states, what is the most likely parameterization of a theory

Prediction: Given a sequence of states and a formula the probability of coming true in steps

I0, . . . , IT TP (I0, ..., IT | T )

I0, . . . , IT

I0, . . . , IT

P (It+d ! F |I0, . . . , IT , T )F

F d

EC

ML

- 08

Toy Example + Sampling

Toy Example: r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

EC

ML

- 08

Toy Example + Sampling

Toy Example:

Sampling successor of state {a,b}:

r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

EC

ML

- 08

Toy Example + Sampling

Toy Example:

Sampling successor of state {a,b}:1. Select rules applicable {r1,r2,r33}

r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

EC

ML

- 08

Toy Example + Sampling

Toy Example:

Sampling successor of state {a,b}:1. Select rules applicable {r1,r2,r33}

2. For each rule select one successor {r1:a,r2:nil,r33:a}

r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

EC

ML

- 08

Toy Example + Sampling

Toy Example:

Sampling successor of state {a,b}:1. Select rules applicable {r1,r2,r33}

2. For each rule select one successor {r1:a,r2:nil,r33:a}

3. yield successor state {a}

r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

Model:

Naive Implementation of e.g..

EC

ML

- 08

Probability Calculation: Naive

r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

P({a,b}|{a})P (St+1|St)

Model:

Naive Implementation of e.g..Enumerate all rules with

EC

ML

- 08

Probability Calculation: Naive

r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

P({a,b}|{a})P (St+1|St)

St ! b1, . . . , bn

b1, . . . , bn ! h1 : p1 " . . . " hm : pm

Model:

Naive Implementation of e.g..Enumerate all rules with

Generate all “selections” s.t. for each rule one is select

Probability of selection is product of ‘s

{r1:a,r31:a}:0.48,{r1:a,r31:b}:0.16,{r1:a}:0.16, {r31:a}:0.12,{r31:b}:0.04,{}:0.02

EC

ML

- 08

Probability Calculation: Naive

r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

P({a,b}|{a})P (St+1|St)

St ! b1, . . . , bn

b1, . . . , bn ! h1 : p1 " . . . " hm : pm

hi

pi

Model:

Naive Implementation of e.g..Enumerate all rules with

Generate all “selections” s.t. for each rule one is select

Probability of selection is product of ‘s

Probability of state is sum of probability of selections generating

{r1:a,r31:a}:0.48,{r1:a,r31:b}:0.16,{r1:a}:0.16, {r31:a}:0.12,{r31:b}:0.04,{}:0.02

EC

ML

- 08

Probability Calculation: Naive

r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

P({a,b}|{a})P (St+1|St)

St ! b1, . . . , bn

b1, . . . , bn ! h1 : p1 " . . . " hm : pm

hi

pi

St+1

{a,b}:0.16,{a}:0.76,{b}:0.04,{}:0.04

Model:

Query: applicable rules r1, r31

Min. one:

Only one:

Generate:

Calculate probability by translating into BDD

EC

ML

- 08

Probability Calculation: Efficient Algorithm

r1 : (.8 :: a ! .2 :: nil) " ar2 : (.9 :: b ! .1 :: nil) " br31 : (.6 :: a ! .2 :: b ! .2 :: nil) " a, br32 : (.2 :: a ! .6 :: b ! .2 :: nil) " a, br33 : (.4 :: a ! .4 :: b ! .2 :: nil) " a, b

(r1 : a ! r1 : nil) " (r31 : a ! r31 : b ! r31 : nil)"(r1 : a ! r1 : b) " (r31 : a ! r31 : b) " (r31 : a ! . . .

(r1 : a ! r31 : a) " r31 : b

P({a,b}|{a})

!1 = {r1:a, r2:b, r3:b}!2 = {r1:a, r2:b, r3:nil}

!4 = {r1:nil, r2:a, r3:b}!3 = {r1:a, r2:a, r3:b}

possible selections:

EC

ML

- 08

Efficient calculation

Efficient calculation using Dynamic Programing

r1:a

r1:

nil

r2:a

r3:b

r3:

nil

01

r2:b

r1:

nil

r2:a

r2:b r2:b

r3:b

r3:

nil

r3:

nil

!1 = {r1:a, r2:b, r3:b}!2 = {r1:a, r2:b, r3:nil}

!4 = {r1:nil, r2:a, r3:b}!3 = {r1:a, r2:a, r3:b}

possible selections:

EC

ML

- 08

Efficient calculation

Efficient calculation using Dynamic Programing

11

p3:b

p3:b + p3:nil

= 1

pr2:b +pr2:ap3:b

P ({a, b}|true) =pr1:a(pr2:b + pr2:ap3:b)+pr1:nilpr2:apr3:b

r1:a

r1:

nil

r2:a

r3:b

r3:

nil

01

r2:b

r1:

nil

r2:a

r2:b r2:b

r3:b

r3:

nil

r3:

nil

pr3:nil

EC

ML

- 08

Learning

Problem: Given observation e.g. {a},{a}Hidden Parameter: Which Rule generated which Element

EM-AlgorithmExpectation: Calculate Probability for each Selection, turning it into fully Observable Problem

Maximization: Set Parameters according to Probability of Selections

r1 : (p1,a :: a ! p1,nil :: nil) " ar31 : (p31,a :: a ! p31,b :: b ! p31,nil :: nil) " a, b

!1 = {r1 : a, r31 : a} p1,a · p31,a

!2 = {r1 : a, r31 : nil} p1,a · p31,nil

!3 = {r1 : nil, r31 : a} p1,nil · p31,nil

p!1,a =

p1,a · p31,a + p1,a · p31,nil

p1,a · p31,a + p1,a · p31,nil + p1,nil · p31,nil=

p1,a

p

EC

ML

- 08

Experiment Blocks-World: Setup

Goal: show that model scales with world complexity

Stochastic Blocks-World (stacks can collapse, stacking can fail,...)

Complexity increases with number of blocks

EC

ML

- 08

Experiment Blocks-World: Setup

Goal: show that model scales with world complexity

Stochastic Blocks-World (stacks can collapse, stacking can fail,...)

Complexity increases with number of blocks

c b

a

c ba a c b

-10000

-9000

-8000

-7000

-6000

-5000

-4000

-3000

-2000

-1000

0 2 4 6 8 10

Log-L

ikelih

ood

Iterations of the EM-Algorithm

10 blocks25 blocks50 blocks

0

2

4

6

8

10

12

10 20 30 40 50

runtim

e [m

inute

s] fo

r 10 ite

rations

Number of blocks

runtime

Blocks-World: convergence behaviour

-10000

-9000

-8000

-7000

-6000

-5000

-4000

-3000

-2000

-1000

0 2 4 6 8 10

Lo

g-L

ike

liho

od

Iterations of the EM-Algorithm

10 blocks25 blocks50 blocks

EC

ML

- 08

Experimental Evaluation I

Blocks-World: runtime behavior

EC

ML

- 08

Experimental Evaluation I

0

2

4

6

8

10

12

10 20 30 40 50

run

tim

e [

min

ute

s]

for

10

ite

ratio

ns

Number of blocks

runtime

EC

ML

- 08

Experiment Travian: Setup

Goal: show that model works on real world problems.

Logged the state of a “live” game server:~30.000 players / ~3 months /snapshot of the world every 24h

Projecting down to a subset of players/cities/alliancesfollowing 10 players (~50 cities) over a period of one month

30 such sequences extracted from distinct parts of the map

Learn a model from December data

Want to predict conquests(P,C) in given... for January and February data (up to five steps in future)

It+5 It

EC

ML

- 08

Experimental Evaluation II

Travian: predict player behavior (attack of a city)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

tru

e p

ositiv

e r

ate

false positive rate

CPT-L, k = 1CPT-L, k = 2CPT-L, k = 3CPT-L, k = 4CPT-L, k = 5

EC

ML

- 08

Experimental Evaluation II

Travian: predict player behavior (attack of a city)

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

1 2 3 4 5

Are

a u

nder

RO

C c

urv

e

Number of steps predicted (k)

January 2008February 2008

EC

ML

- 08

Conclusions

Sequences of relational state descriptions

CPT-L: simple model for sequences of relational state descriptions addresses:1.efficiency

2.rather than maximal expressivity

Experimental evaluation of complex relational sequences 1.able to capture domain dynamics

2.scalability

EC

ML

- 08

Thanks!