Pricing Football Playersusing Neural Networks

Sourya Dey

Final Project - Neural Learning and Computational Intelligence

April 19th 2017, University of Southern California

How the Football Transfer Market Works

2

Club A

Clinical goalscorerReliable defender

Potential youngsterInternational Star

Nominal Price = $2 Million

Club C

Club B

How the Football Transfer Market Works

2

Club A

Clinical goalscorerReliable defender

Potential youngsterInternational Star

Nominal Price = $2 Million

Club C

Club B

Initial Bid = $2.2 Million

Rival Bid = $2.1 Million

How the Football Transfer Market Works

2

Club A

Clinical goalscorerReliable defender

Potential youngsterInternational Star

Nominal Price = $2 Million

Club C

Club B

Initial Bid = $2.2 Million

Rival Bid = $2.1 Million

Asking Price = $2.5 Million

How the Football Transfer Market Works

2

Club A

Clinical goalscorerReliable defender

Potential youngsterInternational Star

Nominal Price = $2 Million

Club C

Club B

Initial Bid = $2.2 Million

Rival Bid = $2.1 Million

Asking Price = $2.5 MillionCan’t match asking price

Counter Bid = $2.3 Million

How the Football Transfer Market Works

2

Club A

Clinical goalscorerReliable defender

Potential youngsterInternational Star

Nominal Price = $2 Million

Club C

Club B

Initial Bid = $2.2 Million

Rival Bid = $2.1 Million

Asking Price = $2.5 MillionCan’t match asking price

Counter Bid = $2.3 Million

Negotiate

How the Football Transfer Market Works

2

Club A

Clinical goalscorerReliable defender

Potential youngsterInternational Star

Nominal Price = $2 Million

Club C

Club B

Initial Bid = $2.2 Million

Rival Bid = $2.1 Million

Asking Price = $2.5 MillionCan’t match asking price

Counter Bid = $2.3 Million

Negotiate

How the Football Transfer Market Works

2

Club A

Clinical goalscorerReliable defender

Potential youngsterInternational Star

Nominal Price = $2 Million

Club C

Club B

Initial Bid = $2.2 Million

Rival Bid = $2.1 Million

Asking Price = $2.5 MillionCan’t match asking price

Counter Bid = $2.3 Million

Negotiate

SOLD for $2.4 Million

Seal deal before transfer window closes!

Money Talks

3

The January I moved from Birmingham to Blackburn, I sat there by the phone for 13 days, worrying, waiting for it to go through - I lost a stone.

“

”-- Robbie Savage

Former Premier League Midfielder Football Focus Special, 2011

Negotiations are a big part of football. What is the optimum price for a player?

Data Source: FIFA 2017

4

A football simulation video game with official licenses for most major teams and players in the world

FIFA 2017 in-game image

Player Attributes

5

Authenticity of Data

6

Season Ticket Holders

Managers

Feedback from 9000 data reviewers

Scouts

300 data points per

player

Stats updated weekly on EA servers as per

real-life performance

Criticisms of Data Authenticity

u Subjectivity by EA

u Not enough data on fledgling players

u Imperfect formula

7

If Messi were playing in the Irish league, his attributes would drop simply because he's not on the highest level anymore.

“

”-- Michael Mueller-Moehring

EA Sports Producer and Ratings Specialist Interview with ESPN FC, September 2016

Gathering Data – sofifa.com

8

Prices

9Original Bottom-Heavy Prices

Rounded Min Price = $43K Rounded Max

Price = $37M

Most Common Price = $1.1M

Rounded and Categorized

Occur as quantized values except for a few outliers

?. . .

x41

.

.

. x119

Network Structure

u 119 output price values (categories)

u 41 input features u 37 on a scale of 0–99

u Age on a range of 16-43

u 3 on a scale of 1-5 stars

u Normalized to 𝜇 = 0, 𝜎 = 1

10

Data Splits

11

0

2000

4000

6000

8000

10000

12000

14000

16000

Training: 71% 10914 samples

Validation: 13% 1926 samples

Test: 16% 2500 samples

Total:

15340 players

Network Experiments

u Activation functions

u Number and size of hidden layers

u Learning rate and its decay

u Regularization (L2)

u Nesterov Momentum

u Early stopping to prevent overfitting

12

Activation Functions

13

Perc

enta

ge A

ccur

acy

0

1.5

3

4.5

6

Number of Hidden Neurons

500 1500 2500 3500 4500

Sigmoid+Softmax Tanh+Softmax ReLU+Softmax ReLU+Sigmoid

ReLU for all hidden layers and softmax for output layer works best

Size of 1st Hidden Layer

14

No real trend. Max accuracy for 3900 neurons. I picked 2000 neurons for final model after optimizing other parameters.

Perc

enta

ge A

ccur

acy

2

3

4

5

6

Number of Hidden Neurons

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

Perc

enta

ge A

ccur

acy

10

13

16

19

22

Learning Rate

0.008 0.01 0.033 0.067 0.1 0.133 0.167 0.2 0.25 0.3

200 1000 2000 3000

Learning rate 𝜂

15

0

5.25

10.5

15.75

21

Learning Rate

1.0E-07 1.0E-06 1.0E-05 1.0E-04 1.0E-03 1.0E-02

Perc

enta

ge A

ccur

acy

Number of neurons in 2nd hidden layer

Picked eta = 0.01

Size of subsequent hidden layers

16

Perc

enta

ge A

ccur

acy

17

18

19

20

21

X = Number of neurons in 2nd hidden layer

500 1000 1500 2000 2500

Perc

enta

ge A

ccur

acy

21

22.5

24

25.5

27

X = Number of neurons in 3rd hidden layer

500 1000 1500 2000 2500

Network config = [41,2000,X,119] Picked X = 1500

Network config = [41,2000,1500,X,119] Picked X = 500

Final network configuration = [41,2000,1500,500,119]

Learning rate decay (Annealing)

17

𝜂𝑡 =𝜂0

1 + 𝒌𝑡

0

3.75

7.5

11.25

15

Learning Rate Decay Coefficient k

1.0E-05 1.0E-04 1.0E-03 1.0E-02

Perc

enta

ge A

ccur

acy

Picked k = 0.001

Nesterov Momentum

18

𝑤𝑡+1 = 𝑤𝑡 − 𝜂𝜕(𝐶𝑜𝑠𝑡)

𝜕𝑤𝑡

𝑤𝑡+1 = (𝑤 − 𝝁 ∆ 𝑤)𝑡− 𝜂

𝜕(𝐶𝑜𝑠𝑡)𝜕(𝑤 − 𝝁 ∆ 𝑤)

𝑡

Ordinary Gradient Descent Update

Nesterov Momentum Update

Perc

enta

ge A

ccur

acy

24

27

30

33

36

Momentum Coefficient mu

0.8 0.9 0.99

Picked Nesterov = 0.99𝜇

𝑤𝑡+1 = (𝑤 − 𝝁 ∆ 𝑤)𝑡− 𝜂

𝜕(𝐶𝑜𝑠𝑡)𝜕𝑤

𝑡

Momentum Update

L2 Regularization

19

𝐶𝑜𝑠𝑡 + = 𝝀∑ 𝑤2

35

36.5

38

39.5

41

L2 parameter lambda

1.0E-08 1.0E-07 1.0E-06 1.0E-05 1.0E-04 1.0E-03

Perc

enta

ge A

ccur

acy

Picked = 5e-4 = 0.0005𝜆

Early Stopping

Prevent overfitting by stopping training when validation accuracy doesn’t improve over some threshold number of epochs

20

Picked threshold = 10 epochs

Final Training Conditions

u Network configuration = [41,2000,1500,500,119]

u ReLU activation for all hidden layers, finally softmax output

u L2 Regularization Coefficient = 0.0005

u Learning rate = 0.01, annealing coefficient = 0.001

u Nesterov momentum coefficient = 0.99

u Minibatch size = 20

21

Early stopping at 60 epochs

Accuracy Metrics

22

Actual Predicted

PredictedActual

PredictedActual

(Top-1) Accuracy: Prediction exactly matches actual price

Top-3 Accuracy

Top-5 Accuracy

40.32%

71.28%

87.2%

Average Percentage Error (APE) in Price

23

𝐴𝑃𝐸 = Avg𝐴𝑙𝑙 𝑇𝑒𝑠𝑡 𝑆𝑎𝑚𝑝𝑙𝑒𝑠

𝑇𝑟𝑢𝑒 𝑝𝑟𝑖𝑐𝑒 − 𝑃 𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 𝑝𝑟𝑖𝑐𝑒

𝑇𝑟𝑢𝑒 𝑝𝑟𝑖𝑐𝑒× 100

APE = 6.32%

Potential Improvements

u Star outliers hard to predict

u Goalkeeping stats not considered

u Training beyond ~60 epochs doesn’t improve u Vanishing gradients: Use different eta for each layer

24

Lionel Messi: $96 Million

Manuel Neuer: $75 Million

Additional Details

u Accompanying report u More explanations

u References

u Code Available on Github

u Tools Used u Python 2.7

u Theano

u Keras

25

Ending Quote

If you have the money and you find the one player who can make you win and make the difference, no matter how expensive he is, you should do it. But there are not many players in the world who will make a real difference.

26

“

”-- Arsene Wenger

Manager, Arsenal Football Club Interview with The Times, 2009