42
IN DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING, SECOND CYCLE, 30 CREDITS , STOCKHOLM SWEDEN 2018 Tracking a ball during bounce and roll using recurrent neural networks FELICIA ROSELL KTH ROYAL INSTITUTE OF TECHNOLOGY SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

  • Upload
    others

  • View
    14

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

IN DEGREE PROJECT COMPUTER SCIENCE AND ENGINEERING,SECOND CYCLE, 30 CREDITS

, STOCKHOLM SWEDEN 2018

Tracking a ball during bounce and roll using recurrent neural networks

FELICIA ROSELL

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE

Page 2: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA
Page 3: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

Tracking a ball during bounceand roll using recurrentneural networks

FELICIA ROSELL

Master in Computer ScienceDate: June 3, 2018Supervisor: Mårten BjörkmanExaminer: Danica KragicSwedish title: Följning av en boll under studs och rull med hjälp avåterkopplande neurala nätverkSchool of Computer Science and Communication

Page 4: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA
Page 5: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

iii

Abstract

In many types of sports, on-screen graphics such as an reconstructedball trajectory, can be displayed for spectators or players in order toincrease understanding. One sub-problem of trajectory reconstructionis tracking of ball positions, which is a difficult problem due to the fastand often complex ball movement.

Historically, physics based techniques have been used to track ballpositions, but this thesis investigates using a recurrent neural networkdesign, in the application of tracking bouncing golf balls. The networkis trained and tested on synthetically created golf ball shots, created toimitate balls shot out from a golf driving range.

It is found that the trained network succeeds in tracking golf ballsduring bounce and roll, with an error rate of under 11 %.

Page 6: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

iv

Sammanfattning

Grafik visad på en skärm, så som en rekonstruerad bollbana, kan an-vändas i många typer av sporter för att öka en åskådares eller spelaresförståelse. För att lyckas rekonstruera bollbanor behöver man först lö-sa delproblemet att följa en bolls positioner. Följning av bollpositionerär ett svårt problem på grund av den snabba och ofta komplexa boll-rörelsen.

Tidigare har fysikbaserade tekniker använts för att följa bollposi-tioner, men i den här uppsatsen undersöks en metod baserad på åter-kopplande neurala nätverk, för att följa en studsande golfbolls bana.Nätverket tränas och testas på syntetiskt skapade golfslag, där bollba-norna är skapade för att imitera golfslag från en driving range.

Efter träning lyckades nätverket följa golfbollar under studs ochrull med ett fel på under 11 %.

Page 7: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

Contents

1 Introduction 11.1 Research Question . . . . . . . . . . . . . . . . . . . . . . 21.2 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Background 32.1 Neural networks . . . . . . . . . . . . . . . . . . . . . . . 3

2.1.1 Feedforward neural networks . . . . . . . . . . . . 32.1.2 Recurrent neural networks . . . . . . . . . . . . . 42.1.3 Loss functions . . . . . . . . . . . . . . . . . . . . . 42.1.4 Output functions . . . . . . . . . . . . . . . . . . . 52.1.5 Optimization algorithms . . . . . . . . . . . . . . . 52.1.6 Teacher forcing . . . . . . . . . . . . . . . . . . . . 62.1.7 Long short-term memory . . . . . . . . . . . . . . 6

2.2 Forces affecting golf ball trajectories . . . . . . . . . . . . 7

3 Related work 83.1 Trajectory tracking . . . . . . . . . . . . . . . . . . . . . . 83.2 Sports ball trajectory tracking . . . . . . . . . . . . . . . . 8

3.2.1 Physical model methods . . . . . . . . . . . . . . . 93.2.2 Machine learning methods . . . . . . . . . . . . . 9

4 Methods 114.1 Software and hardware . . . . . . . . . . . . . . . . . . . . 114.2 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.3 Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.3.1 Inputs . . . . . . . . . . . . . . . . . . . . . . . . . 144.3.2 Saliency map . . . . . . . . . . . . . . . . . . . . . 154.3.3 Hidden layer . . . . . . . . . . . . . . . . . . . . . 164.3.4 Output layer . . . . . . . . . . . . . . . . . . . . . . 164.3.5 Loss . . . . . . . . . . . . . . . . . . . . . . . . . . 16

v

Page 8: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

vi CONTENTS

4.3.6 Optimization algorithm . . . . . . . . . . . . . . . 174.4 Baseline algorithm . . . . . . . . . . . . . . . . . . . . . . 18

5 Results 19

6 Discussion 286.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . 306.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Bibliography 31

Page 9: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

Chapter 1

Introduction

Services in sport analytics have been used commercially for over 15years. They can for instance be used to display explaining graphics ona TV screen, such as marking the trajectory of a ball or a player. Thepurpose can be to help a spectator follow and understand the game,or to give feedback to coaches and players in order for them to im-prove their game faster. Sport analytics can be used to increase boththe viewing and playing experience, and therefore increase the pub-lic’s interest in practicing sports. Since exercise has been shown toincrease both physical and mental health[17] , this benefits society, es-pecially economically.

One important sport analytics task is reconstructing the trajectoryof a ball. Trajectory reconstruction can be divided into three sub-tasks:

• ball detection in a single image• ball tracking through several time steps• interpolation between ball positions to form a full trajectory

Ball tracking can be performed in two dimensions (2D), using asingle camera, to later be combined into a three dimensional trajec-tory. This thesis will investigate ball tracking in 2D, in the applicationof tracking a bouncing golf ball in a setting with several other balls.This is an especially difficult problem since the golf ball is very small,fast moving and has a very complex motion. Traditionally, a physicsdriven approach has been used to track golf balls, but this thesis willinvestigate using a machine learning method, specifically a recurrentneural network.

1

Page 10: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

2 CHAPTER 1. INTRODUCTION

1.1 Research Question

This report will aim to answer the question: "How well can a recurrentneural network track the 2D trajectory of a golf ball during bounce androll, in an environment with multiple golf balls?".

1.2 Delimitations

This thesis aims to track golf balls only during bounce and roll. There-fore it is assumed that there already exists some type of algorithm thathave successfully tracked ball positions before the first bounce occurs.The thesis will further not use any actual image data of golf ball trajec-tories. The data will be synthetically created and directly transformedinto positional data, without using image recognition techniques. Thesynthetically created images will not contain any noise, which sim-plifies training, since it can be performed in a perfect environment.Another benefit that comes from using synthetically created images isthat they will never contain any images of individuals. Storing suchimage data can be ethically problematic, and by this delimitation thisthesis avoids introducing any ethical problems.

Page 11: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

Chapter 2

Background

In this chapter the theory relevant to understanding the rest of thereport will be presented. The first sections will handle the theory ofneural networks, while the last section will handle the theory of theaerodynamics of a golf ball.

2.1 Neural networks

The following section will start by explaining the feedforward neuralnetwork, to continue with the more complex recurrent neural network.It will also introduce the state-of-the art architecture choices for recur-rent neural networks.

2.1.1 Feedforward neural networks

A feedforward neural network [10] consists of a number of neurons,or nodes, typically arranged in layers where information is insertedinto an input layer. The input layer is then connected to the next layerthrough weights and the value of a node in the next layer is computedas a function of the previous layer’s nodes and their correspondingweights. The function is called an activation function and this is re-peated until the last layer, the output layer is reached. An output func-tion is applied to the nodes on the output layer, creating the final out-put of the network. The layers between the input and output layers arethe hidden layers. The nodes in a layer can be represented by a vectorof values, one value for each node.

3

Page 12: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

4 CHAPTER 2. BACKGROUND

A neural network is trained by updating the network parameters,such as the weights, to minimize an error or loss [10]. The optimiza-tion algorithm gradient descent [15] is typically used to perform learn-ing, where the network parameters are updated in the direction ofthe steepest descent in relation to the loss. The update is typicallyalso controlled by a constant called the learning rate, that controls thestep length in the gradient direction [15]. The learning rate is a hyper-parameter [10], that is a parameter that controls learning, but is notlearned by the network.

2.1.2 Recurrent neural networks

A recurrent neural network (RNN) [10] is a feedforward neural net-work with added recurrent connections from a latter layer to a former,or connections forming a loop in a layer. It is often used to processsequences, where a sequence is fed into the network one data pointat a time. When connections are added in a loop, information canbe fed from one time step to the next. The same weights are usedin each time step, enabling parameter sharing over all time steps. Arecurrent neural network architecture often has connections from thehidden nodes in one time step to the next, and therefore informationis passed through from one time step to the next, making the hiddennodes act as a summary of the already processed time steps.

A strength of the RNN is that it is able to use sequences of variablesizes as input. An RNN can be used to produce an output in each timestep, but it can also be used to encode data [10]. An RNN encoderuses entire sequences as input and produces a single fixed sized vectoras output. The encoder output can thereafter be used as input intoanother network, in which case the two networks are trained together,updating the network parameters with respect to the second network’sloss.

2.1.3 Loss functions

A loss function [10] is a type of error measure of a network’s producedoutput compared to the correct outputs, the targets, and training is per-formed by minimizing that loss. A common loss function is the cross-entropy between the training data and model distribution, which canbe interpreted as the negative log-likelihood of the target given the in-

Page 13: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 2. BACKGROUND 5

put and the current model. Another loss function is the mean squarederror, which was commonly used before cross-entropy gained popu-larity in the 2000s. The mean squared error can be interpreted as adistance measure between the output and the target.

2.1.4 Output functions

The output function is applied in the output layer, and the nature ofthe output function depends what the output should represent. Forexample, if each output node represents a probability value we mightwant all outputs in the layer to sum to one. This can be achieved byusing the softmax function, in which the output layer is normalized sothat the nodes sum to one [15]. Another example of an interpretationof the network output is as parameters to a normal probability distri-bution, a mean and a standard deviation [11]. In this case the standarddeviation should always be positive, so an exponential function can beapplied to get positive values.

2.1.5 Optimization algorithms

The optimization algorithm gradient descent has for long been usedto train neural networks. Today the most popular methods for train-ing are in the family of stochastic gradient descent (SGD). In vanillastochastic gradient descent, the training data is divided into mini-batches[15] and in each epoch of training a mini-batch is used as input into thenetwork, rather than the entire data set. Vanilla SGD is often used withmomentum [10], where an decaying moving average of past gradientsis collected in order to speed up training in the right direction. Thereare a number of variants to SGD including Adam, AdaGrad, and RM-SProp.

AdaGrad, RMSProp and Adam [10] all use adaptive learning rates.Adam uses the momentum of the first-order and second-order gradi-ents and therefore introduces two extra hyper-parameters: first-orderdecay rate and second-order decay rate or the gradients. It has beenshown that all these adaptive learning algorithms increase robustnessin learning compared to vanilla SGD, and there is no consensus of asingle best algorithm among them [10].

Page 14: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

6 CHAPTER 2. BACKGROUND

2.1.6 Teacher forcing

Teacher forcing [10] is a common method used to speed up training. Itcan be applied in RNNs where the output in one time step is used asinput in the next time step. Teacher forcing is when, instead of feedingin the generated output, the target is used as input in the next timestep. Strict teacher forcing has the disadvantage that the inputs seenduring training can differ from those seen in testing, and therefore givea higher test error. Therefore, a looser variant of teacher forcing can beused, where only some randomly chosen nodes in each time step useteacher forcing.

2.1.7 Long short-term memory

The hidden layer activation functions of a recurrent neural networkcan be replaced by so called LSTM functions, creating a LSTM net-work. Using LSTM functions, the hidden nodes act as memory cells[15], saving important information from previous time steps. LSTMsare capable of learning longer term dependencies, and also reduces acommon problem called the vanishing and exploding gradient prob-lem, where the gradients become too small or too large which causeslearning to stop [12]. LSTMs have recently been shown to achievegreatly improved performance than RNNs without LSTM nodes [15,11].

Page 15: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 2. BACKGROUND 7

2.2 Forces affecting golf ball trajectories

The flight of a golf ball depends on the forces gravity, lift and drag[7], as can be seen in figure 2.1. The drag force points in the oppositedirection of the ball’s velocity and the lift force points perpendicularlyto the velocity. The lift force depends in the backspin of the ball, themore backspin the more lift. The spin of a golf ball can be divided intotwo parts: backspin and sidespin [16], where the backspin vector pointsout perpendicularly from the figure toward the reader in figure 2.1.

Figure 2.1: Forces affecting a golf ball in flight

Page 16: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

Chapter 3

Related work

In this chapter state of the art in trajectory tracking will be presented,and especially in the application of golf ball tracking. Historically,most systems used for sports ball tracking were based on physicalmodels and not machine learning methods, but recently machine learn-ing methods have been investigated more and more.

3.1 Trajectory tracking

Trajectory tracking and prediction has been performed in areas frompedestrian movement prediction [1] to ball trajectory tracking [3, 5, 6,13, 18, 19, 4]. A large area of trajectory tracking is in the application ofhandwriting prediction [2, 8, 11].

Graves [11] investigated using RNN with LSTM to track a trajec-tory and perform next step-prediction, in 2013. It was successfullyused in the application of handwriting prediction and generation. Graves’network took the pen’s x-y coordinates as input and used mixture den-sity outputs to perform next-step predictions of the positions, withoutthe use of a physical model. Since then, RNN with LSTM and mix-ture density outputs has been used especially for written text trajec-tory tracking applications [2, 14, 21].

3.2 Sports ball trajectory tracking

Computer vision systems have been used for trajectory tracking andreconstruction of sport balls in commercial applications since the 2000s

8

Page 17: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 3. RELATED WORK 9

[20]. It has been used for sports such as golf, tennis, baseball and more.The majority of the systems used and researched for sports ball trajec-tory tracking are based on physics, where a ball’s possible positionsthrough several frames are fitted to physical model and then recon-structed into a 3D trajectory [20]. This section will first present physicsbased ball tracking methods, and then machine learning methods.

3.2.1 Physical model methods

Chen et al. [6], in 2009, and Chakraborty and Meher [4], in 2013, bothused physics-based methods to track and reconstruct ball trajectoriesin the application of basketballs. After finding all possible ball candi-dates in an image, Chen et al. [6] and Chakraborty and Meher [4] usedphysics-based methods to eliminate false 2D trajectories. The meth-ods made the assumption that all trajectories should be near-parabolic,and used a least-squares fitting technique of statistical regression anal-ysis to eliminate trajectories. Chen et al. [6] further performed 2D to3D transformation of the trajectories using multiple cameras, and esti-mated physical parameters of the trajectories. Both Chen et al. [6] andChakraborty and Meher [4] found that their methods performed betterthan a Kalman filter-based method.

Silva et al. [19] performed trajectory estimation in 2011, evaluatedin the application of ball trajectory reconstruction during RoboCup.The method used by Silva et al. [19] also made the assumption ofparabolic trajectories, but transformed the 2D trajectories into 3D tra-jectories using positional information of the camera before eliminatingtrajectories. Trajectory elimination was performed using a maximi-mum likelihood estimator and an Extended Kalman filter. The methodproved to achieve a distance error of under 0.2 meters.

3.2.2 Machine learning methods

Recently, deep learning systems have been investigated for sport balltrajectory reconstruction. One big benefit with machine learning sys-tems is that the network itself can create an implicit physical model ofthe trajectory, eliminating the need of an explicit model of the physicallaws such as an assumption of parabolic trajectories.

Shah and Romijnders [18] investigated the performance of a RNNLSTM network in 2016, in predicting if a basketball was going to hit the

Page 18: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

10 CHAPTER 3. RELATED WORK

basket or not in a three-point shot, an implicit trajectory reconstructiontask. The input data used was a sequence of 3D positions of the ball’strack. The sequence contained 12 data points and represented abouthalf a second in time. The LSTM network’s result was compared totwo static machine learning models using explicit parameters of ve-locity and angle: generalized linear model and gradient boosted ma-chines. They found that the LSTM network outperformed both staticmachine learning models. The same prediction goal of hit or miss, us-ing the same type of input data, was investigated by Zhao et al. [22] in2017, but using a deep bidirectional LSTM network. The performanceof the network was compared to generalized linear model, gradientboosted machines and RNN with and without LSTM. Deep bidirec-tional LSTM showed better performance than all other investigatedmethods in the hit or miss problem. In 2016 Fragkiadaki et al. [9] useda convolutional neural network and LSTM network to predict the ve-locity of a billiard ball given the force applied to it. The performanceof the network was shown to be better than a constant velocity model.

Explicit trajectory prediction has been investigated as early as in2007 by Burgsteiner et al. [3], using a liquid state machine with RNNto predict ball trajectories. It was successful in doing so in the inter-val of 200 ms ahead in time, compared to kernel principal componentanalysis.

Trajectory reconstruction of entire trajectories was investigated in2017 by Jansson [13], who used a RNN LSTM to reconstruct an en-tire 3D trajectory of a golf ball, given some detected 3D ball positions.The network showed an improvement in performance compared to aphysical model using numerical simulations.

Page 19: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

Chapter 4

Methods

This chapter will describe the network architecture that is used in theexperiments of this thesis, as well as the input data and how it is cre-ated. It will begin by describing the software and hardware used.

4.1 Software and hardware

All data used in the project is created using the game engine Unity1

with the programming language C#2. The network was created inPython3 using the software library Tensorflow4 run on the CPU, to de-fine the network and perform training. All data creation and trainingwas performed on a Windows 10 computer with 8 GB RAM and anIntel Core i5-4690K processor, 3.5 GHz.

4.2 Data

All positional data that is used in this project is synthetically created.The data of a single ball trajectory is created by simulating forces of agolf ball, such as an initial shot force, gravitation, lift and drag. A cam-era records the ball positions as projected onto the camera’s 2D view.The trajectory data for one ball is therefore a sequence of 2D positionsas viewed by a camera. The recording is made with 50 frames per

1https://unity3d.com/2https://docs.microsoft.com/en-us/dotnet/csharp/index3https://www.python.org/4https://www.tensorflow.org/

11

Page 20: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

12 CHAPTER 4. METHODS

second and a fixed camera window. Figure 4.1 below shows an illus-tration of the simulation setup when recording data. Table 4.1 showsthe range of values used for each parameter of the ball’s physics as itis shot out. The value of each parameter is randomized for each newball trajectory, in the interval shown in table 4.1.

When a ball is shot out in this experiment, it has a starting positionof x = d, y = 0 and z = 0, where the direction of x and y can be seenin figure 4.1, and z denotes the ball’s elevation. Possible values of dare shown in table 4.1. The ball’s start rotation, that decides in whichdirection the initial shot force should be applied, is zero around the xand y axes. The rotation around the z axis, r, has a value in the intervalshown in table 4.1, where a rotation of zero means that the ball is facingin the direction of the x axis. The magnitude of the initial shot force,f , is also shown in table 4.1, where the ball mass is set to 46 grams.The spin of a ball is created by applying a backward (tb) and right (tr)torque force, of magnitudes shown in table 4.1. The backward torquetranslates to the torque around the negative x axis and the right torqueis the torque around the z axis. The drag force, FD, and lift force, FL,are defined by the following two equations.

FD = −1

2CDρA|v|v

FL =1

2CLρA|v|2

α× v

|α× v|

Here CD and CL denotes the drag and lift coefficients respectivelyand they are set to 0.3 and 0.15 to imitate the physics of a golf balltrajectory. The density of the ambient air is denoted by ρ and set to thevalue 1.1839 kg/m3, to air of temperature of 25 ◦C. A is the projectedarea of the golf ball, where the diameter of the golf ball is 4.267 cm.Further, v denotes the ball velocity and α is the ball’s backspin vector.

The camera has a position of x = 0, y = 0 and an elevation of 20meters during data creation. It also has a rotation of 17 degrees aroundthe x axis and 21 degrees around the z axis.

To emulate multiple balls being in flight during the course of record-ing a main ball, other balls are created at random time intervals andshot out. The recorded image in a single frame is then represented as ablob map, that is a map where each ball’s position is marked. Each balltrajectory therefore has a corresponding sequence of blob maps whereits own position are also included in each blob map.

Page 21: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 4. METHODS 13

Figure 4.1: A conceptual illustration of the simulation setup as seenfrom above. The camera is shown to the left which is recording whenballs to the right are shot from bottom to top in the image. The angle rshows a ball’s starting rotation

In testing, noise is created by randomly dropping out blobs fromblob maps. The purpose of this is to emulate noise, resulting in a blobnot being registered onto the blob map in a certain time step. In eachblob map and for each blob, the probability of dropout in that timestep will be 0.1 %, from here on denoted as the noise probability.

The equation below was used to normalize all positions in the dataset, before being used as input into the network. Here xi denotes a sin-gle two-dimensional data point, xi denotes the normalized data point,and µ and σ denotes the 2-dimensional mean and standard deviationrespectively of the entire data set.

xi =xi − µ

σ

Page 22: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

14 CHAPTER 4. METHODS

Table 4.1: Range of a ball’s physical parameters used during data cre-ation

4.3 Network

This section will describe the neural network proposed in this thesisand its architecture. The thesis proposes a machine learning basedmethod as opposed to a physical model method, due to the flexibilityof the former. Using a machine learning method, no explicit physicalparameters must be chosen, enabling the machine learning network toitself form an implicit physical model. The proposed network consistsof two connected LSTM RNNs trained to track a ball during bounceand roll, given the ball’s previously tracked trajectory and the inputvideo images represented as blob maps. A single ball position is rep-resented as a 2D point and a trajectory as a sequence of 2D points. Thissection will go through each part of both networks, where the secondnetwork is shown in figure 4.2.

4.3.1 Inputs

The inputs to the network in each time step are Pt, the parameters toa mixture distribution, Bt a blob map and c, a context vector.

The context vector, c, is created by the first network, a recurrentLSTM encoder, and is an encoding of the previously tracked part of theball trajectory. As previously discussed it is assumed that some othermethod has successfully tracked the ball trajectory up until a bounceoccurs. The previously tracked trajectory is encoded into a fixed sizevector, a context vector, using a recurrent LSTM encoder in order tolearn long time dependencies. The encoded context from the LSTMencoder is used as input in each time step of the second RNN. The two

Page 23: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 4. METHODS 15

RNNs are trained together.The mixture distribution parameters, Pt, parameterize a bivariate

gaussian distribution. Pr(x;Pt). The probability represents where thenetwork thinks the ball was located in that time step t. Using a mix-ture distribution to represent this enables the network to encode moreinformation such as uncertainty, as opposed to using single values. Pt

is a container containing all mixture parameters, as in the equation be-low where µc denotes the means, σc the standard deviations and w

the weight of each component respectively.

[µ1,µ2, ...,µC ], [σ1,σ2, ...,σC ], [w1, w2, ..., wC ] = Pt

Since P parameterize a bivariate gaussian, each mean vector (µc)and standard deviation vector (σc) have two dimensions. Note thatthe two dimensions are assumed to be independent, and therefore theparameters can be represented with a simple standard deviation ofeach dimension instead of an covariance matrix.

The blob map, Bt, is a representation of a single frame in a videosequence, where multiple balls are being recorded. The goal of thenetwork is to find the position of the main ball given the blob map ofblobs, where the main ball is a blob. The representation of the blobmap is a shuffled list of blob positions in this time step.

In the first time steps of the sequence, blob maps and mixture pa-rameters created from the previously tracked trajectory are used as in-put into the network. The last set of mixture parameters created fromthe previous trajectory are denoted P−1 and are illustrated in figure4.2. P−i is created from a data point in the previously tracked trajec-tory, x−i, by setting all µc to equal x−i, with small standard deviationsand a high weight for the first component.

4.3.2 Saliency map

The distribution, Pt−1, and blob map, Bt, are combined into a saliencymap, St. Each blob, bti, is given a weight wi by using the equationsbelow, where B denotes the number of blobs.

wi = Pr(bti;Pt−1)

wi =ewi∑Bj=1 e

wj

Page 24: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

16 CHAPTER 4. METHODS

The blob positions and their corresponding weights are then com-bined into the saliency map, as can be seen in the equation belowwhere g denotes a simple vector concatenation and w is a vector con-taining all weights wi.

St = g(Bt,w)

4.3.3 Hidden layer

The saliency map, St, and the context vector, c, are used as input intothe hidden layer to produce the output Ht, as can be seen in the equa-tion below, where h denotes the LSTM functions and g denotes a sim-ple concatenation of vectors.

Ht = h(g(St, c),Ht−1)

The network uses a single hidden LSTM layer in each time step,with recurrent connections between time steps. The hidden layer isset to have the same number of nodes as the output layer.

4.3.4 Output layer

The output layer converts the hidden layer output, Ht, into the param-eters of the mixture, Pt. The hidden layer output Ht is divided intomeans, µc, deviations, σc and weights, wc and converted into mixtureparameters by using the following output functions, where C is thenumber of components and k is a constant larger than 1, used as anumerical trick to force down the deviation.

µc = µc

σc = eσc/k

wc =ewc∑Ci=1 e

wi

4.3.5 Loss

In training, the loss of a single data point in a single time step is thenegative log probability of the target position, xt in the mixture dis-tribution. This is then averaged over all time steps, L, as can be seenbelow where T is the sequence length.

Page 25: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 4. METHODS 17

Figure 4.2: A figure of the RNN architecture, unrolled for three timesteps, where the subscript denotes the time step. The input in eachtime step is shown in the top of the figure and the output is shown inthe bottom

L = −∑T−1

t=0 log(Pr(xt;Pt))

T

The loss is further averaged over all data points in a mini-batch.

4.3.6 Optimization algorithm

The optimization algorithm used is Adam, with the learning rate set to0.09, first-order decay set to 0.6 and second-order decay set to 0.8. Themini-batch size is set to 100 data points where all blob sequences arelimited to a length of 300 time steps. A training set of 3000 data points

Page 26: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

18 CHAPTER 4. METHODS

is used. The network is trained with 25 % teacher forcing and the num-ber of output mixture components is set to 3. All hyper-parameterswere experimentally chosen to decrease the training loss.

4.4 Baseline algorithm

A baseline algorithm, is used in comparison to the network. The baselinealgorithm works by trying to follow the ball throughout the blob maps,using minimum distance. In each time step, the baseline algorithmselects the blob closest to the last found blob, and assumes this to bethe blob of the ball for this time step. The baseline error rate in a timestep is the average number of times the baseline does not select thecorrect blob.

Page 27: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

Chapter 5

Results

In this chapter, the results of the network are presented. Testing is per-formed over two data sets, one with and one without noise. Each dataset contains 1000 trajectories that has been cut to a maximum length of300 time steps after first bounce, where the average original trajectorylength after first bounce is 377 time steps, and the average roll lengthis 222 time steps. The average number of blobs in each blob map is 7.5blobs. The network performance is measured in error rate, where thenetwork error rate in a time step is the average number of times theincorrect blob has the highest probability in the output distribution.The baseline error is used in comparison to the network error.

Figures 5.1 and 5.2 display the error rates for the two data sets re-spectively. In figures 5.3 to 5.6 the network error is displayed in rela-tion to bounces and in 5.7 it is displayed during ball roll. In figures5.8 to 5.10 a single ball trajectory is illustrated with network outputand blob map in time steps around a bounce. This chapter will nowcontinue to present the found results in more detail.

In figure 5.1 below the network error rate is displayed comparedto the baseline, for a data set with no noise. The network error rateis shown to lie between approximately 4 % and 11 %, increasing overtime, with the exception of a higher error rate in the first time steps.The baseline error rate is below 2 %. In figure 5.2 the error rates aredisplayed for a data set with noise. For this set, the baseline errorincreases over time and rises to a maximum of approximately 19 %,compared to 2 % for the no noise data set. The network error lies be-tween approximately 4 % and 9 %, with a smaller increase toward theend than for the no noise data set.

19

Page 28: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

20 CHAPTER 5. RESULTS

Figure 5.1: Network and baseline error rate in each sequence time step,averaged over a test set without noise, containing 1000 trajectories

Figure 5.2: Network and baseline error rate in each sequence time step,averaged over a test set with 0.1 % remove blob probability, containing1000 trajectories

Page 29: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 5. RESULTS 21

In figures 5.3 to 5.6 the network error rate is displayed in time stepsaround the second, third, forth and fifth bounce for the no noise dataset. In 5.3 we can see that the error increases slightly at the time stepsaround the bounce. Further, in figures 5.3 to 5.6 we can see that theerror rate increases with the number of bounces. In figure 5.7 the net-work error rate during ball roll is displayed. We can here see that theaverage error rate during roll is smaller than the error rate for the fifthbounce.

Figure 5.3: Network error rate in each sequence time step around thesecond bounce, averaged over a test set with no noise, containing 1000trajectories

Page 30: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

22 CHAPTER 5. RESULTS

Figure 5.4: Network error rate in each sequence time step around thethird bounce, averaged over a test set with no noise, containing 1000trajectories

Figure 5.5: Network error rate in each sequence time step around theforth bounce, averaged over a test set with no noise, containing 1000trajectories

Page 31: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 5. RESULTS 23

Figure 5.6: Network error rate in each sequence time step around thefifth bounce, averaged over a test set with no noise, containing 1000trajectories

Figure 5.7: Network error rate in each sequence time step during ballroll, averaged over a test set with no noise, containing 1000 trajectories

Page 32: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

24 CHAPTER 5. RESULTS

Figure 5.8, 5.9 and 5.10 illustrates the full ball trajectory with thetarget ball position, the blob map and the output mixture distributionin time steps before, during and after a bounce respectively. In figures5.8 and 5.10 we can see that the position of maximum probability isclose to the target blob before and after a bounce, while in 5.9, duringbounce, the position of maximum probability is further away.

Page 33: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 5. RESULTS 25

Figure 5.8: A single ball trajectory as seen by the camera where theaxes are measured in pixels. The main ball position in this single timestep t = 59 is displayed as a black unfilled circle and cyan ellipses indi-cate blobs in the blob map. Each component of the output probabilitydistribution in this time step is displayed as a set of three red ellipses.The inner ellipse has a radius equal to the component’s standard de-viation, and the transparency of each ellipse indicate the probabilityvalue in that area, where a high transparency means a low probability

Page 34: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

26 CHAPTER 5. RESULTS

Figure 5.9: A single ball trajectory as seen by the camera where theaxes are measured in pixels. The main ball position in this single timestep t = 68 is displayed as a black unfilled circle and cyan ellipses indi-cate blobs in the blob map. Each component of the output probabilitydistribution in this time step is displayed as a set of three red ellipses.The inner ellipse has a radius equal to the component’s standard de-viation, and the transparency of each ellipse indicate the probabilityvalue in that area, where a high transparency means a low probability

Page 35: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 5. RESULTS 27

Figure 5.10: A single ball trajectory as seen by the camera where theaxes are measured in pixels. The main ball position in this single timestep t = 79 is displayed as a black unfilled circle and cyan ellipses indi-cate blobs in the blob map. Each component of the output probabilitydistribution in this time step is displayed as a set of three red ellipses.The inner ellipse has a radius equal to the component’s standard de-viation, and the transparency of each ellipse indicate the probabilityvalue in that area, where a high transparency means a low probability

Page 36: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

Chapter 6

Discussion

In this chapter, we will analyse the results, draw conclusions and dis-cuss future work.

6.1 Analysis

We will begin by discussing the results and offer explanations, andthen continue to put the result in greater context.

Firstly, we will offer explanations to why the error rate fluctuatesthrough time for the test set without noise, as can be seen in figure5.1. The network achieved an error rate of under 6 % averaged overthe 100 first time steps, where the error rate was significantly higherin the very first time steps, and increased over time to reach an er-ror rate of approximately 11 % in the last time step. This increase canbe explained by the architecture of the recurrent neural net. In eachtime step, the output from the last time step is fed as input, thereforean error can easily propagate, possibly creating a larger and larger er-ror through the time steps. The high error rate in the first time stepshowever can be a symptom that the network could benefit from usingmore data points from the trajectory before first bounce as step by stepinput. The low baseline error can be explain by the simple fact thatfor the baseline to choose a wrong blob in a data set without noise, awrong blob must have smaller distance to the target blob in the lasttime step, than the actual target blob in this time step. Therefore, ina no noise environment, the baseline error is very low, given that theaverage number of blobs in the blob map is also relatively low.

Secondly, we will offer explanations to the difference in error for

28

Page 37: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

CHAPTER 6. DISCUSSION 29

the two data sets. The baseline error for the data set with noise, as canbe seen in figure 5.2, increased greatly compared to the baseline errorfor the set without noise. The network error however was larger inthe initial time steps than for the noise data set, but did not increaseas much with time. When noise causes a target blob to be removed,the error for that data point is 1 since the error algorithm is forced tochoose a wrong blob. When any other blob is removed however the er-ror would generally decrease slightly since the probability of choosinga wrong blob is less, and the propagation error would probably de-crease as well, since the network has the opportunity to recover froman earlier error. However, if the network was not noise resistant, re-moving the target blob would cause the network to jump to anotherblob and follow that one instead, creating a much larger propagatingerror. Since the network error is larger in the initial time steps, butdoes not increase as much with time, we can draw the conclusion thatthe recurrent network is somewhat, but not completely, resistant tonoise. Further, we can draw the conclusion that the recurrent networkis more resistant to noise than the baseline.

Thirdly, we offer explanations for the error rate surrounding bounces.In figure 5.3, of the error around the second bounce, we could see thatthe error rate increased before the bounce to decrease again after thebounce. This can be explained by figures 5.8, 5.9, 5.10, where we cansee that the output probability does not follow the ball all the way intothe bounce, but "takes a shortcut". This could probably be avoidedwith further training. The same phenomenon is however not as visiblein bounces three to five, displayed in figures 5.4 to 5.6. This could bedue to the position of the camera. Balls are shot out from the camera,and therefore, generally the movements of later bounces are smalleron the camera view.

Fourthly, we will explain the artifacts in the error plot, especiallythat of ball roll in figure 5.7. Note that the the average roll length is 222time steps, therefore the error rate is averaged over increasingly fewerdata points, which result in artifacts. Further, the average trajectorylength after first bounce is 376 time steps. Since the trajectories are cutafter 300 time steps, for performance reasons, the trajectories used asinput into the network does often not represent an entire trajectory.This could cause the network to not generalize as well over longertrajectories.

Page 38: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

30 CHAPTER 6. DISCUSSION

6.2 Conclusions

This thesis investigated how well the positions of a golf ball can betracked in two dimensions, using a recurrent neural network. The net-work was trained and tested on synthetic data and its performancewas compared to that of a simple baseline attempting to follow thecorrect ball through the images. The network was proven to be ableto track the 2D trajectory of a golf ball during bounce and roll with anerror rate of under 11 %, in an environment with multiple golf balls.

6.3 Future work

The network of this thesis performed trajectory tracking, which is asub-problem of trajectory reconstruction. The network was trainedand tested on synthetically created data, with limited sequence length.To get a more reliable result, the method in this network should betested on real data in a real situation, and possibly integrated into a fullreconstruction system. Further, a hyper-parameter search should beperformed in order to find the optimal set of network hyper-parameterssuch as learning rates, number of hidden layers and choice of opti-mization algorithm.

Page 39: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

Bibliography

[1] Akinori Asahara et al. “Pedestrian-movement Prediction Basedon Mixed Markov-chain Model”. In: Proceedings of the 19th ACMSIGSPATIAL International Conference on Advances in GeographicInformation Systems. GIS ’11. Chicago, Illinois: ACM, 2011, pp. 25–33. ISBN: 978-1-4503-1031-4. DOI: 10.1145/2093973.2093979.URL: http://doi.acm.org/10.1145/2093973.2093979.

[2] Daniel Berio et al. “Calligraphic Stylisation Learning with a Phys-iologically Plausible Model of Movement and Recurrent Neu-ral Networks”. In: Proceedings of the 4th International Conferenceon Movement Computing. MOCO ’17. London, United Kingdom:ACM, 2017, 25:1–25:8. ISBN: 978-1-4503-5209-3. DOI: 10.1145/3077981.3078049. URL: http://doi.acm.org/10.1145/3077981.3078049.

[3] Harald Burgsteiner et al. “Movement prediction from real-worldimages using a liquid state machine”. In: Applied Intelligence 26.2(Apr. 2007), pp. 99–109. ISSN: 1573-7497. DOI: 10.1007/s10489-006-0007-1. URL: https://doi.org/10.1007/s10489-006-0007-1.

[4] Bodhisattwa Chakraborty and Sukadev Meher. “A real-time trajectory-based ball detection-and-tracking framework for basketball video”.In: Journal of Optics 42 (2013), pp. 156–170. URL: https://doi.org/10.1007/s12596-012-0108-7.

[5] Hua-Tsung Chen et al. “Ball tracking and 3D trajectory approx-imation with applications to tactics analysis from single-cameravolleyball sequences”. In: Multimedia Tools and Applications 60(2011), pp. 641–667.

31

Page 40: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

32 BIBLIOGRAPHY

[6] Hua-Tsung Chen et al. “Physics-based ball tracking and 3D tra-jectory reconstruction with applications to shooting location es-timation in basketball video”. In: WOS:000264595500004 (2009).URL: https://ir.nctu.edu.tw/handle/11536/7427.

[7] John M. Davies. “The Aerodynamics of Golf Balls”. In: Journal ofApplied Physics 20 (1949), pp. 821–828.

[8] Arka Ujjal Dey and Gaurav Harit. “Generating Synthetic Hand-writing Using N-gram Letter Glyphs”. In: Proceedings of the TenthIndian Conference on Computer Vision, Graphics and Image Process-ing. ICVGIP ’16. Guwahati, Assam, India: ACM, 2016, 49:1–49:8.ISBN: 978-1-4503-4753-2. DOI: 10.1145/3009977.3010042.URL: http://doi.acm.org/10.1145/3009977.3010042.

[9] Katerina Fragkiadaki et al. “Learning Visual Predictive Modelsof Physics for Playing Billiards”. In: CoRR abs/1511.07404 (2015).URL: http://arxiv.org/abs/1511.07404.

[10] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learn-ing. http://www.deeplearningbook.org. MIT Press, 2016.

[11] Alex Graves. “Generating Sequences With Recurrent Neural Net-works.” In: CoRR abs/1308.0850 (2013). URL: http://dblp.uni-trier.de/db/journals/corr/corr1308.html#Graves13.

[12] Sepp Hochreiter and Jürgen Schmidhuber. “Long Short-Term Mem-ory”. In: Neural Comput. 9.8 (Nov. 1997), pp. 1735–1780. ISSN:0899-7667. DOI: 10.1162/neco.1997.9.8.1735. URL: http://dx.doi.org/10.1162/neco.1997.9.8.1735.

[13] Anton Jansson. “Predicting trajectories of golf balls using recur-rent neural networks”. MA thesis. 2017. URL: urn:nbn:se:kth:diva-210552.

[14] Edwin D. de Jong. “Incremental Sequence Learning”. In: CoRRabs/1611.03068 (2016). URL: http://arxiv.org/abs/1611.03068.

[15] Zachary Chase Lipton. “A Critical Review of Recurrent Neu-ral Networks for Sequence Learning”. In: CoRR abs/1506.00019(2015). URL: http://arxiv.org/abs/1506.00019.

Page 41: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

BIBLIOGRAPHY 33

[16] William M. MacDonald and Stephen Hanzely. “The physics ofthe drive in golf”. In: American Journal of Physics 59(3) (1991),pp. 213–218.

[17] Frank J. Penedo and Jason R. Dahn. “Exercise and well-being:A review of mental and physical health benefits associated withphysical activity”. English (US). In: Current Opinion in Psychiatry18.2 (Jan. 2005), pp. 189–193. ISSN: 0951-7367. DOI: 10.1097/00001504-200503000-00013.

[18] Rajiv Shah and Rob Romijnders. “Applying Deep Learning toBasketball Trajectories”. In: CoRR abs/1608.03793 (2016). URL:http://arxiv.org/abs/1608.03793.

[19] Hugo Silva et al. “Robot Soccer World Cup XV”. In: ed. by ThomasRöfer et al. Berlin, Heidelberg: Springer-Verlag, 2012. Chap. Real-time 3D Ball Trajectory Estimation for RoboCup Middle Size LeagueUsing a Single Camera, pp. 586–597. ISBN: 978-3-642-32059-0.URL: http://dl.acm.org/citation.cfm?id=2554542.2554596.

[20] Graham Thomas et al. “Computer vision for sports: Current ap-plications and research topics”. In: Computer Vision and Image Un-derstanding, CVIU (2017), pp. 3–18.

[21] Xu-Yao Zhang et al. “Drawing and Recognizing Chinese Charac-ters with Recurrent Neural Network”. In: CoRR abs/1606.06539(2016). URL: http://arxiv.org/abs/1606.06539.

[22] Yu Zhao et al. “Applying Deep Bidirectional LSTM and MixtureDensity Network for Basketball Trajectory Prediction”. In: CoRRabs/1708.05824 (2017). URL: http://arxiv.org/abs/1708.05824.

Page 42: Tracking a ball during bounce and roll using …kth.diva-portal.org/smash/get/diva2:1267392/FULLTEXT01.pdfTracking a ball during bounce and roll using recurrent neural networks FELICIA

www.kth.se