2011 COURSE IN NEUROINFORMATICSMARINE BIOLOGICAL LABORATORY

WOODS HOLE, MA

Introduction to Spline Models

or

Advanced Connect-the-Dots

Uri Eden

BU Department of Mathematics and Statistics

August 16, 2011

Slide acknowledgements: D. Keren, Dept. of Computer Sci, Haifa U.

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

0 100 200 300 400 500 600 700 800 900 10000

20

40

60

80

100

Time (ms)

Tri

al

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

Fir

ing

Rat

e (

Hz)

Time (sec)0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

Time (sec)0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

Fir

ing

Rat

e (

Hz)

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

Time (sec)0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

Fir

ing

Rat

e (

Hz)

Problem Statement

• Given a set of control points, Pi, known to be on a curve, find a parametric function that interpolates or approximates the curve.

• For interpolating 2 points we need a polynomial of degree 1.

Polynomial interpolation

(x1,y1)(x2,y2)

• For interpolating 2 points we need a polynomial of degree 1.

Polynomial interpolation

2 11y y y

1

2 1

x x

x x

(x1,y1)(x2,y2)

• For interpolating 2 points we need a polynomial of degree 1.

• For interpolating 3 points we need a polynomial of degree 2.

• In general, for interpolating n points, we need a polynomial of degree n-1.

Polynomial interpolation

2 11y y y

1

2 1

x x

x x

(x1,y1)(x2,y2)

Polynomial interpolation

• Let’s try it on our example control points:

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

10

20

30

40

50

60

Time (sec)

Firi

ng R

ate

(Hz)

Polynomial interpolation

• Let’s try it on our example control points:

Time (sec)

Firi

ng R

ate

(Hz)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-250

-200

-150

-100

-50

0

50

100

Polynomial interpolation• Disadvantages:

– Coefficients are difficult to interpret geometrically

– No local operations – adjusting parameters to change one region will change all others

– If polynomial degree is high: • Resulting function will have unwanted fluctuations• Numerical precision problems may arise.

– If polynomial degree is low, the resulting curve will not interpolate the points

Spline interpolation

• Another approach: Polynomial splines

– Advantages:• Rich, flexible representation• Geometrically meaningful coefficients

– Local, piecewise, low-degree, polynomial curves with continuous, smooth joints

Spline interpolation

• Simplest Example: Zeroth order ‘spline’– Constant function between control points

– Simple mathematical representation

1, if i i iy y x x x

Local

(xi+1,yi+1)

(xi,yi)

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

Time (sec)0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

Fir

ing

Rat

e (

Hz)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

10

20

30

40

50

60

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

Time (sec)

Fir

ing

Rat

e (

Hz)

Local but not continuous

Spline interpolation

• Simple Example: Linear ‘spline’– Linear combination of the two nearest control points

– Simple mathematical representation

1 11 , if i i i iy y y x x x

1

i

i i

x x

x x

where

Spline interpolation

• Simple Example: Linear ‘spline’

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 1 1( ) ( ) , if i i i iy f y f y x x x

1( )f

2 ( )f

Spline interpolation

• Simple Example: Linear ‘spline’

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 1 1( ) ( ) , if i i i iy f y f y x x x

1( )f

2 ( )f 0

Spline interpolation

• Simple Example: Linear ‘spline’

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 1 1( ) ( ) , if i i i iy f y f y x x x

0.2

1( )f

2 ( )f

Spline interpolation

• Simple Example: Linear ‘spline’

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 1 1( ) ( ) , if i i i iy f y f y x x x

0.5

1( )f

2 ( )f

Spline interpolation

• Simple Example: Linear ‘spline’

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 1 1( ) ( ) , if i i i iy f y f y x x x

1

1( )f

2 ( )f

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

Time (sec)0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

Fir

ing

Rat

e (

Hz)

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

Time (sec)0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

Fir

ing

Rat

e (

Hz)

Local and continuous, but not smooth

Spline interpolation

• Cubic Splines– Let be cubic functions.

– Produces continuous, differentiable curves– Need to decide how to model derivatives at control points

• Natural Splines: Derivative is zero at endpoints

• Cardinal Splines: Derivative determined by nearby control points

• Hermitian Splines: Derivatives determined by separate parameters

0 1 1 2 1 3 2

1

,

if i i i i

i i

y c y c y c y c y

x x x

0 1 2 3, , , c c c c

Spline interpolation

• Cardinal splines– Linear combination of 4 nearest control points– Features:

• Derivatives determined by adjacent points

• Shape controlled by tension parameter, [0,1]s

Spline interpolation

• Cardinal splines

31

2

11

2

2 2

2 3 3 2, if

0 0

0 1 0 01

T

i

ii i

i

i

ys s s s

ys s s sy x x x

ys s

y

0 1 1 2 1 3 2

1

,

if i i i i

i i

y c y c y c y c y

x x x

Spline interpolation

• Cardinal splines– Basis/blending functions for cardinal spline with s = 0.5.

Spline interpolation

• Cardinal splines

0

0 1 1 2 1 3 2 1, if i i i i i iy c y c y c y c y x x x

Spline interpolation

• Cardinal splines

0.2

0 1 1 2 1 3 2 1, if i i i i i iy c y c y c y c y x x x

Spline interpolation

• Cardinal splines

0.5

0 1 1 2 1 3 2 1, if i i i i i iy c y c y c y c y x x x

Spline interpolation

• Cardinal splines

1

0 1 1 2 1 3 2 1, if i i i i i iy c y c y c y c y x x x

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

Time (sec)0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

Fir

ing

Rat

e (

Hz)

Motivating Problem

• Compute a smooth firing rate function from repeated spike train data

Time (sec)0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

Fir

ing

Rat

e (

Hz)

Fitting Spline Models

• To define a spline model, we must specify– Control point locations (x-coordinates)– Tension parameter

• Often these are selected ad-hoc, but adaptive Bayesian procedures (e.g. BARS) to determine optimal values typically produce better results.

Fitting Spline Models

• Under a specified noise model, the spline parameters (heights of control points) can be estimated by maximum likelihood.

– E.g. If , where is Gaussian white noise, and is a spline function, the ML estimator, , can be found by simple linear regression.

– For spike data, ML spline parameters can be estimated using point process models.

t t ty tt tX

ˆML

Conclusions

• Splines offer one approach to modeling smooth functional relationships in neuroscience data.

• Cubic splines are local 3rd degree polynomials that are piecewise continuous and differentiable.

• Under common noise distributions, spline models can be fit easily using ML methods.

• Goodness of spline model fits depends on selection of control point locations and tension parameters.