Flexible smoothing with B-splines and Penalties or P-splines

• P-splines = B-splines + Penalization• Applications : Generalized Linear and non

linear Modelling ; Density smoothing• P-splines have their grounding in Classical

regression methods and Generalized linear models

• Regression, Smoothing, Splines?• B-splines P-splines?

Smoothing, Regression, Splines, B-splines P-splines?

• In statistics, linear regression refers to any approach to modeling the relationship between one or more variables denoted yand one or more variables denoted X, such that the model depends linearly on the unknown parameters to be estimatedfrom the data. Such a model is called a "linear model."

Smoothing, Regression, Splines, B-splines P-splines?

Regression, Smoothing, Splines, B-splines P-splines?

• Linear model

• Generalized Linear model where x′iβ is the inner productbetween vectors xiand β.

y xα β ε= + +

20 1 2

ni i i n i iy x x x eβ β β β+ + += + +

Regression, Smoothing, Splines, B-splines P-splines?

• The term ei is the residual, . One method of estimation is ordinary least squares. This method obtains parameter estimates that minimize the sum of squared residuals, SSE:

ˆi i ie y y= −

Regression, Smoothing, Splines, B-splines P-splines?

• Smoothing:In statistics and image processing, to smooth a data set is to create an approximating function that attempts to capture important patterns in the data, while leaving out noise or other fine-scale structures/rapid phenomena.

Regression, Smoothing, Splines, B-splines P-splines?

• Many different algorithms are used in smoothing. One of the most common algorithms is the "moving average", often used to try to capture important trends in repeated statistical surveys. In image processing and computer vision, smoothing ideas are used in scale-spacerepresentations.

Regression, Smoothing, Splines, B-splines P-splines?

• Spline : Originally, a spline tool was a thin flexible strip of wood ,metal or rubber used by draftsman to aid in drawing curved lines.

Regression, Smoothing, Splines, B-splines P-splines?

• a spline is a special function defined piecewise by polynomials. In interpolatingproblems, spline interpolation is often preferred to polynomial interpolationbecause it yields similar results

Splines

• Spline of degree zero

0 0 0 1

1 1 1 2

1 1 1

[ , )[ , )

( )

[ , )n n n n

S c x t tS c x t t

S x

S c x t t− − −

= ∈⎧⎪ = ∈⎪=⎨⎪⎪ = ∈⎩

Splines

• A spline of degree 1 0 0 0 0 1

1 1 1 1 2

1 1 1 1

[ , )[ , )

( )

[ , )n n n n n

S a x b x t tS a x b x t t

S x

S a x b x t t− − − −

= + ∈⎧⎪ = + ∈⎪=⎨⎪⎪ = + ∈⎩

Splines

• A cubic spline 3

3

1 [ 1,1/ 2)( )

3 1 [1/2,1)x x

S xx x

⎧ − ∈ −=⎨

− ∈⎩

!B-splines!

• B-splines of degree 010 1 [ , )

( )0

i ii

x t tB x

otherwiswe+∈⎧

= ⎨⎩

!B-splines

• A spline of degree 1 11

0 21 2

2 1

[ , )

( ) [ , )

0

ii i

i i

ii i i

i i

x t if x t tt tt xB x if x t t

t totherwise

++

++ +

+ +

−⎧ ∈⎪ −⎪⎪ −

= ∈⎨ −⎪⎪⎪⎩

B-splines

B-splines

• B-splines are defined recursively

1 111

1 1

( ) ( ) ( )k k ki i ki i i

i k i i k i

x t t xB x B x B xt t t t

− −+ ++

+ + + +

⎛ ⎞ ⎛ ⎞− −= +⎜ ⎟ ⎜ ⎟− −⎝ ⎠ ⎝ ⎠

Properties of P-splines

• No boundary effects• Are a straightforward extension of

(generalized ) linear regression models• Conserve moments like the mean and

variances of the data and fit polynomial data exactly

• Computations and cross validation relatively inexpensive

Fitting curves with B-splines

• A fitted curve

• is the linear combination

( ),ˆ i ix yy to

1

ˆ ( )ˆn

i ii

a B xy=

=∑

Fitting curves with splines

• The corresponding SSE• (quadratic error) is

{ }22

1 1 1

ˆ( ) ( )ˆm m n

i i j ji i j

S x y y a B xy= = =

⎧ ⎫= − = −⎨ ⎬

⎩ ⎭∑ ∑ ∑

O’Sullivan penalty

• Between 1986 and 1988 O’Sullivan introduced a penalty on the second derivative of the fitted curve;

2max

min1 1 1

ˆ ˆ( ) ( ) ( )m n nx

i j j j jxi j j

S x y a B x a B xλ= = =

⎧ ⎫ ⎧ ⎫′′= − +⎨ ⎬ ⎨ ⎬

⎩ ⎭ ⎩ ⎭∑ ∑ ∑∫

Eilers and MarxPenalities

• Eilers /Marx penalty proposal based on finite differences

( )2 2

1 1 1

ˆ ˆ( ) ( )m n n

ki j j j

i j j kS x y a B x aλ

= = = +

⎧ ⎫= − + ∆⎨ ⎬

⎩ ⎭∑ ∑ ∑

Applications

• Generalized linear modeling• Density Smoothing• Example 1 : Motorcycle crash helmet

impact simulation data (Härdle 1994) head acceleration in g units at different times after impact. Smoothed with B-splines of degree 3 and a second order penalty

Example 1 Graph

Density Smoothing

Density Smoothing

Density Smoothing