A Functional Single Index Model

Fei Jiang, Seungchul Baek, Jiguo Cao and Yanyuan Ma

University of Hong Kong, University of South Carolina

Simon Fraser University, Penn State University

Abstract: We propose a semiparametric functional single index model to study the relation-

ship between a univariate response and multiple functional covariates. The parametric part

of the model integrates the functional linear regression model and the sufficient dimension

reduction structure. The nonparametric part of the model allows the response-index depen-

dence or the link function to be unspecified. The B-spline method is used to approximate

the coefficient function, which leads to a dimension folding type model. A new kernel

regression method is developed to handle the dimension folding model, which allows the

efficient estimation of both the index vector and the B-spline coefficients. We also establish

the asymptotic properties and semiparametric optimality for the estimators.

Key words and phrases: B-spline, Dimension reduction, Dimension folding, Functional

linear model, Functional data analysis, Kernel estimation.

1

1. Introduction

The National Morbidity, Mortality and Air Pollution Study (NMMAP) is an im-

portant study aiming at addressing the uncertainties regarding the association

between pollution and health (Samet et al., 2000). In this study, daily measure-

ments of various air pollutants carbon monoxide (CO), nitrogen dioxide (NO2),

sulfur dioxide (SO2) and ozone (O3) are collected in different cities over the

course of a year. The annual death rate caused by cardiovascular diseases (CVD)

is also collected in these cities. Each pollutant has been studied individually with

no significant effect detected on the CVD death rate (Cox and Popken, 2015;

Turner et al., 2016).

This motivates us to develop a single air pollution index which combines the

pollutants in the way that best describes the severity of the overall air pollution

level to the CVD death rate. At the same time, we are also interested in discover-

ing the possible time-varying effect of the single air pollution index to the CVD

death rate. To achieve these goals, we propose a functional single index model

and proceed to devise a novel class of estimators.

More specifically, the NMMAP data contains the measurements of the daily

air pollutant concentrations X(t) ≡ {X1(t), . . . , XJ(t)}T, a J-dimensional func-

tional covariate of t ∈ [0, 1], and the annual CVD death rate in the subsequent

year as the response Y . To measure the overall severeness of the air pollution,

2

we define the single air pollution index as

W (t) = βTX(t) =J∑i=1

βiXi(t),

where β = (β1, . . . , βJ)T is the vector of weights to various air pollutants, cap-

turing the relative importance of various pollutants in determining the pollution

severity. We assume that Y is linked to W (t) through

f{Y |X(t)} = f

{Y,

∫ 1

0

W (t)α(t)dt

}, (1.1)

where f is a conditional probability density function (pdf) or probability mass

function (pmf) and is left unspecified. Here, the functional parameter α(t) cap-

tures the time-varying effect of the air pollution index on the annual CVD death

rate. Note that unlike the usual single index model as discussed in Chen et al.

(2011) and Ma (2016), we make the assumption on the conditional distribution,

instead of the conditional mean.

The estimation and inference of the functional single index model (1.1) is

not simple. The complexity is due to the unspecified bivariate link function f

as well as the unknown coefficient function α(t), in addition to the unknown

index vector β. If α(t) had been known, (1.1) would reduce to a central space

estimation problem and various methods exist to estimate β (Li, 1991; Cook

and Weisberg, 1991; Li and Wang, 2007; Ma and Zhu, 2013). If β had been

known, (1.1) would reduce to a functional dimension reduction problem (Ferre

3

and Yao, 2003, 2005, 2007). Qu et al. (2016) handle the unknown α(t) by using

the reproducing kernel method, where α(t) is approximated by a function in

the reproducing kernel space. Based on a similar idea, we approximate α(t) by

a spline function, which facilitates the estimation and inference procedures on

α(t).

The functional single index model (1.1) is closely related to sufficient di-

mension reduction modelling, where a response depends on the covariate vec-

tor through its linear transformation (Cook, 1998). To this end, we can view∫ 1

0X(t)α(t)dt as the covariate vector in the classical sufficient dimension reduc-

tion model. Moreover, the proposed method also forms an alternative solution

for the dimension folding problem (Li et al., 2010), and our method does not

require any conditions on the covariates as other methods in the literature do.

In summary, the new model and estimators have the following features: (i)

The model contains a single air pollution index to summarize the pollution sever-

ity level; (ii) The time varying coefficient helps to provide timely adjusted health

advice to the general public; (iii) The flexible relation between the CVD death

rate and the overall pollution effect avoids possible model misspecification; (iv)

The model extends the sufficient dimension reduction model to handle multi-

functional covariates.

4

2. Methodology

2.1 Model and Identifiability

The identifiability of Model (1.1) is shown in Proposition 1, which is justified in

the supplementary material.

Proposition 1. Assume α(t) is continuous and set α(0) = 1 and βJ = 1, then

model (1.1) is identifiable.

Under the assumptions in Proposition 1, we approximate α(t) by a B-spline

function∑dγ

k=1 γkBrk(t), where Brk(t) is the kth B-spline basis function with

order r (r ≥ 0). Then the functional single index model (1.1) reduces to the

model

f(Y,βTZγ) = f

{Y,

J∑j=1

dγ∑k=1

βjγk

∫ 1

0

Xj(t)Brk(t)dt

}, (2.2)

where Z is a J × dγ matrix with the (j, k)th entry Zjk ≡∫ 1

0Xj(t)Brk(t)dt,

where f is a unknown density function. We then proceed to estimate β and γ =

(γ1, . . . , γdγ )T simultaneously. Because the B-spline property ensures Br1(0) =

1 and Brk(0) = 0 for k > 1 and our parameterization fixes α(0) = 1, we

automatically obtain γ1 = 1.

Model (2.2) is in the form of the dimension folding models described in Li

et al. (2010) in which the predictors, i.e. Z’s in our form, are matrix-valued. The

5

covariate matrix is sandwiched between the left and right coefficient vectors, i.e.

β and γ in our setting, to generate a univariate quantity. The dimension fold-

ing structure reduces the number of parameters of interest. It has been seen in

our setting that we only have dγ + J − 2 free coefficients to estimate due to the

multiplication from both left and right sides of Z. The dimension folding model

has two main advantages over the standard single index model (Li, 1991; Cook

and Weisberg, 1991; Li and Wang, 2007), where the matrix covariates Z’s are

vectorized and βjγk for j = 1, . . . , J , k = 1, . . . , dγ are Jdγ new coefficients

(Ma, 2016): (1) it automatically takes into account the relation among the co-

efficients; (2) it reduces the dimension of the unknown coefficients from Jdγ to

dγ + J − 2, and hence avoids possible extra high dimensional problems when

the number of covariates are merely moderately large.

As an improvement of the dimension folding method, our proposed estima-

tion procedure relaxes the additional constraints on the covariate matrix Z: our

procedure does not require that E{X | (γ ⊗ β)Tvec(X)} is a linear function

of (γ ⊗ β)Tvec(X) and that var{X | (γ ⊗ β)Tvec(X)} does not depend on

(γ ⊗ β)Tvec(X) as being enforced in Li et al. (2010). Here, ⊗ stands for the

Kronecker product and vec(X) is the vector formed by the concatenation of the

columns of X. These constraints may be violated and are not assumed to hold

for model (2.2) in general.

6

2.2 Doubly Robust Local Efficient Score

To estimate the parameters, we first derive the analytic form of the efficient score.

Let

Seffβ(Y,Z,β,γ, g) =[g(Y,βTZγ)− E

{g(Y,βTZγ) | βTZγ

}](2.3)

×(IJ−1,0){Z− E(Z | βTZγ)}γ,

Seffγ(Y,Z,β,γ, g) =[g(Y,βTZγ)− E

{g(Y,βTZγ) | βTZγ

}]×(0, Idγ−1){Z− E(Z | βTZγ)}Tβ,

where the function g(Y,βTZγ) = f′2(Y,βTZγ)/f(Y,βTZγ) and f ′

2(Y,βTZγ)

is the partial derivative of f with respect to its second argument. Then the effi-

cient score is Seff(Yi,Zi,β,γ, g) ≡ {Seffβ(Y,Z,β,γ, g)T,Seffγ(Y,Z,β,γ, g)T}T.

Our hope is to use the efficient score to construct estimating equation hence to

solve for β,γ from

n∑i=1

Seff(Yi,Zi,β,γ, g) = 0. (2.4)

When g(·) and E(·|βTZγ) are correctly specified, Seff is indeed a function

which falls in the space that is orthogonal to the nuisance tangent space induced

by the unknown conditional density f(·) defined in Proposition 3. Hence, as

shown in Bickel et al. (1993); Tsiatis (2004), Seff is the efficient score (see Propo-

sition S4.1 in the supplementary document) which yields the optimal estimators

7

with the smallest asymptotic variances. In addition, Seff is also a doubly robust

function so that the estimation consistency holds wheneverE{g(Y,βTZγ) | βTZγ

}or E

(Z | βTZγ

)is correctly specified (Ma and Zhu, 2012, 2013).

In reality, the functional form for the conditional density f(·) is usually un-

known, this leads to the difficulty of obtainingE(·|βTZγ) and f ′2(Y,βTZγ)/f(Y,βTZγ)

in (2.3), hence the efficient score cannot be directly used. To retain the best esti-

mation efficiency without imposing more assumptions, we adopt nonparametric

estimation to estiimate the unknown components in the efficient score function.

Specifically, we use the standard kernel smoothing method in nonparametric re-

gression to estimate E(· | βTZγ), i.e.

E{m(Yi,Zi) | βTZγ} =

∑ni=1 m(Yi,Zi)Kh(β

TZiγ − βTZγ)∑ni=1 Kh(βTZiγ − βTZγ)

, (2.5)

for arbitrary function m(Yi,Zi). Here K(·) is a kernel function and Kh(·) =

h−1K(·/h). To estimate f(Y,βTZiγ), we use a local linear estimator. Specifi-

cally, we obtain the estimators f(Y,βTZiγ) = c0, and ∂f(Y,βTZiγ)/∂(βTZiγ) =

c1 from minimizing

J∑j=1

{Kb(Yj − Y )− c0 − c1(βTZjγ − βTZiγ)}2Kh(βTZjγ − βTZiγ) (2.6)

with respect to c0, c1. (2.5) and (2.6) allow us to obtain the unknown quantities

in Seff consistently given any parameter β,γ.

It is clearly a profiling procedure, where unknown nuisance components are

8

estimated as functions of the parameters of interest and then the estimating equa-

tions are solved to obtain the final estimator. This procedure yields the optimal

estimator for β,γ, while it has the drawback that it requires quite heavy com-

putation, especially in solving (2.6) for each value of βTZjγ. Thus, when the

estimation variability is not of great concern, to ease the computation, we may

aim for a possibly non-optimal estimator. Specifically, we may posit a working

model for f and f2, say f ∗ and f ∗2 . Let g∗ = f ∗′

2 /f∗, then Seff(Yi,Zi,β,γ, g

∗)

is a locally efficient score function. Using the locally efficient score function to

construct estimating equations guarantees estimation consistency, while it can

result in efficient estimator when g∗ is the truth.

It is also worth pointing out the issue of obtaining Zi’s. Unlike in the usual

dimension reduction problems, Zi’s are not directly observed and need to be

constructed from the observedXj(t)’s. This involves a numerical approximation

of the integrals∫ 1

0Brk(t)Xj(t)dt. The composite Simpson’s rule (Atkinson,

1989) can be used to approximate the numerical integration, which has the form

∫ 1

0

Brk(t)Xj(t)dt =1

3Q

[Brk(t0)Xj(t0) + 2

Q/2−1∑q=1

{Brk(t2q)Xj(t2q)}

+4

Q/2∑q=1

{Brk(t2q−1)Xj(t2q−1)}+Brk(tQ)Xj(tQ)

],

where tq = q/Q, q = 0, 1, . . . , Q, and Q is an even number.

The estimation procedures can be summarized as follows:

9

Step 1: Choose f and f ′2 by minimizing (2.6) or positing f and f ′2. Denote the

choices by f ∗(Y,βTZγ,β,γ) and f ′∗2 (Y,βTZγ,β,γ) and let g∗(Y,βTZγ,β,γ) =

f ′∗2 (Y,βTZγ,β,γ)/f ∗(Y,βTZγ,β,γ).

Step 2: Replace g in (2.4) by g∗ according to the Step 1 choice.

Step 3: Let Seffγ be the version of Seffγ when replacingE(·|βTZγ) by E(·|βTZγ)

defined in (2.5). Treating γ as a function of β, denoted by γ(β), we solve

n∑i=1

Seffγ [Yi,Zi,β,γ(β), g∗{Yi,βTZiγ,β,γ(β)}] = 0

for γ(β), and denote the estimator as γ(β).

Step 4: Let Seffβ be the version of Seffβ when replacingE(·|βTZγ) by E(·|βTZγ).

We solve

n∑i=1

Seffβ[Yi,Zi,β, γ(β), g∗{Yi,βTZiγ,β, γ(β)}] = 0

for β, and denote the estimator as β.

In the algorithm, we used the last two arguments in g∗(Yi,βTZiγ,β,γ) to

emphasize its possible dependence on β,γ. Obviously, when we posit a model

for f , the functional form does not have to depend on β,γ, while when we esti-

mate the model f , the functional form certainly depends on β,γ as it is the case

in all profiling estimators. The resulting estimators are consistent as discussed

in Proposition 2, because the expectations of the score functions have zero mean

when the parameters are correctly specified. In estimating E(·|βTZγ), we used

10

the variance of βTZγ times n−1/5 as bandwidth. We find that the results are

robust in the range between half of this bandwidth to twice of it.

2.3 Asymptotic Results

The profiling procedures Step 3 and 4 yield the estimators which are asymp-

totically equivalent to the ones from solving the estimating equation based on

the estimating function (STeffγ , S

Teffβ)T. Hence the estimation consistency readily

holds by the following proposition.

Proposition 2. Let β, γ satisfy

n∑i=1

{Seffβ(Yi,Zi, β, γ, g∗)T, Seffγ(Yi,Zi, β, γ, g

∗)T}T = 0

where

Seffβ(Yi,Zi, β, γ, g∗)

=

{g∗(Yi, β

TZiγ)−∑J

j=1Kh(βTZjγ − βTZiγ)g∗(Yj, β

TZjγ)∑Jj=1Kh(βTZjγ − βTZiγ)

}Θβ

×

{Zi −

∑Jj=1Kh(β

TZjγ − βTZiγ)Zj∑Jj=1 Kh(βTZjγ − βTZiγ)

}γ,

Seffγ(Yi,Zi, β, γ, g∗)

=

{g∗(Yi, β

TZiγ)−∑J

j=1Kh(βTZjγ − βTZiγ)g∗(Yj, β

TZjγ)∑Jj=1Kh(βTZjγ − βTZiγ)

}Θγ{

Zi −∑J

j=1Kh(βTZjγ − βTZiγ)Zj∑J

j=1 Kh(βTZjγ − βTZiγ)

}T

β.

11

Let β0 be the true β. Further, let γ0 be a spline coefficient which satisfies

supt∈[0,1] |Br(t)Tγ0 − α0(t)| = Op(h

qb) as stated in Condition (A5). Then β −

β0 = op(1), supt∈[0,1] |Br(·)Tγ −Br(·)Tγ0| = op(1).

In Step 3, at the point of convergence, we show that γ(β0) achieves the non-

parametric spline regression convergence rate, and derive its asymptotic varia-

tion as follows.

Theorem 1. Assume Conditions (A1)–(A8) hold, and let Br(·)Tγ(β0) satisfy

n∑i=1

Seffγ(Yi,Zi,β0, γ(β0), g∗) = 0.

Then n1/2{γ−(β0)− γ−0 } = L + op(L), where

L = −({

E

(Θγ [Zi − E{Zi|β0α0(Xi}]T β0

×∂(g∗{Yi,βT

0 α0(Xi)} − E[g∗{Yi,βT0 α0(Xi)}|βT

0 α0(Xi)]])

∂βT0 α0(Xi)

×βT0 [Zi − E{Zi|βT

0 α0(Xi)}]ΘTγ

)}−1)(n−1/2

n∑i=1

[g∗{Yi,βT0 α0(Xi)}

−E{g∗{Yi,βT0 α0(Xi)}|βT

0 α0(Xi)}]Θγ

[Zi − E{Zi|βT

0 α0(Xi)}]T

β0

).

Here γ−0 = (γ02, . . . , γ0dγ )T, and γ−(β0) = (γ2(β0), . . . , γdγ (β0))T. Further,

for arbitrary dγ−1-dimensional vector with ‖a‖2 < ∞, we have aT{γ−(β0) −

γ−0 } = Op{(nhb)−1/2} .

In addition, we show β from Step 4 is not only root-n consistent, but also

efficient and achieve the information lower bound {E(S⊗2oeff)}−1. Here Soeff is

12

the efficient score for β of the original model (1.1), which contains α(·) instead

of Br(·)Tγ, hence it is different from Seffβ. Its precise expression is given in

Proposition S4.2 in the supplementary document.

To show the asymptotic properties of β, we first define

∆g∗c{Yi,βT0 α0(Xi)}

≡ ∂[g∗{Yi,βT0 α0(Xi)} − E{g∗{Yi,βT

0 α0(Xi)}|βT0 α0(Xi)}]

∂{βT0 α0(Xi)}

and w∗0(t) as a function that satisfies

ΘβE[αc0(Xi)∆g∗c{Yi,βT

0 α0(Xi)}XTic(t)]β0 (2.7)

=

∫ 1

0

E[βT0 Xic(s)∆g

∗c{Yi,βT

0 α0(Xi)}XTic(t)β0]w∗0(s)ds,

where αc0(X) ≡ α0(X) − E{α0(X) | βT0 α0(X)}. We have the following

results.

Theorem 2. Assume Conditions (A1)–(A8) hold, and let β satisfy

n∑i=1

Seffβ(Yi,Zi, β, γ(β), g∗) = 0.

Then√n(β − β0) = A−1B + op(1), where

A = −E[∆g∗c{Yi,βT

0 α0(Xi)}{Θβαc0(Xi)}⊗2

−∆g∗c{Yi,β

T0 α0(Xi)

}∫ 1

0

βT0 Xic(t)w

∗0(t)dtαc0(Xi)

TΘTβ

]

13

and

B = n−1/2

n∑i=1

{g∗{Yi,β

T0 α0(Xi)

}− E

[g∗{Yi,βT

0 α0(Xi)}|βT0 α0(Xi)

]}×{

Θβαc0(Xi)−∫ 1

0

βT0 Xic(t)w

∗0(t)dt

}.

Hence,√n(β − β0) converges to a normal distribution with mean 0 and vari-

ance Σ, where Σ ≡ A−1E(B⊗2)A−1T. Here a⊗2 = aaT for arbitrary vector

or matrix a. In addition, when g∗ = g,√n(β − β0) converges to a normal dis-

tribution with mean 0 and variance {E(S⊗2oeff)}−1, i.e., β is the semiparametric

efficient estimator of β for model (1.1).

Theorem 2 indicates that β is the consistent estimator. Further it is semi-

parametric efficient when g∗ is correctly specified, even though the estimation of

β is devised under the approximate model (2.2). Generally, we can replace g∗

by a consistent estimator of g. The following corollary ensures the asymptotic

efficiency of the resulting β.

Corollary 1. Assume Conditions (A1)–(A8) hold, and let β satisfy

n∑i=1

Seffβ(Yi,Zi, β, γ(β), g) = 0,

where g is a uniformly consistent estimator for the true function g, and Seffβ is

defined in Proposition 2, then β is semiparametric efficient.

14

The corollary readily holds by using the result in Theorem 2 and the consis-

tency of g. We omit the details. In practice, we can use kernel method to estimate

f(Y,βTZγ) and in turn to obtain g, which is guaranteed to be uniformly con-

sistent to g (Mack and Silverman, 1982). Combining the results of Theorems

1 and 2, we are able to establish the theoretical properties of the estimation of

α(t) in Theorem 3. Specifically, Theorem 3 shows that the spline approximation

α(t) = Br(t)Tγ(β) with β,γ estimated using the estimating equation set (3.8)

indeed achieves the usual nonparametric spline regression convergence rate.

Theorem 3. Assume Conditions (A1) – (A8) hold, then

supt∈[0,1]

|Br(t)Tγ(β)− α0(t)| = Op(n

−1/2h−1/2b ).

The proofs for the theoretic results are elaborated in the supplementary doc-

ument.

3. Relation to the semiparametric sufficient dimension reduction

Although performed for the functional model and the dimension folding model,

the proposed estimation procedure is inline with the semiparametric sufficient

dimension reduction techniques discussed in Ma and Zhu (2012). To illustrate

the similarity, following Bickel et al. (1993) and Tsiatis (2004), we first develop

the nuisance tangent space Λ⊥ in the following proposition, which allows us to

15

construct estimators of β,γ from various choices of the function f appeared in

the description of Λ⊥.

Proposition 3. In the Hilbert space H of all the mean zero finite variance func-

tions associated with (2.2), i.e. H = {a(Z, Y ) :∫

a(z, y)f(y,βTzγ)fZ(z)dµ(z, y) =

0,∫

aT(z, y)a(z, y)f(y,βTzγ)fZ(z)dµ(z, y) <∞, a(z, y) ∈ Rdγ+J−2}, where

µ(z, y) is the probability measure of (Z, Y ), fZ(z) is the pdf of Z and f(y,βTzγ)

is given in (2.2), the orthogonal complement of the nuisance tangent space is

Λ⊥ = {f(Y,Z)− E(f | Y,βTZγ) : E(f | Z) = E(f | βTZγ), ∀ f}.

The proof of Proposition 3 is given in the supplementary document. Let

f(Y,Z) =[g(Y,βTZγ)− E

{g(Y,βTZγ) | βTZγ

}]a(Z), where g, a can be

chosen arbitrarily as long as the resulting f contains sufficiently many equations.

Obviously, E(f | Z) = 0 hence E(f | βTZγ) = 0, and f − E(f | Y,βTZγ) ={g − E

(g | βTZγ

)} {a− E(a | βTZγ)

}. Thus, we can construct estimating

equation based on the sample version of

E([g(Y,βTZγ)− E{g(Y,βTZγ) | βTZγ}][a(Z)− E{a(Z) | βTZγ}]

)= 0, (3.8)

and it provides a class of estimators for β,γ.

We now perform a set of analysis somewhat in the spirit of Ma and Zhu

(2012) to illustrate that by different choice of g and a, (3.8) leads to the classical

16

dimension reduction estimators.

3.1 The relation with the sliced inverse regression

As a first choice of g and a, let V = vec(Z), and select g(Y,βTZγ) = E(V |

Y ), a(Z) = VT. This provides an estimator with the flavor of the sliced in-

verse regression (SIR, Li (1991)) in the classical dimension reduction frame-

work. Specifically, under this choice of g, a, (3.8) has the form

E([E(V | Y )− E{E(V | Y) | βTZγ}

] {VT − E(VT | βTZγ)

})= 0.(3.9)

The above estimating equation set contains J2dγ2 equations while we only have

J+dγ−2 free parameters. We can use GMM to reduce the number of equations

in practice. We can also construct g or a or both using only a subset of V.

3.2 The relation with the sliced average variance estimator

As a second choice of g, a, we select g1(Y,βTZγ) = IJdγ − cov(V | Y ),

g2(Y,βTZγ) = g1E(V | Y ), a1(Z) = −V{V − E(V | βTZγ)}T, a2(Z) =

VT. We then construct a classical sliced average variance estimator (SAVE,

Cook and Weisberg (1991)) flavored estimator based on

E[{g1 − E(g1 | βTZγ)}{a1 − E(a1 | βTZγ)}

](3.10)

+E[{g2 − E(g2 | βTZγ)}{a2 − E(a2 | βTZγ)}

]= 0.

17

3.3 The relation with the directional regression

The third choice of g, a that we would like to present is g1(Y,βTZγ) = IJdγ −

E(VVT | Y ), g2(Y,βTZγ) = E{E(V | Y )E(VT | Y )}E(V | Y ), g3(Y,βTZγ) =

E{E(VT | Y )E(V | Y )}E(V | Y ), a1(Z) = −V{V − E(V | βTZγ)}T,

a2(Z) = a3(Z) = VT. This leads to a classic directional regression (DR, Li and

Wang (2007)) flavored estimator from

3∑i=1

E[{gi − E(gi | βTZγ)}{ai − E(ai | βTZγ)}

]= 0. (3.11)

The three estimators given in (3.9), (3.10) and (3.11) are respectively of the

flavours of SIR, SAVE and DR, because if we had worked in the classical suf-

ficient dimension reduction context, and if further equipped with the additional

linearity condition and constant variance condition, the choices of g and a that

led to the three estimating equations above would have further led to SIR, SAVE

and DR (Ma and Zhu, 2012). Further, the choices of g, a in (3.9), (3.10) and

(3.11) only depend on the moments of Z instead of the conditional density as

the one used in Seff defined in (2.3). Hence, these estimators can serve as al-

ternatives to the proposed efficient estimators when the conditional density is

difficult to obtain.

18

4. Simulation Studies

We carry out three simulation studies under the following settings in order to

assess the finite sample performance of our estimation method. In each simula-

tion, we generate 1000 data sets with the sample size n = 500.

Simulation 1

(1) J = 9, β = (1, 1.2, 1.5, 0.5,−0.5,−1.5,−1.2,−1, 1)T α0(t) = sin(πt) + 1,

t ∈ [0, 1];

(2) Xji(t), j = 1 . . . , 4 follows U(−5, 5), where U [a, b] denotes a random vari-

able from the uniform distribution in the range of [a, b];

(3) Yi follows a normal distribution with mean∫ 1

0Wi(t)α0(t)dt and variance 1,

where Wi(t) = βTXi(t).

Simulation 2 investigates the ability of our methods in handling the nonlinear

mean and variance.

(1) Yi follows a normal distribution with mean sin{2∫ 1

0Wi(t)α0(t)dt}+ log[1+

{∫ 1

0Wi(t)α0(t)dt}2]− 3 and variance 0.5[1 + {βT

∫ 1

0Xi(t)α0(t)dt}2]1/5.

Simulation 3 resembles the air pollution data structure in Section 5.

(1) J = 4, β = (−0.2,−1,−1.5, 1)T and α0(t) = 1 − 26.76t + 145.3t2 −

227.27t3 + 107.99t4;

(2) Xi1(t) = 0.66 − 4.84t + 5.12t2 + U [−4, 6], Xi2(t) = 0.43 − 2.95t +

19

3.11t2 + U [−5, 5], Xi3(t) = −1.61 + 10.40t − 10.85t2 + U [−4, 4], Xi4(t) =

0.58− 3.59t+ 3.52t2 + U [−4, 8];

(3) Yi is from a normal distribution with mean 0.075 + 0.53(∫ 1

0Wi(t)α0(t)dt +

1.23) and variance 0.05.

We applied the proposed method to estimate both β and α(t), where α(t) is

approximated with cubic B-spline basis functions using three equally spaced in-

ternal knots. For comparison, we implemented three estimators: the oracle, the

efficient, and the locally efficient estimators. In the oracle estimator, we specified

f(Y,βTZγ) using the normal pdf form, and used the true g(Y,βTZγ) in the es-

timation. In the efficient estimator, E(·|βTZγ), f(Y,βTZγ), and f ′2(Y,βTZγ)

are estimated via nonparametric method. In the local estimator, we specified

an incorrect model of f(Y,βTZγ), hence a misspecified g∗(Y,βTZγ) function

was used, and estimated E(·|βTZγ) nonparametrically. Note that the form of

f(Y,βTZγ) is generally unknown, hence the oracle estimator is unrealistic and

is only included here as a benchmark for a comparison purpose.

The numerical performances of the estimation of β in Simulation 1, 2 and

3 are summarized in Table 1, 2 and 3, respectively. Based on the asymptotic

results in Theorem 2, the average of the estimated standard error is obtained and

the coverage of the 95% confidence interval is also provided. As expected, both

the efficient and the locally efficient estimators have very small bias, the esti-

20

mated variances are close to their empirical ones, and the 95% coverage are also

reasonably close to the nominal level. The variances of the efficient estimators

are smaller than those of the locally efficient estimators. In fact, the performance

of the efficient estimators is very close to the oracle estimators. We illustrate the

performance of the estimation of α0(t) in Figure 1, where we show the mean

estimated curves and the pointwise 90% confidence bands. The performance

shown in Figure 1 is rather typical for spline approximations.

In functional data analysis, a simple stacking approach is often used to study

the effect of the functional covariates (Ramsay and Silverman, 2005) in a less

structured model

E{Y |X(t)} =

∫ 1

0

X(t)Tη(t)dt, (4.12)

where η(t) = {η1(t), . . . , ηp(t)}T. The stacking approach is a special case of

the proposed functional single index model. We thus implemented the stacking

approach and compared the two estimators in Figure 2. It is easy to see that our

estimator performs better than the stacking approach, with narrower confidence

bands. This pattern also applies to simulations 2 and 3, and we provide the

corresponding plots in Figures S1 and S2 of the supplementary document.

21

5. Application

We apply the proposed method to study the effect of various air pollutants on

the rate of death caused by CVD, where we adopt the model in (1.1) without

specifying any special link function.

In the NMMAP data (Peng and Welty, 2004), all four pollutants (CO, NO2,

SO2, and O3) were recorded on a daily basis in 108 U.S. cities. The measure-

ments unit is parts per billion (ppb) by volume and spans the range from 1987

to 2000. 400 observations with a relatively small portion of missing values are

used for the analysis and each observation has 365 daily median measurements

of four air pollutants, where we imputed a few missing days in some obser-

vations by linear interpolation. We also standardized each pollutant across the

whole year so that the 365 observations yield a sample mean 0 and sample vari-

ance 1. The time interval is normalized to [0, 1]. Figure S3 of the supplementary

document displays the mean trajectories for four pollutants.

We fit model (1.1) to estimate the air pollution index directly related to the

following year’s CVD death rate. Throughout the implementation, we set the

kernel bandwidth h to be n−1/5range(βTziγ) and b = n−1/7range(yi), where

the unknown parameters β and γ are updated during each iteration. The func-

tional parameter α(t) is estimated using a linear combination of cubic B-splines

with three equally spaced internal knots in [0, 1], where the optimal number of

22

internal knots is determined through a ten-fold cross-validation. We calculated

the confidence band for α(t) using the asymptotic results in Theorem 1.

Figure 3 shows the time-varying effect α(t) of the estimated air pollution

index to the CVD death rate. The time-varying effect of the air pollution index

is significantly positive in the spring, summer, and fall seasons but insignificant

in the winter. The air pollution index has the largest positive effect on the CVD

death rate in the summer.

Figure 4 displays the air pollution index for three major cities: Boston, New

York and Chicago together with their CVD annual death rates. With the largest

air pollution index in the summertime, New York has the largest CVD death rate.

On the other hand, Boston has the lowest air pollution index in the summer and

hence the CVD death rate is the smallest in Boston, although Boston has the

highest air pollution index in the winter.

Table 4 displays the estimated coefficients for all four air pollutants β. The

standard errors and the p-values are obtained based on the asymptotic normality

of β shown in Theorem 2. All estimated coefficients β are statistically signifi-

cant, which indicates that CO, NO2, O3 and SO2 are all significant risk factors

for the air pollution index related to the CVD death rate. It reaffirms that all of

the pollutants have a significant effect on the CVD death rate. The estimated

coefficients for CO, NO2, and SO2 are negative, which is caused by the corre-

23

lation of these three air pollutants with O3. The correlation coefficients among

these four air pollutants are provided in Section S5 of the supplementary files.

We also study the time-varying effect of each individual pollutant on the CVD

death rate by fitting a simple functional linear regressionE(Y ) =∫ 1

0η(t)X(t)dt

to the air pollution data, where the response variable Y is the annual CVD death

rate, and the functional covariate X(t) is the daily concentration of the air pol-

lutant CO, NO2, SO2, and O3, respectively. Figure S4 in the supplementary file

displays the estimated functional coefficient η(t) with the 95% pointwise con-

fidence interval. It shows that no significant effect of each individual pollutant

is found on the CVD death rate. This is another motivation for us to estimate a

comprehensive air pollution index to measure the contributions of air pollutants

simultaneously.

For comparison, we implemented the stacking approach to estimate the

functional linear model (4.12). Figure 5 compares the estimated ηk(t) for the

stacking functional linear model (4.12) and the estimated βkα(t) for our func-

tional index model (1.1), where k = 1, . . . , 4. While there is slight disagreement

between the two sets of estimations from the two models, it is clear that the un-

structured model has very large variability and can hardly deliver any statistically

significant results.

We further assessed the prediction performance of our proposed method in

24

comparison with three other methods, including the stacking functional linear

model (4.12), the functional additive model (Muller and Yao, 2008), and a sin-

gle index model where each covariate is simply the yearly average of each pol-

lutant. The evaluation is conducted through a ten-fold cross-validation. Table

5 displays the mean squared prediction errors (MSPE) of our proposed method

and the three comparison methods. It shows that our proposed functional sin-

gle index model has the smallest MSPE among all four methods. For instance,

MSPE is decreased by 31% when using our proposed functional single index in

comparison with the stacking functional linear model (4.12).

6. Discussion

We proposed a functional single index model to study the relation between the

pollutants and CVD death rate. The model contains a single index which sum-

marizes the pollution severity level and a time-varying coefficient which cap-

tures the seasonality of the pollution effects. Furthermore, the model is robust

against the misspecification of the conditional density function fY |X(t)(·). When

replacing the function α(·) by its B-spline approximation, the model reduces

to a dimension folding model, and our estimator yields a new estimator as a

by-product. This new estimator requires much more relaxed conditions on the

covariates while at the same time performs much better than all existing meth-

25

ods. Finally, the model and method can be used in the high dimensional settings

thanks to the fact that the numbers of covariate functions and spline basis are

added. In contrast, the traditional functional single index described in (4.12)

would result in multiplication of these two numbers.

In our analysis, we assume the functional covariate Xi(t) is known to sim-

plify the problem. However, in practice, the measurements for the functional

covariate Xi(t) may contain errors. To take into account the errors, model (1.1)

should be further augmented. The resulting model falls into the measurement

error framework and deserves careful investigation in future work.

Supplementary Materials

The supplementary document online includes the comprehensive proofs of all

theoretic results. The computing codes for our simulation studies and application

can be downloaded from https://github.com///sbaek306/FSIM .

References

Atkinson, K. E. (1989). An Introduction to Numerical Analysis (2nd ed.). New York: John Wiley & Sons.

Bickel, P. J., C. A. Klaassen, P. J. Bickel, Y. Ritov, J. Klaassen, J. A. Wellner, and Y. Ritov (1993). Efficient

and adaptive estimation for semiparametric models. New York: Springer.

Chen, D., P. Hall, H.-G. Muller, et al. (2011). Single and multiple index functional regression models with

nonparametric link. The Annals of Statistics 39(3), 1720–1747.

26

Cook, D. (1998). Regression Graphics: Ideas for Studying Regressions through Graphics. New York:

Wiley.

Cook, R. and S. Weisberg (1991). Discussion of “sliced inverse regression for dimension reduction”.

Journal of the American Statistical Association 86, 28–33.

Cox, L. A. T. and D. A. Popken (2015). Has reducing fine particulate matter and ozone caused reduced

mortality rates in the united states? Annals of epidemiology 25(3), 162–173.

Ferre, L. and A. Yao (2003). Functional sliced inverse regression analysis. Statistics 37(6), 475–488.

Ferre, L. and A. Yao (2005). Smoothed functional inverse regression. Statistica Sinica 15(3), 665–683.

Ferre, L. and A. Yao (2007). Reply to the paper ‘a note on smoothed functional inverse regression’ by

liliana forzani and r. dennis cook. Statistica Sinica 17(4), 1534–1544.

Li, B., M. K. Kim, and N. Altman (2010). On dimension folding of matrix or array valued statistical

objects. Annals of Statistics 38, 1094–1121.

Li, B. and S. Wang (2007). On directional regression for dimension reduction. Journal of the American

Statistical Association 102, 997–1008.

Li, K. C. (1991). Sliced inverse regression for dimension reduction (with discussion). Journal of the

American Statistical Association 86, 316–342.

Ma, S. (2016). Estimation and inference in functional single-index models. Annals of the Institute of

Statistical Mathematics 68(1), 181–208.

Ma, Y. and L. Zhu (2012). A semiparametric approach to dimension reduction. Journal of the American

27

Statistical Association 107, 168–179.

Ma, Y. and L. Zhu (2013). Efficient estimation in sufficient dimension reduction. Annals of Statistics 41,

250–268.

Mack, Y. and B. W. Silverman (1982). Weak and strong uniform consistency of kernel regression estimates.

Zeitschrift fur Wahrscheinlichkeitstheorie und verwandte Gebiete 61(3), 405–415.

Muller, H. and F. Yao (2008). Functional additive models. Journal of the American Statistical Associa-

tion 103(4), 1683.

Peng, R. and L. Welty (2004). The nmmapsdata package. R News 4, 10–14.

Qu, S., J.-L. Wang, X. Wang, et al. (2016). Optimal estimation for the functional cox model. The Annals

of Statistics 44(4), 1708–1738.

Ramsay, J. and B. W. Silverman (2005). Functional data analysis (Second ed.). New York: Springer-

Verlag.

Samet, J., F. Dominici, S. Zeger, J. Schwartz, and D. Dockery (2000). The national morbidity, mortality,

and air pollution study. part i: Methods and methodologic issues. Research report (Health Effects

Institute) (94 Pt 1), 5–14.

Tsiatis, A. A. (2004). Semiparametric Theory and Missing Data. New York: Springer.

Turner, M. C., M. Jerrett, C. A. Pope III, D. Krewski, S. M. Gapstur, W. R. Diver, B. S. Beckerman, J. D.

Marshall, J. Su, D. L. Crouse, et al. (2016). Long-term ozone exposure and mortality in a large

prospective study. American journal of respiratory and critical care medicine 193(10), 1134–1142.

28

F. Jiang

Department of Statistics and Actuarial Science,

University of Hong Kong, Hong Kong, China. E-mail: feijiang@hku.hk

Seungchul Baek

Department of Statistics, University of South Carolina,

Columbia, SC 29208, USA. E-mail: sbaek@stat.sc.edu

Jiguo Cao

Department of Statistics and Actuarial Science,

Simon Fraser University, Burnaby, BC V5A 1S6, Canada. E-mail: jiguo cao@sfu.ca

Yanyuan Ma

Department of Statistics, Penn State University,

University Park, PA 16802, USA. E-mail: yzm63@psu.edu

29

0.0 0.2 0.4 0.6 0.8 1.0

1.0

1.5

2.0

truemean95%5%

0.0 0.2 0.4 0.6 0.8 1.0

1.0

1.5

2.0

2.5

truemean95%5%

0.0 0.2 0.4 0.6 0.8 1.0

−1

01

23

truemean95%5%

Figure 1: The mean and point wise 90% confidence bands of the estimated α(t)

for the functional single index model (1.1) in Simulations 1 (left), 2 (middle) and

3 (right). The true α0(t) is plotted in the solid curve.

30

Table 1: The average (AVE), the sample standard deviations (STD), the average

of the estimated standard deviation (STD), the square root of the mean squared

error (MSE) and the coverage of the estimated 95% confidence interval (CI) from

the oracle (Ora), efficient (Eff) and Locally efficient (Loc) estimates of β,γ in

Simulation 1.

β1 β2 β3 β4 β5 β6 β7 β8

1 1.2 1.5 0.5 -0.5 -1.5 -1.2 -1

Ora AVE 1.0089 1.2070 1.5134 0.5067 -0.5033 -1.5081 -1.2141 -1.0062

STD 0.1294 0.1364 0.1588 0.1020 0.1007 0.1553 0.1412 0.1304

STD 0.1244 0.1365 0.1577 0.0994 0.0996 0.1570 0.1373 0.1242

MSE 0.0168 0.0186 0.0254 0.0104 0.0101 0.0242 0.0201 0.0170

CI 0.9470 0.9520 0.9540 0.9420 0.9470 0.9530 0.9420 0.9330

Eff AVE 1.0264 1.2279 1.5398 0.5160 -0.5122 -1.5344 -1.2347 -1.0240

STD 0.1368 0.1460 0.1719 0.1065 0.1056 0.1669 0.1522 0.1389

STD 0.1349 0.1495 0.1744 0.1051 0.1052 0.1734 0.1503 0.1349

MSE 0.0194 0.0221 0.0311 0.0116 0.0113 0.0290 0.0243 0.0198

CI 0.9640 0.9650 0.9660 0.9500 0.9510 0.9630 0.9580 0.9520

Loc AVE 1.0335 1.2427 1.5533 0.5194 -0.5139 -1.5479 -1.2444 -1.0341

STD 0.1535 0.1674 0.1962 0.1229 0.1192 0.1903 0.1728 0.1559

STD 0.1544 0.1700 0.1997 0.1208 0.1207 0.1969 0.1702 0.1538

MSE 0.0247 0.0298 0.0413 0.0155 0.0144 0.0385 0.0318 0.0254

CI 0.9550 0.9540 0.9580 0.9520 0.9570 0.9640 0.9550 0.9600

31

Table 2: The average (AVE), the sample standard deviations (STD), the average

of the estimated standard deviation (STD), the square root of the mean squared

error (MSE) and the coverage of the estimated 95% confidence interval (CI) from

the oracle (Ora), efficient (Eff) and Locally efficient (Loc) estimates of β,γ in

Simulation 2.

β1 β2 β3 β4 β5 β6 β7 β8

1 1.2 1.5 0.5 -0.5 -1.5 -1.2 -1

Ora AVE 1.0037 1.2019 1.5031 0.5015 -0.5030 -1.5015 -1.2001 -1.0014

STD 0.0611 0.0677 0.0752 0.0503 0.0493 0.0750 0.0682 0.0568

STD 0.0603 0.0662 0.0764 0.0484 0.0485 0.0763 0.0662 0.0604

MSE 0.0037 0.0046 0.0057 0.0025 0.0024 0.0056 0.0046 0.0032

CI 0.9500 0.9430 0.9430 0.9370 0.9480 0.9550 0.9310 0.9580

Eff AVE 1.0027 1.2018 1.5042 0.5024 -0.5037 -1.5005 -1.2004 -1.0011

STD 0.0752 0.0835 0.0930 0.0571 0.0564 0.0966 0.0801 0.0718

STD 0.0734 0.0805 0.0931 0.0579 0.0578 0.0937 0.0805 0.0730

MSE 0.0057 0.0070 0.0087 0.0033 0.0032 0.0093 0.0064 0.0051

CI 0.9460 0.9460 0.9470 0.9470 0.9500 0.9410 0.9470 0.9500

Loc AVE 0.9944 1.1907 1.4877 0.4963 -0.4973 -1.4868 -1.1889 -0.9913

STD 0.1494 0.1720 0.2098 0.0830 0.0842 0.2126 0.1736 0.1426

STD 0.1571 0.1855 0.2282 0.0893 0.0895 0.2296 0.1854 0.1568

MSE 0.0223 0.0296 0.0441 0.0069 0.0071 0.0453 0.0302 0.0204

CI 0.9440 0.9400 0.9440 0.9530 0.9470 0.9460 0.9430 0.9460

32

Table 3: The average (AVE), the sample standard deviations (STD), the average

of the estimated standard deviation (STD), the square root of the mean squared

error (MSE) and the coverage of the estimated 95% confidence interval (CI)

from the oracle, efficient, and locally efficient estimates of β,γ in Simulation 3.

β1 β2 β3

TRUE -0.2 -1.0 -1.5

Oracle AVE -0.2009 -1.0015 -1.5005

STD 0.0497 0.0650 0.0842

STD 0.0493 0.0634 0.0860

MSE 0.0025 0.0042 0.0071

CI 0.9520 0.9480 0.9520

Efficient AVE -0.2017 -1.0057 -1.5058

STD 0.0502 0.0662 0.0851

STD 0.0497 0.0642 0.0871

MSE 0.0025 0.0044 0.0071

CI 0.9480 0.9440 0.9540

Locally AVE -0.2020 -0.9893 -1.4849

Efficient STD 0.0769 0.1002 0.1246

STD 0.0790 0.1069 0.1508

MSE 0.0059 0.0101 0.0157

CI 0.9630 0.9420 0.9530

33

Table 4: The estimated coefficients for the air pollutants CO, NO2, SO2 and

standard errors for the functional single index model (1.1) in the air pollution

data using the efficient method. The coefficient for O3 is fixed to be 1 for iden-

tifiability, as introduced in Section 2.1.

β1 (CO) β2 (NO2) β3 (SO2) β4 (O3)

Coefficients -0.286 -0.971 -1.833 1.000

Standard Errors 0.080 0.006 0.002 -

p-values 3e-4 <5e-5 <5e-5 -

Table 5: The mean squared prediction errors of the four methods for the CVD

death rate.

Methods Mean Squared Prediction Errors

Functional single index Model (1) 2.14×10−6

Stacking functional linear model (12) 3.11×10−6

Functional additive model 2.56×10−6

Single index model 2.44×10−6

34

0.0 0.2 0.4 0.6 0.8 1.0

−2

02

46 true

proposedsimple

0.0 0.2 0.4 0.6 0.8 1.0

−2

02

4

trueproposedsimple

0.0 0.2 0.4 0.6 0.8 1.0

−4

−2

02

46

trueproposedsimple

0.0 0.2 0.4 0.6 0.8 1.0

−2

02

4

trueproposedsimple

0.0 0.2 0.4 0.6 0.8 1.0

01

23

4 trueproposedsimple

0.0 0.2 0.4 0.6 0.8 1.0

−2

02

46

trueproposedsimple

0.0 0.2 0.4 0.6 0.8 1.0

−2

02

46

trueproposedsimple

0.0 0.2 0.4 0.6 0.8 1.0

−2

02

4

trueproposedsimple

0.0 0.2 0.4 0.6 0.8 1.0

−1

01

23

45 true

proposedsimple

Figure 2: The estimated βkα(t), k = 1, . . . , 9, and their point-wise 90% confi-

dence bands for the proposed functional single index model (1.1), in comparison

with the estimated ηk(t) for the simple stacking functional linear model (4.12)

in Simulation 1.

35

0.0 0.2 0.4 0.6 0.8 1.0

−2

−1

01

23

t

cbin

d(es

tc, e

st95

, est

5)

Efficient95%5%

Figure 3: The estimated α(t) for the functional single index model (1.1) from

the air pollution data. It captures the time-varying effect of the air pollution

index on the annual CVD death rate. The pointwise 90% confidence band of the

estimated α(t) is also provided.

36

-4-2

02

4

Month/Day

Air

pollu

tion

inde

x

01-01

02-01

03-01

04-01

05-01

06-01

07-01

08-01

09-01

10-01

11-01

12-01

Boston 0.27%New York 0.39%Chicago 0.35%

Figure 4: The pollution indices for Boston, New York and Chicago. The CVD

death rates are shown in the legend.

37

0.0 0.2 0.4 0.6 0.8 1.0

−3

−2

−1

01

proposed95%5%simple95%5%

0.0 0.2 0.4 0.6 0.8 1.0−

3−

2−

10

12

proposed95%5%simple95%5%

0.0 0.2 0.4 0.6 0.8 1.0

−6

−4

−2

02

proposed95%5%simple95%5%

0.0 0.2 0.4 0.6 0.8 1.0

−2

−1

01

2

proposed95%5%simple95%5%

Figure 5: Comparison of the estimated βkα(t) in the proposed functional single

index model (1.1) with their 90% confidence bands, and the estimated ηk(t)

with their 90% confidence bands in the simple stacking functional linear model

(4.12), for k = 1, . . . , 4. The top left panel is β1α(t) and η1(t), the top right

panel is β2α(t) and η2(t), the bottom left panel is β3α(t) and η3(t), and the

bottom right panel is β4α(t) and η4(t).

38