1 Date

Name, department

Lecture 3Empirical Bayes and

Proc MixedZiad Taib

Biostatistics, AZ

April 24, 2009

Date

Name, department

2

Outline of the lecture

1. Reminder

2. Inference for the random effects

3. Proc Mixed

Date

Name, department

3

1. Reminder

Vi

Date

Name, department

4

2. Inference for the Random Effects - Empirical Bayes Inference

Date

Name, department

5

)()/(

)(

)()/(

)(

),()/(

iii

i

iii

i

iiii

bfbyf

yf

bfbyf

yf

byfybf

Date

Name, department

6

Date

Name, department

7

Date

Name, department

8

Comments

The above EB estimate of the random effect can be obtained using a set of equations

It can be shown that using the EB estimate lead to Best Linear Unbiased Prediction of linear combination of the form:

When trying to predict the response of an individual, we can use:

and we see that the observed data are shrunken towards the prior average profile.

ibu bβ ''

data Observed

1

average Population

1 ˆ...ˆˆˆiiiniiiiiii yVIXVbZXY

i

Date

Name, department

9

3. Statistical software

Date

Name, department

10

Software (cont’d)

SAS – SPSS – BMDP/5v – ML3 – HLM – Splus – R can handle correlated data but some are more restricted than others.

Most packages offer a choice between ML and REML and optimisation is often based on Newton-Raphson, the EM algorithm or Fisher scoring.

The user has to specify a model for the mean response that is linear in the fixed effects and to specify a covariance structure. The user can select a full parameterisation of the covariance structure (unstructured) or choose among given covariance structures.

The covariance structure is also influenced by the inclusion of random effects and their covariance structure.

Date

Name, department

11

Software (cont’d)

Output often includes: history of optimisation iterations

estimates of fixed effects

covariance parameters with standard errors

estimates of user specified contrasts

Graphics is often limited but can be done in another software

Date

Name, department

12

SAS PROC MIXED and Repeated Measures PROC MIXED of SAS offers greater flexibility for the modelling of

repeated measures data than PROC GLM. (Firstly, the procedure provides a mechanism for modelling the covariance structure associated with the repeated measures. Secondly, it can handle some forms of missing data without discarding an entire subject’s-worth of data. Thirdly, it has some capability to handle the situation when each subject may be measured at different times and time intervals.)

In PROC GLM, repeated measures are handled in a multivariate framework and it requires a multivariate view of the data. PROC MIXED, on the other hand, requires a univariate or stacked-data view of the data. In other words, there is only a single response variable. The repeated information, including all of the information about the subjects, is contained in other variables. Proc GLM assumes that the covariance matrix meets a sphericity assumption compound symmetry.

Date

Name, department

13

Proc mixed was designed to handle mixed models. It has a large choice of covariance structures (unstructured, random effects, autoregressive, Diggle etc)

PROC MIXED can be used not only to estimate the fixed parameters, but also the covariance parameters.

By default, PROC MIXED estimates the covariance parameters using the method of restricted maximum likelihood (REML).

PROC MIXED provides empirical Bayes estimates. Separate analyses for separate groups can be run using the BY

statement. Approximate F tests for class variables are obtained using Wald’s test. All components of the output can be saved as a SAS data set for

further manipulation using other internal (SAS) or external procedures.

SAS PROC MIXED

Date

Name, department

14

PROC MIXED: Syntax

PROC MIXED < options > ; BY variables ; CLASS variables ; ID variables ; MODEL dependent = < fixed-effects > < / options > ; RANDOM random-effects < / options > ; REPEATED < repeated-effect > < / options > ; PARMS (value-list) ... < / options > ; CONTRAST 'label' < fixed-effect values ... >                                    < | random-effect values ... > , ... < / options > ; ESTIMATE 'label' < fixed-effect values ... >                                    < | random-effect values ... >< / options > ; LSMEANS fixed-effects < / options > ; MAKE 'table' OUT=SAS-data-set ;

Proc

Date

Name, department

15

Data structure of Proc Mixed Consider the example where arm strength is measured on 8

patients at 3 different times and where patients have been randomized to one of 2 treatment groups. The multivariate view associated with e.g. PROC GLM code: would look like below

Date

Name, department

16

For analysis of this data set using PROC MIXED, the univariate or stacked-data view will be required. The univariate view below was obtained by Proc Transpose:

Date

Name, department

17

Date

Name, department

18

Date

Name, department

19

Date

Name, department

20

Date

Name, department

21

Date

Name, department

22

Date

Name, department

23

Date

Name, department

24

PROC MIXED data=prostate method=REML asycov asycorr covtest ic; CLASS id group timeclss ; MODEL lnpsa = group age group*time age*time group*time2 age*time2

/ noint solution ddfm=satterth covb chisq; ID id time ; RANDOM intercept time time2 /type=un subject=id g gcorr v vcorr solution; REPEATED timeclss / type=simple subject=id r rcorr ; CONTRAST ‘Final model' age*time 1,

age*time2 1,group*time2 1 0 0 0,group*time2 0 1 0 0,group*time2 0 0 1 –1 /chisq;

ESTIMATE ‘Diff L/R-BPH, t=5yr’ group 0 –4 4 0group*time 0 -2 2 0,group*time2 0 0 1 0,/ cl alpha=0.05 divisor=4chisq;

MAKE ‘solutionR’ out=randeff;RUN;

Proc

Date

Name, department

25

Proc mixed

Proc mixed invokes the mixed procedure. Method= specifies the estimation methods (ML. REML. MIVQUE0)

Asycov and Asycorr can be used for printing the asymptotic covariance and correlation matrices for the marginal model

Covtest prints the asymptotic standard errors and associated Wald tests for the variance components

ic calculates some information criteria

Class specifies which variables are considered as factors.

Date

Name, department

26

Proc mixed

Model specifies the model (i.e response and fixed effects Xi). Intercept included by default.

Solution prints estimates of the fixed effects in the model together with standard errors, t-statistics and p-values for significance.

Covb gives the whole covariance matrix for the estimates.

ddfm= specifies the number of degrees of freedomin the t- and F-approximations. One of many options is Satterthwaite.

Chisq is used to make SAS include Wld tests next to the default t- and F-tests for all effects specified in the model.

Date

Name, department

27

Proc mixed

Id In general, SAS uses the same order as the original data but it does not hurt to have an extra column helping identifying the records and the subjects. This is nice to have e.g. when using predmeans or predicted to get predicted values.

Random specifies the random effects (Zi). Notice that random intercept is not default.

Solution needed to calculate empirical Bayes estimates

Subject= id

G, gcorr, v, vcorr print correlation matrices D and Vi. Default is first subject but number of subjects can be specified.-

Date

Name, department

28

Proc mixed

Repeated used to specify the i. The repeated effects must be classification variables.

Type= specifies the structure of i. Simple meabns independence.

r and rcorr print residual covariance, i , and correlation matrices.

Contrast eeallows testing hypothesis of the form

Several contrasts can be specified and thereby we can run several tests at the same time. A label is needed in single quotes as well as the linear combinations (the rows in L). F-test is default but the Wald test can be run using the chisq option.

0: versus,0:0 LHLH A

Date

Name, department

29

Proc mixed

Estimate permits the estimation of one or several linear combinations of the fixed effects. A label is needed in single quotes as well. Very similar to contrast but output also includes confidence intervals.

Use the option cl alpha = 0.05 when you require an t-type test with = 0.05

Make is used to convert parts of the output into a sas data set. In later versions Make is replaced by ODS.

Date

Name, department

30

Modelling the Covariance Structure Using the

RANDOM and REPEATED Statements in PROC

MIXEDMeasures on different individuals are independent, so covariance needs attention only with measures on the same individuals. The covariance structure refers to variances at individual times and to correlation between measures at different times on the same individual. There are basically two aspects of the correlation.

First, two measures on the same individual are correlated simply because they share common contributions from that individual. This is due to variation between indivduals.

Second, measures on the same individual close in time are often more highly correlated than measures far apart in time. This is covariation within indivduals.

.

Usually, when using PROC MIXED, the variation between indivduals is specified by the RANDOM statement, and covariation within indivduals is specified by the REPEATED statement

Date

Name, department

31

PROC MIXED fits many different structures (some are listed here). Note also that a particular structure may be fit using more than one “TYPE” designation, and with combinations of the RANDOM and REPEATED statements.

Date

Name, department

32

Confusing Proc mixed? The simple answer to why SAS's PROC MIXED can seem so confusing

is that it's so powerful, but there's more to it than that. Early on, many guides to PROC MIXED presented an example of fitting a compound symmetry model to a repeated measures study in which subjects (ID) are randomized to one of many treatments (TREAT) and then measured at multiple time points (PERIOD). The command language to analyze these data can be written as

proc mixed; class id treat period; model y=treat period treat*period; repeat period/sub=id(treat) type=cs;

or proc mixed; class id treat period; model y=treat period treat*period; random id(treat);

Date

Name, department

33

Because both sets of command language produce the correct analysis, this immediately raises confusion over the roles of the repeated and random statements. In order to sort this out, the underlying mathematics must be reviewed. Once the reason for the equivalence is understood, the purposes of the repeated and random statements will be clear, cf. the following:

http://www.jerrydallal.com/LHSP/mixedq.htm

Date

Name, department

34

Summary:

In Proc Mixed, the mixed model is specified by means of a number of statements like CLASS, MODEL, RANDOM and REPEATED.

The CLASS statement identifies the classification variables (for example, gender, person, age, etc.).

The MODEL statement specifies the model’s fixed effects equation, Xiβ. Thus, the design matrix Xi is defined and the model’s intercept is included by default.

The RANDOM statement is used to specify random effects and the form of covariance matrix D. (Useful options: SOLUTION: print random effects solution).

The REPEATED statement models the intra-individual variation and includes the structure of i=Cov(ei), where i is a block diagonal matrix for each subject. (If the REPEATED statement is not included it is

assumed that i=σ2I). LSMEANS Calculates least squares mean estimates of specified

fixed effects.

Date

Name, department

35

The rat data

proc mixed data=rat method=reml;class id group;model y = t group*t / solution;random intercept t / type=un subject=id ;run;

Date

Name, department

36

?

Date

Name, department

37

Date

Name, department

38

Results Using the option nobound

Date

Name, department

39

Non convergence or non positive definiteness can be indications of negative variance components. Usually Proc mixed would not allow that to happen. But using the option nobound in Proc mixed will result in a new set of estimates where d22 is negative. Consider the fitted variance function:

Hence, the negative variance component suggestsa negative curvature in the variance function.

Date

Name, department

40

Date

Name, department

41

Date

Name, department

42

The prostate data

Age could not be matched

Date

Name, department

43

SAS code

Date

Name, department

44

ML and REML estimates:

Date

Name, department

45

ML and REML estimates (cont’d)

Date

Name, department

46

Fitted average profiles

Date

Name, department

47

Date

Name, department

48

In practice, histograms and scatter plots of certain components of the estimate of bi can be used to detect model deviations or subjects with exceptional evolution over time.

Date

Name, department

49

Histograms and scatter plots

Correlations between components of estimate of b

Date

Name, department

50

Date

Name, department

51

Date

Name, department

52

Mixed Models using R vs. SAS:Do they give the same answers? When you perform a mixed model analysis using R, the

question arises are we using the ”right model”?

The syntax for lme() –is not as transparent as its SAS counterpart and its documentation is not as good.

Therefore it is interesting to compare the two approaches.

R and SAS give different answers! One of the reasons is that they apply different restrictions to achieve uniqueness of estimates.

It is possible to ”force” R to get the same answer as PROC MIXED in SAS.

Date

Name, department

53

R commands for linear mixed models Commands for linear mixed models are in the library

nlme.

data <- read.table(file.choose(), header = T)attach(data)Time = factor(Time)Group = factor(Group)Subj = factor(Subj)library(nlme)model <- lme(y ~ Time + Group + Time*Group, random = ~1 | Subj)summary(model)anova(model)

# This model is very close to the one produced by SAS using compound symmetry,# when it comes to F values, and the log likelihood is the same. But the AIC # and BIC are quite different. The StDev for the Random Effects are the same# when squared. The coefficients are different because R uses the first level# as the base, whereas SAS uses the last.

Date

Name, department

54

PROC MIXED: Applied to Clustered Data (SAS/STAT User’s Guide )

In the following example, the response variable height measures the heights (in inches) of 18 individuals who are classified according to their respective family and gender. We will peform two analyses:1. A traditional two-way analysis of variance (ANOVA) of these

unbalanced data (which produces output similar to what you get with PROC GLM) The assumptions concerning the residuals from an analysis of variance include:

1. normally distributed

2. independent

3. have constant variance

Date

Name, department

55

2. A mixed model which includes both random and fixed effects. The random effects, family and family*gender, are now listed only on the RANDOM statement (notice the two terms must not appear on the MODEL statement with PROC MIXED). The type=vc option specifies the variance components model for both family and family*gender. The residual matrix is assumed to equal σ2*I18 where I is an 18x18 identity

matrix. Declaring family as a random effect in this model sets up a common correlation among all heights measured from the same family. The interaction of family*gender as a second random effect also accounts for the correlation between all observations that have the same level of both family and gender. One interpretation is that females will have a higher (or lower) correlation with other females in the same family than males will have with other males in the same family. With the height data, this random effects model seems reasonable.

Date

Name, department

56

PROC MIXED: Applied to Clustered Data (SAS/STAT User’s Guide )DATA heights;

INPUT family obs gender $ height @@;

CARDS;

1 1 F 67 1 2 F 66 1 3 F 64

1 1 M 71 1 2 M 72

2 1 F 63 2 2 F 63 2 3 F 67

2 1 M 69 2 2 M 68 2 3 M 70

3 1 F 63 3 1 M 64

4 1 F 67 4 2 F 66

4 1 M 67 4 2 M 67 4 3 M 69

;

PROC TABULATE NOseps;

CLASS family gender obs;

VAR height;

TABLE family, gender*obs=' '*height='

'*sum=' '*f=6.0

/ rts=10 BOX=' Heights' misstext=' ';

RUN;

Date

Name, department

57

Date

Name, department

58

PROC MIXED DATA=heights NOitprint;

CLASS gender family;

MODEL height = gender family family*gender;

RUN;

Date

Name, department

59

PROC MIXED DATA=heights NOitprint;

CLASS gender family;

MODEL height = gender;

RANDOM family family*gender / subject=family type=vc vcorr ;

RUN;

Date

Name, department

60

Conclusions

In the second analysis we perform a significance test for the fixed effect, gender. Note that its p-value (p=0.0667) is larger than the one observed in the first statistical model that assumed all fixed effects (p=0.0139). The contrast in these two results illustrates the importance of modeling family as a random, rather than a fixed, effect. In fact, if 0.05 is applied as the cutoff point for significance, the fixed effects model shows a significant effect, whereas the model with random effects does not.

An additional benefit of a random effects analysis is that it enables you to make inferences about gender that apply to a population of families, whereas the inferences about gender from the analysis where family and family*gender are treated as fixed effects apply only to the particular families present in the dataset.

This simple example was designed to show you how PROC MIXED lets you model correlation in your data directly and make inferences about fixed effects that apply to entire populations of random effects.

Date

Name, department

61

References

1. 1. Littell, R.C., Milliken, G.A., Stroup, W.W., and Wolfinger, R.D. (1996). SAS System for Mixed Models, SAS Institute, Cary, NC.

2. 2. SAS/STAT User’s Guide: The Mixed Procedure. Chapter 41, Section 6 “Clustered Data Example,”

3. http://www.id.unizh.ch/software/unix/statmath/sas/sasdoc/stat/chap41/sect6.htm

4. http://sas.uoregon.edu/sashtml/stat/chap41/sect6.htm

Date

Name, department

62

Any Questions?