Regression and Programming in R
Anja Bråthen Kristoffersen
Biomedical Research Group
R Reference Card
http://cran.r-project.org/doc/contrib/Short-refcard.pdf
Simple linear regression
• Describes the relationship between two variables x and y
• The numbers α and β are called parameters, and ϵ is the error term.
2014.01.15 3
xy
Example data
• Waiting time between eruptions and the duration of the eruption for the Old Faithful geyser in Yellowstone National Park, Wyoming, USA.
head(faithful)
eruptions waiting
1 3.600 79
2 1.800 54
3 3.333 74
4 2.283 62
5 4.533 85
6 2.883 55
2014.01.15 4
2014.01.15 5
P-value
Estimated
parameters
Coefficient of Determination
Coefficient of determination
• the quotient of the variances of the fitted values and observed values of the dependent variable.
2014.01.15 6
2
2
2ˆ
yy
yyr
i
i
Prediction
• develop a 95% confidence interval of the mean eruption duration for the waiting time of 80 minutes
newdata <- data.frame(waiting=80)
predict(eruption.lm, newdata, interval="confidence") fit lwr upr 1 4.1762 4.1048 4.2476
2014.01.15 7
Residuals
• The difference between the observed data of the dependent variable y and the fitted values ŷ.
2014.01.15 8
yyResidual ˆ
eruption.res <- resid(eruption.lm)
plot(faithful$waiting, eruption.res, ylab="Residuals",
xlab="Waiting Time", main="Old Faithful Eruptions")
abline(0,0, col = "red", lwd = 2)
2014.01.15 9
Standardized residual
• the residual divided by its standard deviation
eruption.stdres = rstandard(eruption.lm)
plot(faithful$waiting, eruption.stdres,
ylab="Standardized Residuals", xlab="Waiting Time",
main="Old Faithful Eruptions")
abline(0, 0, col ="red", lwd = 2)
2014.01.15 10
i
i
i
residual of deviationstandard
residualresidualedStandardiz
2014.01.15 11
Normal Probability Plot of Residuals
qqnorm(eruption.stdres,
ylab="Standardized Residuals",
xlab="Normal Scores",
main="Old Faithful Eruptions")
qqline(eruption.stdres, col = "red", lwd=3)
2014.01.15 12
Generalized additive model
• Can replace the linear relationship between response and variable, as in linear regression, with a non-linear relationship. A spline.
2014.01.15 13
GAM
install.packages("mgcv")
library(mgcv)
This is mgcv 1.5-5 . For overview type
`help("mgcv-package")'.
eruption.gam <- gam(eruptions ~ 1+s(waiting), data=faithful)
plot(eruption.gam)
2014.01.15 14
plot(eruption.gam)
2014.01.15 15
2014.01.15 16
Coefficient of
Determination
p-value
eruption.res.gam <- resid(eruption.gam)
plot(faithful$waiting, eruption.res.gam, ylab="Residuals",
xlab="Waiting Time", main="Old Faithful Eruptions")
abline(0,0, col="red", lwd=2)
2014.01.15 17
eruption.res.gam <- resid(eruption.gam)
qqnorm(eruption.res.gam)
qqline(eruption.res.gam, col="red", lwd=3)
2014.01.15 18
Multiple regresion
• describes a dependent variable y (response) by independent variables x1, x2, ..., xp (p > 1) is expressed by the equation
where the numbers α and βk (k = 1, 2, ..., p) are the parameters, and ϵ is the error term.
2014.01.15 19
k
kkxy
Multiple regresion Explore your data
• Explore your data before starting the analysis
– Are the responses correlated?
– Use plot (trellis graphics, boxplot)
– Use correlation (pearson, spearman)
2014.01.15 20
2014.01.15 21
DLBCL patient data
• Response: Germinal.cnter.B.cell.signature
• Explanatory variables – Lymph.node.signature
– Proliferation.signature
– BMP6
– MHC.class.II.signature
2014.01.15 21
pairs(dat[,8:11])
2014.01.15 22
cor(dat[,8:11])
2014.01.15 23
2014.01.15 24
Adjusted Coefficient of Determination
p-vaules
Not significant
p values
Adjusted Coefficient of Determination
2014.01.15 25
1
1)1(1
22
pn
nRR
adj
Hvor n er antall observasjoner og
p er antall parametere brukt i modellen
Comparing models
• As long as analysis are done on the same responses you can compare your models by information criteria:
• AIC (Akaike's An Information Criterion)
-2*log-likelihood + 2*npar
• BIC (Schwarz's Bayesian criterion)
-2*log-likelihood + log(n)*npar
• Goal: as small AIC or BIC as possible, i.e. explain most with less parameters
2014.01.15 26
27
The AIC value is less, chose this model
All p values are significant
Adjusted coefficient of
determination hardly
changed
fit2.res <- resid(fit2) fit2.fitted <- fitted(fit2) pairs(fit2.fitted, fit2.res, col = "darkgreen", pch="*")
2014.01.15 28
Logistic regression
• We use the logistic regression equation to predict the probability of a dependent variable that takes on only the values 0 and 1. Suppose x1, x2, ..., xp (p > 1) are the independent variables, α, βk (k = 1, 2, ..., p) are the parameters, and E(y) is the expected value of the dependent variable y
2014.01.15 29
k
kkx
e
yE
1
1)(
DLBCL patient data
• Response: alive or dead
• Explanatory variables – Subgroup
– IPI.Group
– Germinal.center.B.cell.signature
– Lymph.node.signature
– Proliferation.signature
– BMP6
– MHC.class.II.signature
2014.01.15 30
DLBCL.glm <- glm(Status.at.follow.up ~ Subgroup + IPI.Group + Germinal.center.B.cell.signature + Lymph.node.signature + Proliferation.signature + BMP6 + MHC.class.II.signature, family= "binomial", data = dat)
2014.01.15 31
Here
logistic
regression
is chosen
DLBCL2.glm <- glm(Status.at.follow.up ~ IPI.Group, family= "binomial", data = dat) summary(DLBCL2.glm) High Low Medium missing NA's 32 82 108 1 17
2014.01.15 32
The survival of
those containing
the IPI.group Low
is significantly
different from
those that are in
the group High
The estimate
is negative,
hence pations
in group Low
have less
probability to
die then those
in group High
Backward an forward inclusion of response variables
• Forward
– Start with all response variables, exclude the one that is least significant, compare AIC values.
• Backward
– Start with one regression for each response variable
– Include more and more response variables that are significant in the model, compare AIC values
2014.01.15 33
Programming in R
2014.01.15 34
Function
panel.hist <- function(x){
usrx <- par("usr")
on.exit(par(usrx))
par(usr = c(usrx[1:2], 0, 1.5) ) # indicates the position in the plot
hi <- hist(x, plot = FALSE) # calculate histogram without plotting it
Breaks <- hi$breaks # define breaks used in h
nB <- length(Breaks) # count the number of breaks
y <- hi$counts # find the counts in each interval
y <- y/max(y) # scale y for plotting
rect(Breaks[-nB], 0, Breaks[-1], y) #plots rectangulars in existing plot
}
pairs(dat[,7:11], col = dat[,5], cex = 0.5, pch = 24, diag.panel = panel.hist,
upper.panel=NULL)
2014.01.15 36
2014.01.15 37
2014.01.15 38
apply()
2014.01.15 39
• Use apply when you want the same thing done on every row or column of a matrix.
• apply(d, 1, functionA) will use functionA on every row of dataset d.
• apply(d, 2, functionA) will use functionA on every column of dataset d.
Example: apply()
2014.01.15 40
d <- matrix(runif(90), ncol = 10, nrow = 9)
head(d)
par(mfrow = c(3,3))
apply(d, 1, hist)
for – loop
• for (name in expr_1) expr_2
– name on variable: i
– expr_1 should be 1:antall
– expr_2 should be
sum(dbinom(1:6, 8, p[i])), but you have to save it somewhere:
beta8[i] <- sum(dbinom(1:6, 8, p[i])) then you have to create beta8 before the loop.
for – loop
p <- seq(0.6, 1, 0.01)
antall <- length(p)
beta8 <- rep(NA, antall) #allocate space for beta8
for(i in 1:antall){
beta8[i] <- sum(dbinom(1:6, 8, p[i]))
}
power8 <- 1 - beta8
plot(p, power8, type = "l")
2014.01.15 43
if statment
• if (expr_1) expr_2 else expr_3 • expr_1 is a statement that is either true or
false • expr_2 is preformed if expr_1 is true • expr_3 is optional, but preformed if expr_1 is
false and else statement is included
Example: if()
2014.01.15 45
x <- runif(1,0,1)
y <- rnorm(1,0,1)
if(x > y){
print(paste("x =", round(x,3), "is greater
than y =", round(y,3), sep = " "))
} else {
print(paste("x =", round(x,3), "is less than
y =", round(y,3), sep = " "))
}
[1] "x = 0.783 is greater than y 0.536"
When finished
• You could save your workspace in R, you could then save the workspace in different project folders. I do not do that!
• You could just save your script and run it once again if you want to work further on it.
• You could write your partial results (matrix) that you want to work further on to a .txt file, use write.table(objectname, ”path”, sep = ”\t”)
– objectname is the name of your object in R
– ”path” is your path to where you will save the object including the file name of the object.
– sep = ”\t” ensure that your txt file is tabulate separated and makes it easier to open in excel.