39
Predicting the quality of a survey question from its design characteristics Daniel Oberski Dept. of Methodology & Statistics, Tilburg University, The Netherlands UN IVERSI T A T PO M P E U FABR A Predicting the quality of a survey question from its design characteristics Daniel Oberski

Predicting the quality of a survey question from its design characteristics

Embed Size (px)

Citation preview

Predicting the quality of a survey questionfrom its design characteristics

Daniel OberskiDept. of Methodology & Statistics,Tilburg University, The Netherlands

U N I V E R S I T A T

P O M P E U F A B R A

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Question #1: Where is Tilburg?

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Summary

• We wanted to predict how “good” a survey question isbased on its “design characteristics”, i.e. choices such as #categories, DK present/not, use of showcards, etc.(Willem’s talk 1 & 2)

→ Trained a random forest of regression trees to predictthe ESS multitrait-multimethod estimates of reliabilityand method effect (Cf. Melanie’s talk) from these questioncharacteristics, as coded by teams of human coders.

• Prediction powers a webapp allowing users to codequestion & get predictions ± uncertainty (Diana’s talk).

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Simple example

Predicting the quality of a survey question from its design characteristics Daniel Oberski

European Social Survey, 2002

Method A:

3

ENTER START TIME: A1 TvTot CARD 1 On an average weekday, how much time, in total, do you spend watching television? Please use this card to answer.

No time at all

Less than ½ hour

½ hour to 1 hour

More than 1 hour, up to1½ hours

More than 1½ hours, up to 2 hours

More than 2 hours, up to 2½ hours

More than 2½ hours, up to 3 hours

More than 3 hours

(Don’t know)

A2 TvPol STILL CARD 1 And again on an average weekday, how much of your time watching television is spent watching news or programmes about politics and current affairs1? Still use this card. No time at all 00

Less than ½ hour 01

½ hour to 1 hour 02

More than 1 hour, up to 1½ hours 03

More than 1½ hours, up to 2 hours 04

More than 2 hours, up to 2½ hours 05

More than 2½ hours, up to 3 hours 06

More than 3 hours 07

(Don’t know) 88

1 About “politics and current affairs”: about issues to do with governance and public policy, and with the people connected with these affairs.

00 GO TO A3

01

02

03

04 ASK A2

05

06

07

88

Method B:!!""#$%&'()*%)+&#!)&,%$#!-&.# !"#$"#$%&'$(&#)&&*+$,-#./)#012.#340&-#4"#3/3$5-#+/#,/1#67&"+#)$32.4"(#

3&5&%464/"89:##

#

# # # ,$/+%#/)#;!<=>0#### ###?@A#BC@<DE>0# # # #

# # # #

-&1# #!"#$"#$%&'$(&#)&&*+$,-#./)#012.#340&-#4"#3/3$5-#+/#,/1#67&"+#5463&"4"(#3/#

3.&#'$+4/8F:##

#

# # # ,$/+%#/)#;!<=>G## ?@A#BC@<DE>G# # # #

# # # # # #

#

#

-&2# !"#$"#$%&'$(&#)&&*+$,-#./)#012.#340&-#4"#3/3$5-#+/#,/1#67&"+#'&$+4"(#3.&#

"&)67$7&'688:##

#

# # # ,$/+%#/)#;!<=>G# #?@A#BC@<DE>G# #

#

#

#

'34564#7897:5;4#;<#=>5;#4?;48;#@<A#5BC44#<C#9765BC44#=7;>#45:>#<D#;>4#D<33<=78B#6;5;4E48;6F#

# # #

-&G# H>/0&340&6#7/543426#6&&06#6/#2/07542$3&+#3.$3#C#2$"H3#'&$55,#1"+&'63$"+#).$3#

46#(/4"(#/"8IJK#

# '34564#;7:I#<84#J<?F#! A46$('&&#63'/"(5,# F#

# A46$('&&# 8#

# @&43.&'#+46$('&&#"/'#$('&&# I#

# ?('&&# L#

# ?('&&#63'/"(5,# M#

#

-&K# NC#3.4"*#C#2/15+#3$*&#$"#$234%&#'/5&#4"#$#('/17#4"%/5%&+#)43.#7/54342$5#4661&68LJK#

# '34564#;7:I#<84#J<?F# #

! A46$('&&#63'/"(5,# F#

# A46$('&&# 8#

# @&43.&'#+46$('&&#"/'#$('&&# I#

# ?('&&# L#

# ?('&&#63'/"(5,# M#

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"#!$%%!&'!()!*+()!,-%./(0))+(1%!"'!$%%!&2!""!$%%!&3!"2!$%%!4"!

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Design characteristics for method A

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Design characteristics for method B

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Reliability and common method variance estimatesfrom an MTMM experiment

For the TV watching example:

Reliability Methodvariance

Method A: 8 categories 0.796(0.01)

0.098(0.01)

Method B: write (after outlier del’n) 0.819(0.02)

0.140(0.01)

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Goal of this research:

obtain a rough estimate of the reliability and method effects ofthese kinds of questions without needing amultitrait-multimethod experiment.

Predicting the quality of a survey question from its design characteristics Daniel Oberski

ESS multitrait-multimethod experiments

• In the European Social Survey (ESS), every round aboutsix MTMM experiments are done;

• Each experiment usually estimates the quality for 9questions (Method-Trait combinations).

• Range of topics is reasonably diverse, though factualquestions are underrepresented.

• In total about 1051 method-trait combinations available,3483 “questions” when the analysis is done separately percountry

Predicting the quality of a survey question from its design characteristics Daniel Oberski

“Old” multitrait-multimethod experiments

• In addition to the ESS, an older series of experiments alsoexists (Andrews; Koltringer; Saris; Scherpenzeel, 1990’s)

• These add another 1089 questions for which reliability andcommon method coefficients are estimated

• Combining the two datasets (ESS question qualities andOld experiment qualities, we created a database of 3483questions with their reliability and common methodestimates (1051 unique method-trait combinations).

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Reliability and common method variance estimates of3483 questions (1051 unique method-trait comb’s)

Reliability coefficient

Reliability coefficient

0.4 0.6 0.8 1.0

0

200

400

600

800

Common method variance

Common method variance

0.0 0.2 0.4 0.6 0.8

0

500

1000

1500

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Design characteristics of questions

• Social Desirability• Centrality• Reference period• Question

formulation• WH word used• Use of gradation• Balance of the

request• Encouragement• Showcards

present• Showcards have

pictures• ...

• Emphasis on subjectiveopinion in request

• Information about theopinion of other people

• Use of stimulus orstatement in the question

• Absolute or comparativejudgment

• Response scale: basicchoice

• Number of categories• Labels full, partial, or no• Labels full sentences• Knowledge provided• Survey mode• ...

• Order of the labels• Correspondence between

labels and numbers of thescale

• Theoretical range of thescale

• Neutral category• Number of fixed reference

points• Don’t know option• Interviewer instruction• Respondent instruction• Extra motivation, info or

definition available?• Agree-disagree scale• . . .

(Saris & Gallhofer 2007)

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Coding design characteristics of the 3483 questions

• For each of the 3483 questions in all countries, a team ofcoders coded 40 design characteristics of the question;

• Some codes were automatically generated by NaturalLanguage Processing software (syllables, words, etc).

• Coders were students, assistants to the local coordinatorsof the ESS, and two experts;

• For English source version, experts double-codedquestions independently, then created consensus codes;

• Non-expert codes were quality-controlled by detailedcomparison with consensus codes for the English source;

• In a meeting between the experts and each other coder,the discrepancies were discussed and either corrected orleft in as true differences.

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Domain of question # questions # unique comb’sInternatl politics 64 64Health 244 88Living conditions 433 234Other beliefs 292 292Work 509 119Family 109 7Personal relations 308 62Consumer behavior 26 26Leisure activts 131 63National gvt 169 31Institutions 412 35Political parties 30 30Trade unions 12 12Economy 266 16Other 370 89

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Concept of question # questions # unique comb’s

Evaluative belief 703 136Feeling 1044 161Importance 96 96Expectation 39 30Facts, behavior 111 60Complex concept 44 44Other simple 1338 608

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Meta-analysis dataset

• For each of the 3483 questions, we have in the database:• The estimated quality (reliability and common method

coefficients)• About 50 design characteristics (through hand- and

automatic coding)

• Predict reliability and method effect estimates from designcharacteristics

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Meta-analysis using random forests of regression trees

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Example of one (1) regression tree (CART).

Predicting the quality of a survey question from its design characteristics Daniel Oberski

|

domain=3,4,7,11,13,14,112

domain=3

gradation>=0.5 position< 339.5

position>=410

concept=1,2 position< 404.5

concept=1,73,78

position< 322.5

ncategories>=4.5

domain=6,101,103,120

domain=4,7,11,13,14,112

gradation< 0.5 position>=339.5

position< 410

concept=73,75,76 position>=404.5

concept=2,76

position>=322.5

ncategories< 4.5

1.955n=1988

1.724n=1303

0.9636n=108

0.4959n=36

1.198n=72

1.793n=1195

1.642n=722

2.023n=473

1.544n=108

1.28n=76

2.17n=32

2.165n=365

1.97n=217

2.45n=148

2.394n=685

1.489n=138

2.622n=547

2.384n=233

2.799n=314

2.681n=260

3.364n=54

Example regression tree for reliability coefficient

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Obtaining a prediction from a random forest

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Training the forest of regression trees

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Random forest algorithm (Breiman 2001)

1 Randomly sample (w/ replacement) cases (questions)→ some “in-bag” cases;→ Rest is “out-of-bag”.

2 Randomly sample features (characteristics);3 Grow a regression tree on the in-bag cases using the

subset of selected features;4 Calculate mean square prediction error on the out-of-bag

cases→ built-in cross-validation;5 Back to (1) until 1500 trees have been grown.

Predicting the quality of a survey question from its design characteristics Daniel Oberski

joined.data

Ordered by number of missing items

avgabs_intronabst_intro

nnouns_introshowc_boxesshowc_questshowc_startopinionothershowc_horizshowc_lettershowc_over

positionncategories

range.correspondencescal_neutral

absoluteconceptdomain

dont_knowform_basic

futureinstr_interv

instr_responmotivation

pastscale_basic

stimulususedshowcard

interviewervisual

fixrefpointslabels

scale_corresscale_trange

labels_grammcomputer.assisted

intr_requestavgabs_total

avgsy_totalavgwrd_introavgwrd_total

balancecentrality

countryencourage

gradationintropresentknowledge

labels_orderlanguage

nabst_totalnnouns_total

nsub_questnsyll_total

numsub_intronwords_intronwords_totalquestiontype

rel.estrel.est.orig

repetitionscale_urange

socdesirstudy

subjectiveopsymmetry

used_WH_wordval.est

val.est.origfrom

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Missing data• Imputed using multiple imputation with chained equations

(MICE) (van Buuren & Groothuis-Oudshoorn 2011).• A small random forest (bush?) is grown for each imputed

dataset• These bushes are joined to form the final forest• “raimforest” (RAndom IMputation forest)

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Out-of-bag crossvalidation

• R2 for logit(reliability coef.) prediction: 0.64• R2 for logit(validity coef.) prediction: 0.84

Predicting the quality of a survey question from its design characteristics Daniel Oberski

●●

●●●●

●●●

●●

●● ●

●●

●●

● ●●●

●●

●●

●●●

●● ●

●●●

● ●

●●

●●

●●

●●

●●

● ●●●

●●

●●●

●●●

●●

●●

● ●

●●

● ●

● ●●

●●

●●

● ●

●●

●●

●●

●●

● ●●

●●

●●

●●

●● ●

●●

● ●● ●

●●

●●

●●●

●● ●

●●

●●

●●

●●

●●

●●

●●

●●● ●●

●●

●●

●●

●●

● ●

●●

●●

●●●

●●

●●

● ●

● ●

● ●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

● ●

●●

●●

●●●

● ●

●●

●●

●●●

●●●

● ●

●●●

●●

●●

●●

●●

●●●

●●

●●

●●

●●

●●

● ●●

●●

●●●

●●

●●

●●

●●● ●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●● ●

● ●

● ●

●●

●●

●●

●●

●● ●

●●

● ●

●●

●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●●

●●

●●

●●

●●

●●

●●

●●

●●

●●●

● ●

●●

●●

●●

●●

●●

●●

●●

●●

●●

● ●

●●

●●

●●

●● ●

●●●

●● ●●

●●

●●

●●

●●

●●●

●●●

●●

●●●

●●

●●

●●

●●●

●●

●●●●

●●

●●

●●

●●

●●

●●

●●

●● ●

●●●●

●●

●●

● ●

●●●

●●

●●

●●

●●●

●●

●●

● ●

●● ●

Quality prediction

Quality predicted from random forests

MT

MM

est

imat

e of

qua

lity

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Fitted versus residuals for random forest predictions

Reliability

!

!

!

!

!

!!!!!

!

!!

!

!

!

!!!

!!

!

!!

!

!!!

!

!!!

!

!

!!

!!!

!

!!

!!!

!!

! !

!! !

! !! !

!!

!!!!

!!!! !!!!!

!! !!

!!!!!! ! !! !!!!!! ! !

!

!!!!!!

!

!

! !!

! !!! !! !!!

!!!!!

!

!!!!

!

!!!!

!

!! !!!!!

!! !!!

!!! !! !!!!!!!!

!!

!!!

!

!! !

!

!!!

!

! !!!

!

!

!!!

!

! !!

!!!!

!!!!!!

!!

!

!!

!! !!

!!!

!!

!! ! !

!

! !

!

!

!!

!!!!

!

!!!!

!!!!

!

!!

!!!!! !!!!!

!

!!!!

!!

!

!!

!! !! !!! !!!

!!

! !!

!

!

!

!

!!

!!

!

!

!

!!

!

!

!!

!

!

!!

!

!

!!!

!!

! !!

!

!!

!

!

!!!

!

!

!!!

!

!!

!

!

!!

!!

!!

!

!!! !

!!! !

! !

!!

!!

!!

!

!

!! !!!

!!!

!

!!

!

!!!

!

!!

!

!!!

!

!

!

!

!!

!!!!

!

!!

!!!

! !!!

!! !

!

!!!

!

!!

!

! !!!!

! !!!

!!

!!

!

! !!!

!! !!!

!

!!

!

!!! !

!

!!

!!

!

!!!

!!

!

!

!

!!!!

!!

!!!

!!! !

!!

!

!

!!!

! !

!!!!

!

!! !!

!

!!!!

!

!!!

!

!

!

!

!!!

!

!!!

!

!!

!

!

!!

! ! !!!!! !!!

!!!!

!

!! !!! !! !

!!

!!

!

!

! !!

!

!!

!

!!!

!

!

!

!!

!

!

!! !

!

!!

!

!

!! !! ! !! !! !

!!!!! !

!

!!! !

!!!!!

!

!!!

!!!

!

!!

!!

! !

!!! !!

! !! !!!!

!!

!!! !! !!

!

!!!! !!

!!

!! !!

!

!

!!! !!!!

!!

!! !

!

!

! !!!

!!!!!!

!! !!!!

!

!

!!!

!

!!!

!

!

! !!!

!

!!!!!

!!!!!

!

!!

!! !!

!!!!!!

!!!!

! !!

!

!!

!!!

!!

!!!

!!!!

!

!!

!! !!

!

!!!

!!! !! !

!!!

!

!!

!

!!

!

!!

!!

!!

!

!!

!!!! !

!

!!

!!

!

!!

!

!

!!!!

!

!!! !!!!

!

! !!

!

!!

!

!!

! !!

!

!

!

!

!

!

!

!

!!

!

!

!!!

!

!

!!

!

! !

! !

!

!

!!

!

!!

!

!

!

!

!

!

!!

!

!!!!

!

!!

!

!! !!!

!

!!!!!

!

!!

!

!!

!

!

!!!!

!

!!!

!!

!

!

!

!

!!

!

!!!!

!!!!!!

! !!

!

!!

!!

!!

!

!

!

!

!!!!

!

!

!

!!! ! !!

!!!

!!!!!!!

!

!

!

!!!

!

! !!

!!

!!

!

! ! !!!

!

!

!!

!

!

!!

!!!

!

!

!

!

!!!

!

!

!

!

!!!

!!

!

!!

!

! !

! !

!

!

!!

!

!!!!

!!

!!

!

!!!!!!

! !

!!!

!!! !

!!

!!!!!

!

!!!

!

!!

!

! !!

!

!

!

! !!

!

!

! !!!! !!!

!

!!

!! !

!! !!!! !

!!

!

!!!!

!

!!

!

!! !!

! !!!!

!

! !!!

!! !

!

!

!

!!!!! !!

!

! !!!

!! ! !!

! !!

!!!! !

!! !!! !!! !

!!

! !!!

!!!!! !!

!! !! !!!! !!!!! !

!!!!!

!!

! !!!!

!

!!!

!

!!

! !!!

!

!! !!!

! !!!

!!!!!

!!!!! !!!!

!! !!!

!!

!

!

!!!!! !!!!!!

!

!! !

!

!! !!

!!!

!

!

!

!

!!!!!!

!

!!!!!!!

! !! !! !

! !!!!

!!!

!!!!

!

!

!!! !!

!!!

!

!!!!!!! ! !!!

!!

! !!!

!!!!!!!!! !!

!!!!!

!!!! !!!!!!!!!

!

!!!!!!! !!

!!!!!!!!!! !! !

!!

! !!

!

!!! !!!!!

! !! !! !

! !!

!!!

!!

!

!!!

!!!

!

!!

!!!! !! !!

!!

!

!! !! !!!!

!!! !!

!!

!

!!

!

!!!

!!

!!

!!!

!

!!

!

!!

!!

!!

!

!!!!

!

!!!!!

!

!

!!!! !

!!!! !

! !!

!

!! !! !!!

!

!!

!!!!! !

!

!!!

!!!! !!!!

!!!!

!! !!!

! !!!

!!!! !

!!!!

!

!

!

!!

!!

! !!!!!!!!!!!!

!!

!!

!!!

!!

!!

!!

!!!!

!!

!!!

!!

!

!!!

!!

!

!! !!!

!!!

!!!!!!! !

!

!!!!

!! !! !!

!!!!! !!

!!

!!! !!

!

! !!!!

!

!!!

!!!!

!

! !!

!

!!!!

!

!

!!!!

!! !!!!! !! !

!!

! !!!!

!!

!!

!

! !!

!

!

!!!!!!!

!!

! !!

!!

!!

!

!! !!

!

! !! !

!!!

! !! !!

!

!! !!

!! !! !

!

!!

!!!!!

!! !!!!! !!!

!!!!

!

!

!! !!!!!!

!!! !!! !!! !

!! !!!

!!!

!

!!!!

!!! !

!!!!!!

!!!!!!!

!!!

!!!

!!!!!

! !!

!!!!!

! !!

!!

!!

!!!!

!!!!!!!

!!

! !

!

!!!!!

!!!

!!!!!!

!

!!!!

!!

!!! ! !! !!!

!!

!

!

!

!!!

!

!! !

!

!! !

!

! !!

!!

!!

!

! !!

! !!!

!

!

!!!!!! !!!!! !!!!

!!!!!

!! !!!!!!!!!!!!!!!!

!! !! !!!! !!!

!!!!

!!!!

!!! !!!

!!

!!!!

!

!!!!

!!

!

!

!

!!

!

! !!! !!

!

! !! !!

!!!!

!!!

!!!!

!! !!!!!!!!

!!!

!!

!!!!

!!!!!!

!!

!!

!

!

!!

!

!!!

!

!

!

!

!!!

!

!!

!!

!

!!!

!!!

!

!!!!!!

!! !!!

!! !

!

!! !

!!! ! !!

!

! ! !!!! !!

!

! !!!! ! !! !

!!!

!!! !!!! !! !!

!

!! !! !!!

!!!! !

!

!

!

!!

!

!!

!

!!!!! !

!

!

!

!! !!!! !

!!

!

!!

!!!

!

!

!

! ! !!

!!!!! !!!

!!!! !

!

!!!

!!

!!

!!!!

!!! !

!

!

!

!

! !!

!!!!! !!!

! !

!

!

!!

!

!!

!!

!!

!!!

!

!

!!!

!!!

! !!! ! !

!!!

! !!

!

!!

!!

!!!

!! !

!!

!!! !!! !!

!!!

!

!

!

!

!!!

!!!

!

!!

!

!

! !!

!

!!!! !

!!

!!

! !!

!

!

! ! !

!!

!

!!

!!!

!!

!!!

!!!

!!

!!

!!

!!!!

!

!

!!!

! !!!!

!

!!!!

!

!

!!

!! !

!

!

!

!!

!! !

!

!!!!

!!

!!

!!! !!!!!! ! !! !!

!!

!!

!!

!! !!!

!!!!! !

!! !

!

!!

!!

!!

!

! !! !!!

!!!

!

!!! !!! !!! !

!

!!!

!!!!

!!

!!!

!!

!

!

!!

!

!!!

!

!

!

!!!!

!

!!

!

! !

!!!

!!

!

!

!!!!

!!

!

!

!!!!! !!

!!

!

!!

!

!

!

!

! !!!!

!

!!

!

!!

!

!

!

!

!!

!

!

!

!

!!

! !!!!

!!

!! !

!

!!

!! !

!! !!!

!

!

!

!!!

!!!

!!

!

!

!!

!

!!

! !

! !

!!

!

!!

!!!!

!!

!

!

!!

!

!! !!

!

!!!!

!!

!!!

!!!

!!! !!!

!

!

!!

!! ! !

!!

!

!

!

!! !

!! !

!

!! !

!

!!

!!

!!! !

!!

!!

!!!!

!

!!

!

!

!!

!

!

!!

!!

!

!

!! !!!

!!!

!

!

!

!

!!

!

!

!

!

!

! !

!

! !!

!

!

!!

!

!!

!! !

!

!!

!

!

!

!

!!

!!

!!!

!!!

!!

!!

!!

!

!

!!

!!

!

!

!!

!

!

!

! !!!!!

!!!!

!!

!

!!!! ! !! !!

!!

!!!

! !

!

!!

!!

!

!

!!

! !!!!

!!

!!!

!! !!!!

!!!

!

!!!!

!!

!!

!

!!

!

! !!!

!

!!!

!!

!!!

!

!!

!

!

!

!! !!

!!!

!!

!

!!

!! ! ! !! ! !

!!! !!

! ! !!

!!

!

!!

!!!

!!!

!!

!

!

!

!! !!!

!!! !

!!!! !

!

!

!

!

!!!

!!!!!!!!

!! !!

!!! !!

! !!

!! !

! !!

!!

!!!

!!!!!

!!!

!!! !

!!!!!

!!!!

!! !!

!!!!

!!!! !!!! !!

!!

!!

!!!

!!! !!

!!!! !

!!! !

!!

!!! !

!!

!!

!

!!!!

!!!!!

!!! !!!

!

!

! !!!

!!!

!!

! !!!

!!

!!

!!

!!

!

1 2 3 4

−4−2

02

predict(rf)

resi

d(rf)

!

!

!

!

!

!!!!!

!

!!

!

!

!

!!

!

!!

!

! !

!

!!! !

!!

!

!

!

!!

!! !!

!!

!!!!

!!!

!! !

! !! !

!

!

!!!!

!!

!! !!!!!

!!

!!

!!!

!!!! !! !! !!! !!

!!!!!!!!!

!

!!

!!

!!! !! !!!

!!!!! !!

! !!!

!!

!!

!

!!!!

!!! !!!

!!!!!

!!!!!!!!

!! !!! !

!

!

!! !

!!!!

!

! !!!

!

!

!!!

!

!!

!!!!!!! !!!

!!!

!!

!!! !

!!!!

!

!

!! ! !

!

! !

!

!

!!

! !!!

!

!!

!!

!!!!

!

!!!

!!!! !!!!!

!

!

!!! !!

!

!!

!! !!!!!

!

!

!!

!! !!

!

!

!

!

!!

!!

!

!

!

!!

!

!

!!

!

!

!!

!

!

!!

!!!

!

!!

!

!!

!

!

!!!

!

!

! !!

!!

!!

!

!!

!!!!

!

!!! ! !!! !! !

!!!

!! !

!

!

!! !!!! !!

!

!!

!

!!

!

!

!!

!!

!! !!

!

!

!!

!!!!

!

!!

!!!

! !!

!!

! !

!

!!!

!

!!

!

! !! !!!

!!!

!!!!

!

! !!!

!!!!

!

!

!!!

! !!!

!

!!

!!

!

!!!

!!

!

!

!

!!!

!

!!

!

!!!

!! !

!

!

!

!

!!!

! !

!!!

!!

!! !!

!

!

!!!

!

!!!

!

!

!

!!

!!

!

!!!!

!!

!

!

!!

!!

!!

!!! !!

!!

!!!

!

!! !! ! !!!!!

!!!

!

!!!

!

!!

!

!!!

!

!

!

!

!!

!

!! !!

! !

!

!

!! !!

! !! !! !

!!!!!

!

!

! !!!!

!!! !

!

!!!

!!

!

!

!!

!!

! !!!

! !!

! !! !!!

!!

!

!!

! !! !!

!

!!!! !!

!!

! ! !!!

!

! !! !!!!

!!

!

! !!

!

! !!!

!!!!!! !! !!!!

!

!

!!!

!

!!!

!

!

! !!!

!

!!!!!

!!! !!

!

!!

!! !!

!! !!!!

!!!!

! !!

!

!!

!!!

!!

!!!

!!!!

!

!!

!! !!

!

!!!

!!!!

! !!!

!!

!!

!

!!

!

!!

!!

!!

!

!!

!!!! !

!

!!

!!

!!

!

!

!

!!!!

!

!! ! !! !!

!

! !!

!

!!

!

!!

! !!

!

!

!

!

!

!

!

!

!

!

!

!!!!

!

!

!!

!

! !

!!

!

!

!!

!

!!!

!

!

!

!

!

!!

!

!!!!

!!

!

!

!! ! !!

!

!!! !!

!

!!

!

! !!

!

!!!!

!

!!!

!!

!

!

!

!

!!

!

!!!!

!!!! !!

!!!

!

!!

!!

! !

!

!

!

!

!

!!!

!

!

!

!!! !!

!

!

!!

!!!!!

!!

!

!

!

!!!

!

! !!!

!!

!

!

! !!!!

!

!

!!

!

!

!!

!!!

!

!!

!

!

!!

!

!

!

!

!!!

!!

!

!!

!

!!

! !!

!

!!

!

!!

!!

!!

!!

!

!!!! !!

! !

!!!

!!! !

!!

!!! !!

!

!

!!!

!!

!

! ! !!

!

!

! !!!

!

! !!!! !!!

!!

!!

! !

!! !!!! !

!!

!

!!!!

!

!!

!

!!! !! !

!!!

!

! !!

!!! !

!

!

!

!!!

!! !!

!

! !!

!!! ! !!! !

!!!

!! !!! !!! !!

! !!!! !

!!!!!!! !!

!!!! !!!! !!! !! !

!!!!

!!!! !

! !!

!

!!!

!

!!! !!!

!

!! !!!

! !!!

!!!

!!

!!!!! !!!!

!!!!

!!

!!

!

!!!!

! !!!!!!

!

!! !

!

!! !!

! !!

!

!

!

!

!!!!!

!!

!!!!!!!

! !! !! !

!!!!

!!

!!!!!

! !

!

!!! !

!!!!

!

!!!!!! ! ! !!!!!

! !!!

!!!

!!!! !! !!!!!!!!!!! !!!!!!! !!

!!!!!!!

! !!!

!! !!!!!!! !

! !!

!!

!!

!

!!! !!!!!! !! !

!!! !!

!!!

!!

!

!!!!!

!

!

!!!

!!! !!!!

!!

!

!! !! !!!!

!!!!!

!!

!!

! !

!!!

!!

!!

!!!

!

!!

!

!!

!!

!!

!

!!!!

!

!!!!!

!

!!

!!! !

!

!!! !

! !!

!

!! !! !!!

!

!!

!!!!

!!

!

!!!

!!!!!

! !!!!!!!

!!!

!! !!!

!!!! !

!!!!

!

!

!

!!

!!

! !!!!!

!!!!

! !!

!!!!

!!!

!

!

!!

!!

!!!!

!!

!!!

!!

!

!!!

!!

!

!! !!!

!!!

!!!!!!

! !

!

!!!!!

! !!!!

!!!!!!!

!!!

!! !!

!

!!!!!

!

!!!

!!! !

!

! !!

!

!!!!!

!

!!!!

!! !!!! ! !! !!!

! !!!

!!

!

!!

!

! !!

!

!

! !

!!!!!

!!

! !!

!!

!!

!

!! !!

!

! !! !

! !!

! !!

!!

!

!! !!

!

! !! !!

!!! !!!

!!!!!!!! !

!!!!!! !

!

!!!!!!!!

!

!! !!!!

!! !!!

!!!!!

!

!

!!!!

!!! !

!!! !!!

!!! !!!!!!

!! !

!!! !!!

! !!

!!!!!

! !!

!!

!!

!!!!

!!!!!!!

!!

! !

!

!!!!!!

!!

!!!

! !!

!

!!!!

!!

!!! ! !! !!!

!!

!

!

!

!!!

!

!! !!

!! !

!

!!!

!!

!!

!

! !!

! !!!

!

!

!! !! !!

!!!

!! !!!!!!!

!!!

! !!!!!!!!!!

!!!

!!!

!! !!!!!! !!!!!!!

!!!!

! !!!

!!

!!

!!!

!

!

!!!

!!

!

!

!

!

!!

!

! !!! !!

!

! !! !!

!!!!!!

!!!!!

!!!!!!!!!!!

!!

!!

!!!!

!!!!!!

!!

!!

!

!

!!

!

!!!

!

!

!

!

! !!

!

!!

!!

!

!!!

!

! !

!

!!

!!!!!! !!!

!! !

!

!!!!

! ! ! !!!

!! !!!! !!

!

! !!

!! !!! !

!!!

!! !!!!!

!!!!

!

!! !! !

!!!!!

!!

!

!

!

!!

!

!!

!!!!!! !

!!

!

!! !!!

! !

!

!

!

!!

!!!

!

!

!

! ! !!

!!!!! ! !!

!!!! !

!

!!!

!!

!!

!!

!!

!!! !

!

!!

!

! !!

!

!!!! !!! ! !

!

!

!

!

!

!!

!

!!

!

!!!

!

!

!!!

!!!

! !!! ! !

!!!

! !!

!

!

!!

!!!

!

!! !

!

!!!! !!

! !!!

!!!

!

!

!

!!!

!

!!

!

!! !

!

!!

!!

!!!! !

!!

!!! !

!!

!

! !!

!

!

!

!!!

!!!!

!!!

!

!

!

!!

!! !

!!!!!

!

!

!!!

! !!!!

!

!!!!

!

!

!!

!! !

!

!

!

!!

!!!

!

!!!!

!!

!

!

!!!

!!! !!! !!! !!

!!

!!

!!

!! !!

!

!!!!! !

!! !!

!!

!!

!!

!

! !! !!!

! !!!

!!! !!! !!

! !!

!!!

!!!!

!!!!

!!!

!

!

!!

!

!!!

!

!

!

!!!

!

!

!

!!

! !

!!!

!!

!

!

!!!!

!!!

!

!!!!! !!

!!!

! ! !

!

!!

! !! !!

!

!!

!

!!

!

!

!

!

!!

!

!

!

!

! !! !

!!!

!!

!!

!

!

! !

!! !!! !!

!

!

!

!

!!!

!!!!!

!

!

!!

!

!!

! !

!!

!! !

!

!

!

!!!

!!!

!

!!

!

!! !

!!

! !!

!!!

!!!!!

!

!!!

!!!

!

!

!!

!! !!

!!

!

!

!

!! !!! !

!

!!!

!

!!

!!

!!!

!

!!

!!

!!! !

!

!!

!!

!!

!

!

!!

!!

!

!

!! ! !!!!

!!

!

!

!

!!

!

!

!

!

!

! !

!

! !!

!

!

!!

!

!!

!

!!

!

!!

!

!

!

!

!! !!

!!

!

!! !!

!!!

!!

!

!

!!

!!

!

!!!

!

!

!

! !!!!

!!!!!

!!

!!!!!

! !!!

!!!

!!!

! !

!

!!

!!

!

!

!

! !!

!!!

!!

! !!!! !!!

!

!!!

!

!

!!!

!!

!!

!

!!

!

! !!!

!

!!!

!!

!!

!!

!!

!!

!

!! !!

!!!

!!

!!

!!

! ! ! !! ! !!

!! !!

! ! !!!

!!

!!

!!!

!

!!!

!!

!

!

!! !

!!

!!! !

!!! ! !

!

!

!

!

!!!

!!!!

!!!! !! !!

!!! !!!

!!

!! !

! !!!!

!!!

!!!!!

!!!

!!! !

!!

!!!!!

!!!! !!

! !!!!!!!!!!! !!

!!

!

!!

!!!!! !!

!! !! !

!!! !

!!

!!! !

!!

!!

!

!!!!

!!!!!

!!! !!!

!

!

! !!!

!!!

!!

! !!!

!

!!!

!!

!!

!

1 2 3 4

−4−2

02

predict(rf)

resi

d(rf)

!

!

!

!

!

!!!!!

!

!!

!

!

!

!!

!

!!

!

!!

!

!!!!

! !!

!

!

!!

!!

!

!

!!

!!!

!!

! !

!!

!

! !!!

!!

!!!!

!!!! !!!

!!!

!!!

!!!!!! ! !

!!! ! !!! !!!

!!!

!!!

!

!

! !!

!!!! !

! !!!

!!!!!

!

!!

!!

!

!! !!

!

!!!!

!!! !! !

!!

!!! !!!!!!!!!!

!!!

!!

!

!! !

!

!!!

!

! !!!

!

!

!

!!

!

! !!!!

!!!!!!!

!! !

!!

!!! !

!

!!!

!

!

!! ! !!

! !!

!

!!

!!!!

!

!

!!!

!! !

!!

!!

!!!!! !

!!!!

!

!!!!!!

!

!!

!! !! !!! !! !!!

!!!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!! !!

!

!!

!

!

!!

!

!

!!!

!

!

!!!!

!!

!

!

!

!

!!!

!!

!!!

! !

!

! !! !

!!

!!

! !

!

!

!! !!! !!!

!

!!

!

!!!

!

!!

!!

!! !!

!

!

!!

!!!!

!

!!!!

!!

!!!

!! !

!

!!!

!

!!

!

! !! !!

!!!

!!!

!!

!

! !!!

!! !!

!

!

!!!

!!!!

!

!!

!

!

!

!!!

!!

!

!

!

!!!

!

!!

!!!

!!

! !

!

!

!

!!!!

!!

!!!

! !

!!

!!

!

!!!!

!

!!!

!

!

!

!

!

!!

!

!!! !

!!

!

!

!!

!!!!!!

! !!!

!!!!

!

!! !! ! !! !

!!

!!

!

!

! !!

!

!!

!

!!!

!

!!

!

!!

!

!! !

!

! !

!

!

!! !!! !! !! !

!!!!!!

!

! !! ! !!!! !

!

!!!

! !!

!

!!

! !

! !!!!!

!! !! !

!!

!

!!

!!!!! !!

!

!!!! !!!!

! ! !

!!

!

!!

! !!!!

!!

!! !

!

!

! !!

!!

!!!

!!!! !!!!

!

!

!!!

!

!!

!!

!! !!!

!

!!!!!

!!!!!!

!!

!! !!

!!!!!!

!!!!

! !!

!

!!

!

!!!

!

!!!

!!!!

!

!!

!! !!

!

!!!

!!! !! !!!

!!

!!

!

!!

!

!!

!!

!!

!

!!

!!!! !

!

!!

!!

!

!!!

!

!!!!

!

!!! !! !!

!

!!!

!

!!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!

!!

!

! !

!!

!

!

!!

!

!!!

!

!

!

!

!

!!

!

!!!!

!!

!

!

!! !!!!

!!! !!

!

!!

!

! !!

!

!!!!

!

!!!

!!

!

!

!

!

!!

!

! !!!

!!!! !

!!!!

!

!! !

!

! !

!

!

!

!

!!!!

!

!

!

!!! !!!

!!

!

!!!!!!!

!

!

!

!!!

!

! !!

! !!

!

!

! ! !!!

!

!

!!

!

!

!!

!!!

!

!

!

!

!!!

!

!

!

!

!

!!

!!

!

!!

!

! !

! !

!

!

!!

!

!!!

!

!!!!!

!!!!!!

! !

!!!

!!!

!!

!!!

!!!

!

!!!

!

!!

!

! ! !!

!

!

! !!

!

!

! !!!! !!!

!

!!

!! !

!! !!!! !

!!

!

!!!!

!

! !

!

!!! !! !

! !!!

! !!

!!! !

!

!

!

!!!!! !!

!

! !!!

!! !!!

!!!

!!

!! !!! !!! !!

! !!!! !

!!!!!!! !!

!! !! !!!! !!! !!

!!

!!!

!

!!!

!!!!

!

!!!!

!!

! !!!!

!! !!!

!!!!

!!!

!!

!!!!!!!!

!!!

!!! !!

!

!

!!! !

! ! !!!

!!

!

!! !

!

!! !!! !!

!

!

!

!

!!!!!

!!

!!!!!!!

! !! !! !

!!!

!!

!!!!!!!!!

!!! !!

!!!

!

!!!!!!!! !!!!!

!!!!

!!!!!!!!! !! !! !!!

!!!! !!!!!!!

!!

!!!!!!!! !!

! !!!!!!!!! !

! !!!

! !!

!

!!!!

!!!!! !! !

!!! !!

!!!

!!

!

!!!!!

!

!

!!

!! !! !!!!

!!

!

!! !! !!!!

!!

!!!!!!!

!!

!!!

!!

!!

!!!

!!

!

!

!!

!!!!

!

!!!!

!

!!!!!

!

!

!!!! !

!

!!! !

! !!

!

!! !! !!!

!

!!

!!!!! !

!

!!!

!!!! !! !!

!!!!!

! !!!! !!

!

!!

!! !

!!!!

!

!

!

!!

!!

!!!!!!!

!!!!!!

!!!!

!!!

!!

!!

!!

!!!!

!!

!!!

!!

!!!

!!!

!

!!!!!

!!!

!!!!!!

! !

!

!!!!!

! !! !!

!!!!!!!

!

!!!!!!

!

!!!!!

!

!!!

!!! !

!! !

!!

!!!!!

!

!!!!

!! !!!!! !! !!

!

! !!!!

!!

!!

!

! !!!!

!!!! !!!

!!

! !!

!!

!!

!

!! !!

!

! !! !

! !!

! !!!!

!!! !!

!

!!! !!

!!! !

!!!

!!

!!!!! !!!!!!!!

!

!!!!!!!!

!

!!!!! !

!! !!!! !

!! !

!

!

!!!!

!!! !

!!!!!!

!!! !!!!!!!

! !!

!! !!!

! !!

!!!!!

! !!

!!

!!

!!!!

!!

!!!!

!!

!! !

!

!!!!!!

! !!!!

!!!

!

!!!!

!!

!!! ! !

! !!!!

!

!

!

!

!!!

!

!! !

!

!! !

!

! !!

!!

!!

!

! !!

! !!!

!

!

!!!!!! !!!!! !!!!!!!

!!!! !! !

!!!!

!!!

! !!!!!

!! !!!!!!!!!

!!!!

!!!!

! !!!!!

!!

!!!!

!

!!!!

!!

!

!

!

!!

!

! !!! !!!

! !!!

!!!!!!!

!!!!!!!!

!!!!!!!

!!! !

!!!!

!!

!! !!!

!!

!!

!

!

!!

!

!!

!!

!

!

!

!!!

!

!!

!!

!

!!!

!

! !

!

!!! !!!

!! !!!!!!

!

!!!

!! !! !!

!!! !!

!!!!

!

! !!

!! !!! !

!!!

!!!! !!!

!!!

!

!

!! !!

!

!!!!!

!!

!

!

!

!!

!

!!

!!!!!! !

!!

!

!!!!!

! !

!!

!

!!

!!! !

!

!

! !!

!!

!!!! !!!!

!!! !

!

!!

!

!!

!!

!!!!

!!! !

!

!!

!

! !!

!!!!! !!!

!!

!

!

!

!

!

!!

!!!

!

!!!

!

!

!!!

!!!

!!!! ! !

!!!

! !!

!

!!

!

!!!

!

!! !

!

!!!! !!

! !!!

!!!

!

!

!

!!!

!

!!

!

!!!

!

!!

!!

!!!! !

!!

!!

! !!!

!

! !!

!

!

!

!!

!!!

!!

!!!

!!

!

!!

!!

!!

!!!!

!

!

!!!

! !!!!

!

!!!!

!

!

!!

! ! !

!

!

!

!!

!! !

!

!!!!

!!

!!

!!!

!!! !!! ! !! !!

!!

!!

!!!! !!

!

!!!!!!

!! !

!

!

!!

!!!

!

! !! !!!

! !!!

!!! !!! !!! !

!

!!

!!!!!

!!!!

!!!

!

!

!!

!

!!!

!

!

!

!!!!

!

!

!!

! !

!!!

!!

!

!

!!!!

!!!

!

!!!!! !

!

!!

!

!! !

!

!

!

! !! ! !

!

!!

!

!!

!

!

!

!

!!

!

!

!

!

! !!

!!!!

!!

!!! !

!!

!!!!

!!

!!

!

!!

!!

!

!!! !

!!

!

!!

!

!!

! !

! !

!! !

!

!

!

!!!

!!

!

!

!!

!

!!!

!!

!!

!

!!!

!! !

!!!

!

!!

!!!

!

!

!!

!! ! !

!!

!

!

!

!!

!!! !

!

!!!

!

!!

!!

!!! !

!!

!!

!!!!

!

!!

!

!

!!

!

!

!!

! !!

!

!!!!!

!!!

!

!

!

!

!!

!

!

!

!

!

! !

!

! !!

!

!

!!

!

!!

!

!!

!

!!

!

!

!

!

!!

!!

!!!

!! !

!

!

!

!

!!!

!

!!

!!

!

!

!!

!

!

!

! !!!!

!!!!!

!!

!

!!!! ! !!

!!

!!

!!!

! !

!

!!

!!

!

!

!

!! !!!!

!!

! !!!! !!

!!

!!!

!

!

!!!

!!

!!

!

!!

!

! !!!

!

!!!

!!

!!

!!

!!

!!

!

!! !!

!!!

!!!

!!

!! ! ! !! ! !

!!! !!

! !!!

!! !

!!

!!!

!

!!!!

!

!

!

!! !

!!

!!

! !

!!!

! !!

!

!

!

!!!

!!!!!!

!!

!!!!

! !! !!! !

!

!! !

! !!

!!

!!!

!!!!! !

!!

!!

! !!!

!!!!!

!!!! !!

!!!! !!!!!!!

! !!!!

!!

!

!!

!!! !!

!!!! !

!!! !

!!

!!! !

!!!!

!!!!!

!!!!!

!! ! !! !!

!

! !!!

!!!

!!

! !!!

!!!!

!!

!!

!

1 2 3 4

−4−2

02

predict(rf)

resi

d(rf)

!

!

!

!

!

!!!!!

!

!!

!

!

!

!

!!

!!

!

!!

!

!!!

!

!!

!

!

!

!

!

!!!

!

!!

!!!

!

!

! !

!

! !

! !!!

!!

! !!!

!!

!! !!

!

!!!

! !!!

!!

!!!

! !! !!! !!

! !!

!

!!

!!

!!!

!

!!

!

!!!!

!! !!!

!!!!! !

!!!! !

!!

!!

!

!!!!

!!!

!! !

!!

!!!

!!

!!!!!!!

! !!

!!!

!!

!!

!

!!!

!

! !!!

!!

!!!

!! !

!!

!!!

!! !!!!

!!

!

! !!

! !!

!!!!!

!! ! !

!

! !!!

! !!!!!

!!

!!!

!!!!

!!

!!

!!!! !

!!!!

!!!!

!!!

!

!!

!! !! !!! !!

!!

!! !!

!!

!!

!!

!

! !

!

!!!

!!

!!

!

!! !

!

!

!!!!

!

! !!

!

!!!!

!!!

!

!!!!

!

! !!

!

!!

!!

!!

!

!!!

! !!! !

! !

!!

!!

!!

!

!

!! !!!

!!!

!!

!

!

!!

!

!

!

!

!

!

!!!

!

!

!

!!

!!!!

!

!

!!!

!

! !!

!!

! !

!

!!!

!

!!

!

! !!!

!!

!!

!!!

!!

!

! !

!!!

! !!

!

!

!!

!

!!! !

!

!

!

!

!

!

!

!!!

!

!

!

!

!!!!

!!

!!!

!!! !

!!

!

!!

!!

! !

!!!!

!

!! !!!

!!!!

!

!!!

!

!

!

!

!!

!

!

!!!

!

!!

!

!

!!

! ! !!!!!

!!!

!! !

!

!

!! !!

! !! !

!

!

!!

!

!!!!

!!!

!

!! !

!

!!

!!!

!!

!!!

!!

!

!

!! !! ! ! ! !! !!!

!!!

!

!

!!! !

!! !! !

!

!!!

!!

!

!

!

!

! !

! !

!!! !

!

!!! !

!!!

!

!

!!! !! !!

!

!! !!

!!

!!

! ! !

!

!

!

!!

! !!!

!

!!

!! !

!

!

! !!!

!!!!

!!

!! !!!!

!

!

!! !!! !

!!

!! !!!!

!!!!!

!!! !!

!!!!! !

!! !!

!!!!! !!! !!

!!

!

!!! !

!! !!!!!

!

!

!!!

! !!!

!!!

!!! !! !!!

!!!!

!!

! !!

!!!!!

!

!! !!!! !

!!

!!

!!

!!!

!!!

!!

!!!! !!

!!

!

! !!

!!

!!

!!

! !!

!

!!

!

!!!

!

!!

!

!!!!

!

!

!!

!

!!

! !!

!

! !

!

! !!!

!

!

!

!!!!!!!! !

! !!

! ! !!!!

!!!!!!!!

!! ! !

!!!!!!

! !!!!!

!!

!

!!

!!!!!!!

!!!

!

!!

!

!

!!

!

!

! !

!

!

!

!

!

!

!!

!

!

!

!

!! ! !

!

!

!!

! !!

!!!

!

!

!

!

!!

!

!

! !!

!!

!!

!

!! !!!

!

!

!!

!

!

!! !!!

!

! !

!

!!!!

!!

!

!!!

!!!

!!

!

! !

! !!

!

!!

!

!!!

!

!!

!!

!

!!!!!!!!

!!!

!!! !

!! !!!!! !

!!!!

!!!

! ! !!

!!

! !!!

!! !!!! !!!

!! !

!! !

!! !!!! !

!!

!

!!!!!! !

!

! !!!! !! !!!! !!

! !! !!

!!

!!!!! !!!

! !!!

!! !!!

!!!!!!! !!! !!!!!! !

!!

! !!! !!!!!!!

!! !! !!!!!!!!! !

!!!!!

!!! !!!!

!!!

!!

!!! !

!!

!

!! !

!

!! !!!

!

!!

!!

!!!!! !!!

!

!! !!

!!

!!

!

!!! !

! !!!

!!!

!

!! !

!

!! !!

!!!

!

!

!

!

!!!!!

!!

!!!!!!!

! !! !

! !! !!!!

!!!!!!!!!

!!! !! !!!

!!!!

!!!!!!!!

!!! !

!!

!!!!!!!!! !!

!!!!!

!!!! !!!!!!!!!

!

!!!!!!

!!

!

!!! !!!!!!! !! !

!!

! !!

!

!!! !!!!!

! !! !! !

! !!!

!!

!!!

!!!

!!!

!

!!

!! !! !! !!

!!

!

!! !! !!!

!

!

!!

!!

!!

!

!! !

!!!!!

! !!!!

!

!!

!!!!!!

!!!!!!!!!!!!!!!!!! !!!!!!! !!

!!!!! !!!!!!!!!!! !!!!!

!!!!

!!!!

!!!!

!

! !!

!!

!!!

!!

!! !

!!!!

!

!

!

!!

!!

! !!!!!

!!!!!!!

!!

!!

!!!!

!

!!

!!

!!!!

!!

!!!!!!

!!! !!

!

!! !! !!!!!!!!!!! !

!!

!!!!

! !! !! !!!!! !!

!!!

!!!!

!! !

!!!

!

!

!!

!

!!!

!

! !

!!

!

!!!

!

!

!!!

!

!! !!!! ! !! !!

!

! !!!!

!!

!!

!

! !! !!

!!!!!!!

! !

! !!!!

!!!!! ! !!! !! !

!!!!!!!!!

!!!!!

! !! !!!!!!!!

!!

!!!!!! !

!!!

! !!!

!

!! !!!!!!

!!!!!! !!! !

!! !!! !!!

!

!!!!!!! !

!!!!!!

!!!!!!!

!!

!

!!! !!!!

!! ! !

!!!!!

! !!

!!

!!

!!!!

!!!!!!!

!!

! !

!

!!!!!

!!!

!!!!!!

!

!!!!

!!

!!! ! !

! !!!

!!

!

!

!

!!

!!

!! !

!

!! !

!

! !!

!!

!!

!

! !!! !! !

!

!

!!!!!!

!!!

!! !!!

!!!!!!

!! !!!!!!!!!

!!!!!!

!!! !! !!!! !!!

!!!!

!!!!

! !!!

!!

!!

!!!

!

!

!!!!

!!

!

!

!!!

!

! !!!

!!!

!!! !

!

!!!!

!!!

!!!!

!! !!!

!!!!!!

!! !

!!!!

!!!!!!!

!!

!

!!

!

!!

!

!!!

!!

!

!

!!!

!

!

!!

!

!

!!!!

!!

!

!!!!!!

!! !!!

!! !

!!

! !!! ! !!!

!

! ! !!!! !!

!

! !!!!! !! !!

!!

!!!!!!! !! !! !!

!!! !

!!!!!

!!

!

!

!!

!

!

!!

!

!

!!

!

! !

!

!

!

!!!

! !

!!

!

!

!

!!

!!!

!!

!

! !!!

!!!!! !!!

!!!! !

!!!

!!!

!!

!!!!

!!! !

!!

!

!! !!

!!!!! !!!

! !!

!!

!!

!!

!!!!!!!

!

!!!!

!!!

!!

!! !

!!!!

! !!

!

!!!

!!!

!!

! !!!!!! !!

! !!

!

!!

!

!

!

!

!!!

!!!

!

!!

!

!

!!!

!

!!!! !

!

!

!!! !

!

!!

! !!!

!

!!!!

! ! !!

!!!

!

!!

! !

!

!

!!

!!!!

!

!

!!!

! !!!!

!

!!!!

!!

!!

!!

!

! !

!! !!

! !

!!!

!!!

!!!!!!!!! !!! ! !! !!

!!

!!

!

!!! !!

!

!!!!! !!! !

!!

!!

!!!

!

! !! !!! !!!

!

!!! !!! !!! !!

!!

!!!!!!!

!!!

!!

!

!

!!

!

!!!

!

!

!

!!!!

!

!

!

!

! !

!!!

!!

!

!

!

!

!!

!!

!

!

!

!!!! !

!

!!!!!!

!!

!

! !!!!!!!

!

! !!!!!!!

!

!!!!!!!

!!!

! !

!! !

!

!!!

! !!!

!!!!

!!

!!!!!!!!!

!!

!

!!

!

! !!

!!!

!

!!!

!!!

!!!!

!!!!! !!!

!!!!!!!!! !!!

!!! !!!!!

!!!! !!!!!

! !!

! !!! !!!

!!!!!

!! !! ! !!

! !!!!! ! !

!

!!

!!

!

!

!!

!

!!!

!!!! !!

!!!

!!

!

!

!!

!

!

!!

!

! !

!! !

!

!

!

!!

!

!

!

!! !

!

!!

!

!!

!

!! !!

!

!

!

!!!

!

!!!

!!

!!

!!!

!

!

!

!

!

!

!

!

! !!!!

!

!!!

!!

!

!

!!!! ! !! !!

!! !

!!

! !

!

!!!

!

!

!

!

!! !

!!!

!!

!!!

!!!!!!

!!!

!

!! !!

!

!

!!

!

!!

!

!!!!

!

!!!

!

!!!

!!

!!!!

!!! !!

!!!

!!

!

!!

!! ! ! !! ! !

!!! !!

!! !

!!

! !

!!

!!!

!

!!!

!!

!

!!! !

!!

!!! !

!!!! !

!!

!!

!! !!!!!

!!!!

!! !!

!!! !!

!!

!

!

! !! !

!!!

!!!

!!!!

!

!!

!

!!

!!

!!!!!

!!

! !!! !!

!!!!

!!!

! !!!! !!!!!!!!!

!!! !!

!!!! !!!! !

!!

!

!! ! !!!!

!

!!!

!!

!!!!

!

! ! !!!!

!

! !!!

!!!

!

!

! !!!

!!

!!

!

!!!

!

0.6 0.7 0.8 0.9 1.0−0

.20.

20.

6

invlogit(predict(rf))

resi

d(rf,

inv

= TR

UE)

!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!!

!

!!

!

!!

!

!!! !

!!

!

!

!

!

!

!! !

!

!!

!!!!

!

! !

!

! !

! !!

!

!!

! !!!

!!

!! !!!

!!!

!!!

!

!!

! !!! !! !!

! !! !!

!!

!!!!

!!!

!

!!

!

!!!!

!! !!!

!!!!

! !!

! !! !

!!

!!

!

! ! !!

!!! !!

!

!!

!!!

!!!!!!!!

!! !!! !

!

!!

! !

!!!!

!

! !!!

!!

!!!

!!!

!!!!!!! !!!

!!!

!

! !!! !

!!!!

!!

!! ! !

!

!!!!

! !! !!!

!

!!

!! !!!!!

!!!

!!!! !!!!!

!!

!!! !!

!

!!

!! !

! !!! !

!

!!

!! !!

!!

!

!!!

!

!!

!

!!

!

!!

!!

!

!

! !

!

!

!!!!

! !

!!

!

!!!

!

!!!

!

!

! !!

!! !

!

!

!!!

!!!

!

!!! ! !!! !

! !

!!!

!

!!

!

!

!! !!!! !!

!!

!

!

!!

!

!

!!

!

!

!! !

!

!

!

!!

!!!!

!!!

!!

!!

!

!

!!

! !

!

!!!

!

!!

!

! !! !!

!!!

!

!!

!!

!

! !

!!!

!!!

!

!

!!

!

! !!!

!

!

!

!

!

!

!

!!!

!

!

!

!

!!!

!!!

!!

!!

!!!

!

!

!

!!!!

! !

!!!

!

!

!! !!!

!

!!!

!

!!!

!

!

!

!!

!!

!

!!

!!!!

!

!

!!

!!

!!

!!!

!!!

!!!

!

!

!!!!! !

!!

!!

!!

!

!! !

!!

!!

!

!! !

!

!

!

!!!

!!! !!

!!

!

!

!! !!

! ! ! !! !!!!!!

!

!

! !!!!

! !! !

!

!!!

!

!!

!

!

!

! !

!!

!!

! !

!

! !! !!

!!

!

!

!!

! !! !!

!

!!!

!!

!!

!

! ! !

!!

!

! !

! !!!

!

!!

!! !

!

!

! !

!!

!! !!!

!!

! !!!!!

!

!! !!

! !!!

!! !!!

!!!!!!

!!! !!

!!!

!! !!

!! !!!!!!!!

! !!

!!

!

!!! !

!! !!!

!!!

!

!!!

! !!!

!!!

!!!!! !!!

! !!!

!

!! !

!!!!!!

!

!!

!!!! !

!!

!!

!!

! !!

!!!

!!

!

!! ! !!!

!!

! !!

!

!!

!

!!

! !!

!

!!

!

!!

!

!

!!

!

!!!!

!

!!!

!

!!

!!

!

!

!!

!

!!! !

!

!

!

!!!!!!!! !

! !!

! !!!!!!

!!!!!!!

!! ! !!

!!!!!! !!!!

!!

!

!

!!

!!!!

!!!!! !

!

!!!

!

!

!!

!

! !

!

!

!

!

!

!

!!

!

!

!

!

!! !

!

!

!

!

!

!!

!!

!!

!

!

!

!

!!

!

!

! !!

!!

!!

!

! !!!!

!

!

!

!!

!

!!!

!!

!

! !

!

!!!

!

!!

!

!!!

!!!

!!

!

! !

! !!

!

!!

!

! !!

!

!!

!!

!

!!!!!!

! !!

!!!!! !

!! !!! !!!

!!!!

!!!

! ! !!

!!

! !!!!

! !!!! !

!!!

!! !

! !!! !!!! !

!!

!

!!!!!! !!

! !!!! !! !!!! !!

! !! !!!

!!!!!

! !!!

!!!!

!! !!!!!!!!!! !!! !!! !!

! !!!! !!! !!!!! !!!!!! !!!!!!!!! !

!!!!!

!!! !

! !!!

!!!

!

!!! !!!

!

!! !!

!!

!!!

!

!!

!!

!!!!!!!!

!!

! !!

!!

! !

!

!!! !

!!!!!

!!

!

!

! !!

!! !!

!!!

!

!!

!

!!!

!!!

!

!!!!!!!

! !!

!! !

!!!!! !!!!!!!!!!!!!! !!

!!

!!!!!!!!!!!!!

! !!!

!!!

!!!! !! !!!!!!!

!!!! !!!!!!! !!

!

!!!!!!

!!!

!!! !!!!!

!!!

! !!

!! !!

!

!!! !!!!!! !! !

!!! !!!

!!

!!!

!!!!!

!

!

!!

!!!!!!!

!

!!

!

!! !! !!!!

!

!!

!!

!!

!

!! !

!!!!!

! !!!!

!

!!

!! !!!!

!!!!!!!!!!!!!!!!!! !!!!!!! !!

!!!!! !!!!!!!!

!!!!!!!!!!!

!!

! !!!

!!!

!!!

!

!! !!!

!!

!! !

!!!!

!!

!

!

!

!!

! !!!!!

!!!!

! !!

!!!!

!!!!

!

!

!!!

!!!!

!!

!!!!! !

!!! !!

!

!!!! !!!!!!!!!!

! !!

!!!!!

! !!!! !!!!!!!

!!!!! !!

!!!!!!

!

!

!!

!

!! !

!

! !

!!

!

!!

!!

!

!!!

!

!! !!!! ! !! !!

!

! !!!

!!

!

!!

!

! !! !

!

! !

!!!!!! !

! !!!!

!!!!! ! !!! !!!

! !!! !!!!!

!!!!!

!!! !!!!!!!!

!!

!!!!!! !

! !!!!

!!

!

!!!!!!!!

!

!! !!! !

!! !!! !!!!!

!!

!!!!

!!!!!

!! !!!

!!!

!!!!!

! !! !

!!! !!!

! !!

!!!!!

! !!

!!

!!

!!!!

!!!!!!!

!!! !

!

!!!!!!

! !!!!

! !!

!

!!!!

!!

!!! ! !! !

!!!

!

!

!

!

!!

!!

!! !

!

!! !

!

! !

!

!!

!!

!

! !!! !!

!

!

!

!! !! !!

!!!

!! !!!

!!!!

!!!

! !!!!!!!!!!

!!!!!!!! !

!!!!! !!!!!!!

!!!!

! !!!

!!

!!

!!!

!

!

!!!!

!!

!

!

!

!!

!

! !!! ! !

!

!!! !

!

!!!!!!

!!!!

!!!!

!!!!!! !

!!

! !!

!!!!

!!!!!!

!!!

!

!

!

!!

!

!!

!

!!

!

!

!!!

!

!

!!

!

!

!!!!

! !

!

!!

!!!!!! !!!

!! ! !

!!!

!! ! !!!

!

!! !!! ! !!

!

! !!!!

!!! !!!

!!! !!!!

! !!!! !!! !! !

!!!! !

!!

!

!

!!

!

!!

!

!

!

!!!

! !

!

!

!

!!

!

!!

! !

!

!

!

!!

!!

!!

!

!

! ! !!

!

!!!! !!!

!

!!! !!!!

!!!!

!!!!

!

!!! !

!!!

!!!!

!

!!!! !!!! !

!!

!!

!!

!!

!!

!

!!!

!

!!!!

!!!

!!

!! !

!!!!

! !!!

!!!

!!!

!!

! !!!!!! !!

!!!

!

!!

!

!

!

!

!!!

!!!

!

!! !

!

!!!

!

!!!! !

!

!

!!! !

!!

!

! !!!

!

!!!!

!! !!

!!!

!

!!

!!

!

!!

!!!!!

!

!

!!

!! !

!!!

!

!!!!

!!

!!

!!

!!!

!! !!

!!

!!!

!!!

!!

!!!

!!!! !!! !!! !!

!!

!!

!

!!! !!

!

!!!!! !!! !

!!

!!

!!!

!

! !! !!! !!!

!

!!! !!! !!

! !!!!!

!!!!!!!!!

!!

!

!

!!

!

!!!

!

!

!

!!

!

!

!

!

!!

! !

!!!

!

!

!

!

!

!

!!

!!

!

!

!

!!!! !!

!!!!!!!!

!! !!!!!!!

!

! !!!!!!!

!

!!!! !!!

!!!

! !

!! !

!

!!

!! !!! !!

!

!

!!

!!!

!!! !!!

!!

!

!!

!

! !!

!!! !

!

!

!

!!!

!!!!

!!!!! !!!

!!!!!!

!!! !!!

!!! !!!!!

!!!! !!! !!

! !!

! !!! !!!

!!!!

!!

! !!! !

!!

!!!

!! !!

!

!!

!!

!

!

!!

!!

!!!

!!! !!!!

!!

!

!

!

!!

!

!

!!

!

! !

!! !

!

!

!

!!

!

!

!

!

!!

!

!!

!

!

!

!

!! !!

!!

!

!! !!

!!!

!!!!

!!

!

!

!

!

!

!

!

!

!

! !!!!

!!!!!

!

!

!

!!!!

! !! !!!! !

!!

! !

!

!!

!!

!

!

!

!!

!!!!

!!

! !!

!!!!

!!

!!!

!

!!!!

!

!

!!

!

!!

!

! !!!

!

!!!

!

!!!

!!

!!

!!

!!!!!

!!!

!!!

!!

!! ! ! !! ! !

!!! !!

!! !

!!! !!

!!

!!

!

!!

!

!!

!

!!

!!

!!

!!! !

!

!! ! !!

!!

!

!! !!!!!

!!

!!

!! !!

! !! !!

!!

!

!

! !! !!!!

!!!

!!!

!!

!!

!

!

!

!!

!!

!!!

!!

!!!! !

!

! !!!!!!!!!!! !!!

!!

!!!!

!!! !!

!! !! !!!! !

!!

!

!! ! !!!!

!

!!!!

!!!!!

!! ! !! !

!

!

!!!!

!!

!

!

!

! !!

!

!!

!!

!

!!!

!

0.6 0.7 0.8 0.9 1.0

−0.2

0.2

0.6

invlogit(predict(rf))

resi

d(rf,

inv

= TR

UE)

!

!

!

!

!

!

!!!!

!

!!

!

!

!

!

!!

! !

!

!!

!

!!!

!

! !

!

!

!

!

!

!!

!

!

! !

!!!

!

!

! !

!

! !

! !!

!!!

!!!

!

!!

!! !!!

!!!

!!!

!

!!

!!!! !

! !!

! !!! !!!

!!

!!

!!!

!

!!

!

!!!!

!! !!!

!!!!! !

!!

!! !

!!

!!

!

!!!!

!!! !!!

!!

!!!

!!!!!!!!!! !!!

!!

!!

! !

!

!!!

!

! !!!

!

!

!!!

!!!

!!!!!

!!!!!!

! !!

! !!! !

!

!!!

!!!! ! !!

! !!!!

!!!!!

!

!!!!

!! !!

!

!!

!!!!! !

!!!!

!!!!

!!!

!

!!

!! !! !!! !

! !!!

!!!

!!

!!

!!

!

! !!

!!!

!

!

!!

!

!!!

!

!

!! !!

!

! ! !

!

!!!!

!!!

!

!!!!!

! !!

!

!!

!!

!!

!

!!

!! !

!

! !! !

!!

!!

! !

!

!

!! !!! !!!

!!

!

!

!!

!

!

!

!

!

!

!!!

!

!

!

!!

!!!!

!

!

!!!

!!

!!

!!

! !!

!!!

!

!!

!

! !! !!

!!!

!

!!

!!

!

! !

!!!

!!!

!

!

!!

!

!!!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!! !

!!

!!!

!!

! !

!

!

!

!!!!

!!

!!!

!!

!!

!!!

!!!

!

!

!!!

!

!

!

!

!

!!

!

!!! !

!!

!

!

!!

!!!

!!!

!!!

!!

!!

!

!

!!!!! !! !

!

!

!!

!

!! !

!!

!!

!

!! !

!

! !

!!!

!!

!!!

!!

!

!

!! !!! ! ! !! !!!!!

!!

!

!!!!

!! !! !

!

!!!

!!

!

!

!

!

! !

!!

!!!!

!

!!! !!!

!

!

!

!!! !! !!

!

!!!

!!

!

!!

! ! !

!

!

!

!!

!!!

!

!

!!

!! !

!

!

! !

!!

!!

!!

!!!! !!!!!

!

!! !!

! !!

!

!! !!!!

!!!!!

!!!!!!!!

!! !!

!!!!!!!! !!

! !!

!!

!

!!! !

!! !!!!

!!

!

!!!

! !!!

!!!

!!!!! !!!

! !!!

!!

! !!

!!!!!

!

!!

!!!! !

!!

!!

!!

!!!

!!!

!!

!

!!! !!

!!

!

!!!

!

!!

!

!!

! !!

!

!!

!

!!

!

!

!!

!

!!!

!

!

!

!!

!

!!

!!

!

!

!!

!

!!!!

!

!

!

!!!!!!!! !! !

!! ! !!! !!

!!!!!!!

!! ! !!

!!!!!! !!!!

!!!

!

!!

!!!!

!!!!! !

!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!! !

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!!

!

!

! !!

!!

!!

!

!! !!!

!

!

!!

!

!

!!

!!!

!

! !

!

!!!

!

!

!

!

!!!

!!!

!!

!

! !

! !!

!

!!

!

!!!

!

!!!!!

!!!!!!

! !!

!!!!! !!

! !!!!! !

!!!

!!!

!! ! !!

!!

! !!!

!! !!!! !

!!!

! !!

! !!! !!!! !

!!

!

!!!!!! !

!

!!!!! !! !!!! !!! !! !

!!

!!!!!

! !!!

! !!!

!! !!!!!

!! !

!! !!! !!! !!! !

!!! !!! !!!!! !!

!! !! !!!! !!! !!!!

!!!!

!!! !!!!

!!!

!!

!!

! !!!

!

!! !!

!!

!!!

!

!!

!!

!!!!!!!!

!

!! !!

!!

! !

!

!!! !

!! !!!

!!

!

!! !

!

!! !!

! !!

!

!

!

!

!!!!!

!!

!!!!!!!

! !! !!

!!!!!! !!!!!!!!!!!! !! !!

!!

!!!!!!!! !!!!!

!!!!!!!!!!!!! !!

!! !!!

!!!! !!!!!!!

!!

!

!!!!!!! !!

!!! !!!!!!! !

! !!!

! !!

!

!!!!! !!!! !! !

!!! !!!

!!

!!!!!

!!!!

!

!

!

!! !! !!!!

!!

!

!! !! !!!!

!

!!!!!!

!

!! !

!!!

!!! !!!!

!!

!!

!!!!!!

!!!!!!!!!!!!!!!!! !!!!!!! !!

!!!!!!!!!!!!!!!! !!!!!

!!!!

!! !!

!!!!!

! !

!

!! !!

!

!!

!! !

!!!!

!!

!

!

!

!!

! !!!!!!

!!!! !!

!!!!

!!!!

!

!!

!!

!!!!

!!

!!!!! !!!! !!

!

!!!! !!!!!!!!

!!! !

!!

!!!!

! !! !!

!!!!! !!!!!

!!!!

!

!!!!!

!

!

!!

!

!!!

!

!!

!!

!

!!

!!

!

!!!

!

!! !!!!! !! !!

!

!!!!!

!!

!!

!

! !!!!

!!!!!!!

! !

! !!!!

!! !!! ! !!

!!! !! !!!!!!!

!!!!!!

! !! !!!!!!!!

!

!

!!!!!! !

! ! !! !

!!

!

!! !!!!!

!

!

!! !!! !

!! !!!!!!

! !!

!

!!!!

!!!!!

!!!!!

!!! !!!!!!

!! !

!!! !!!

! ! !!!!!!

! !!

!!

!!

!!!!

!!!!!!!

!!

! !

!

!!!!!

!! !!!!!!!

!

!!!!

!

!

!!

! ! !! !!!

!!

!

!

!

!!

!!

!! !

!

!! !

!

! !

!

!!

!!

!

! !!

! !!!

!

!

!!!!!! !!!

!! !!!

!!

!!!!

!! !! !!!!!!!

!! !!!!!!! !!!!!! !!!

!!!!

!!!!

! !! !

!!

!!

!!!

!

!

!!!!

!!

!

!

!!!

!

! !!! !!!

!!! !

!!!!!!!

!!!!

!!! !!!!

!!!!!

!! !!

!!!!

!!! !

!!

!!

!!

!

!

!!

!

!

!!

!!

!

!

!

!!

!

!

!!

!

!

!!!

!! !

!

!!! !!!

!! !!!!!!

!!

! !!

! !! !!

!

!! !!!! !!

!

! !!!! !!! !!

!!

!!!!!!!

!! !!!!

! !!!

!!!!!

!!

!

!

!!

!

!!

!

!

!

!!!

! !

!

!

!

!! !

!!

! !

!

!

!

!!

!!! !

!

!

!! !

!

!

!!!! !!!

!

!!! !!!!

!!!

!!

!!!!

!!! !

!!!

!!!!

!!!!! !!!

!!

!!

!!

!!

!!

!!

!!!!

!

!!!!

!!!

!!!

! !!!!

!

! !!

!

!!!

!

! !!

!! !!

! !!! !!

!!!

!

!!

!

!

!

!

!!!

!!!

!

!! !

!

!!!

!

!!!! !

!

!

!!! !

!!

!

! !!!

!

!!!

!! ! !!

!!!

!

!!

!!

!

!

!!

!!!!

!

!

!!

!

! !!!!

!

!!!!

!!

!!

!!

!

! !

!! !!

! !

!!!

!!!

!!

!!!

!!!! !!! ! !! !!

!!

!!

!

!!! !!

!

!!!!!!

!! !!

!!!

!!!

!

! !! !!! !!!

!

!!! !!! !!! !!!!

!!!

!!!!!!!

!!

!

!

!!

!

!!!

!

!

!

!

!

!!

!

!

!!

! !

!!!

!

!

!

!

!

!

!!

!!

!

!

!

!!!! !

!

!!!

!!!!!

!

! !!!!!!!

!

! !!!!!!!

!

!!!

! !!!!!!

! !

!! !!

!!

!! !!!!

!!!

!!

!!

!!!! !

!!!

!!

!!

!

! !!

!!! !

!!

!

!!!

!!!

!!! !!! !!

!!!!!!!

!! !!!!

!!!!!!!!

!!!! !!! !!

!!

!! !!! !

!!!!

!!!

!! !! ! !!

! !!!!!

!!

!

!!

!!

!

!

!!

!

!!!

!!!! !!

!!!

!!

!

!

!!

!

!

!!

!! !!

! !!

!

!

!!

!

!

!

!

!!

!

!!

!

!!

!

!! !!

!

!

!

!!

!

!

!!

!

!!!!

!!

!

!

!

!

!

!

!

!

!

! !!!!

!

!!!

!!

!

!

!!!! !

!! !!

!! !

!!! !

!

!!!

!

!

!

!

!! !!!!

!!

!!!

!!!!

!!

!!!

!

!

!!!

!

!

! !

!

!!

!

!!

!!

!

!!!

!

!!!

!!

!!

!!

!!!!!

!!!

!!!

!!

!!

! ! !! ! !!!! !!

! !!

!!

! !

!!!

!!

!

!!!

!!

!

!!

! !

!!

!!

! !

!

!!! !

!!

!

!

!! !!!!!

!!

!

!

!!!!

! !! ! !

!!

!

!

! !! !

!!!

!!

!!! !

!!

!!

!

!

!

!!

!!

!!!!!

!!!! !

!

!!!! !!!!!!!

! !!!!!!

!!!

!!! !!

!! !! !!!! !

!!

!

!! ! !!!!

!

!!!

!!

!!!!

!! ! !!

!!

!

! !!!

!!!

!

!

! !!!

!!

!!

!

!

!!

!

0.6 0.7 0.8 0.9 1.0

−0.2

0.2

0.6

invlogit(predict(rf))

resi

d(rf,

inv

= TR

UE)

!

!

!

!

!

!

!

!!!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!!!

!

!

!!

!

!

!

!

!!

!

!

! !

!!

!!

!

!!

!

!!

!!

!

!

!!

!!

!

!

!!

!

!!!

!

!!

!

! !!

!

!

!

!!

!

!!

!!

!

! !!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!!!

!!

!!!

!!

!!

! !

!!

!!

!

!!

!!

!

!

! !!

!

!!!

!!

!!

!

!

!

!

!

!!!

!!!

!!!

!

!

!

!

!

!

! !

!

!

!!

!

! !

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!!

!!

!

!

!

! !

!

!

!

!

!!!

!!

!! !

!

!

! !

!

!

!

!

!!

!!

!

!

!

!

!

!!

!!

!

!

!

!

!

!!! !

!

!!!

!

!!

!

!

!

!!

!

!

!

!

!

! !!!

!!

!

!

!

! !!

!

!

!

!

!!

!

!

!

!

!! !

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!

!!

!

!

!!

!

!

! !

!

!

!

!

!

!

!

!

!

!

!!!

!

!

!!

!!

!!

!

!

!

!

!

!

!

! !!

!

!!!

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!!

!

!

!

!!

!

!!

!

!

!

! !

!

!

!!

!

!!

!

!!

!

!!

!

!!

!

!

!

!!

!

! !

!!

!! !

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!!

!

!

!!!!

!

!!

!!

!

!!!!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!!!

!

!

!

!

!

!!

!

!

!!!

!

!!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!! !

!

!!

!

!

!!

!!

! !

! !!

!

!!

!

!

!!

!

!

!!

!

!

!!!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!!!

!

!

!!

! !

!

!

!

!!

!

!

! !

! !

!

!

!!!

!

!

!

!

!

! ! !

!

!

!

!

!

!

!!!

!

!!

!

!!

!

!

!!

!!

!!!!

!!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!

!

! !!!

!

!

!

!!

!

!

!

!

!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

! !!

!

!

!

!

!

!!!

!!

!!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!! !!

!

!

!

!!

!

!!

!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

! !

!

!

!

!

!

! !!

!

!

!

!

!

!

!

!

!

!!! !

!

!

!

!

!!!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

! !!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!!

!!

!

! !

! !

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

! !!

!!

!

!!

!

!

!

!

!!

!

!

!

!!!

!

!

!

!

!!

!!

!

!

!

!

!

!!!

! !

!

!!

!

!

!!

!

!!

!

!

!

!

!

!!

!

! !

!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!

!!

!

!

!

!!!

!

!

!

!

!

! !

!!

!!

!!!!

!

!

!

!! !!!!

!!

!

!

!

!

!

!!!

!

!

!

!!

!

!!

!

!!

!

!

!!

! !

!

!

!

!

! !

!

!

!!

!!

!

!

!

!

!

!!!!!!!

!

!

!

! !!

!

!

!!

!

!

!! !

!

!

!!

!

!

!

!

!

! !

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

! !

!

!!

!

!

!

!!

!

!!

!

!

!

!!! !

!

!!

!!

!!

!

!!

!!

!!!

!

!!

! !

!

!

!

!

!

!!

!!!

!

!!

!

!

!

!

!

!

!!

! !

!!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!!

!! !

!!

!

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!!

!

!!

!

!

!

!

!

! !!

!! !

!

!!

!

!! !

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!!!!

!

!

!

!

!! !

!

!

!!

!

! !!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!!

!

!

!!

!!!!

!

!

!!

!

!!

!

!

! !

!

!

!!

!

!

!

!

!! !

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!

!

!

!!!

!

!

!

!!

!!

!

!

!!

!!

!

!!

!

!!

!

!

!!

!!

!

!

!!

!

!

!

!! !

!

!

!!!!!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!!

!

!

!!

!

!

!

!

!!

!!

!! !!!

!! !

!

!

!

!

!

!

!!

!

!

!

!!

!

!!

! !

!

!!

!!

!!!

!!

! !!

!!

!

!

!

!!

!

!!

! !

!!

!!

!

!!

!

!!

!

!!!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!!!

!! !

!!

!

! !

!

!

!

!

! !!!

!!

!

!

!!

!

!

!

!

!!

!

!! !!

!

!!

!

!

!

!

!

!

!

!!

!

!

!!!

!!

!!

!!!

!!

!

!

!

!!

!

!!

!!

!

!!

!

!!

!!

!

! !!

!

!

!

!

!

!

!!

!!!

!!!

!

!

!

!!

!

!

!!

!!

!!

!

!

!!

!!

!

!

!

!

!

!

!

!

!!

! !!

!

!

!!

!

!

!

!

!

!!

!

!

!

! !

!

!! !

!

! !

!

!

!

!

!

!

!!

!

!

!!

!

!

!

!

!!!!

!

!!!

!

! !

!!

!

!

!!

!!

!

! !

!

!

!!!!

!!

!

!

!!!!

!

!!!

!

!

!!!

!!!

!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

! !!

!

!!

!

!

!!

!

!

!

!

!!

!!

!

!!

!

!

!!

!

!

!

!

!

!!!

!

!

!

!

!

!!!

!

!

!!!

!!

!!

!

!!

!

!!

!

!!!

!

!

!

!

!!!

!

!

!

!

!

!

!!

!!

!!

!

!

!

!!

!!

!!

!

!!

!

!!

!

!! !

!

! !!!!

!

!!

!!

!

!!!

!

! !

!

!

!!

!

!

!

!!

!

!!!

!

!!

!!

!

!!

!

!

!!!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!!!

! !

!!

!

!

!

!!

!

!!

!

!

!

! !!

!

!

!!!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!

!

!

!

!

!

!

!!!

!!!

!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!!

!

!

!!

! !

!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!!

!

!!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

! !

!

!

!

!

!

!!!

!

!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!!

!

!!

! !

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

! !

!

!

!!

!

!

!!

!

!!

!!

!

!!

!

!

!

!

!!

!

!

!

!

!!

!

!!

!

!

!!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

! !

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!!!!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!!

!!

!

!

!

!

! !

!

! !

!

!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

! !

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

! !

!

!!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!

!

!!

!

!

!

!

!!

!! ! !!

!!

!

!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!!

!!

!!

!!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

! !!

!!!

!

!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!! !

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!!!

!

!

!

!

!!!!

!

!!

!

!!

! !

!

!!

!

!

!

!

!!

!

!!

!!

!

!

!!

!

!

!

!

!

!

!

! !

!

!

!!!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!!

!

! !!

!

!

!

!

!

!!

!

!!

!!

!!!

! !

!

!!

!

!!

!

!

! !

!

!

!

!

!

!

!!

!

! !

!!!

!

!!

!!!

!

!

!!!

!

!!!

!

!

! !!

!

!

!

!

!

!

!!

!

!

0.6 0.7 0.8 0.9 1.0

−3−1

12

3

invlogit(predict(rf))

resi

d(rf)

/resi

d.se

(rf)

!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!!

!

!!

!

!!

!

!!! !

!!!

!

!

!

!

!!!

!

! !

!!!

!

!

! !

!

! !

! !

!

!

!

!

! !!

!

!!

!! !!!

!!

!

!!

!

!

!

!

! !!! !

! !!! !

! !!

!!

!!

!!

!!!

!

!

!

!

!!

!!!!

!!!

!!

!!! !

!!

!!

!

!

!

!!

!

!! !

!!

!!!

!!

!!

!!

!

!!!!

!

!!

!

!!!

!!

!

!

!

!

!!

!!

!!

!

! !

!!

!

!

!

!!

!!

!

!

!

!

!!

!! !!!

!

!

!

!

!!

!

!!

!!!!

!!

!! !!

!

! !

!

!

!

!

! !!!

!

!

!

!

!!

!!!

!

!

!

!

!

!!! !

!

!!!

!

!

!

!

!

!!

!

!

!

!

!

!

! !!!

!

!

!

!

!

! !

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!!!

!

!

!!

!

!!

!

!

!!!

!

!

!!

!!

!!

!

!

!

!

!!

!

!

!

!!! !

!!!

!

!!

!!

!

!

!!

!

!

!! !!!

! !!

!

!

!

!

!!

!

!

!

!

!

!

!! !

!

!

!

!

!

!

!!!

!

!

!

!!

!!

!

!

!!

! !

!

!

!!

!

!!

!

! !! !!

!!!

!

!!

!!

!

! !

!!

!!!!

!

!

!!

!

! !!!

!

!!

!

!

!

!!!

!

!

!

!

!

!!

!!

!!

!

!!!

!

!!

!

!

!

!!

!!

!!

!!!

!!

!!!

!

!

!

!!!

!

!!!

!

!

!

!

!!

!

!

!!

!!

!!

!

!

!!

!

!

!!

!!

!

!!

!

!

!!

!

!

!!!

!!

!!!

!

!

!

!

!

!

!!!

!

!!

!

!!

!

!

!

!

!

!!

!

!!!!

!!

!

!

!! !

!

! !! !

!!

!!

!

!

!!

!

!!!!

!! !!

!

!

!!

!

!

!

!

!

!

!! !

!

!

!!! !

!

! !!

!

!

!!

!!

!!

! !

! !!

!

!!

!!

!!

!

!

! !!

!

!

!

! !

!!!!

!

!!

!

! !

!

!

!!

!!

!!

!!!

!!

! !!!

!

!

!

!!

!

!

!

!

!

!

!

! !!!

!!

!!!

!

!

!

! !!

!

!!

!!

!

!

!! !

!!!

!!

!!

!!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!

!

!!

!

! !!

!

!

!

!

!

!!!

! !

!

!

!

!

!!

!

!

!!

!

!

!!

!

!

!

!

!

!!

!! !

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!

!!

!!

!

!!

!

!

!!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!!!

!

!

!

!

!

!!

!!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!

!

!!! !

!

!

!

!

!!!

!

!

!

!!!!

!

!!

!

!!

!

!

!

!

!!

!

!

!!

!!

!

!

!

!

!

!

!

!!!

!

!!!

!!

!

!!!

!

!

!!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!! !

!

!

!

!!

!!

!!

!

!

!

!

!

!

!!

!

!

! !!

!

!!

!

!

! !!!

!

!

!

!

!

!

!

!

!

!

!!

!

! !

!

!

!!

!

!

!

!

!

!!

!!!

!!

!

!!

! !

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!!

!

! !

!

!!

!

!

!!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!!

!

!

!

! !!

!

!

! !!

!! !

!

!!

!

!

!

!!

!

!!

!!!

!

!

!

!

!!

!!

!

!!

!

!

!!!

! !

!

!!

!

!!

!

!

!!!

!

!

!

!

!!

!! !

!

!

!!

!

!

!!!!

!

!!

!

!!

!!!!

! !!! !!

!!

!

!

! !

!!!

!

!!! !!!!!!

!!!!!!

!!

! !

!

!!!

!

!!! !

!!!

!

!!

!

!

!!

! !!

!

!

!!

!!

!

!!

!!

!

!

!

!

!

!!!!!!!

!

!

!!

!!

!

!

! !

!

!

!! !

!

!!!

!

!

!

!

!

!!

!

!

!!!

!!!

!

!

!

!

!!

!

!!

!

!

!!!!

!

!!

!

!

!

!

! !

!

!!!

!

!

!!

!!!

!

!

!

!!!!

!

!!

!!

!!

!!!!!

!!!

!

!!

!!

!

!

!!

!

!!

!!

!!!

!

!

!!

!

!

!!!

! !

!!!

!

!!

!!

!

!!!

!!!

!

!!

!

!! !!

!!!

!!!

! !

!

!

!!

!

!

!!!!

!!

!

!

! !!

!

!!

! !

!

!

!!

!

!

!

!

!

!!!

!

!

!

!

!!!

!!!!

!

!!

!

!! !

!!!!

!

!

!

!!

!

!

!

!

!

!!

!

!!

!!

!

!!!

!

!

!

!

!

! !

!

!!

!

!

!!

!!

!

!

!!!!

!

!

!

!!!

!

!

!

!!!

! !!

!

!

!!

!!

!!

!

!!

!!

!

!

!

!

!

!

!!

!!!!

!!

!!

!!!!

!

!!

!

!!

!!

!

!

!

!!!

!!

!!

!

!

!

!

!

!

!

!!!

!

!!

!

!!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!

!

!

!!!

!!

!

!!

!!

!

!

!!!!

!

!!!

!!!!

!!

!!

!

!

!!!

!

! !!!

!!!!!!

!!

!

!!!

! !!

!

!!

!!!

!

!

!!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!!

!!

!! !!!! ! !!

!!

!

!!

!!!

!

!

!!

!

! !! !

!

! !

!!!!!

!!

! !!

!!

!!

!

!!!

!!

! !!!

! !!

! !!!!

!

!!!

!

!

!!!

!!

!

!

!!!

!

!

!

!

!!!!

! !

! !!!!

!

!

!

!

!!!

!

!!!

!

!!!

!!

!

!!

!

!!!

!!!!

!

!

!!

!!

!!!!

!

!! !

!!

!

!!

!!

!!

!

!!

! !

!

!!!

!

!

!!

!

!!

!!

!

!!!

!

!

!

!

!

!

!!

!!

!!!!

!

!

!

!!

!

!!!!

!

!

!!

!!!

! !

!

!

!

!!

!

!

!

!!! !!

!!

!!

!

!

!

!

!

!!

!!

!! !

!

!! !

!

! !

!

!

!

!!

!

!!!

! !!

!

!

!

!

!!

! !!

!!!

!! !

!!

!

!

!!

!!

!

!!!!

!!

!!

!!

!

!

!!!!

!

!!

!

!!!

!! !!!

!

!!

!

!!!!

!

!!

!

!

!

!

!

!!

!

!

!

!!!

!

!

!

!

!

!

!!

!

! !!

!! !!

!

!!!

!

!!

!!

!!

!

!!!

!

!!!

!!!!

!! !

!

!

!

!

!

!!!

!

!

!!!

!!

!!

!

!

!

!

!!

!

!!

!

!

!

!

!

!!

!

!

!

!

!!

!

!!!

!

! !

!

!

!

!!!!

!!!!

!

!

!!

!

!!!

!

! ! !!!

!

!!

!!

!!

!!

!

! !

!

!!

!!!

!

!!

!

!! !!!!

! !

!

!!

!

!

! !!!

!!

!! !

!!

!

!

!

!

!

!

!

!

!

!

!!

!

! !

!

!

!

!!

!

!!

! !

!

!

!

!!

!

!!

!

!

!

! !!

!

!

!!!! !

!!

!

!!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!!!

!

!

!

!

!!

!

!

!!!

!!!

!! !

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!!

!

!

!!

! !!

!

!!

!!

!!

!

!

!

!

!!

!

!

!!

!

!

!!! !

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!!

!

!

!!

!!

!!

!

!

!

!

!

!

!

!!!

!

!

!

!!

!

!!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

! !

!

!

!

!

!

!!!

!

!

!

!

!! !

!

!

!

!

!

!

!!

!

!!

!!

!

!

!

!

!

!!

!!!!

!!

!!!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!!

!!

!! !

!

!

!

!

!

!

!

!

!!

! !

!!

!!

!

!

!!

!!!

! !!

!!

!

!

!

!

!!

!!

!!

!!

!

!!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

! !

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!

!!

!

!!

!!!

!

!!!

!!!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!!

!

!

!!

!

!

!!

!

! !

!! !!

!

!

!

!

!!

!

!!!

!!

!

!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!! !

!

!

!

!

!

!

!!

!!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

! !

!

!!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!!!

!

!

!!

!

!

!

!

!!

!

!!

!

!

!

!

!!

!!

!!

!

!!!

!

!!!

!

!

!

!

!!

!

!

!

!!!

!

!

!

!

!!!

!

!

!!

!

!

!

!

!

!

!

!!

! !!!

!

!

!

!!

!

! !

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!!!

!!

!!

!!

!!

!

!

!

!!!

!!

!

!

!

!

!

!

! !

!!

!

!

!!

!!

!!

!

!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!!

!

! !!

!! ! !

!

!

!!

!

!! !

!!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!! !

!

!

!!!

!

!

!!!

!

!

!

!

!

!!!

!

!

!!

!! !!

!

!!

!

! !!

!!

!!

!

!

!!

! !!!!

!!

!

!!!

!

!

!

!

!

!

!

!!

!

!

!!!

!!

!!

!!!

!

!

!!

!

!

!

!

!!!

!

! !!

!

!

!

!

!

!!

!

!! !!

!! !

! !!

!! !

!!

!

!!

!

!

!

!

!

!

!

!!!

!!

!!!

!

! !!!

!!

!

!!!

!

!!!

!

!

! !!

!

!

!!

!

!

!

!!

!

0.6 0.7 0.8 0.9 1.0

−4−2

02

invlogit(predict(rf))

resi

d(rf)

/resi

d.se

(rf)

!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!

! !

!!

!

!

!

!!

!

! !

!!

!!!

!

!!!

!

!!

!! !!

!

!!

!

!

!!

!

!

!

!!

!

!!

! !!

!!!

! !!

!

!

!

!

!

!!

!

!

!!

!

!

!!!

!!

!!!

!!

!!

! !

!

!

!

!

!

!!

!!

!

!!!

!!

!! !!

!

!!

!

!

!

!!!!!

!

!!

!!

!!

!

!

!

!

!

! !

!

!

!!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!!

!!

!!

!

! !

!

!!

!

!

!!

!!

!!

!

!!

! !

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!! !

!

!!!

!

!!!

!

!

!!

!

!

!

!

!

! !!!

!! !

!

!

!!!

!!

!

!

!!

!

!

!

!

!

! !

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!!

!

!

!

!!

!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!!

!!

!!

!!

!!

!

!

!!

!!!

!!!

!

!

!

!

!!

!

!

!

!

!

!

!! !

!

!

!

!

!

!

!!!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!!

! !!

!

!!

!

!

!

!!

!

! !

!!

!!

!!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!

!!

!

!!

!

!! !

!

!

!

!!

!!

!!

!!!

! !

!!

!!

!

! !!

!

!

!!!

!

!

!

!

!

!!

!

!

!

!!

!!

!

!

!!

!

!!

!

!!

!

!!

!

!

!!

!

!

!!

!! ! !! !

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!!

!

!! !!

!!

!

!

!!

!!!!

!!!

!

!!

!!

!!

!

!

!!

!!

! !!!

!

!!

!

!!

!

!

!

!

! !

!

!

!!

!!

!

!!

! !

!

!

!

!!

!!

!!

! !

!

!

!!

!!

!!

!

!

! ! !

!

!

!

!!

!!!

!

!

!!

!

! !

!

!

! !

!

!

!

!!

!

!!!! !!

!!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!

!!!

!

!

!

!

!!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!!

!!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!

!

!!

!

! !!

!

!

!

!

!

!!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!!

!

!!

!

!

!

!!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!

!

!

!

!

!!!!

!

!

!

!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!

!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!!

!!!

!

!

!

! !

! !

!

!

!

!

!

!!

!!

!!!

!

!

!!

!!!

!

! !

!

!!

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!!

!

!

!

! !!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!

!

!

!

!!

!!!

!!

!

!

!!!

!!

!

!!

!

! !

!

!

!!

!

!

!

!

!

!!

!!

!!

!

!!

!

!

!!

!!

!

!!

!

!

!

!

!!

!

!

!

!! !!

!

!

!

!

! !

!!

!!

!!! !!

!

!!

!!!

!! !!!

!

!

!

!

!!

!

!

!

!!

!!

!!

!

!

!

!!

!

!

!!

!

!

!

!! !

!

!

!

!!

!

!

!

!

!

!

!!!

!!!!!

!

!!

!!

!

!

!!

!

!

!! !

!

!!!

!

!

!

!

!

! !

!

!

!!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!!!

!

!

!

!

!

!!

!!

!

!!!

!

!

!!

!

!

!

!

!

!

!!!

!

!

!!

!

!

!

!

!

!!!!!

!!

!!!

!!

!

!

!

!

!

!!

!!!

!

!!

!

! !

!

!

!

!!

!!

!

!!

!

!

!

!!

!

!!

!

!!

!!

!

!

!

!!!!

!!!

!! !

!!

!!

!!

!

!

!!!

!

!!

!

!

!!

!!

!

!! !

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

! !!!!!

!

!!

!

!! !

!!!!

!

!

!

!!

!!

!

!

!

!!

!

!!

!!

!

!

!!

!

!

!

!

!

!!

!

!!

!

!

!!

!!

!

!

!!!!

!

!

!

!

!! !

!

!

!!

!

!!

!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!!

!

!

!!

!!!!!

!!!

!!!

!

!

! !

!

!!

!

!

!

!

!

!! !

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!!

!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!

!

!

!!!

!

!!

!!

!

!!

!

!!!!

!

!!

!

!!

!

!

!!

!!

!

!

!!!

!

!!! !

!

!!!!!

!!

!

!!

!

!

!!

!

!

!

!!!

!

!

!!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!!

!!

!! !!

!

!! !

!

!

!

!

!

!

!!

!

!

!

!

!

!

! !

!!

!

!!

!!!!!

!!

! !!

!!

!!

!

!!!

!

!

!!!!

!!!

!!!

!!

!

!!!

!

!

!!

!!!

!

!

!

!

!!

!

!

!

!

!!!

! !

!!

!! !

!

!

!

!

!!!!

!!

!

!

!!

!

!!

!

!!

!

!!!!!

!!

!

!

!

!

!!

!

!!!

!

!!!

!!

!

!

!!

!

!!

!

!

!

! !

!

!!!

!

!

! !

!

!!

!!

!

!!

!

!

!

!

!

!

!

!!

!!

!!!!

!

!

!

!!

!

!!

!!

!

!

!!

!!!

!!

!

!

!

!

!!

!

!

!

!! !

!

!!

!!

!

!

!

!

!

!!

!

!

!! !

!

!! !

!

! !

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!!!!

!

!!!

!!!

!!

!

!

!!

!!

!! !

!

!

!!

!!

!!

!

!!

!!!

!

!!

!

!

!!!! !!!

!

!!!

!!!!

!!!!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!!

!

! !!

!!!

!

!

!!

!

!

!!

!!

!!

!

!!!

!

!!!

!!!

!

!!!

!

!

!!

!

!!!

!

!

!!!

!!

!!

!

!

!

!

!!

!

!

!!

!

!

!

!

!!!

!

!

!

!

!

!

!!

!

!

! !

!

!

!

!!!

!

!!

!!

!

!

!!

!

!! !

!

!!

! !!

!

!!

!!

!

!

!!

!

! !

!

!!

!!!

!

!

!

!

!!!

!

!!

!

!

!

!!

!

!

! !!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!!

!

!!

! !

!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!!!

!!!!

!

!!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!

!

!

!

!!

!

!

!!!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!!

!

!

!!

! !

!

!

!

!

! !

!

!

!

!

!

!

! !

!

!

!!

!

!

!!

!!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

! !

!

!

!

!

!

!!!

!

!

!

!

!

! !

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!!

!!

!!

! !

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!!

!

!!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!

!!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

! !

!!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!!

!!

!

!

!

!

!

!!

!

!!

!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!!

!

!

!!

!!

! !

!!

!!

!

!

!!

!

!!

!

!

!

!!

!!

!!

!

!

!

!

!!

!!

! !

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!!

!!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!

!

!

!!

!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!!

!!

!!

!!

!

!

!

!!!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!!

!!!

!!

!

!

!!

!

!

!!

!

!

! !

!

!

!

!

!

!

!!

!

!

!

!

!

!! !

!!

!

!!

!

!

!!

! !

!

!

!

!

!!

!

!

!

!!

!

!

!

! !

!

!

!

! !!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!!

!!

!

!

!

!

!!!

!

!

!

!

!!!

!

!!!

!

!

!

!

!

!!

!

!

! !!

!! !

! !

!

!! !

!!

!

!!

!

!

!

!

!

!

!

!!

!

! !

!!

!

!

!!

!!!

!

!

! !!

!

!!!

!

!

! !!

!

!

!

!

!

!

!

!!

!

0.6 0.7 0.8 0.9 1.0

−3−1

12

3

invlogit(predict(rf))

resi

d(rf)

/resi

d.se

(rf)

!

!

!

!

!

!

!

!!!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!!!

!

!!

!

!

!

!

!

!!

!

!

! !

!!

!

!

!

!!

!

! !

! !!

!

!!

!!

!

!

!!

!

!!!

!

!!

!

! !!

!

!!

!!

!

!!

!!

!

! !!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!!!

!!

!!!

!!

!!

! !

!!

!!!

!!

!!

!

!

! !

!!

!!

!!

!

!!

!

!

!

!

!

!!!

!!!!!

!

!

!

!

!

!

!

! !

!

!

!!

!

! !!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!

!

!

!

!

! !

!

!

!

!

!!!

!!

!! !!

!

! !

!

!

!

!

!!

!!

!

!

!

!

!

!!

!!

!

!

!

!

!

!!!!

!

!

!!

!

! !

!

!

!

!!

!

!

!

!

!

! !

!!!!

!

!

!

! !!

!!

!

!!!

!

!

!

!

!!!

!

!

!!

!

!!

!

!

!

!

!!

!

!

! !

!

!

!!

!

!

!

!!

!

!

!!

!!

! !

!

!

!

!

!

!

!

!

!

!!

!!

!!

!!

!!

!!

!

!

!

!

!

!

!

! !!

!

!!!

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!!

!

!!!

!

!

!

!!

!

!

!

!

!

!

! !

!

!

!! !

!!

!

!!

!

!

!

!

!!

!

!

!

!!

!

! !

!!

!! !

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

! !

!

!

!

!

!

!!

!!

!!!

!

!

!!

!!

!

!! !!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!!

!

!

!

!

!

!

!!

!

!

!!!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!

!

!

!!

!

!!

!

!

!!

!

!

!! !

!! !

! !!

!

!!

!

!

!

!

!

!

!!

!

!

!!!

!

!

!!

!

!

!

!

!

!

!! !

!

!

!!

!!

!

!!

!!

!

!

!

!

!

!

!

! !

! !

!

!

!!!

!

!

!

!

!

! !!

!

!

!

!

!

!

!!

!

!

!!

!

! !

!

!

!

!

!!

!

!!!

!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!

!

! !!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

! !!

!

!

!

!

!

!!!

!!

!!

!

!

!!

!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!

!

!!! !!

!

!

!

!!

!

!

!

!

!

!!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

! !

!

!

!

!

!

! !!!

!

!

!

!

!

!

!

!

!!!!

!

!

!

!

!!!!

!

!

!

!

!!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

! !!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!!

!

! !

!

!

!!

!

!

!

!

!

!!

!!!

!!

!

!!

! !

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!!!

!

!!

!

!!

!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

! !

!

!!

!

!!

!

!

!

!

! !

!

!

!

!!!

!

!

!

!

!!!

!!

!!

!

!

!!!

! !

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!!

!!

!

!!

!

!

!

!

!!

!

!

!

!!

!!

!

!

!

!

!!

!

!

!!

!!!!

!

!

!

!! !!!!!

!

!

!

!!

!!!!

!

!

!!

!!!!

!

!!

!

!

!!

! !

!

!

!

!

! !

!

!

!!

!!

!

!

!

!

!

!!!!!!!

!

!

!

! !!

!

!

!!

!

!

!! !

!

!

!!

!

!

!

!

!

! !

!

!

! !!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!

!!

!

!!

!

!

!

!!

!

!!

!

!

!

!!! !

!

!!

!!

!!

!!!

!!

!!!

!

!!

! !

!

!

!

!

!

!!

!!!

!

!!

!

!!

!

!

!

!!

! !

!!!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!!

!! !

! !

!

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!! !

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

! !!

!! !

!

!!

!

!! !

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!!

!

!

!!

!

!

!

!

!

!!

!

!!

!

!

!!

!!

!

!

!!!!

!

!

!

!

!! !

!

!!!

!

!!!

!

!

!!

!

!!

!

!

!!

!!

!

!

!!

!

!

!!

!

!!!!

!

!!

!

!!

!

!

! !

!

!

!

!

!

!

!

!

!!!

!!

!

!

!

!

!

!

!

!

!

!!!

!

!!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!

!

!

!!!

!

!

!

!!

!

!

!

!

!!!!

!

!!

!

!!

!

!

!!

! !

!

!

!!

!

!

!

!! !

!

!

!!!!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!!

!!

!! !!!

!

! !!

!

!

!

!

!

!!

!

!

!

!!

!

!!

!

!

!

!!

!!

!!!

!!

! !!

!!

!

!

!

!! !!

!

! !

!!

!!!

!!

!

!!

!

!!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!!

! !

!

!

!

! !

!

!

!

!

! !!!

!!

!

!

!!

!

!

!

!

!!

!

!!

!!

!

!!

!

!

!

!

!!

!

!!

!

!

!!!

!!

!!

!!!

!!

!

!

!

!

!

!

!!

!!

!

!!

!

!!

!!

!

!!

!

!

!

!!

!

!

!!

!!!

!!!

!

!

!

!!

!

!

!!

!

!

!!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!! !!

!

!

!

!

!

!

!

!

!

!!

!

!

!

! !

!

!! !

!

! !

!

!

!

!

!

!

! !

!

!!

!

!

!

!

!

!!!!

!

!!!

!

! !

!!

!

!

!!

!!

!

! !

!

!

!!!!!!

!

!

!!!!

!

!!!

!

!

!!!

!!!

!

!!!

!

!

!!

!!!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

! !!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!

!

!!!

!

!

!!!

!!

!!

!

!

!

!

!!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!!

!

!!

!

!

!

!!!!

!!

!

!!

!

!!

!

!

! !

!

! !!!!

!

! !!!

!

!!!

!

! !

!

!!

!!

!

!

!!

!

!!!

!

!!! !

!!

!!

!!

!!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!!!

! !

!!

!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!!!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!!!

!!!

!

! !

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!

!!

!!

!

!

!

!

! !

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!!

!

!

!!

! !

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!!

!

!!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!!

!

!

!

!

!

! !

!!

!

!!

!

!

!

!

!!

!

!

!

!

!

!

!!!

!!!

!

!!

! !

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!

!!

!

!

!!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!!!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

! !

!

!

!

!

! !

!

!!

!

!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!!

!!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!

!

!

!

!

!

! !

!

!!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!

!

!

!

!!

!

!

!

!

!!

!! !

!!

!!

!

!

!

!

!

! !

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!!

!!

!!

!!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

!

!!

!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!

!

!

!

!!

!!!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!! !

!

!

!

!!

!

!

!

!

! !

!

!

!

!

!!!

!

!

!

!

!

!!

!!

!!

!

!!

! !

!

!

!

!

!

!

!

! !

!

!!

!!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!!!

!

!

!!

!

!!

!

!!

!

!

!

!

!

!!!

!

! !!

!

!

!

!!

!!

!

!! !!

!!

!

! !

!

!!

!

!!

!

!

! !

!

!

!

!

!

!

!!

!

! !

!!!

!

!!

!!!

!

!

!!!

!

!!!

!

!

! !!

!

!

!

!

!

!

!

!

!

!

0.6 0.7 0.8 0.9 1.0

−20

12

3

resi

d(rf,

inv

= TR

UE)

/resi

d.se

(rf, i

nv =

TRU

E)

!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!!!

!

!

!!

!

!

!

!

!!!

!

! !

!!!

!

!

! !

!

! !

! !

!

!

!!

!!!

!

!

!

!! !!!

!!

!

!!!

!

!

!

! !!!

!! !!

! !! !

!

!!

!!

!!

!!

!

!

!

!

!

!

!!!

!!!!!

!!

!!

! !

!!

!!!

!

!

!!

!

!! !

!

!

!!!

!!

!!

!!

!

!!!

!!

!!!!!

!!

!!

!

!

!

!

!

!!

!

!

!

! !!!

!

!

!!!

!

!

!

!

!

!

!!

!

! !!!!

!!

!

!!

!

!!

!!!!

!!

!!! !

!

! !

!

!!

!

!!

!!

!

!

!

!

!!

!!!

!

!

!

!

!!!! !

!

!!!

!

!

!!

!

!!

!

!

!

!

!

!

! !!!

!

!

!

!

!

! !

!

!!

!

!!!

!

!!

!

!!!

!

!

!!

!

!

!!

!!

!!!!

! !

!!

!

!!

!

!

!!!

!

!

!!

!!!

!!

!

!

!

!!

!

!

!

!!! !

!!!!

!!

!!

!

!

!!

!

!

!! !!!

! !!

!

!

!

!

!!

!

!

!

!

!

!

!! !

!

!

!

!!

!

!!!

!

!!

!!

!!

!

!

!!

! !

!

!

!!!

!!

!

! !! !!

!!!

!

!!

!!

!

! !

!!

!!

!!

!

!

!!

!

! !

!!

!

!

!

!

!

!

!

!!!

!

!

!

!

!!!

!

!!

!

!!!

!

!!

!

!

!

!

!

!!

! !

!

!!

!

!

!!!

!

!

!!

!!

!

!!!

!

!

!

!!

!!

!

!!

!!

!!

!

!

! !

!

!

!!

!!

!

!!

!

!

!!

!

!

!!!

!!

!!!

!

!

!

!

!

!

!!!

!

!!

!

!!!

!

!

!

!

!!

!

!!

!!

!!

!

!

!! !

!! ! !

!! !

!!!!

!!

!

! !!!!

! !!!

!

!!

!

!

!

!

!

!

!! !

!

!

!!

! !

!

! !!

!

!!

!

!

!

!!

! !

! !

!

!

!!

!

!

!!

!

!

! !!

!

!

!

! !

!!!

!

!

!!

!

! !

!

!

!!

!!

!

!!!

!!

!! !!!

!

!

!

!!

!

!

!!

!

!

!

! !!!

!!

!

!!!

!

!

! !!

!

!!

!!

!

!

!! !

!!!!!!!

!!

!

!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!!

!

! !!

!

!!

!

!

!!!

! !

!

!

!

!

!!

!

!

!!

!

!

!!

!

!

!

!

!

!!

!! !

!

!

!

!

!

!

!!

!

!

!!

!!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!!!

!

!

!!

!

!!

!!

!

!

!

!

!

!!! !

!

!

!

!

!

!

!

!

!!! !

!!

!

!!!

!!!

!

!!!!

!

!!

!!

!!

!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!

!!!

!!

!

!!!

!

!

!!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!! !

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

! !!

!!

!

!

!

! !!!

!

!

!

!

!

!

!

!!

!

!!

!

! !

!

!

!!

!

!

!

!

!!!

!!!

!!

!

! !

! !

!

!

!!

!

!

!

!!

!

!

!

!

!

!!

!!!

!

! !

!

!!

!

!

!!

!

!!!

!

!!

!

!

!!

!

!

!

!

!

! !

!

!

!

! !!

!

!

!!

!

!!!

!

!!

!

!!

!!

!

!!

!!! !

!!

!

!!!!

!

!!

!

!!!!! !

!!!

!

!

!

!

!

!! !

!

!

!

!

!!!

! !!

!

!!

!

!

!! !!

!

!!

!

!!

!!!!! !!! !!

! !

!!! !

!!!

!

!!! !!!!!! !!!!!!!!

! !!

!!!!

!!!!! !!

!!!

!

!

!!

! !!

!

!

!!

!

!

!

!!

!!

!

!

!

!

!

!!!!!!!

!

!

!!

!!

!

!

! !

!

!

!! !

!

!!!

!

!

!

!

!

! !

!

!

!!!

!

!!

!

!

!

!

!!

!

!!

!

!

!!!!

!

!!

!

!

!!

! !

!!!!

!!

!!

!!!

!!

!

!!!!

!!

!

!!

!!!!!!!

!!!

!

!!

!!

!

!

!!

!

!!

!!

!!!

!

!

!!

!

!

!!!

! !

!!!

!

!!

!!

!

!!!

!!!

!

!!

!

!! !!

!

!!!!

!

!!

!!

!!

!

!

!!!!

!!

!!

!!

!!!!

! !!

!

!!

!

!

!

!!

!!!

!

!

!

!

!

!!!

!!!

!

!!

!

!! !

! !!!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!!

!!!!

!

!

!

!

!

!!

!!

!

!

!

!!!!

!

!

!!!!

!

!

!!!! !

!

!!!!

!!

!

!

!!!

!!!!

!

!!

!!!!!!

!

!

!!

!!!

!

!!

!!

!!!!

!

!!

!

!!

!

!

!

!

!

!!!

!!

!!

!

!

!

!

!

!

!

!!!!

!!

!

!!!

! !!

!

!

!

!

!

!!

!

!

!

!

!!

!!!

!

!

!

!!!

!!

!!!

!!!

!

!!!!!!!!

!!!!

!!

! !

!

!

!!!

!

!!!!

!!!!!

!!!

!!

!!

!!!

!

!!

!!!

!

!

!!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!!

!!

!! !!!

! ! !!!

!

!

!!!!

!

!

!

!

!

!

! !! !

!

! !

!!!!!

! !

! !!

!!

!!!!! ! !!! !!!

! !!! !!!!

!!!!!

!

!!!

!!

!!

!!!

!

!

!

!

!

!!!

! !

!!!

!!

!

!

!

!

!!!

!

!!!

!

!!!

!!

!

!! !

!!!

!!!!

!

!

!!

!!!

!!!

!

!! !

!!

!!!

!!

!!

!

!!

!!

!!! !!

!

!!

!

!!

!!

!

!!

!

!

!

!!

!

!

!!

!!

!!!!

!

!

!

!!

!

!!!!

!

!

!!

!!!

! !

!

!

!

!!

!

!

!

!!!!

!

!!

!!

!

!

!

!

!

!!

!!

!! !

!

!! !

!

! !

!

!

!

!!

!

!!!

! !!

!

!

!

!

!!

! !!!!!

!! !

!!

!!

!!

!!

!

! !!!

!!

!!!!

!

!!!!!

!

!! !

!!!

!! !!!

!

!!!

!!!!

!

!!!

!

!

!!

!!

!

!

!

!!!

!

!

!

!

!

!

!!

!

! !!

!! !!

!

!!!

!

!!

!!

!!

!

!!!

!

!!!

!!!!

!! !

!

!

!!

!

!!!

!

!

!!!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!!

!

! !

!

!

!

!!!!

!!!

!!

!

!!

!

!!!

!

! ! !!!

!

!!

!!

!!

!!

!

! !!

!!!!!

!!!

!

!! !!!!

! !

!!!

!!

! !! !

!!

!! !

!

!

!

!

!!

!

!

!

!

!

!

!!

!

! !

!

!

!

!!

!

!!

! !

!

!

!

!!

!

!!

!!

!

!!

!

!

!

!!!!

!!!

!

!!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!! !

!

!

!

!

!!!

!

!

!!!

!!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!!

!

!

!

!!

!!

!

!

!

!

! !

!!

!

!

! !

!

!

!

!

!

!

!!

!

!

!!

!

!

!!! !

!

!!!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!

!

!!

!

!

!

!

!!!

!

!

!

!

!!

!!

!!

!

!!

!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!

!

! !

!

!

!

!

!

!!!

!

!

!

!

!!

!

!!

!

!!

!

!!

!

!!

!!

!

!

!

!

!!

!

!!!!

!!

!!!

!!

!

!

!

!

!

!!!

!!

!

!!

!!

!!

!! !

!

!

!

!

!

!!

!

!!

! !

!!

!!

!

!

!!

! !!

! !!

!!

!

!

!

!

!!

!

!!

!

!!

!

!!

!

!

!!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!!!

!

!!

!

!!

!!

!! !!!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!!

!

!!

!

!

!!

!

!

! !

!

! !!

! !!

!

!

!

!

!!

!

!!!

!!

!

!

!!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

! !

!

!

!

!!

!

!!

!!!

!!

!

!

!!

!!

!

!

!

!

!

!

! !!

! !

!

!

!

!

! !

!!

!

!!

!

!

!

!

!

!

!!

!

!!

!

!

!!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!!!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!!

!

!

!!

!

!

!

!

!!

!

!!

!

!

!

!

!! !

!

!!

!

!!!

!

! !!

!

!

!

!

!!

!

!

!

!!!

!

!

!

!!!!

!

!

!!

!

!

!

!

!

!!

!!

!!!

!!

!!

!!

!! !

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!!!

!!

!!

!!

!!!

!

!

!!!

!

!

!

!

!

!!

!

! !!!

!

!

!!

!

!

!!

!

!

!

!

!!

!!

!!

!

!

!!

!

!

!

!!

!

! !!

!! ! !!

!

!!

!

!! !

!!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!! !

!

!

!!!

!

!

!!!

!

!

!

!

!

!!!

!

!

!!

!!

!!

!

!!

!

! !!

!!

!

!

!

!

!!

!!!

!!

!!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!!

!!

!!

!!!

!

!!!

!!

!

!

!!!!

! !!

!

!

!

!!

!!

!

!! !!

!!

!

! !!

!!

!!

!

!

!! !

!

!

!!

!

!

!!!

!!

!!!!

! ! !!!

!

!

!!!

!

!! !

!

!

! !!

!

!

!!

!

!

!

!!

!

0.6 0.7 0.8 0.9 1.0

−20

24

resi

d(rf,

inv

= TR

UE)

/resi

d.se

(rf, i

nv =

TRU

E)

!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!

!

!

! !

!

!!

!

!!!

!

!

!!

!

!

!

!

!

!

!

!

! !

!!

!

!

!

!!

!

!!

!!

!!!!

!!!

!

!!

!! !

!!

!!

!

!

!!

!

!

!

!!

!

!!

! !

!

!!!! !

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!!

!!

!!!

!!

!!

!!

!

!

!!

!

!

!

!!

!

!

!!

!

!

!!

!!

!

!!

!

!

!

!!!!

!

!

!!

!!

!!

!

!

!

!

!

! !

!

!

!!

!

! !

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!!!

!!

!!

!

! !

!

!!

!

!

!!

!

!

!!!

!!

! !

!

!

!

!

!!

!

!

!

!

!

!

!

!! !

!

!

!

!

!

!!!! !

!

!!

!

!

!!!

!

!!

!

!

!

!

!

!

! !!!

!! !

!

!

!!

!

!!

!

!

!!

!

!!

!

!! !

!

!

!!

!

!

!!

!

!

!

!!

!

!

!! !

!

!!

!

!

!

!!

!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

! !

!!

!!

!

!

!!

!

!

!!

!!! !!!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!!!

!!

!

!!

! !!

!

!!

!

!

!

!!

!

! !

!!

!

!

!!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!

!!

!

!!

!

!

! !

!

!

!

!

!

!!

!!

!!!

!

!

!

!

!!

!

! !!

!

!

!!!

!

!

!

!

!

!!

!

!

!

! !

!!

!

!

!!

!

!!

!

!!

!

!!

!

!

!!

!

!

!!

!!!

!! !

!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!

!

!

!!

!

!!

!!

!!

!

!

!!!!

!!

! !!!

!!

!!

!!

!

!

!!

!!

!!!

!

!

!!

!

!

!

!

!

!

!

! !

!

!

!!

!

!

!

!!

!!

!

!

!

!!

!!

!!

! !

!

!

!!

!

!

!!

!

!

! ! !

!

!

!

!!

!

!!!

!

!!

!

! !

!

!

! !

!

!

!

!

!!

!!!! !

!

!!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!!

!!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!

!

!

!!

!

!

!

!

!

!

!!

!!

!

!

!

!!

!

! !!

!

!

!

!

!

!!

!

!

!

!!

!

!

!!

!

!

! !

!

!

!!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!!!

!

!

!

!

!

!!

!

!

!

!

!!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!!!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

!!

!!!

!

!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!!

!

!

!!!

!

!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!!

!

! !

!

!

!!

!

!

!

!

!

!!

!!!

!

!

!

! !

! !

!

!

!

!

!

!!

!!

!!!

!

!

!!

!!!

!

! !

!

!!

!

!

!

!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!

! !

!

!

!

! !!

!

!

! !!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!

!

!

!!

!!!

!!

!

!!!!!!

!

!!

!

! !

!

!

!!

!

!

!

!

!

!!

!

!

!!

!

!!

!!

!!

!!

!

!!

!

!

!

!

!!

!

!

!!

!!!

!!

!

!

! !

!!

!!

!!! !!

!

!!

!!!

!! !!!

!

!

!

!

!!!

!

!!

!!

!

!!

!

!

!

!!

!!

!!

!

!

!

!! !

!

!

!

!!!

!

!

!

!

!

!!!

!!!!!

!

!

!!!

!

!

!!

!

!

!! !

!

!

!!

!

!

!

!

!

! !

!

!

!!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!!!

!

!

!

!

!

!!

!!

!

!!!

!

!

!!

!

!

!

!

!

!

!!! !

!

!!

!!

!

!

!

!!!!!

!!

!!!

!!

!

!

!

!

!

!!

!!!!

!!

!

! !

!

!

!!

!

!!

!!!

!

!!

!

!

!

!!

!

!

!!

!!

!

!

!!!!

!

!!

!! !

!!

!!

!!

!

!

!!!

!!

!

!

!

!!

!!

!!

! !!

!

!

!

!

!

!

!

!

!!!

!

!

!

!

!

! !!

!!!

!

!

!

!

!

! !

!!!!

!

!

!

!

!

!!

!

!

!

! !

!

!!

!!

!!

!!

!

!

!

!

!

!!

!!

!!

!

!

!

!!

!

!!

!!!

!

!

!

!

!!!

!

!

!!

!

!

!!

!

!

!!

!

!!!

!

!

!

!!

!

!

!!

!

!

!!

!!!!!

!!!

!!!

!

!

! !

!

!

!!

!

!

!

!

!!!

!!

!

!

!

!

!

!

!

!

!

!!!

!

!!

!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!

!

!

!!!

!

!!

!!

!!!

!

!!!!!

!!!

!!!!

!!

! !

!

!

!!!

!

!!! !

!

!!

!!!!!

!

!!

!

!

!!

!

!

!

!!!

!

!

!!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!!

!!

!! !!

!

!! !

!

!!

!

!!

!!

!

!

!

!!

!

! !

!!

!

!!

!!!!!

!!

! !!

!!

!!!!!

! !

!

!!!!

!!!

!!!!!

!

!!!!

!

!!

!!!

!!

!!

!!

!

!

!

!

!!!

! !

!!

!

! !

!

!

!

!

!!!!

!!

!

!

!!

!

!!

!

!! !

!!!!!

!!

!

!

!!

!!

!

!!!

!

!!!

!!

!

!

!!

!

!!

!

!

!

! !

!!!!

!

!

! !

!

!!

!!

!

!!

!

!

!

!!

!

!

!

!

!!

!!!

!

!

!

!

!!

!

!!

!!

!

!

!

!

!!!

!!

!

!

!

!

!!

!

!

!

!! !

!

!

!

!!

!

!

!

!

!

!!

!

!

!

! !

!

!! !

!

! !

!

!

!

!!

!

!!

!

! !!

!

!

!

!

!!!!

!

!!!

!!!

!!

!!

!!

!!

!! !

!!

!!!!

!!

!

!!

!!!

!

!!

!!

!!!! !!!

!

!!!

!!!!

!!!!

!

!

!!

!!

!

!

!

!!!

!

!

!

!

!

!

!!

!

! !!

!!!

!

!

!!

!

!

!!

!!

!!!

!!!

!

!!!

!!!

!

!

!!

!

!

!!

!

!!!

!

!

!!!

!!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!!

!

!

!

!

!

!

!!!

!

! !

!

!

!

!!!

!

!!

!

!!

!

!!

!

!!

!

!

!!! !!

!

!!

!!

!

!

!!

!

! !

!

!!

!!!

!!

!

!

!!!

!

!!

!!

!!!

!!

! !!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

! !

!

!

!

!!

!

!!

! !

!

!

!

!!

!

!! !

!

!

!

!

!

!

!

!!!

!!

!!

!

!!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!

!

!

!

!!

!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!!

!

!

!!

!!

!

!

!

!

! !

!

!

!

!

!

!

!!

!

!

!!

!

!

!!

! !

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

! !

!

!

!

!

!

!!!

!

!

!

!

!

! !

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!!!!

!!

! !

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!!

!!

!! !

!

!

!

!

!

!!

!

!!

!!

!

!

!!!

!

!!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!!

!

!!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!!

!

!

!

!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!!

! !

!!!!

!!

!!

!!

!

!

!!

!

!!

!

!

!

!!

! !

!!

!

!

!

!

!

!

! !!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!!

!! !

!

!

!

!

!

!

!!

!

!

!

!

!!

! !!

! !

!

!

!

!

! !

!

!!

!!

!!

!

!

!

!

!

!!

! !

!

!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!

!

!

!! !!

!

!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

! !!!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!

!

!

!

!

!! !

!

!!

!

!

!

!

!

!

!!

!

!!

!

!

!

!!

!!

!!

!!

!!

!

!

!

!!!

!

!

!!

!

!

!

!

!!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!!

!!

!

!!

!

!

!!

!

!

! !

!

!

!

!

!

!

!!

!

!

!

!

!

!! !

!!

!

!!

!

!

!!

!!

!

!

!

!

!! !

!

!

!!

!

!

!

!!

!

!

!

! !!

!!

!

!

!

!

!

!

!!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!

!

!!

!

!

!

!

!!!

!

!

!

!

!!!

!

!!!

!

!

!

!

!

!!

!

!

! !!

!!

!

! !

!

!! !

!!

!

!!

!

!

!

!!

!

!

!!

!

! !

!!!

!

!!

!!!

!

!

! !!

!

!!!

!

!

!!

!

!

!

!

!

!

!

!

!!

!

0.6 0.7 0.8 0.9 1.0

−20

12

3

resi

d(rf,

inv

= TR

UE)

/resi

d.se

(rf, i

nv =

TRU

E)

13

Common method variance

!

!!!!

!!

!!

!

!

!!

!

!!

!

!

!

!!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!

!

!

! !

!

!

!

!

!

!

!

!

!

!!

!

!!!

!

!

!

!!

!!

!!

!

!!

!

!

!!!!!!

!

!

!

!!

!!!

!

!

!

!

!!!!!!

!!

!

!!

!!!!

!

!

!

!

!!!

!!

!

!

!

!!

!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!

!!!!!

!

!

!!!!!!!!!! !!!!!!!

!

!!!!!!!!!

!

!!!!!!! !

!!

!!!!!!

!!

!!!!!!!!!!!!!!!

!!

!!!!!!

!!!!!!!!!!

!!

!

!!!

!!

!

!!!!!!!! !

!

!!!! !

!

!

!

!!

!!

!!!

!!

!

!

!! !!

!!

!!

!

!! !!!

!!

!!

!!

!

!

!

!!

!

!!

!!!

!!

!

!!! !!!!

!

!!!!!

!

!!!

!!!

!!

!

!

!

!!!!

!!

!!!

!

!!!!!

!

!

!!!!!

!!!!

!!!!!!!

!

!!!!!

!!

!

!

!

!!

!

!

!!

!

!

!!

!!!!

!

!! !

!!

!!!

!!

!!!!

!

!

! !

!

!!

!!!!

!

!

!

!!!

!

!

!!

!

!!

!

!!!

!!

!!!

!!

!!!

!

!!

!

!!!

! !

!!!

!

!!!

!

!

!!

!

!

! !!

!!

!

!!

!

!

!

!! !!! !

!

!!!!

!!!!!

!!

!!!!

!!

!

!!!!!

!

!!!

!!

!!!!

!!! !!!

!

!!!

!

!

!

!

!!

!

!!

!

!!!!

!!!!!!

!

!

!

!!

!

!

!!!!!!!!!!!!!!!!!!

!!!

!!

!

!!!

!

!!

!!!

!

!!

!!!!!!!!!!!!!!!

!

!! !!!!!!!!!

!!

!

!!

!

!

!!!!!!!!!!

!!

!

!!!!

!!

!!!!!!

!

!

!!!!

!!

!!!!

!!

!!!

!!

!

!

!!!!!

!!!!

!!!!!!

!

!

!

!!

!!!!!!!

!!!

!!!

!!

!

!!

!

!

!!

!

!

!! !!!

!

!!!!!! !

!

!!

!!

!!!! !

!

!

!!!!!!! !!

!

!

!! !

!!!!

!

!

!!!

!

!!! !

!!!!

!

!

!

!

!!!!!!!!

!

!!!

!

!!!

!

!

!

!!!!!!!

!

!

!!!

!!!!

!

!

!

!!!!

!!!!

!

!!!!!!!!!!!!

!!!!!!!!!!!!!!!

!

!!

!

!!!

!!!!

!

!!!!!!

!

!!

!

!!

! !

!! !!!!!!!

!

!!!!!!

!

!

!

!

!

!!!!

!!!!!!!!!!!!

!

!!

!

!!!

!

!

!!!

!

!!!!!! !!

!!

!!!!!!!!!!!!!!!!!

!!!!

! !!!!!!!!

!!!!!!!

!!!!!!!!!!!!!!!!

!!

!!

!

!

!!!! !!!

!

!!!!!!!

!

!

!

!!

!!!

!!

!

!

!!!!!!

!

!!!

!

!!!

!!!

!!!

!

!

!

!

!!!!! !!!!!!!!!

!

!!!!!!!!!

!

!!!!!!!

!

!!!!

!!!

!

!!!!

!

!!

!!

!

!!!

!

!!

!!!

!

!!

!!

!!

!!!!!

!

!!!!

!

!

!!

!!

!!

!

!

!!

!!

!!

!!

!

!

!!!

!!!

!

!

!!

!

!

!!

!!!!!!!!!!!!

!!

!!

!!!

!

!

!!

!

!

!!!!

!!!!

!

!!!

!!!!!!!!!!

!!!!!!!!

!!!!

!!!

!!

!!

!!!

!

!

!

!

!

!

!

!

!!!

!

!!!!!

!

!!

!

!!

!!!

!

!!!!!!!!!!!

!

!!!!!!

!!!!!!!!!

!!

!

!!

!!

!

!!!

!!!!!!

! !

!!

!

!

!!

!!!!

!

!!!!

!!

!

!

!!!!

!!!!

!!!!

!

!!

! !

!!!!!

!!!!! !!!!

!!

!!!!!!

!

!!

! !!!

!

!

!!

!!

!

!!!

!!!!!

!!!!!

!

!

!

!!!

!!

!!!!!!!!

!

!!

!!!!!!!!!

!!!

!

!

! !! !!!

!!!

!!!!!

!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!

!!!

!!!!!!!!!!

!!!

!!!!!!!!!!!!!

!!!!!!

!!!

!

!

!!!!!!!!

!

!!! !!!!

!!!!

!!!!!!!!

!

!!

!!!!!!

!!

!!!!!!!!!

! !!

!!

! !!!!!!!!!!!!

!!!

!

!!

!!!!!!

!!!!

!

! !!!!!!!!!!

!

!

!!!

!

!

!!!

!!!

!!!

!

!

!!

!

!

!!!

!!!

!!!!

!

!

!!! !

!

!!!

!

!!!

!!!

!

!

!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!

!!!!!!! !!

!!!!!!!!!!!!!

!

!!!

!!!!! !!!!!!

!

!!! !!!!

!!!!

! !!!!

!!

!!!!!!

!

!!!!

!!!!!!!!!! !

!!!!!

!!!!

!

!!!!!

!!

! !

!

!!!!!!!

!

!!

!

!!!! !! !

!!!

!!!!

!

!!!!!!!!!

!

!

!

!!!!!! !!!

!

!

!!

!

!

!!

!!!

!! !

!!!!

!!

!

!!

!!!

!!

! !!!

!

!!!

!

!

!!

!

!

!!!!!

!!!!!!!

!!

!!

!!

!!!

!!!!

!!

!!!!!!

!

!!!!!!!

!!

!!!

!!

!!

!!

!!

!!

!!

!!

!

!

!!

!

!

!

!

!!

!!!!

! !

!!

!

!

!!

!!

! !

!!!

!!!

!! !!

!

!

!!!!!!!

!!! !!!!!!!!

!

!

!!

!

!

!

!

!

!

!!

!

!!!

!!!!

!!

!

!

!

!!!

!

!!!

!

!!

!

!!!!

!!

!!

!!

!

!

!!!!!!!

!

! !!!!!!!

!!!!

!!!!

!!!!!!

!!!!!!!!!

!

!!

!!!!

!!!!!!!!!!!

!!!

!!!

!!!!!!! !

!!!!

!

!!!!

!!

!

!!!

!!

!!!

!!

!!!!!

!!!!! !!!

!

!!!

!!!!

!!!!

! !!!

!!!!

! !!! !!

!!

!!!

!

!!!

!!! !!

!!!

!

!!!!!

!!!

! !!!

!!!!

!!!!!!!!

!!!!

!!!!

!!!!

!!! !!!

!!! !!

!!

!!

!!

!

!!!!!!

!!!

!!!!

!

!!

!

!!!!

!!!

!!!

!!

!

!!!

!!!!

!!

!!! !!! !

!

!

!!!!

!!

!!!!!!

!

!!!!!

!!!

!!! !

!!!!! !

!

!

!!!!!!

!!!

!!!

!!!

!!!

!!!

!!!!

!!

!!!

!!!

!!!

!

!! !!!!!!

!!!

!!!

!!!

!!!!!

!!!! !!!

!!

!

!

!!

!!! !!!

!!!!

!!

!!!!!!!!!

!!

!

!!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!!

!!!!!!

!

!!

!

!!!

!! !!!!!!

!!!

!!! !!

!!!

!

!!!

!!!!

!

!!!!!!!

!

!

!!!! !

!! !!!!!!!!!!!!

!!!!!!!!!!!!

!!!!!!!!!! !! !!!!!!!!!

!!!

!!! !!!!!!!!!!!! !!!!!!

!!!

!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!

!!!

!!!!!!!

!

!!!!

!!

!

!!

!!!

!

!!!

!!!

!!!

!!

!

!!!

!

!!

!!!

!!

! !!!!!

! !

!

!!

!!

!!

!

!!!

!!

!

!!!

!

!!!!!

! !!

!

!

!

!!!

!!! !!!

!

!!

!

! !

!

!

!!!

!! !!!!!!!!!!!

!!

!!!

!!!!

!!

!!

!

!!

!!!!

!!!

!!!!!!!!!!

!!!!!!!!

!

!!!

!!!

!!

!

!!

!!!!

!!

!!

!!!!

!

!!!!!!!

!!!!

!!!!

!!!!!!!

!!!!

!

! !!!

!!!!!!!!!!!!

!!! !

!

!!!!

!!!

!!!!!!

! !!

!

!!

!!

! !

!! !!!!

!!!!

!!!!

!!!!

!!

!

!

!

!

!

!

!

!

!

!!

!!

!!!

!!

!!!!!!

!

!!!!!

!!!!!!

!!!!!!

!

!

!

!

!

!!

!

!

!!

!

!

!!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!

!!!

!

!!!!!

!!!!!!

!!!

!!! !

!!

!!

!!

!!!!

!!!

!!!!

0 1 2 3 4 5 6

−3−1

12

predict(rf)

resi

d(rf)

!

! !!!

!!

!!

!

!

! !

!

!!

!

!

!

!!!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!!!

!!

!

!!

!

!

!!

!

!

!

!

!

!!!

!!!

!

!

!

!!

!!!!

!

!

!

!

!

!!!!!

!

!

!

!

!!

!!!

!!

!

!

!!!!!

!

!!

!

!

!

!!!!!

!

!

!

!!!!!

!

!

!

!

!!!!

!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!!!!!!!!!!

!!

!!!!!

!

!!!

!!!!!!

!

!!!!!!! !

!!!!!!!!

!!

!!!!!!!!!!

!!!!!

!!!

!!!!

!!!!!

!!!!!!

!!!

!!!

!!

!

!!!

!!!!! !

!

!!!! !

!

!

!

!

!

!!!!!!

!!

!

!! !!

!!

!!

!

!!!!

!

!!

!!

!!

!

!

!

!!

!!!

!!!!

!

!

!!! !!!!

!

!!

!!!

!

!!! !!!

!!

!

!

!

!!!!!!!!!

!!!!

!!

!

!

! !!!!!!!

!!!!!!!!

!

!!!!!

!!

!!

!

!!

!

!!!

!

!

!!

!!!

!!

!! !

!!!!!

!!

!!!!

!

!

! !

!

!

!!

!!!

!

!

!!!!

!

!

!!

!

!

!

!!

!!

!!

!!!

!

!!!!

!

!!

!

!!!

!!

!!!

!

!!!

!

!

!!

!

!

!!

!

!!

!

!!

!!

!

!!!!! !

!

!!!

!!

!!!!

!

!

!!!!

!!

!

!

!!

!

!!!!!

!!

!!!!!!

!!!!

!

!!!!!

!

!

!!

!

!!

!

! !!!!!!!!!

!

!

!!

!

!

!

!!!!!!!!!!!!!!!!!!! !!

!!

!

!!!

!

!!

!!!!

!!

!!!!!!!!!!!!!!!

!

!! !!!!!!!!!!!

!

!!

!!

!!! !!!!!!

!!

!!

!!!!

!!

!!!!!!

!

!!!!!

!!

!

!!!!!!!!!

!!

!

!!!!

!!!!

!

!!!!!!!

!

!

!!

!!!!!!!

!!

!!!

!!

!

!

!!

!

!!

!

!

!

!! !!!

!

!!!!!! !

!

!!

!!

!

!!! !

!

!!!!

!

!!! !!

!

!

!! !

!!! !!

!

!!!

!

!!! !!!!!

!

!

!

!

!!!!!!!!

!

!!!

!

!!!

!

!

!

!!!!!!!!

!

!!!

!!!!

!

!

!

!!!

!!!!!

!

!!!!!!!!!!

!!!

!!!!!!!!!!

!!!!

!

!!

!

!!!

!!!!

!!!!!!!

!

!!

!

!!

! !

!!!!!!!

!!

!

!!!!!!

!

!

!

!

!

!!!!!

!!

!!!!!!!!!!

!!

!!!

!

!

!

!!!

!

!!!

!!! !!!

!

!!!!!!

!!!!!!!!!!!

!!!!

! !!!

!!!!

!!!!!!!!

!!!!!!!!!!!!!!!!

!!

!!

!

!!!!

! !!!

!

!!!!!! !

!

!

!

!!

!!!

!!

!

!!

!!!!!!

!!!

!

!!!

!!!

!!!

!

!

!

!

!!!!!!

!!!!!!!!!

!!!!!!!!!

!

!!!!!!!

!

!!!!

!!!

!

!!!!

!

!!!!

!

!!!

!

!!

!!!

!

!

!

!!!

!

!!!

!!

!

!!!

!!

!

!

!!!

!!

!

!!!!

!

!!

!!

!

!

!!!

!!

!!

!

!!

!

!

!!

!!!!!!

!!!!!

!!!

!!

!!!

!

!

!!

!!

!!!!

!!!!

!

!!

!

!!!!!!!!!!

!!!!!!

!

!!!

!!!!!!

!

!!!!!

!

!

!

!

!

!

!

!

!!!

!

!!!!!

!

!

!

!

!!

!!!

!

!!!!!!!!! !!

!

!!!!!

!

!!!!!!!!!

!

!

!

!

!!!

!

!!!

!

!!!!!

! !

!!

!

!

!!!

!!!

!

!!!!

!!

!!

!!!!

!!

!!

!!!!

!

!!

! !

!!!

!!

!!!

!!!!

!!

!!

!!!

!!!

!

!

!

!!!

!

!

!!

!

!!

!

!!!

!!!!!

!!!!!

!

!

!

!!!

!!

!!!!!!

!!

!

!!

!!!!!!!!!!!!!

!

! !!

!!!!!

!!!!!!

!!

!!

!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!

!!

!!!!!!!!!!

!!!!!! !

!!!!!

!!!!!!!!

!!!!!!

!!

!

!

!!

!!!!!!

!

!!! !!!!

!!!!

!!!!

!!!

!!

!!!!!!!!

!!

!!!!!!

!!!! !!

!!

! !!!!!!!!!!!!

!!

!

!

!!

!!!!!!

!!!!

!

! !!!!!!!!!!

!

!

!!!

!!

!!!

!!!

!!!

!

!

!!

!

!

!!!!

!!!!!!

!

!

!!! !

!

!!!

!

!!!

!!!

!

!

!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!

!

!!!!!!!!!!!!!

!

!! !

!!!!! !!!!!!!

!!! !!!!

!!!!

! !!!!

!!!!!!! !

!

!!

!!

!

!!!!!!!!! !!

!!!!

!!!!

!

!!!!

!

!!

! !

!

!!!!!!!

!

!!

!

!!!! !! !

!!!

!!!!

!

!!!!!!!!!

!

!

!!

!!!!! !!!!!

!!

!

!

!!

!!!!!

!!!

!!

!!

!

!!

!!!!!

!!

!!

!

!!!!

!!

!

!

!

!

!!!!!!!!!!!

!

!!

!

!

!

!!!!!!! !

!!!!!

!!

!

!! !

!!!!!!!!!

!!

!!

!!

!

!

!!

!!

!!

!

!

!!

!

!

!

!

!

!

!

! !!

!!

!!

!

!

!!

!!

!!

!!!!!!

!!!!

!

!

!!!!!!!

!!! !!!!!!!!

!

!

!!

!

!

!!

!

!!

!!

!!!

!!!

! !

!

!

!

!

!!!

!

!!!

!

!!

!

!!!!!

!

!!

!!

!

!

!!!!!!!

!

! !!!!!!!

!!!! !! !!

!!!!!!

!!!!

!!!!! !

!!

!!!!

!!!!!!!

!

!!!!!!

!!!

!!!!!!! !!

!!

!

!

!! !!!!!

!!

!!! !!

!

!!

!

!

!!!

!!

!!

!!!!

!

!!!

!!!!

!!!!

! !!!

!!!!

!!!! !!!!

!!!

!

!!!!

!!!!

!!

!!

!!!!!

!!!

! !!!

!!!!

!!!!

!!!!

!!!!

!!!!

!!!!

!!! !!!

!!! !!

!

!!!!

!

!

!!!!!!

!!!

!!!!!!

!

!

!!!!

!!!

!!!

!!

!

!!

!

!!!!

!!

!!! !!! !

!

!

!!!!

!!

!!!!!!

!

!!!

!!

!!!

!!! !!!!!

! !

!

!

!!!!!!

!!!

!!!

!!!

!!!

!!!

!!!!

!!

!!!

!!!!!!

!

!!!!!

!!!

!!!

!!!

!!!

!!!!!

!

!!! !!!

!!

!

!

!!

!!! !!!

!!!

!!!

!!!!!!!!!

!!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!

!!!

!!!!!!

!

!!

!

!!!

!! !!!!!!

!!!

!!! !!!!

!!

!!!

!!!!

!

!

!!!!!!!

!

!!!! !!

!!!!!!!!!!!!

!

!!!!!!!!!

! !! !!!!!!!!!!!

! !!

!!!!!!!!!!

!!! !!!!!!!!!!!!

!!!!!!

!!!

!!! !!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!

!!!

!!!!!!!!

!!!!

!!

!

!!

!!!

!

!!!

!!

!!!!

!!!

!!!

!!!!

!!

!!! !!!

!!! !

!

!

!!!!

!!

!

!

!!!!

!!!

!!!!!!

!!

!!

!

!!!!

!!!

!!!

!!!

!

!

!!

!

!!!!! !!

!!!!!!!!!

!!!!! !!!!

!!!!

!

!!!!

!!

!

! !

!!!!!!!!!!!

!!!!!!!

!

!!!

!!!

!!

!

!!

!!!!

!!!!

!!!!!!!

!!!!!!!!!

!!!!

!!!!!!!!

!!!!

!!!!

!!!!!!!!!!!!

!!! !!

!!!!!

!! !!!!

!!

! !!

!

!!!!

!!

!

!!!!!

!! !!

!!!!

!!!!

!!

!

!

!

!

!

!

!

!

!

!!

! !

!!

!

!!

!!!!!!

!

!!!!

! !!!!!!!!!!!!

!

!!

!

!

!!

!

!!

!!

!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!

!!!

!

!!!!!

!!!!

!!!!!

!!! !!

!!!!

!

!!!!!!

!

!!

! !

0 1 2 3 4 5 6

−3−1

12

3

predict(rf)

resi

d(rf)

!

!!!!

!!

! !

!

!

! !

!

!!

!

!

!

!!!!!

!

!

!

!

!

!!!

!

!

!

!

!

!!!

!!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!!!

!

!

!

!

!

!

!!!

!

!!

!

!

!!!!!

!!

!

!

!!

!!!

!!

!!

!!!!!

!

!!

!

!!

!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!!!!!!!!!

!

!!!

!!!!

!

!!!!!!!!!

!

!!!!!!! !!!!!!!

!!

! !

!!!!!!!!!!!

!!!!!

!

!!!!!!

!!!!!!!!!!

!

!!

!!!

!!

!

!!!!!!!! !

!

!!!! !

!

!

!

!!

!!

!!!!!

!

!

!! !!

!! !

!

!

!!!!

!

!!

!!

!!

!

!

!

!!

!

! !!! !!!

!

!!! !!!!

!

!!!!!

!!

!! !!!!

!!

!

!

!!!!!!!!!

!

!!!!!

!

!

! !!!!!!!!

!!!!!

!!

!

!!!!!!

!

!!

!

!!

!

!!!

!

!

!!

!! !!

!

!! !

!!

!!!

!!

!!!!

!

!

! !

!

!

! !!!!

!

!

!

!!!

!

!

!!

!

!!

!!!!

!!

!!!

!

!!

!!

!

!!

!

!!!

!!

!!!

!

!!

!

!

!

!!

!

!

!!!

!!

!

!!

!

!

!

!! !!!!

!

!!!

!

!!!!!

!

!

!!!!!!

!

!!!!

!!!!!

!!

!!!!!

!!!!!

!

!!!!!

!

!!

!!

!!

!

!!!!!!!!!!

!

!

!

!!

!

!

!!!!!!!!!!!!!!!!!!!

!! !!

!

!!!!

! !!!

!!

!!

!!!!!!!!!!!!!!!

!

!! !!!!!!!!!!!

!

!!

!!

!!! !!!!!!!

!!!

!!!!

!!

!!!

!!!

!

!

!!!!!

!!!!!!!!!!

!!

!

!

!!!!!!!!

!

!!!!!!!!!

!!

!!!!

!!!

!!!

!!!

!!

!

!!

!

!

!

!

!

!

!! !!!

!

!!!!!! !

!

!!

!!

!!!! !

!

!

!!!!!!! !!

!

!

!! !

!!! !!

!

!!!

!

!!! !!!!!!

!

!

!

!!!!!!!!

!

!!!

!

!!!

!

!

!

!!!!!!!!

!

!!!

!!!!

!

!

!

!!!!

!!!!

!

!!!!!!!!!!!!

!!!!!!!!!!!!!!!

!

!!

!

!!!

!

!!!

!!!!!!!

!

!!

!

!!

! !

!!!!!!

!!!

!

!!!!!!

!

!

!

!

!

!!!!

!

!!!!!!!!!!!

!

!!

!

!!!

!

!

!!!

!

!!!

!!! !!!

!

!!!!!

!!!!!!!!!!!!

!!!!

!!!!!!!!

!!!!!!!

!

!

!!!!!!!!!!!!!!!

!!

!!

!

!!!!

! !!!

!

!!

!!!! !

!

!

!

!!

!!!

!!

!

!

!!!

!!!!

!!

!

!

!!!

!!!

!!!

!

!

!

!

!!!!!!

!!!!!!!!!

!!!!!!!!!

!

!!!!!!!

!

!!!

!!!!

!

!!!!!

!!!!!

!!!

!

!!

!!!

!

!

!

!!!

!!

!!

!!

!

!!!!!

!

!

!!!

!

!!

!!!!

!

!!!!

!

!

!!

!!!!

!

!!!

!

!

!!

!!!!!!!!!!!!!!

!!

!!

!

!

!

!!

!!

!!!!

!!!!

!

!!!

!!!!!!!!!!!!!!!!

!!

!!!!!!

!!!

!!!!!

!

!

!

!

!

!

!

!

!!!

!

!!!!!

!

!!

!

!!!!!

!

!!!!!!!!! !!

!

!!!!!

!

!!!!!!!!!!!

!

!

!!!

!

!!

!

!!!!!!

! !

!!

!

!

!!!

!!!

!

!!!!!!

!

!

!!

!!

!!!!!!!!

!

!!

! !

!!!!!

!!!

!!

!!!!

!!

!

!!!!!

!

!!

!!!!

!

!!

!!!

!

!!!

!!!!!!!!!!

!

!

!

!!!!

!!!!!!!!!

!

!!

!!!!!!!!!

!!!

!

!

! !!!

!!!!!

!!!!!!!!

!!

!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!

!!!!!!!!!

!!!! !! !!!!!!!!!!!

!!

!!!!!!!

!!!!

!!

!!!!!!

!

!!! !!!!

!!

!!!!!!

!!!

!!!

!!

!!!!

!!!

!!!!!!!!!

!!!

!!

! !!!!!!!!!!

!!!!

!

!

!!

!!!!!!

!!!!

!

! !!!! !!!!!!

!

!

!!!

!!

!!!!

!!

!!!

!

!

!!

!

!

!!!!

!!!!!

!

!

!

!!! !!

!!!

!

!!!!!!

!

!

!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!

!!!!!!!! !

!

!!!!!!!!!!

!!!

!

!!!

!!!!! !!!!!!

!

!!!

!!!!!!!!

! !!!!!

!!!

!!! !

!

!!!!

!!!!!!!!!! !

!!!

!!

!!!!

!

!!!!!

!!

!!

!

!!!!!!!

!

!!

!

!!!! !

! !!!

!

!!!!

!

!!!!!!!!!

!

!

!!

!!!!! !!!

!

!

!!

!

!!

!!

!!!

!!

!!!!

!!

!

!!

!!!!!! !!

!

!

!

!!!

!

!!!

!

!!!

!!!!!!!!!

!!

!

!

!

!

!!!

!!!!

!

!!!!

!!!

!!

!!!!!!!

!!!!

!!

!!

!!

!!

!!

!!!

!

!

!

!!

!

!

!

!

!!

!

!!

!

!!

!!

!

!

!!

!!

!!

!!!!!!

!!

!!

!

!

!!!!!!!!! ! !!!!!!!!

!

!!!

!

!

!

!

!

!

!!

!

!!!

!!!

!!

!!

!

!

!!!

!

!!!

!

!!

!

!!!!

!!

!!

!!

!

!

!!!!!!!!

!!!!!

!!!!!!! !! !!

!!!!!!

!!!

!!!

!!!!

!!

!!!!

!!!!!!!

!!!!

!!!

!!!

!!!!!!! !!

!!!

!

!!!!

!!!

!

!!!! !!

!

!!

!!!!

!

!!!!! !!!

!

!!!

!!!!

!!!!

! !!!

!!!!

!!!! !!!!

!!!

!

!!!!!! !!

!!!!

!!!!!

!!!

!!!!

!!!!

!!!!

!!!!

!!!!!!!!

!!!!

!!!!!!

!! ! !!

!

!!!!!

!

!!!

!!!!!!

!!!!!!

!

!

!!!!

!!!

!!!

!!

!

!!!

!!!!

!!

!!! !!! !

!

!

!!!!!!

!!!!!!

!!!

!!!

!!!!!

! !!!!!! !

!

!

!!!!!!

!!!

!!!

!!!

!!!

!!!

!!!!!!

!!!

!!!

!!!

!

!!!!!

!!!

!!!

!!!

!!!

!!!!!

!!!! !!!

!!

!

!

!!

!!! !!!

!!!!!!

!!!!!!!!!

!!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!

!!!

!!!!!!

!

!!

!

!!!

!! !!!!!!

!!!

!!! !!!!

!!

!!!

!!!!

!

!

!!!!!!!!

!!!! !!! !!!!!!!!!!!

!!!!!!!!!!

!!! !!!!!!!!!!!! !!!!!!!!!

!!!!!! !!!!!!!!!!!! !!!!!!

!!!

!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!

!!!

!!!!!!!

!

!!!!

!!!

!!

!!!

!

!!!

!!!

!!!!!!

!!!

!!!

!!!

!!! !!!

!

!

! !

!

!

!!!

!!

!

!!!

!!!

!!!

!!

!!!!

! !!!

!

!

!!

!

!!

!!!!!!!

!

!

!!

!

! !!!! !!!!

!!!!!!!!

!

!!! !!!!!!!!

!

!!

!!!!

!! !

!!!!!!!!!!

!!!!!!!!

!

!!!

!!!!!

!

!!

!!!!

!!!!

!!!!

!!!!!

!!!!!!!

!!!!

!!!!!!!!

!

!!!!

!!!

!!!!!!!!

!!!!!

!! !!

!!!!

!!!

!!!!!!!!!

!

!! !!

! !

!!!!!

!!!!

!

!!!!

!!!!

!!

!

!

!

!

!

!

!

!

!

!!

!!

!!!

!!

!!!!!!

!

!!!!! !!!!!!

!!!!!!!

!!

!

!

!

!!

!!

!! !!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!!!!

!

!!!!!

!!!!

!!!!!

!!! !!

!!!!!!

!

!!!

!!

!!

!!

0 1 2 3 4 5 6

−3−1

12

predict(rf)

resi

d(rf)

!

!!!!!!

!!

!

!

!!!!!!

!

!!!!!!!!!!!!!!!!!!

!

!!! !!!!!!

!

!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!

!!!!!!

!

!

!!!!!!!!

!! !!!!!!

!

!

!!!!!!!!

!!

!!!!!!

!

!

!!!!!!!!!

!!!!!!!!!! !! !!!!!

!!

!!!!!!!!!

!!!!!!! !!!!!!!!!

!! !!!!!!!!!!

!!!!!

!!

!!!!!!

!!!!! !!!!!

!!!

!! !

!!

!

!!!!!!!!

!!

!!!! ! !!

!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!

!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!! !!

!

!!!! !

!!

!

!

!

!!

!

!

!!

!

!

!!

!!!!

!

!! !

!!

!!!

!

!

!!!

!

!!

!!

!

!

! !!!!

!

!!!!!!!!!

!

!!!!!!

!!

!!!!!!!!

!

!!!!!!

!!

!!!!!!!!

!

!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!

!

!!!

!

!!!! !!!!!!!!!!!

!!!! !

!!!!!!!!!!!!!!!!!!

!!!

!!

!

!!!

!

!!

!!!

!

!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!

!!

!!!!!!!!!!!!!!!!!!

!

!!

!

!!

!! !!!!

!!!!!! !

!

!!!!!

!!! !

!

!!

!!!!!! !! !

!

!! !!

!!!!

!

!! !

!

!!! !!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!

!!!!!!!!

!!

!!!!!!!!!

!!!!!!!!

!!!!!!

!!!!!!!!!! !!!!!

!!!!!!

!!!!

!!!!!!!

!

!! !!!

! !!! !!!!!!!!!!!!!!

!

!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!

!!

!!!

! !!!!

!!!!!! !

!

!!!

! !!!

!!

!

!!

!!!!!!

!! !!

!! !!!

!!!!

!!

!!

!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!

!!!!

!!!

!

!!!!!!! !!

!!!!

!!! !!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!

!!!

!

!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!

!!

!

!

!!!

!

!!!!!

!

!!!!!

!!! !!!!!!!!!!!!!!!!!!!

!!!!!!!! !!!

!!!

!!

!

!!

!

!!!!!!

! !

!!

!

!

!!

!!!

!

!

!!!!

!!

!!!!!!!!!!!!!!!!

!!

!!!!!!!!!!!!!!!!!

!!!!!!

!

!!

! !!

!

!

!!

!! !

!

!!!!!!!!!!!!!

!

!

!

!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!!

!!

!!

!!!

!

!!!

!

!

!!!!!

!

!

!

!!!

!!! !

!!!

!!

!!

!

!

!

!!

!!!!!

!

!!!!!!!

!!!!!!!

!!!

!!

!

!!

!

!

!!!!

!!

!!!!!

!

!!!

! !

!

!!

!!

!!!!!!!!!

! !

!!!

!

!!

!!!!!

!!!!

!

!

! !!!!!!!!!!

!

!

!!!!

!!!!!!! !!!

!

!!!

!!!!!!!!!!!!

!

!!!! !

!

!!!

!

!!

!

!!!

!

!

!!!!!!!

!

!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!

!!!!!!!!

!

!!!!!

!!!!!

!

!!

!

!!!

!

!!!

! !!!!!!

!

!

!! !!! !

!!!!

! !!!!

!!

!!!!!

!

!

!!!!

!

!!!!!!

!!! !!!!

!!

!!!!

!

!!!!!

!

!! !

!

!

!!!!!!

!

!!

!

!!! ! !!

!!!!

!!!!

!

!!!!!! !!

!

!

!

!

!!!!!! !!!

!

!

!

!

!

!

!

!

!!!

!

! !!!

!!

!! !

!!

!!!

!!

!!!

!

!

!!!!

!!

!

!

!

!!!!! !

!!!! !!!!!

!

! !!!

!!! !! !!!!!

!!!

!

!!!!!!!!! !!! !!

!!

!!!!!!!!!!

!

!

!!

!!

!

!

!!!!!!!!!!

!

!

!!

!!!!!!!!!!!!!!!!

!!!!!!!!

! ! !!!!!!!! !!!!!!

!!

!

!

!!

!

!!!

!!!!

!!!!!

!!!

!

!!! !!!!!!!!

!!!!!!!!!!!!

!!!!!!!!!

!!!!!!! !!!!!!!!!!

!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!! !!!!

!

!

!! !!!!!!!!!! !!!!!

!

!!!!

!!!!! !!! !!!!!!!!!!!!

!!

!!

!!!!

! !!!

!! !!

!!!!!!!!

!! !!

!!

!

!

!!!!!!!!

!!!!

!!!!!!!!!!!!

!!!!!

!!!

!!!!!!!!!!

!!!

!!!!!!

!!

!

!!!!!! !!!!!!!

!!

!

!

!!!!

!!!

!!!

!

!

!

!!!!!!

!!!

!!

! !!! !!!!!!!

!!

!!!!!!!!!!

!! !!!!!!

!!!!

!!

!!!!!!!!!

!!! !!!!!!

!!!

!!!

!!!!

!!

!!!!!!!!!

!

!!

!!!!!!!!!!

!! !!!!!!!!

!!!!!!!!!!

!

!!

!!!!!!!!!

!!

! !!!!!!!!!

!!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!

!!

!!

!

!

!!

!

!!!

!!!

!!!!! !!!

!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!! !!!!!!!!!

!!! !!!!!!!!!!!! !!!!!!!!!!

!!!!! !!!!!!!!!!!! !!!!!!!!!!!! !!!!!!!!!!!! !!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!

!!!!!!!!

!

!!!!!!

!!

! !

!

!

!!

!!

!

!

!!

!

!!!!

!!

!!!

!!!

!!

!

!

!

!

!!!!!!

!!!!!!

!

! !

!

!

!

!!!! !!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!

!

!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!

!

!!

!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!! !! !

!!!!

!!!!!!!!!

!!!

!!!!!!!!

!! !

!

!

!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!! !

!!!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!

!!

!!

! !!!! !!!!!!!!!!!!!!!!!!! !!!! !!!

!!!!

0.5 0.6 0.7 0.8 0.9 1.0

−0.1

0.1

0.3

invlogit(predict(rf))

resi

d(rf,

inv

= TR

UE)

!

!!!!!!

!!

!

!

!!!!!!

!

!!!!!!!!!!!!!!!!!!

!

!!! !!!!!!

!

!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!

!!!!!!

!

!

!!!!!!!!

!! !!!!!!

!

!

!!!!!!!!

!!

!!!!!!

!

!

!!!!!!!!!

!!!!!!!!! ! !! !!!!!!

!!!!!!!!!

!

!!!!!!! !!!!!!!!!

!! !!!!!!!!!!

!!!!!

! !!

!!! !

!!!!!

!!!!!!

!!!!

! !

!!

!

!!!

!!!!! !

!

!!! ! ! !

!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!! !!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!

!!

!

!!!!!

!!

!!

!

!!

!

!!!

!

!

!!

! ! !!

!

!! !

!!!

!!

!

!

!!!!

! !

!!

!

!

! !!!!

!

!!!!!!!!!

!

!!! !!!

!!

!!!!!!!!

!

!!!!!!

!!

!!!!!!!!

!

!! !!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!

!

!!

!!

!!!!!!!!!!!!!!!!!!! !

!!!!!!!!!!!!!!!!!!! !!!!

!

!!!

!

!!

!!!!

!!

!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!

!

!!

!

!!

!! !!!!

!!!!!! !

!

!!!!!

!!! !

!

!!!!!

!!! !! !

!

!! !!!! !!

!

!! !

!

!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!

!!

!!!!!!!!!

!!!!!!!!!!

!!!!

!!!!!!!!!!!!!!

!!!

!!!!!!!!!

!! !!!!

!

!! !!!! !!! !!!!!!!!!!!!!!

!

!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!! !!

!!!

! !!!!

!!!!!! !

!

!!!

! !!!!

!

!

!!

!!!!!!

!! !!

!! !!!

!!!!

!! !

!

!!!!!!!!!!!! !!!!!!!!!!!!

!

!!!!!!!!

!!!!!!!

!

!!!!!!!!!

!!!

!

!

!! !!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!

!!

!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!

!!

!

!

!!!!

!!!!!

!

!!!!!

!!! !!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!

!!

!!!

!

!!

!

!!!!!!

!!

! !

!

!

!!!

!!!

!!!!

!!!

!!!!!!!!!!!!!!!!

!!

!!!!!!!!!!!!!!!!!

!!!!

!!

!

!

!

!!!

!

!

!!

!

!! !

!!!!!!!!!!!!!

!

!

!

!!!!!!!!!! !!!

!

!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!!

!!! !

!!

!!

!!!

!!

!!!! !

!!

!

!!!

!! !!

!

!!

!!

!!

!

!

!

!!

!! !!!

!

!!! ! !!!

!! !!

!!!!

! !!

!

!

!!!

!

!!!!!

!

!!!!!

!!!!

!!

!

!!

! !! !!!!!!!!

! !

!!!

!

!!

!!!!!!

!!!!

!

! !!!!!!!!!!

!

!

!!!!!!!!!!!!!!

!

!!!

!!!!!!!!!!!!

!

!!!! !

!

! !!

!

!!!

! !!

!

!

!!!!!!!

!

!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!

!!!!!!!

!

!

!!! !!!!

!!!

!

!!

!

!! !

!

!!!! !!! !!!

!

!!! !!!! !!!

!

! !!!!

!

!

!!!!! !

!

!!

! !

!

!! !!!!

!!!!!

!!!!

!!!!

!

!!!!

!

!!

! !

!

!

!!

!!

!!

!

!!

!

!!!! !

!!!

!!

!!!!

!

!!!!!!!

!!

!

!

!

!!!!!! !!!

!

!

!

!

!

!

!

!

!!!!! !!!

!!

!! !

!!

!!!

!!

!!!

!

!

!!!!

!!

!

!!

!!!!!!!!!!!!!

!!

!

!!!!! !!!

! !!!! !!!!

!

!!!!!!!!!!!! !!

!!

!!!!!!!!!!

!

!

!!

! !

!

!

!!!!!!!!!!

!

!

!!

!!!!!!!!!!!!!!!!

!!!!!!!!

!! !!!!!!!! !

!!!!!

!!

!

!

!!

!!!!

!!!

! !!!!!

!!!

!

!!! !!!!!!!!

!!!!!!!!!!!!

!!!!! !!!!!!! !!!! !!!!

!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!! !!!!

!

!

!!!!!!!!!!!! !!!

!!

!

!

!!!

!!!!! !!! !!!!

!!!!!!!!

! !!!

!!!!

!!!!

!!!!

!!!!!!!!!!!!

!!

!!

!!!!!!!!

!!!!

!!!!

!!!!

!!!!!!!!!

! !!!!!!!!!!!!

!!!

!!!!!!!

!

!

!!!!!! !!!!!!!!!!

!

!!!!!!!

!!!

!!

!

!!!

!!!!!!

!!! !!! !!!!!!!

!!

!!!!!!!!!!

!! !!!!!!

!!!!!

!!!!

!!!!!!!

!! !!!!!!!!!

!!!

!!!!

!!

!!!!!!!!!

!

!!!!!!!!!

!!!

!! !!!!!!!!

!

!!!!!!!!!

!

!!

!!!!!!!!!

!!! !!!!!!!!!

!!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!

!!

!!!

!

!!

!

!!

!

!!!

!!!!! !!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!! !!!!!!!!!!!

!!!!!!!!!!

! !! !!!!!!!!!!! ! !!!!!!!!!

!!! !!! !!!!!!!!!!!! !!!!!!!!!!!! !!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!

!!!!!!!!!

!!! !!!

!!

! !

!

!

!!!

!

!

!

!

!

!!!!!

!!

!!!!

!

!

! !!

!

!

!

!!!!!! !!!!!!

!

!

!!

!

!

!!!! !!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!

!

!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!

!

!! !!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!

! !!

!!!!!!! !!

!!!!!!! !!! !

!

!

!

!

!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!

!!!!

!!!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!!

!!!

!

!!!!! !!!!!!!!!

!!!!!!!!!!

!!!!!!!

!!!!

0.5 0.6 0.7 0.8 0.9 1.0

−0.2

0.0

0.2

0.4

invlogit(predict(rf))

resi

d(rf,

inv

= TR

UE)

!

!!!!!!

!!

!

!

!!!!!!

!

!!!!!!!!!!!

!!!!!!!

!

!!! !!!!!!

!

!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!

!!!!!!

!

!

!!!!!!!!

!! !!!!!

!!

!

!!!!!!!!

!!!!!!!

!!

!

!!!!!!!!

!!!!!!!!!!

!!!!!!!!

!!

!!!!!!!!

!

!!!!!!!

!!! !!!!!!

! !!!!!!!!!! !!

!!!!! !

!!!

!! !!

!!!!!!!!!

!!!

!!!

!!

!

!!!!!!!! !

!

!!!! ! !

!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!

!!!!!!!!!

!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!

!!

!!

!!

!!

!!

!

!

!

!!!

!

!

!!

!! !!

!

!

! !

!!

!!!

!

!

!!!!

! !

! !

!

!

! !!!!

!

!!!!!!!!!

!

!!! !!!

!!

!!!!! !!!

!

!!!!!!

!!

!!!!!!!!

!

!! !!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!

!

!!!!

!!!!!!!!!!!!!!

!

! !!!!

!!!!!!!!!!!!!!!!!!!

!!

!!

!

!!!!

!!

!!!

!!!

!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!

!!

!!!!!!!!!!!!!!!!!!!

!

!

!

!!

!! !!!

!!!!!!! !

!

!!

!!!

!!! !

!

!!!!

!!!! !! !

!!! !

!!! !!

!

!! !

!

!!!!!!!!!!!

!!!!!!!!!

!!!!!!!!!!

!!!!!!!!!

!!!!!!!!

!!

!!!!!!!!!

! !!!!!!!!!!!!

!!!!!!!!!!!!!!!

!

!! !!!! !!!!!

!!!!!!

!

!! !!!! !

!! !!!!!!!!!!!!!!!

!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!

!!!

!! !!!

!!!

!!!! !

!

!!!!!

!!!

!

!

!!

!!!!!!

!! !

!!! !

!!!

!!!

!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!

!!!!

!!!

!

!!!!!!!!! !!!

!

!!! !!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!!

!

!

!!!!

!!!!!

!

!!!!!

!!!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!

!!

!!!

!

!!

!

!!!!!!

! !

! !

!

!

!!!

!!!

!

!!!!

!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!

!!!!!!

!

!!

!!!!

!

!! !!

!!

!!!!!!!!!!!!!

!

!

!

!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!!

!!

!!!!!

!

!!! !!

!! !!!

!

!

!

!

!!!

!

!!!

!!!

!

!

!!

!

!

!!

!!!!!

!

!!! !!!!

!!

!!!!!

!!!!

!

!

!

!!

!

! !!

!!

!

!! !!!

!

!!!! !

!

!!

! !!!!!!!

!!!

! !

!!!

!

!!!!!!!!

!!!!

!

! !!!!!!!!!!

!

!

!!!!

!!!!!!!!!!

!

!!!

!!!!!!!!!!!!

!

!!!! !

!

!!!

!

! !!!!

!

!

!

!!!!!!!

!

!!!!!

!!!!

!

!!!!!!!!!!!!!!!!!!

!!!!!!! !

!

!!!!!!!!!!

!

!!

!

!!!

!

!!!

! !!! !!!

!

!

!!!!!

!!!!

!! !!

! !

!

!

!!!

!!!

!

!

!

!!

!

!!!!!!

!!!!!!!

!!

!!!!

!

!! !!

!

!

!! !

!

!

!!

!!!!

!

!!

!

!!

!! !

!!!

!!

!!!!

!

!!!!!!!

!!

!

!

!

!!!!!! !!!

!

!

!

!

!

!

!

!

!

!!!

!!

!!

!!

!! !

!!

!!!!! !

!!

!

!

!

!!!!!

!

!

!

!!! !!!!!!!!!!

!!

!

!!!!!

!!!! !!!!

!!!!

!

!!!!!!!!

!!!! !!

!!!!!!!!!!!!

!

!

!!

!!

!

!

!!!!!!!!!!

!

!

!!

!!!!!!!!!!!!!!!!

!!!!!!!!

! ! !!!!!!!! !

!!!!!

!!

!

!

!!

!!!!

!!! !

!!!!!

!!!

!

!!! !!!!!!!!!!!!!!!!

!!!!!!!!! !!!!!!!!!!! !!!!

!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!! !!!!!

!

!! !!!!!!!!!! !!!!!

!

!!!!

!!!!! !!! !!!!!!!!!!!!

! !!!

!!!!

!!!

!!!!!

!!!!!!!!!! !!

!!!!

!!!!!!!!

!

!!!

!!!!

!!!!

!!!!!!!!!

!!!

!!!!!!!!!!

!! !!!!!!!!

!

!

!!!!!! !!!!!!!!!

!

!

!

!!!!!!

!!!

!!

!

!!!!!!

!!!

!!! !!! !!!!!!!!

!

!!!!!!!!!!

!! !!!!!!

!!!!!!

!!!

!!!!!!!!! !!!!!!

!!!

!!!

!!!!!

!

!!!!!!!!!

!

!!!!!!!!!!!!

!! !!!!!!!!

!

!!!!!!!!!

!

!!

!!!!!!!!!

!!

! !!!!!!!!!

!!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!

!!!!!

!

!!

!

!!

!

!! !

!!!!! !!!

!!!!!!!! !!!!!!!!!!!!!!!!!

!!!!! !!!!!!!!!!!!!!

!!!!!!!!!!

!!! !!!!!!!!!!!! !!!!!!!!!

!!!!!! !!!!!!!!!!!! !!!!!!!!!!!! !!!!!!!!!!!! !!!!!!!!!!!!!!!!!!

!!!

!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!

!!!!!!!!

!!!!!!!

!

!

! !

!

!!!

!!

!

!

!!

!

!!!!!!

!!!

!

!!

!!!

!

!

!

!!!

!!

! !!!!!!

!

!

!!

!

!

!!!! !!!!!!!!!!!!!!!! !!!!!!!!

!!!!!!!

!

! !

!!!!!!!!!!!!!!!!!!

!!!!!!!!!

!

!!

!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!

!!!!

!!!!

!!! !!!!!!

!!!

!!!!!!!!

!! !

!

!

!

!

!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!! !!!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!

!!!

!

!!!!! !!!!!!!!!

!!!!!!!!!!!!

!!!!!

!!!!

0.5 0.6 0.7 0.8 0.9 1.0

−0.1

0.1

0.3

invlogit(predict(rf))

resi

d(rf,

inv

= TR

UE)

!!!!!!!

!!!!

!!!

!!!!

!!!!!!!!!!

!!

!!!!!!

!!!! !!!!!!

!

!!!

!!!

!!!!!!!

!

!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!

!!

!!!!!!

!!

!!!!!!!!

!!!!!!!

!!

!!!!!!!!

!!

!!!!!!

!!

!!!!!!!!

!!!!!!!!!!! ! !! !!!!!!!!!!!!!!!

!!!!!!!

! !!!!!!!!!!

! !!!!!!!!!!!!!!!!! !!!!!!

!!!!! !!!!!

!! !!

! !

!!

!!!!!!!!

!!

!!!

!! ! !!!!!!!!!!

!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!

!!!!!!!!!!

!

!!!!!!!!!!!!!! !!

!

! !!! ! ! !!!!

! !!

!!!

!

!!!

!!!!

!!! !

!!!!!

! !!!!!

! !

! !!

!! !!!!

!

!

!!!!

!!

!!!

!!!!!!

!!

!!!!!!!!

!!!

!!!!

!!!!!!!!!

!!!! !!!!

!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!! !!!!!!

! !

!! !!!!!!!!!!!

! !!! !!!!!!!!!!!!!!!!!!!!

!!!!!!! ! !

! !!! !! !! !!!!!!!!!!!!!!!

!!! !!!!!!!!!

!!!!!

!!!!!!!!!!!!!!!!!!!

!!!!!!!!

!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!

!!

!!!!!!!!!!!!!!!!

!!

!

!!!

!!!! !!!

!!!!!!! !

!

!!

!!!

!!! !!

!!

!!!!!! !!

!

!!! !

!!!!!

!

!! !

!!!

! !!!!!!!!

!!!!!!!!!

!!!!

!!!!

!!

!!!!!!!!!

!!!!!!!!

!!

!!!!!!!!!

!!!

!!!!!!!!!!

!!!!!!!!!!! !!!!! !!

!!!!!

!!!

!

!!!!!!!!!

!!!

! !!! !!!!!!!!

!!!!!!!

!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!

!!!!

!!

!!!! !!!

!!!!!!! !!

!!

!! !!!

!!

!

!!

!!!!!!

!! !!

!! !!!

!!!!

!

!!!

!!!!!!!!!!!!!!!!!!!!!!!

!!

!!!!

!!!

!!!!

!!!!

!

!!!!

!

!! !!!

!! !

!!!

!!!

!

!!!!!!!!!!!

!

!!!!!

!!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!!!!!

! !!!

!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!

!!

!!!!!!!

!

!!!!!!

!!

!!!

!!!!

!!!!!!!!!!!

!!!!!!!

!!!!!!!! !! !!

!!!!!

!!!

!!!!!!! !

! !!

!!!!

!!!

!

!!!!!! ! !

!!!!!!!!!!!!!

!!! !

!!!!!!!!!!!!!!!!

!!!!!!!! !! !! !! !! !! !!!

!!!!!!!!!!!!

!!

!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!! !! !

!!! !!! !! !!! !! !

!!!!!!

!!

!! !! ! !!!! !

!!!

!!

!! !!!!!!

!

!!! !!!!!!!!!!!

!!!!!!!

!!

!!!!!

!!!!!! !!!!!

! !!

! !! ! !!!!

!!!!!! !

!!!!

!!!!!!!!

!!!!

!! !!!!!!!!!!

!

!!!! !

!!

!!!!! !!!

!

!!!

!!

!!!!!!!!!!

!

!!!

! !!

! !!

!! ! !!!!!

!

!!!!!!!!

!!!! !!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!! !

!!!!!!!!!!!

!!!

!!! !

!!!! ! !!!!!!

!

!! ! !!! ! !!! !! !!! !!

!!! !!! !!

!!!!

!!!!!!!

!!! !!!!!!!!!!

!!!!!

!

!!

! !

!!

!!!!!!

!!!

!!!

! ! !! !!

!!

!!!!

!!!!!!

! !! !

!

!!

!!!!!! !!!!!! !

!!

! !! !!! ! !!!!!!!

!!!

! !!! !! !!!

!!

!!!!!!

!

!

!!!!! !!!!! !!

!!!!

! !!!

! !! !! !!!!!

!!!

!! !!!!!!!! !!!

!!

!!

!!!!!!!!!!

!

!

!!

! !

!!

!!!!!!!!!!

!

!

!!

!!

!!

!!!!!!!!!!!

!

!!!!!!!!

! ! !!!!!!!! !!!!!

!

!!

!!

! ! !!!!

!!!! !!!

!

!!!!

!!!! !

!!!!!!!

!!!!!!!!

!!!!!!!!!!!!!

!!!!!!! !!!!

!!!!!!!!!!!!

!!! !!!!!!!!!!!!!!!!!!!!!

!!!

!!!!!!! !!

!!!

!!! !!!!!

!!!

!! !!!

!!

!!!!!!!!!! !!! !!!!

!!!!

!!!!

! !!!!!!!

! !!! !! !!!!!

!

!!!!!! !!

!!! ! !!!!!!!!! !!!

!!!!!!!!!!!!

!!!!! !!!!!!!!!!!!!

!! ! !!!!!!!!

!!!!!!!!!!!!!

!

!

!!

!!

!!!

!!!

!!!

!!!

!!!!!!! !!

!!!

!!! !!

!

!!!!!!

!!!!!!

!!!!

!!!!!

!!!!!!!!

! !!!

!!!!!!!!!

!!!!!!

!!!

!!!

!!!!!!!!!

!!!!!!

!!!

!!!!!!

!!!!!! !!!

!!!!!! !!!!!!!!!

!

! !

!!!!!!

!!!!

!!

!!!!!!!!!

!!

!

!!!

!

!! !

!

!!

!!

!

!

!

!

!!!!

!!!!

!

!

!!

!!!

!

!!!

!!!!! !!!

!!!!!!!

!!

!!!

!!!!!!!!!!

!!!

!!!!!!!!!!!!!!!!!!!

!

!!!!

!

!!!

!

!!!!!!!

!

!!!!!!! !!!!!!!!!!!!

!!! !!!!!!!!!!!!

!!!!!!

!!!!!!

!!!!!!!!!

!!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!!

!

!!!!

!!!

!!!!!!!!!

!!!!!!

!!!

!!!

!!!!!!!!!!!!!!! !

!

! ! !!!!

!!!

! !!!

!!!! !!!!!

! ! !!!! !!!

!!! !!!!

!!

!! !

!

!! !!!! !!!!!!

!!!!!!!!!! !!!!!!!!

!!!!!!!

!! !!!!!!!!!!!!!!!!!!!!!!!

!!!!!

!!!

!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!

!

!!!!!!! !!!!!!! !! !

!!!!

!!!!!!!!

!!!!

!!!!!!!!!

! !

!

!

!

!

!!

!

!!!!!!!!!!

!!!!!!!

!!!!!!!!!!!

!!!!!!!

!!!

!!!!!

!! !!!!!!

!!

!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!!!!!!!

!! ! !! !!!! !!!!!!!!!

!!!!!!!!!! !!!! !!!

!!!!

0.5 0.6 0.7 0.8 0.9 1.0

−20

−10

0

invlogit(predict(rf))

resi

d(rf)

/resi

d.se

(rf)

!

!!!!

!!

!!

!

!

!!!

!!

!

!

!

!!!

!!

!

!

!

!

!

!

!!

!

!

!

!

!

!!!

!!

!

!!

!

!

!!

!

!

!

!

!

!!

!

!!!

!

!

!

!

!

!!!!

!

!

!

!!

!!!!!

!

!

!

!!!!!!!!!!

!!!!!!!!

!

!!!!!!!

!

!

!

!!!!

!

!

!

!

!!!!!!!

!

!!

!!!!!

!

!

!

!!

!!!!!

!

!

!

!!!!!

!

!

!

!!

!!!!!

!

!!!!!!!!!

!!

!!

!!!!!

!!

!!!

!!

!!!!

!!!!!!

!!

!!!!!!

!!

!

!!

!!!!!

!!!!

!

!!!!

! !!

!!

! !

!

!!!!

!!!!!!

!!!

!! !

!!

!

!!

!!!!!

!

!

!

!!! !

!

!

!

!

!!

!

!

!!!

!!

!

!

!!!

!

!

!!!

!

!!!

!

!

!!

!!

!!

!

!

!

!!

!

!

!

!!

!!

!

!

!!!!!!!

!

!!!!!

!

!!!!!!!!

!

!

!

!

!!!!!!!!

!!!

!

!!

!

!

!!!!!!!

!

!!

!

!!! !!

!

!!!!!

!!

!!

!

!!

!

!!!

!

!

!!

! ! !

!!

!! !

!!!!!

! !

!!!!

!

!

! !

!

!

!

!

!!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!

!!

!!

!

!

!

!

!!!

!!

!

!!

!

!!

!

!

!

!!

!

!

!!

!

!!

!

!

!

!

!

!

!!

!!!!

!

!!!

!!!!!!!!

!!!!!

!

!!

!!

!

!

!

!!!

!!

!!!!!

!

!!!!

!

!!!!!

!

!

!!

!

!

!

!

!

!!!!!!!!!

!

!!!

!

!

!

!!!!!!

!

!!!!!!

!

!!!!!

!!!!

!

!!!

!

!

!

!!!

!

!!

!!!!!!!!!!!!!!!

!

!!

!!!!!!!!!

!!

!

!!

!!

!!!!!!!!!

!

!!

!

!!!!

!!

!!!!!

!

!

!

!

!!!!!

!

!!!

!!!!!

!

!

!

!

!!!!

!

!!

!!

!!!!!!

!

!

!

!!

!

!

!!!!!

!

!!!!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!!!

!

!

!!

!

!

!

!!

!

!

!

!

!

!!

!

!!!!

!

!

!

!! !

!

!

!!

!

!

!!

!

!

!!

!

!!!!!

!

!

!

!

!!!!!!!!

!

!!!

!

!!!

!

!

!

!!!!!!!!

!

!!!!!!!

!

!

!

!!!!!!!!

!

!

!!

!

!

!!!!

!!!

!

!!!!

!!!!

!!

!!!!

!

!!

!

!!

!

!

!!

!

!

!!

!!!

!

!

!!

!

!!

!!

!!

!!!!

!!!

!

!!!!!!

!

!

!

!

!

!

!

!!

!

!

!!!!!!!!!!

!

!!

!

!!

!

!

!

!!!

!

!!

!

!!!!!

!

!

!!!!

!

!!

!!!!!

!!!

!!

!

!

!!

!!

!

!

!!!

!

!!!!!!!

!

!

!!!!!!!!!!!!!!!

!!

!!

!

!

!!

!

!!

!!

!

!!

!!!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!

!!

!

!

!!

!

!!

!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!!!

!

!!

!

!!!!!!!

!!

!

!!!!

!!!

!

!!

!!

!!!

!

!!

!!

!

!

!!!

!

!!

!

!

!!

!!!

!

!

!!!!!

!!!!!

!

!!!!!

!

!!!!!!!

!!!!!!!

!!!

!

!!!!!!!!

!!!

!

!!

!!!!!!!!!

!!!!!

!!

!!!

!

!

!!

!

!

!!!

!

!!!!!

!

!!

!!!!!!!!!!

!!!!!!

!

!!!!!!!!!

!

!!!!!

!

!

!

!

!

!

!

!

!!!

!

!!!!!

!

!

!

!

!

!

!!!

!

!!!!!

!!!!!!

!

!

!!!!

!

!!!!!!!!!

!

!

!

!

!

!!

!

!!

!

!

!!!!

!

! !

!!

!

!

!!

!

!!!

!

!!!

!!

!!

!

!!

!!

!

!!!

!

!!!!

!!

! !

!

!!

!!

!!!!!!!!!

!!

!

!!!

!!

!

!

!

!!!

!

!

!

!

!

!!

!

!

!!

!!!!!!!!!!

!

!

!

!

!!!!

!!!

!!

!!!

!

!!

!!!!!!!

!!!!

!!

!

!!!

!!!

!!!!!!!!!!!!

!!!

!!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!

!

!

!

!!!

!!

!

!!

!

!

!!!!

!

!

!!!!

!

!

!

!

!!

!! !

!!

!

!!

!

!!

!

!

!

!

!

!!!

!!

!

!!!! !!!

!

!!!

!!!

!!

!

!

!

!

!!!!

!!!!!!

!!!!!!

!!!

!

!!

!

!

! !!!

!!

!!!!!

! !!!

!

!

!

!

!!!!

!

!

!!!

!

!

! !!!!!!!!!!

!

!

!!

!

!

!

!

!!

!!

!

!!

!

!

!

!!

!

!

!

!!

!

!!

!

!!!

!

!

!!

! !

!

! !!

!

!!!

!!!

!

!

!

!!!

!!!

!

!!!!!!!!!

!

!!!!!!!!!!!!!!!!!

!

!!!!!!!

!

!

!!! !!!!

!!!

!

!!

!

!!

!

!

!!!! !!!!!!

!

!

!! !!!! !!!!

! !!!!

!!

!!!!

! !

!

!

!

! !

!

!!!!!!

!!!

!

!

!!

!!

!!

!!

!

!!!

!

!

!

!

!!

!

!

!!

!!

!!

!

!!

!

!!!! !

!

!!

!!

!

!!!

!

!!!!!

!!

!!

!

!

!!

!!!!! !!!

!

!

! !

!

!

! !

!!!

!!

!!!!!

!

!

!

!!

!!!

!!

!

!!

!

!

!

!!!

!

!!

!

!

!

!!!

!!

!!!!!!!

!

!

!

!

!

!!!!

!!!

!

!!!!!

!!

!

!

!!!!!

!!!!!!

!!

!!

!!

!

!!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!!!

!!

!!

!

!

!!

!!

!!

!!

!

!!!

!!

!!

!

!

!!!

!

!!!

!

!! !!!!!!!!

!

!

!!

!

!

!

!

!

!

!!

!

!!!

!

!!

!!

!

!

!

!

!!!

!

!!!

!

!!

!

!!!!

!

!

!!

!

!

!

!

!!

!

!!!!!

!!!!

!

!!!

!

!!

! !!!

!

!!

!

!!!

!!!!

!!

!!! !

!

!

!!!!

!!!!!!!

!

!!!

!!

!

!

!

!

!!

!!

!

!!!!!

!!

!

!! !!

!!

!

!

!

!

!!

!!

!

!

!

!

!

!!!

!!!!

! !

!!

!

!!!

!!!!

!

!!!

! !!!

!!

!

!

!!!! !

!!!

!!!

!

!!!

!

!!!

!

!!

!!

!

!!

!

!

!!!

! !!!

!

!

!!

!

!!!

!!!!

!

!!!

!

!!!

!

!!

!

!!

!!!!

!!

!!!

!

!

!!

!

!

!

!

!

!!

!!

!

!

!

!!!

!

!

!

!

!

!

!!

!

!

!

!

!!!

!

!

!

!

!

!

!!!!!

!

!

!

!!!!

!!

!

!!!

!

!

!

!!!

!!!

!

!!

!

!

!

!!!

!!

! !!

!!!

! !

!

!

!!!

!!!

!

!!

!!!

!!!

!!

!

!!!

!!!!

!!

!!!

!!

!

!

!!

!

!!

!!!

!

!!

!!!

!

!

!

!!!

!

!

!!!

!

!!!!!!

!!

!

!

!

!

!!!!

!!

!!!

!

!

!

!!!

!!!

!

!!

!!

!

!!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!

!!!

!

!!

!!!

!

!

!

!

!

!

!!!

!

!

!

!!!!

!!

!

!

!!!! !

!

!

!!!!!!!!!!!!

!

!!!!!!!!

! !!

!!!!!!!!!

!

!

! !!

!!!!!!!

!!!

!!

!!!!!!!!!!

!!

!

!!

!!!!

!

!!

!!

!

!!!!!!!!!

!!

!

!!!!!!

!!!

!!!!

!!!!!

!

!!

!!!!!!!

!

!

!!!

!!!

!

!

!!

!

!

!!!

!

!

!

!!!

!!!

!

!!

!!!

!

!!

!!! !

!!!!

! !

!

!

!!

!!

!

!

!

!

!

!!

!

!

!!

!!!!!!

!!

!!

!

!!!!

!!!

!!!!

!!

!

!

!

!

!

!!!

!!!!!!

!!

!!!!!

!

!

!!!!!!!!!!!

!

!!!!!!

!

! !

!!!

!!!

!!!!!!!!!

!!!

!!!!

!!!!

!

!

!!

!!!!

!

!

!!

!!!!!!!!!!!!!!!!

!!

!

!

!!!!!!!!!!!!

!

!!!

!!!!!!!!

!!!!

!

!!!!

!!!!

!!!!!!!!!

!!!

!

!!!!

! !

!

!!!!

!

!! !!

!!!!

!!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!!

!!!!

!

!!!

!

!!!!!!!!!!!!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!

!!!

!

!!

!!!

!!!!!!!!!

!!!!

!

!!!!!!!!!!!!

!!!!

0.5 0.6 0.7 0.8 0.9 1.0

−20

12

invlogit(predict(rf))

resi

d(rf)

/resi

d.se

(rf)

!

!

!!!

!

!

!!

!

!

!!

!

!!

!

!

!

!!!!!

!

!

!

!

!

!!!

!

!

!

!

!

!!

!

!!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!!

!

!

!

!!

!!!!!

!!

!

!!!

!!!!!!

!!!!!!

!

!!

!

!!!!!!!

!

!

!

!!!!

!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!!!

!!!

!!!

!

!!

!

!!!!

!!

!

!!

!!

!!!

!

!!!!

!

!

!

!!

!

!!

!!

!!

!!

!!!!!!

!!

!!

!

!!!!

! !

!

!

!

!!

!

!

!!!

!!!

!!

!

!

!!

!

!!

!

!

!

!!!!!!!

!

!

!

!!

!!

!!

!

!

!!

!

!

!!!

!!

!

!

!!!

!

!

!!

!

!

!!!

!

!

!!

!

!

!!

!

!

!

!!

!

!!

!!!!

!

!

!!!!!

!

!

!

!!!!!!!!!!!!!!

!

!

!

!

!!!

!

!!!!!

!!

!!!

!

!

!!!!!!!

!

!

!!!

!!!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!!

!

!

!!

!

! !!

!

!

!!

!

!

!

!!

! !

!!!

!

!!

!!

!

!

! !

!!!

!

!

!

!!

!

!

!

!!

!

!

!

!

!!!

!!

!

!!

!

!!

!!

!

!

!

!

!!!

!!

!

!!

!

!!

!

!

!

!!

!!!!

!

!!

!

!

!

!!

!

!!!!!!

!

!!

!

!

!!!!!

!!

!!!!!!

!

!!!

!

!

!

!!!

!!

!!!!!

!!!!!

!

!!!!!

!

!

!

!!

!!

!

!

!!!!!!!!!

!

!

!

!!

!

!

!!!!!!

!

!!!!!!

!

!!!!

!

!

!!

!

!

!!

!!

!

!

!! !

!

!!

!!!!!!!!!!!!!!!

!

!!!!!!!!

!!!!!

!

!!

!!

!!!!!!!!!

!

!

!

!

!!!!

!!

!!!

!!

!

!

!

!

!!!!!

!

!!!

!!

!!

!!

!

!

!

!!!!!

!!!!

!!!!!!!

!

!

!!

!

!

!!!!!

!

!

!

!

!!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!!!

!

!

!

!

!

!

!

!!

! !

!

!

!

!!

!

!!

!

!

!

!

!

!! !

!

!!

!!

!

!!

!

!

!!

!!!!!!

!

!

!

!

!!!!!!!!

!

!!!

!

!!!

!

!

!

!!!!!!!!

!

!!!!!!!

!

!

!

!!!!!!!!

!

!

!

!

!

!

!!!!

!!!

!

!!!!!!!!

!!

!!!!

!

!!

!

!!

!

!

!!

!

!

!!

!!!!

!

!!

!

!!

!

!

!!

!!!!

!

!!

!

!

!!!!!

!

!

!

!

!

!!

!!

!

!

!

!!!!!!!!!!

!!

!

!!

!

!

!

!!!

!

!!

!

!!!!!

!!

!!!!

!

!!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!

!!

!

!!

!

!!!!!

!

!

!

!!

!

!!!!!!!!!!!!

!

!

!

!

!

!

!!!

! !

!!

!

!

!

!!!! !

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!!

!

!

!

!

!

!!!

!!

!

!!

!

!

!

!

!

!!

!!

!!

!

!!!!!!!

!

!

!!!!!!!!

!

!!!!

!!!

!

!!

!

!

!!!

!

!

!

!!

!

!!!!

!

!!

!

!

!!

!!!

!

!

!!!!!!!!!!

!

!!!!!

!

!!!!

!!!

!!!!!

!!!!!

!

!!!!!!!!!!

!

!

!!

!!!!!!!!!!!!!!

!!

!

!!

!

!

!!

!

!

!!!!

!

!!!

!

!!

!

!!!!!!!!!!

!!!!!!

!!

!!

!!!!!!!

!

!!!!

!

!

!

!

!!

!

!

!!

!

!

!!!!

!

!

!

!

!

!!

!!!

!

!!!!!!!!!!!

!

!!

!!!

!

!!!!!!!!!!

!

!

!

!

!!

!

!!

!

!

!!!!

!

!!

! !

!

!

!!

!

!!!

!

!!!!

!!!

!

!!

!!

!!!!!!!!!

!!

!!

!

!!

!!

!!!!

!!!!!

!!

!

!!!!

!

!

!!

!!!!

!

!! !

!!

!

!

!!!!!!!

!!!!!

!

!

!

!

!!!

!!!!!!!!!

!

!

!

!!!!!!!!!!!

!!

!

!!!!!!!!!!!!!!!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!!

!!!!!

!

!! !!! !!

!

! !!!

!

!

!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!

!

!!! !!!

!

!

!

!!!

!!

!

!!

!

!

!

!

!!

!

! !!!

!!

!!!!!

!!!!

!!!

!

!

! !!!!!

!!!!!

! !!!

!

!

!

!

!!!!!

!!!

!

!

!

! !!!!!!!

!!!

!

!

!!

!

!

!

!

!!!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!!!

!!

!

!

!

!!!

!

!

!!

!

!

!!!

!!!

!

!

!!!!

!!!!

!!!!!

!!!!

!

!!!!!!!!!!!!!!!!!

!

!!!!!!! !

!

!!!

!!!!!

!!

!

!!

!

!!!

!

!!!

! !!!!!!

!

!

!!

!!!

!!!!

!! !!

! !

!!

!!!

!!!

!

!

!

!!

!

!!!!!

!

!!!

!

!

!!

!!

!!!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!!

!!

!

!!

!

!!

!! !

!

!!

!!

!!!

!

!

!!!!!

!!

!!

!

!

!

!!!!!!

!!!

!

!

! !

!

!

!!

!

!!

!!

!!!

!!

!!

!

!!

!!

!!!! !!

!

!

!

!!!

!

!!

!

!

!

!!!

!!!!!!!!!

!

!

!

!

!

!!!

!!!!

!

!!!!!

!!

!

!

!

!!!!!!

!!!!

!!

!!

!!

!!!

!

!!

!!

!

!

!!

!

!

!

!

!!

!

!!

!

!!

!!

!

!

!!

!

!

!!

!!

!

!!!

!

!

!!

!

!

!!!!!!

!

!

!!

!!!!!!!

!!

!

!!

!

!

!

!

!

!

!!

!

!!!

!

!!

!!

!

!

!

!

!!!

!

!!!

!

!!

!

!

!!!

!!

!!

!

!

!

!

!!

!

!!!!!

!

!!!!

!!!

!!!!

!!

!!

!!!

!!!

!!!

!

!!

!!! !

!

!

!!!!

!!!!!!!

!

!!!!!

!

!

!

!

!!

!!

!

!! !

!!

!!

!

!! !!!!!

!

!!

!!

!!

!

!

!

!

!

!!!

!!!!

! !!!

!

!!!

!!!!

!!!!

! !!!

!!

!

!

! !!! !

!!!

!!!

!

!!!!

!

! !

!

!!!!

!!!!!

!!

!

!!!!

!

!!!

!

!!!

!

!!!

!!!!

!!!!

!!!!

!!

!!!!

!!

!!!

!

!!!

!

!

!

!

!

!

!!!

!

!

!

!!!

!

!

!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!!

!!! !

!

!

!!!

!

!

!

!!!

!

!

!

!

!!

!

!

!

!!!

!

!

!

!

!!

!!

! !

!

!

!!!

!!!

!

!!

!!!

!!!

!!

!

!!

!

!!!!

!

!

!!!

!!!

!!!

!

!!

!!!

!

!

!

!!

!

!

!

!

!!!

!

!

!!!

!

!!!!!!

!

!

!

!

!

!

!!!!!!

!

!!

!

!

!

!!!

!!!

!!

!

!!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!

!

!

!

!!

!

!!!

!

!!!!

!!!

!

!!

!!

!!

!

!

!

!

!

!!!

!

!

!

!!!!

!!

!

!

!!!!

!

!

!

!!!!!!!!!!!!

!!!!!!!!!

!!!

!!!!!!!!!!

!! !!!!!!!!!

!!

!

!!

!!!!!!!!!!

!

!!

!!!

!!!

!!!

!!!

!!!!!!!!!

!!

!

!!!!!!

!!!

!!!!!!!!!

!

!!

!!!!!!

!

!

!

!!!

!!!

!

!

!!!!

!!!

!!!

!!!

!!!

!

!

! !

!

!

!!

!

!!

!!!!

!!

!!

!

!

!

!!

!

!

!

!!

!

!!!

!

!!

!!!

!

!!

! !!

!

!

!

!

!

!

!

!

! !!!!

!!

!

!

!

!

!

! !!!!!!!!

!!

!!!!!

!

!

!!

!!!

!!!!

!!

!

!!

!!!!

!

! !

!!!

!!!

!!!!!!!!!

!!!

!!!!

!!!!!

!

!!

!!!!

!!!!

!!!!

!!!!!

!!!!!

!

!

!

!

!!

!

!

!!!!!!

!

!!!!

!!!

!!

!

!!

!!!

!!!!

!!!

!!

!!!!

!!!

!!!!!!

!!!

!

!!!!

!!

!

!!!!!

!!

!!

!!!!

!!!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!

!

!!!

!

!!!!!!!!!!!!!

!

!

!

!

!

!

!

!

!

!!!

!!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!

!

!!! !

!

!

!!!

!!!!!!!!!

!!!!!!!

!!!!

!

!!

!!!

!!!!

0.5 0.6 0.7 0.8 0.9 1.0

−20

12

invlogit(predict(rf))

resi

d(rf)

/resi

d.se

(rf)

!

!!!!

!!

!!

!

!

!!

!

!!!

!

!

!!

!

!!

!

!!

!

!!

!!!!

!

!

!

!!!!!!

!!

!

!

!

!

!

!!!

!

!!!!!!

!

!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!!

!

!!!!!

!

!!!

!!!

!!!!!

!

!

!

!!

!!!!!

!

!

!!!!!

!

!!

!

!!

!

!

!!

!

!!!!!!

!

!!!!

!

!!!!

!

!

!

!!

!

!

!

!

!!!

! !!!!

!

!! !

!

!!

!!

!

!!!!!!!

!

!

!

!!

!!

!!

!

!

!!

!!!!!

!!

!!

!!!!

!!!!

!

!!!!

!!!

!!

!!

!

!

!

!!

!

!!!!

!

!!

!

!!!!!!!

!

!!!!!

!

!!!!!!!!!!

!

!!!!!!

!!!!!!!!!

!

!

!!!!!!!!!!!!

!!!!

!

!!

!! !

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!!

!

!

! !

!!

!!

!

!!

!!!

!

! !

!!

!

!

!!!!!

!

!

!

!!!

!

!

!!

!

!!!!!!

!

!

!!!!!

!!!

!

!!

!

!!!

!!

!!!

!

!!

!

!

!

!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!

!!!!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!!!!

!

!

!

!!!

!

!!!!!!

!

!!!!!!

!

!!!

!

!

!

!!!

!

!!

!

!

!

!

!!

!

!

!!

!!!!!!!!!!!!!!!

!

!

! !!!!!!!!!!!!!!

!!

!!!!!!!!!!

!!!!!!!

!!

!!!!!!

!

!

!!!!!!

!!!!!!!!!!

!!

!

!!!!!!!!!!!!!!!!!!

!!

!!!!!!!!

!!!!!!!!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!!!!

!

!!

!

!

!

!!! !

!

!

!

!!

!

!!

! !!

!

!

!! !

!

!

!

!

!

!

!!!

!

!!

!

!!!!!!

!

!

!

!!!!!!!!

!

!!!

!

!!!

!

!

!

!!!!!!!!

!

!!!!!!!

!

!

!

!!!!!!!!

!

!!

!

!

!!!!

!

!

!!

!

!!!!!!

!

!

!

!!!

!!!

!!

!!!

!

!

!!

!

!

!!

!!!!

!

!!

!

!!

!!

!

!

!!!!!!!!!!!!!!

!

!

!

!

!

!

!!!

!

!

!!!!!!!!!!!

!!

!

!!!

!

!

!!!

!

!!

!!!!!!

!

!

!!!!!!!!!!!!!!!!!

!!

!!!!!!!!!!!!!!!!!

!

!

!!!!!!!!!!!!!!!

!!

!!

!

!

!!

!

! !!!

!

!!

!!!!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!!

!!

!!

!! !

!

!

!!

!!

!

!!

!

!

!

!

!

!!

!!

!!

!!!!!!!!

!

!!!!!!!

!

!

!

!!

!!

!

!!

!

!!

!!

!!

!

!

!!!!

!

!

!

!!

!

!!

!

!

!

!

!!!

!

!!!!!!!!!!!

!

!!!!!

!

!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!!

!!!

!

!

!!

!

!

!!!!!!!!

!

!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!

!!

!

!

!!

!

!

!!!!!

!

!

!

!

!

!

!!!

!

!!!!!!!!!!

!

!

!

!!!!

!

!!!

!!!!!

!

!

!

!

!

!

!!

!

!!

!

!!!!!

!

! !

!!

!

!

!!

!

!!!

!

!!!!

!!

!

!

!!!!!!!!!!!!!

!!

!!

!!!

!!!!!!!!!!!!

!

!!

!!!!

!

!!

!!

!

!

!

!

!!

!

!

!

!

!!!!!!

!!!!!!

!

!

!

!

!!

!

!!!!!!!!!

!

!!

!!!!!!!!!!!!

!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!!

!!!

!!!

!!

!!!

!

!

!

!!!!

!

!

!

!

!!!!

!!

!

!

!!

!

!

!

!

!

!

!

!

!!!!

!

!

!!!!

!!!

!!!!

!

!!

!

!!

!

!

!

!

!

!

!

!!!!

!

!

!!!!

!

!

!!!

!!

!

!!

!

!!!

!!

!!

!!!

! !

!!

!

!

!!

!!!!

!

!

!!

!

!

!

! !!!!!!!!!!

!

!

!!

!

!

!

!

!

!!

!

!

!!!

!

!

!!

!

!

!!!

!

!!!!!!

!

!

!!!

!

!

!!!

!

!

! !!!!

!

!

!!!!

!!!

!

!

!!!

!!!!!

!

!

!!

!!

!!!!

!!!!!!!!

!

!!!

!!!

!

!

!

!!!!!

!!!!!

!

!

!

!

!!!

!

!

!!

!

!!!!!!

!

!

!

! !!

!

!

!!!!

!!!!

!!

!

!!

!

!!!

!

!

!

!!

!

!!!!

!

!

!!!

!

!

!!

!!

!!!

!

!

!!!

!

!

!

!

! !

!

!

!!

!!

!!

!

!!

!

!!

! ! !

!!!

!

!

!!!!

!

!!!!!

!!!

!

!

!

!

!!!!!!

!!

!

!

!

!

!

!

!

!!

!!!

!

! !!!

!

!

!!

!

!

!

!!!

!!

!!!

!

!

!

!!!

!

!

!

!

!

!

!!

!!

!

!!

!! !!

!

!

!

!

!!

!!

!

!!

!! !

!

!!!

!

!!

!

!

!

!!!

!!!

!

!!!

!!

!!

!!!!!!

!!!!

!

!

!!

!!

!

!

!!

!!!!

!!

!!

!

!

!!

!!

!!

!!!!!!!!!!

!

!

!!

!

!

!!!

!

! ! !!!!!

!!!

!

!

!!

!

!

!

!

!

!

!!

!

!!!

!!!

!

!!

!

!

!

!!!

!

!!! !

!!

!!!!!

!!

!!

!!!

!

!!!

!!!!

!

!!!!!

!!!

!!!! !!!!

!!!

!!!

!!!

!

!!

!!! !

!!!!!!!!!!!!!!!!!!!!

!

!!

!!!!

!

!! !!!!

!

!

!!

!!!!!

!

!!!

!!!

!

!!

!

!

!!!

!!!!! !!!

!!!!

!

!!!

!!!!

!!

!!

!!

!

!

! !!!

!!

!!

!!!

!

!!!!!! !!

!!

!

!

!!!!!

!!!

! !!!

!

!!!

!

!!!

!!!!

!

!!!

!

!!!

!!!!

!!!!!!

!!!

!!

!!

!!

!

!

!

!

!

!!

!!

!

!

!

!!!

!

!

!

!

!

!

!!

!

!

!

!

!!

!

!

!

!

!!!

!

!!!

!!

!

!

!

!

!! !

!

!

!!!!

!

!

!!!

!!!

!

!!

!

!

!

!!!

!!! !

!!!!

! !

!

!

!!

!!!

!

!

!!

!!!

!!!

!

!!

!!!

!!!!

!

!

!!!

!!!

!

!!

!

!

!

!!!!

!!

!!!!

!

!

!!!

!

!

!!!

!

!!!!!!

!!

!

!

!!

!!

!!

!!

!!

!

!

!

!

!!!

!!!

!!

!

!!

!

!!!

!

!!

!

!

!

!

!

!

!

!

!

!

!!!

!

!!

!!

!

!

!

!

!

!!

!

!!

!

!

!!!!!!!

!

!!

!!!!

!

!

!!!

!!!!

!

!

!!!!

!

!

!

!

!!!!!

!

!

!!!!!!!!!!

!

!

!!!!!!!

!

!!

!!!!!!!!

!

!

!

!!!!!!!!!!

!!

!!

!!!!!!!!!!

!

!!

!!!!!!

!!!

!!!

!!!

!!!!!!

!!

!

!!!

!!!

!!!

!!!!!!!!!

!!!

!!!!!!

!

!

!

!!!

!

!

!

!!!!!!

!

!!

!!!

!!!

!

!

!

!

!

!

!!!!!!

!!!!!!

!!

!!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!!

! !!!

!!

!

!

!!

!

!

!!!

!!!!!!!

!!

!

! !

!

!

!!!!!

!!!!!!

!!!!!

!

!!!! !!!!!!!!

!

!!!!!!

!

!!

!!!

!!!

!!!!!!!!!!!!

!

!!!

!!!!!

!

!

!

!!!!

!!!!

!!!!!!!!!!!!!!!!

!!

!!

!!!!!!!!!!!!!

!!

!

!!!!!!!!!!!!

!!!!

!

!!!!!!!

!!!!!!

! !

!!

!!!!

!!

!!

!!!!

!

!

!

!

!!!!

!!

!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!!!!

!

!!

!

!

!!!!!!!!!!!!!

!

!

!

!

!

!

!

!

!

!!

!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!

!!

! !

!!!!!

!!!!!!!!!

!

!!!!!!!!!!!!! !!!

!!!!

0.5 0.6 0.7 0.8 0.9 1.0

−3−1

13

resi

d(rf,

inv

= TR

UE)

/resi

d.se

(rf, i

nv =

TRU

E)

!

!!!!!!

!!

!

!

!!!

!!

!

!

!

!!!

!!

!

!!

!

!

!!!

!!

!

!

!

!!! !!!

!!

!

!

!!!

!!!

!

!!

!

!!!

!

!

!

!!!!!!

!

!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!!

!!!!!!!!!!

!!!

!!!!!

!!

!!!

!!

!!!

!

!!!!!!

!

!!!!!!!

!!

!

!!!!!!

!

!!!!

!

!!!!

!!!

!!

! !

!

!!!!! !!!

!!

!!!

!

! !

!!

!

!!!

!!!!

!

!

!

!!

! !!

!

!

!

!!

!!!!!!!!

!

!!!!

!!!!

!

!!!!

!!!

!!

!!

!

!

!

!!

!

!!

!!!!

!

!

!!!!!!!

!

!!!!!!!!!!!!!!

!

!

!

!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!!

!!!!

!

!!!!!

!

!

!

!

!

!

!

!

!!!

!

!

!!

!!

!

!

!

!

! !

!!

!

!!

! !

!!!

!

! !

! !

!

!

!

!

!!!

!

!

!

!!!

!!

!!

!

!

!

!!!!

!

!

!!!

!!!!!

!

!!

!

!!!

!!

!!!

!!!!

!

!

!!

!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!

!

!!!

!!

!

!

!

!

!

!

!

!

!

!!

!!!!!!!

!

!!!!

!

!

!!!!!!

!

!!!!!!

!

!!!!

!!

!!!

!

!!

!

!

!

!

!!!

!

!!

!!!!!!!!!!!!!!!

!

!! !!!!!!!!!

!!

!

!!

!!

!!!!!!!!!!!!!!!!!

!!

!!!!!

!

!

!

!!!!!!

!

!!!!!!!!!!

!

!

!!!!!!!!!!!!!!!!

!!

!!

!!!!!!!

!!!!!!!!

!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!!!

!

!

!!

!

!

!

!!

!

!

!

!

!

!!

!

!!! !!

!

!

!! !

!

!

!!

!

!

!!

!

!

!!

!

!!!!!!

!

!

!

!!!!!!!!

!

!!!

!

!!!

!

!

!

!!!!!!!!

!

!!!!!!!

!

!

!

!!!!!!!!

!

!!!

!

!!!!!

!!

!

!

!!!!

!!!!

!!

!!!!

!

!!

!

!!

!!

!!

!

!

!!

!!

!!

!

!!

!

!!

!

!

!!!!!!!!!!!!!!!!

!

!

!

!

!!!!!!

!!!!!!!!!!!!!!

!

!!!

!

!

!!!

!

!!!!!!!!

!

!

!!!!!

!!!!!!!!!!

!!

!

!

!!!!!

!!!!!!!!!!!!

!

!

!!!!!!!!!!!!!!!

!!

!!

!

!

!!

!

!!

!!

!

!!

!!!!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

!!

!!

!

!

!

! !

!!

!

!!

!

!

!

!

!

!!

!

!

!!

!

!!!!

!

!!

!

!!!!!!!

!!

!

!!!!

!!!

!

!!

!!

!!

!

!

!

!

!!

!

!

!

!!

!

!!

!

!

!!!!!

!

!!!!!!!!!!!

!

!!!!!

!

!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!!

!!!

!

!

!!

!

!

!!!!

!!!!!

!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!

!!

!

!

!!!

!

!!!!!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!!!!

!

!

!

!

!!!!

!

!!!!!!!!!

!

!

!!

!

!!

!

!!

!

!!!!!

!

!!

! !

!

!

!!

!

!!!

!

!!!

!

!! !

!

!!!!!!!!

!!!!!

!!

!

!

!!!!!!!!!!!!!!!!

!!!!

!!

!

!

!

!

!!

!

!

!

!

!

!

!!

!

!!!

!!!!!!!!!

!

!

!

!

!!!!

!!!!!

!!!

!

!!

!!!!!!!!!!!

!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!!

!!

!

!

!!

!

!

!!!!

!

!

!!!!

!

!

!

!

!!

!! !!!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!

!!!!

!!!

!!

!!

!

!!

!

!!

!

!

!

!

!!

!

!!!!

!!

!!!!!

!

!!!

!

!!

!!

! !!!

!!

!!!!!

! !

!!

!

!

!!

!!

!!!

!

!!!

!

!

! !!!!!!!!!!

!

!

!!

!!

!

!!!!!

!

!!!

!

!

!!

!

!

!!!!!!!!!!

!

!!!

! !

!

! !!

!

!!

!

!!!

!

!

!

!!!

!!!

!

!!!!!!!!!

!

!!!!!!!!!!!!!!!!!

!

!!!!!!!

!

!

!!!!!

!!!!!

!

!!

!

!!

!

!

!!!!

!!!!!!

!

!

!! !!

!!

!!!

!

!!!!

!

!

!

!!

!!! !

!

!

!

!!

!

!!!!

!

!

!!!

!!

!!

!!

!

!!!

!

!!!

!

!

!

!

! !

!

!

!!

!

!

!!

!

!!

!

!!

!

! !

!

!!

!

!

!

!!!

!

!!!!!

!!

!!

!

!

!

!!!!!

! !!!

!

!

!!

!

!

!!

!!!

!!

!!!!!

!! !

!!

!!!

!!

!

!!

!

!

!

!!!

!

!

!

!

!

!

!!!

!!

!!!!!!!

!

!

!

!

!

!!!

!!!

! !

!!!!!

!!

!

!

!!

!!!

!!!!!!

!!

!!

!!!!!!

!!!!

!

!

!!

!!

!

!

!!

!

!!!

!!

!!

!

!

!!

!!

!!

!!!!!!!!!!

!

!

!!!

!

!!!

!

!! !!!

!!!!!!

!

!!

!

!

!!

!

!

!!

!

!!

!

!

!

!

! !

!

!

!

!

!!!

!

!!!

!

!!

!

!!!!!

!

!!!!!

!

!!!!!!!

!

! !!!!!!!

!

!!! !!!!

!!!

!!!

!!!

!

!!!!! !

!!!!!!!!!!!!!!!!!!!!

!

!

!

!!

!!

!

!!!!!

!!

!

!!

!!!!!

!!

!!

!!!

!

!!

!

!

!!!

!!!!! !

!!

!!!!

!!!!

!

!

!!

! !!!

!!

!

!

!!!!

!

!!!

!!!

!

!!!

!!

!!!

!!

!

!

!

!!!

!

!!!

! !!!

!

!!!

!

!!

!

!!!!

!

!!!

!

!!!

!!!!

!!!!!!

!!

!!!!

!!!!

!

!

!

!

!!

!!

!

!

!

!!!

!

!!

!

!

!

!!!

!

!

!

!

!!

!

!

!

!

!

!

!!!

!!

!

!

!

!!!! !!

!

!!!

!

!

!

!!!

!!!

!

!!

!

!

!

!!!

!!

! !!

!!!

! !

!

!

!!!

!!!

!

!!

!!!

!!!

!

!

!

!!!

!!!!

!

!

!!!

!!

!

!

!!

!

!!

!!!

!

!!

!!!

!

!

!!!!

!

!

!!!

!

!!!!!

!

!!!

!

!

!

!

!!!

!!

!!!

!

!

!

!!!

!!!

!

!!

!!

!

!!!

!

!

!!

!

!

!

!

!

!

!

!

!

!!

!

!

!!

!!

!

!

!

!

!

!!

!

!

!

!

!

!!

!! !!!

!!!

!!!!

!!

!!!

!!!

!

!

!

!!!!!!!

!

!!!! !!

!

!!!!!!!!!!!

!

!!!!!!!!!

!!!

!!!!!!!!!!

!

! !!!!!!!!!

!!!

!!

!!!!!!!!!!

!!

!

!!!!!!

!

!

!

!!

!

!!!!!!!!!

!!

!

!!!!!!

!!!

!!!!!!!!!

!!!

!!!!!!!!!!!!

!

!

!!!!!!!

!!!

!!!!!!!!!

!!!

!!!!

!!

!!! !

!!

!!

! !

!

!

!!

!!

!

!

!

!

!

!!

!

!

!!

!!!!

!

!

! !!!

!

!

!!!

!!! !!!!!!

!

!

!!

!

!

!!!! !!!!!!

!!!!!

!

!!!! !!!!!!!!

!

!!!!!!

!

! !

!!!

!!!

!!!!!!!!!!!!

!

!

!!

!!!!!

!

!!

!!!!

!

!

!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!!!

!

!!!!!!!!!!!!!

!!!

!

!!!!

! !

!

!!!!

!

!!

!!

!!!!

!!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!!

!!!!!!!

!!!

!

!!!!!!!!!!!!!

!

!!

!

!

!!!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!!

!!!

!

!!

!!!

!!!!!!!!!

!!!!!

!!!!!!!!!!!!

!!!!

0.5 0.6 0.7 0.8 0.9 1.0

−20

12

3

resi

d(rf,

inv

= TR

UE)

/resi

d.se

(rf, i

nv =

TRU

E)

!

!!!!

!!

!!

!

!

!!!

!!!

!

!

!!!!!

!

!

!

!

!

!!!

!!

!

!

!

!!!!!!

!!

!

!

!!

!

!!!

!

!

!

!

!!!

!

!

!

!

!!!!!

!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!

!

!

!

!!!!!

!

!

!

!!!!!!!!

!

!!

!!!

!

!!!

!

!!

!

!!!! !

!

!

!!

!

!

!!!

!

!

!!!

!

!

!

!!

!

!!

!!

!!

!

!

!!!!!!!!

!!

!

!!!!

!!

!

!!

!!

!

!

!!!

!!!

!!

!

!

!!

!

!!

!

!

!

!!!!

!!!

!

!

!

!!

!!

!!

!

!

!!

!

!!!!!!!

!

!!!

!

!!!!

!

!!!!

!!!

!!

!!

!

!

!

!!

!

!!!!!!!

!

!!!!!!!

!

!!!!!!!!!!!!!!!

!

!

!!!!!!!!!!!!!!!

!

!

!!!!!!!!!!!!

!!!!

!

!

!!!

!!

!

!

!

!

!

!

!

!!!

!

!

!!

!!

!!

!

!

! !

!!

!

!!

! !

!!!!

! !

!!

!

!

! !!!!

!

!

!

!!!

!!

!!

!

!

!

! !!!

!

!

!!!

!! !

!!

!

!!

!

!!!

!!

!!!

!!!!

!

!

!!!!!!!!!!

!

!!!!!!!!!!

!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!

!!

!

!!!!!

!

!

!

!

!

!

!

!

!

!!

!!!

!!!!

!

!!

!

!!

!

!!!!!!!!!!!!!

!

!!!!!

!

!!

!

!

!!

!!

!

!

!!

!

!

!!

!!!!!!!!!!!!!!!

!

!! !!!!!!!!!!!

!

!!

!!

!!!!!!!!!!!!!!!!!

!!

!!!!!

!

!

!

!!!!!!

!!!!!!!!

!!

!

!

!

!!!!!!!!!!!!!!!!

!

!

!!

!!

!!!!!!!!!!!!!!

!

!

!

!

!

!

!

!

!

!!

!!

!

!!

!

!!!

!

!

!

!

!

!

!!!

!!

!

!

!

!!

!

!!!

!

!

!

!

!! !

!

!!

!!

!

!!

!

!

!!!

!!!!!!

!

!

!

!!!!!!!!

!

!!!

!

!!!

!

!

!

!!!!!!!!

!

!!!!!!!

!

!

!

!!!!!!!!

!

!!

!

!

!!!!!

!!!

!

!

!!!!!!

!

!!

!!!!

!

!!

!

!!

!

!

!!

!

!

!!

!!

!!

!

!!

!

!!

!

!

!!

!!!!!!!!!!!!!!

!

!!

!

!

!!!!

!

!

!!!!!!!!!!!!!

!

!!!

!

!

!!!

!

!!!!!!!!

!!

!!!!!!!!!!!!!!!!!

!

!

!!!!!!!!!!!!!!!!!

!

!

!!!!!!!!!!!!!!!

!

!

!

!

!

!

!!

!

! !

!!

!

!!

!!!! !

!

!

!

!

!!

!!

!

!

!

!

!

!!

!!!

!

!

!

!

!

!! !

!!

!

!!

!

!

!

!

!

!!!!

!!

!

!!!

!!!!

!

!

!!!!!!!!

!

!!!!

!

!!

!

!!

!

!

!!

!

!

!

!

!!

!

!

!!!

!

!!

!

!

!!

!!!

!

!!!!!!!!!!!

!

!!!!!

!

!!!!!!!!!!!!!!!!!

!

!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!

!!

!!!

!

!

!!

!!

!!!!!!!!!

!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!

!!

!

!

!!

!

!

!!!!

!

!

!

!

!

!!

!!!

!

!

!!!!!!!!!

!

!

!

!!!

!

!

!!!!!!!

!!!!

!

!

!

!!

!

!!

!

!!!!!

!

! !

! !

!

!

!!

!

!!

!

!

!

!!!

!! !

!!!!!

!!!!!!!!!

!!

!!

!!!!!!!!!!!!!!!

!

!

!!!!

!

!

!

!

!

!!

!

!

!!

!

!!

!

!

!!!!!!!!!!!!

!

!

!

!

!!!

!!!!!!!!!

!

!

!

!!!!!!!!!!!!!

!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!

!

!

!!

!!

!!!

!

!

!!!

! !!

!

! !!!

!

!

!

!

!!

!

!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!!!

!

!!! !

!!

!

!!

!!

!

!!

!

!!

!

!

!

!

!

!

!

!!

!

!!

!

!!!

!!

!

!!!

!

!

!

!

!

! !!!

!!

!!

!!!

! !

!!

!

!

!!

!!

!!

!

!

!!!

!

!

! !!!!!!!

!!!

!

!

!!

!

!

!

!!!!

!

!!!

!

!

!

!!

!

!

!

!!!

!!!!!

!

!

!

!!!!

!

!!

!

!

!

!!

!!!

!

!

!!!!

!!!

!

!!!!!

!!!!

!

!!!!!!!!!!!!!!!!!

!

!!!!!!! !

!

!!!

!!!!!

!!

!

!!

!

!!!

!

!

!!

!!!!

!!!

!

!

!!

!!!

!!!!

!

!!!!

!

!

!

!!

!

!!!

!

!

!

!!

!

!!!!!

!

!!!

!!

!!

!!

!!!

!

!

!

!

!!

!

!

!

!!

!

!

!!

!!

!!

!

!!

!

!!

!

! !

!

!!

!

!

!!!!

!

!!!!!

!

!!!

!

!

!

!

!!!!!

!!!

!

!

!!

!

!

!

!

!

!!

!

!!!!

!

!

!! !

!!

!!

!!!!

!!

!

!

!

!!!

!

!

!

!

!

!

!!

!

!!

!!

!!!!!

!

!

!

!

!

!!!

!

!!

!!

!!!!!

!!

!

!

!

!!!!!!

!

!!!

!!

!!

!!!!!!

!!!!

!

!

!!

!

!

!

!

!!

!!!!

!!

!!

!

!

!!

!!

!!

!!!!!!

!!!!

!

!

!!!!!!

!

!

!

! !!!!!

!!! !

!!!

!

!

!!

!

!

! !

!

!!!

!

!!

!

!!!

!

!

!!!

!

!!!

!

!!

!

!!!!

!!

!!

!!!

!

!!!

!!!!!

!!!!!

!!!!

!!! !!!!

!!!

!!!

!!!

!

!!!!! !

!!!!!!!!!!!!!!!!!!!!

!!

!

!!!!

!

!! !!!!!

!

!!!!!!!

!

!!!! !!

!

!!

!

!

!!!

!!!!! !!!

!

!!!

!!!!

!!!!

!!!!

!!

!

!

! !!

! !

!!!

!!!

!

!!!!!

!!

!

!

!!

!

!!!!!

!!!

!

!!!

!

!!!

!

!!

!

!!!!

!!!!

!

!!!

!!!!

!!!!!!

!!

!

!!

!

!!!

!

!

!

!

!

!

!

!!

!

!

!

!!!

!

!

!

!

!

!

!!!

!

!

!

!

!

!

!

!

!

!

!

!

!!

!

!

!

!

!

!

!

!!

! !

!

!

!!!

!

!

!

!!!

!

!!

!

!!

!

!

! !!!!

!

!!

!!

!!

!!

!

!

!!!

!!!

!

!!!!!

!!!

!

!

!

!!!

!!!

!!

!

!!!

!!!

!!!

!

!!

!!!

!

!

!

!!

!

!

!

!

!!!

!

!

!!!

!

!!!!!!

!

!!

!

!

!

!!!!!!

!

!!

!

!

!

!!!

!!!!!

!

!!

!

!!!

!

!!!

!

!

!

!

!

!

!

!

!

!!!

!

!!

!!

!

!

!

!

!

!!

!

!!!

!

!!!!!!!

!!!

!!!!

!

!

!!

!

!!!!

!

!

!!!!!

!

!

!

!!!! !

!

!

!!!!!!!!!!

!!

!!!!!!!!!

!

!!

!!!!!!!!!!

!!!!!!!!!!!

!

!

!

!!! !!!!!!!!!

!

!!!!!!!!

!!

!!!!

!!!!!!!!!

!!!

!!!!!!!!!!!!!!!!!!

!!!

!!!!!!!!!!!!

!

!

!

!!!!!!

!

!

!

!!!!!!!!!

!!! !

!!

!!!

!!!!!!

!

!

!!

!

!

!

!!

!

!

!

!!

!

!!

!

!

!!

!!!

!

!!

!!

!!

!

!

!!

!

!

!

!!!!!!!

!

!

!!

!

!!!!!

!!!!!!!!!!!!

!!!! !!!!!

!!!

!

!!!!!!

!

! !

!!!

!!!!!!!!!!!!!!!

!!!!

!!!!!

!

!

!

!!!!

!!!!

!!!!!!!!!!!!!!!!!

!!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!!! !!

!!!!!!!

!!!!!!!!

!

!

!!!!

!!

!

!

!!!!!

!

!

!

!!!!!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!

!!

!!!!!!

!!!!!!!!!!!!!!!!!!!

!

!

!

!

!

!!

!

!!!

!!

!!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!

!!!!!

!

!!! !

!

!

!!

!

!!!!!!!!!!!!!!!!!!!!

!!!!!!

!!!!

0.5 0.6 0.7 0.8 0.9 1.0

−20

12

3

resi

d(rf,

inv

= TR

UE)

/resi

d.se

(rf, i

nv =

TRU

E)

13

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Conditional variable importanceStrobl et al. (2008), BMC Bioinformatics 2008, 9:307.

Predicting the quality of a survey question from its design characteristics Daniel Oberski

●●

● ● ● ● ● ●●

●● ● ● ●

● ● ●

● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

Conditional importance of variables for prediction of reliability coefficientC

ondi

tiona

l im

port

ance

mea

sure

domain

concept

labels

scale_corres

questiontype

stimulusrange_corres

0.00

0.02

0.04

0.06

0.08

dom

ain

conc

ept

labe

lssc

ale_

corr

esqu

estio

ntyp

est

imul

usra

nge_

corr

esnc

ateg

orie

sin

terv

iew

ersh

owc_

horiz

grad

atio

nfu

ture

posi

tion

labe

ls_o

rder

bala

nce

cent

ralit

yfo

rm_b

asic

repe

titio

nco

untr

yav

gwrd

_tot

alns

yll_

tota

lfix

refp

oint

sus

ed_W

H_w

ord

visu

alsc

ale_

basi

csc

al_n

eutr

alop

inio

noth

ersh

owc_

star

tsc

ale_

tran

geus

edsh

owca

rdav

gsy_

tota

lnw

ords

_tot

alsc

ale_

uran

gesh

owc_

ques

tso

cdes

irdo

nt_k

now

com

p_as

sist

show

c_le

tter

nnou

ns_i

ntro

nnou

ns_t

otal

nsub

_que

stsy

mm

etry

know

ledg

eav

gabs

_int

rosh

owc_

boxe

ssh

owc_

over

nabs

t_in

tro

mot

ivat

ion

nabs

t_to

tal

intr

opre

sent

inst

r_re

spon

abso

lute

avgw

rd_i

ntro

avga

bs_t

otal

nwor

ds_i

ntro

subj

ectiv

eop

enco

urag

ela

bels

_gra

mm

inst

r_in

terv

past

intr

_req

uest

num

sub_

intr

o

Predicting the quality of a survey question from its design characteristics Daniel Oberski

●●

● ● ● ● ●● ● ● ●

● ●● ● ● ● ● ●

● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

Conditional importance of variables for prediction of validity coefficientC

ondi

tiona

l im

port

ance

mea

sure

ncategories

domain

concept

labels

scale_corres

range_corres

fixrefpoints

0.00

0.02

0.04

0.06

0.08

ncat

egor

ies

dom

ain

conc

ept

labe

lssc

ale_

corr

esra

nge_

corr

esfix

refp

oint

sst

imul

usin

terv

iew

erba

lanc

efu

ture

labe

ls_o

rder

form

_bas

icpo

sitio

ngr

adat

ion

show

c_ho

rizre

petit

ion

coun

try

scal

e_ba

sic

nsyl

l_to

tal

avgw

rd_t

otal

cent

ralit

ysc

al_n

eutr

alus

ed_W

H_w

ord

show

c_st

art

scal

e_tr

ange

dont

_kno

wsc

ale_

uran

gevi

sual

nwor

ds_t

otal

used

show

card

avgs

y_to

tal

com

p_as

sist

nnou

ns_i

ntro

nnou

ns_t

otal

show

c_le

tter

opin

iono

ther

show

c_bo

xes

show

c_ov

ersh

owc_

ques

tns

ub_q

uest

nabs

t_in

tro

sym

met

rykn

owle

dge

socd

esir

ques

tiont

ype

intr

opre

sent

mot

ivat

ion

avga

bs_i

ntro

abso

lute

nabs

t_to

tal

avgw

rd_i

ntro

nwor

ds_i

ntro

inst

r_re

spon

avga

bs_t

otal

enco

urag

esu

bjec

tiveo

pin

tr_r

eque

stin

str_

inte

rvla

bels

_gra

mm

num

sub_

intr

opa

st

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Back to the example

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Evaluation and future work

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Some of the challenges encountered

• (Coding process)• (MTMM estimation process)• Mapping the old coding system into the new...• Missing data; non-applicable characteristics (→ raimforest)• Multicollinearity, aliasing of interactions (→ RF)• Propagating the uncertainty about the MTMM estimates

(not solved)• Coming up with a reasonable prediction interval (→ RF)• Dealing with main vs. supplementary (estimate an order

effect and assume the coded question is like the mainquestionnaire)

• ...

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Things that still need doing in my opinion• Publish journal articles to let the world know about this

meta-analysis!• Model was cross-validated on ESS R1–3. Also validate

predictions against new ESS MTMM results.• Evaluate practical effect on estimates of interest before &

after correction for reliability and method effect: howdifferent are the estimates in practice?

• Seriously reduce the number of codes• Planned experiments to reduce multicollinearity, aliasing in

the meta-analysis• Propagate uncertainty about the MTMM estimates• Perhaps compare w/ linear regression/other meta-analysis

techniques• ...

Predicting the quality of a survey question from its design characteristics Daniel Oberski

Thank you for your [email protected]

Predicting the quality of a survey question from its design characteristics Daniel Oberski