View
224
Download
0
Tags:
Embed Size (px)
Citation preview
ChristianDormann1
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Substantive & Methodological Problems
in the Study of Stress at Work!
ChristianDormann2
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Overview
1. Introduction
2. The Big 5 Substantive & Methodological
Challenges in Organizational Stress Research
2.1 Overlap (Conceptual & Measurement)
2.2 Confoundedness (Congeneric & Measurement)
2.4 Change (Intraindividual & Interindividual)
2.3 Directedness (Reversed & Reciprocal)
2.5 Time (Longitudinal & Shortitudinal)
3. Take Home Messages
ChristianDormann3
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Why this topic
Demonstrating and understanding causal relations = Central aim of work
and organizational psychology research on stress
Experimental designs are frequently impossible to implement.
Circumventing problems associated with non-experimental designs along
two pathways:
1) Substantive considerations tailored to W&O stress research
2) Methodological considerations useful for various content domains
Introduction
ChristianDormann4
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Why this topic
Three commonly accepted conditions must hold for a scientist to claim
that X causes Y:
1. Time precedence
2. Relationship
3. Nonspuriousness
Experiment
1. The DV is measured after the IV is established
2. Systematic variations in the DV conditional upon the IV
3. Participants are assigned to the IV by random
(Ken
ny, 1979)
Introduction
ChristianDormann5
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Why this topic
(Ken
ny, 1979)Getting Rid of the Relationship Condition
StressorStress
Reaction+ .25
ThirdVariable
+ .50 - .50 r = +.25 + (+.50 * -.50) = +.25 - .25 = 0.00
Introduction
Suppres-sor
Mediator
ChristianDormann6
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
The Content of the Subsequent Presentation
What
- substantive (i.e. related to the content) and
- methodological (i.e. related to design and analysis)
considerations are worthwhile in order to establish
- time precedence (i.e. the ordering of IV and DV) and
- nonspuriousness (i.e., rejection of third variable explanations)
among
- variables in work and organizational stress research allowing for
- causal explanations?
=> let’s go ahead
Introduction
ChristianDormann7
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Conceptual& Measurement
Overlap
Overlap
ChristianDormann8
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Two Questions Addressed in this Section
1) How should stress and related concepts be defined?
2) How should stress and related concepts be measured?
Overlap
ChristianDormann9
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Defining „Stress“Defining Stress
- stimulus/situation based
- reaction/person based
- transactional
Examples:
- Kobasa (1979): Stressful life events cause changes in, and
demand readjustment of, an average person’s normal routine.
- Selye (1936): Stress is the body’s nonspecific response to a
demand placed on it.
- Lazarus (1966): Stress occurs when an individual perceives that
the demands of an external situation are beyond his or her perceived
ability to cope with them.
Overlap
ChristianDormann10
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Defining „Stress“Problems with Stimulus-based Definitions of Stress
- no possibility to describes qualitatively different types of stress on
the same scale (e.g., relocating and divorcing)
- no model available to allow combinations of different types of stress
- no consideration of interindividual differences in stress appraisal
- no consideration of interindividual differences in responses to stress
- everything might be regarded as stress response (even a non-
response)
Problems with Response-based Definitions of Stress
- no universal stress response pattern identified yet
- if universal stress responses existed, their meaning were probably
not unique (e.g., blood pressure due to fear vs. joy vs. exercise)
Overlap
ChristianDormann11
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Defining „Stress“Solutions:
- Probabilistic model: “Average” person; (cf. Holland’s 1986
fundamental problem of causal inference and his concept of average
causal effects):
Stressors represent stimuli that cause stress reactions, on average
- Different conceptual perspectives:
+ Defining a processes versus
+ Defining variables
- Different terms:
+ stressor (or demand – take care!)
+ resources
+ stress reaction (strain)
Overlap
ChristianDormann12
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Defining „Stress“
Example: Work-Family Conflict
(Stressor, Stress, Stress Reaction, Resource, Process)?
Work-family conflict is a form of interrole conflict in which the role pressures
from the work and family domains are mutually incompatible in some respect.
That is, participation in the work (family) role is made more difficult by virtue of
participation in the family (work) role.
Overlap (Gre
enhaus & B
eutel, 1985)
ChristianDormann13
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Measurement of „Stress“The exact definition of a variable represents a prerequisite for developing
adequate measures.
Exercise: Develop 1-2 items in order to measure the following:
- family role-pressure
- work role-pressure
- mutual incompatibility
- difficulty
- participation in the work role
- participation in the family role
Definition: Work–family conflict is a form of interrole conflict in which the role
pressures from the work and family domains are mutually incompatible in some
respect. That is, participation in the work (family) role is made more difficult by virtue
of participation in the family (work) role
Overlap
ChristianDormann14
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Measurement of „Stress“Examples:
Carlson, Kacmar, & Williams (2000): Due to all the pressures at work, sometimes when I come home I am too stressed to do the things I enjoy.
Netemeyer Boles, & McMurrian (1996): I have to put off doing things at work because of demands on my time at home.
Carlson, Kacmar, & Williams (2000): I am often so emotionally drained when I get home from work that it prevents me from contributing to my family.
Haun & Dormann (2008): I feel I have not enough time for both my work and my private life.
Haun & Dormann (2008): To what extent is your work life (nonwork life) to blame, when you encounter problems reconciling your work and your nonwork life?
What do these items actually measure?
What are potential problems?
Overlap
ChristianDormann15
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Avoiding Overlap
Conclusion
We have to define non-overlapping concepts and measure them without
overlap in order to avoid obtaining trivial (i.e. spurious) relations!
Solutions
- Measuring stressors, resources, stress reactions objectively (take
care!)
=> The End of this Section
- Measuring (adequately!!) confonding variables and controlling (i.e.,
partialling) them:
=> The Transition to the Next Section
Overlap
ChristianDormann16
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Objective Measurement of „Stress“
Conceptions of Objective Stressors and Resources:
1) Measurement without psychological processes involved
=> problem then: psychological stressors and resources can never be
objective (only useful for material objects and processes)
2) not illusory or unreal
=> problem then: the concept of objectivity is too broad (everything not
illusionary) to be useful. Applies only when psychopathological phenomena
are proposed
3) not being related to a specific individual's cognitive and emotional
processes (perceptions, appraisals etc).
=> problem then: concepts such as “average person” is required to
conceptually leave out “individual” cognitive processes.
Overlap
ChristianDormann17
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Ways to Measure „Stress“ Objectively
Measuring stress-variables that are unrelated to one specific individual's cognitive and emotional processes (perceptions, appraisals etc):
1) independent observers (i.e., randomly assigned) => several problems enumerated on the next slides
2) descriptive instead of evaluative items=> more or less easy to develop, amount of objectivity unclear
3) dependent observers (e.g., colleagues, supervisors, spouses)=> difficult to obtain, and amount of objectivity unclear. For example
+ colleagues use their own (related) working conditions to infer the level of stress reactions of the target’s stress reactions
+ halo errors of supervisors (e.g., an attractive subordinate feels excellent even though he was given the most demanding
tasks)+ similarity in personality characteristics (e.g., N or NA) of spouses
cause similar biases in measurement (e.g., my wife’s time pressure is very high; my wife feels poor)
Overlap
ChristianDormann18
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Objective Measurement of „Stress“
Problems with observer ratings:
1) Limited time of observation (work shift, work cycles, any kind of cycle)
2) Unobservability of mental processes (e.g., high concentration)
3) Effects of observation on work behavior
4) Representativeness of workplaces (sampling frequently restricted to
good/typical workplaces/hours etc)
=> Conservative estimates:
All of these four reasons lead to a decrease of the correlation between
observed stressors and stress reactions (compared with the true
correlation). Reason: The true variance of stressors is underestimated.
Overlap (Frese &
Zapf, 1
988)
ChristianDormann19
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Objective Measurement of „Stress“ in Questionnaires
Note: Any kind of 'subjective' questionnaire report can be placed somewhere on a dimension from 'low in dependency on cognitive and emotional processing' to 'high in dependency on cognitive and emotional processing.
Examples:
Time Pressure: - How frequently do you have to skip regular breaks because of high time pressure?
+ The time pressure at work makes me feel fatigued.
Social Support: - When I have too meet tough deadlines I can count on my colleagues.
+ My colleagues are very responsive when I need help
Burnout: - Working all day is really a strain for me.+ I have headaches.
Overlap (Frese &
Zapf, 1
988)
ChristianDormann20
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Objective Measurement of „Stress“ in Questionnaires
Overlap
ObjectiveStressor
PerceivedStessor
WellBeing
Appraisal Coping
obj./perc.Resource
Stressor R
eduction
Appra
isal
Opt
ions C
oping Options
Well Being Enhancement
ChristianDormann21
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Measurement of „Stress“There ain’t such thing as method variance without variance in the method.
Since we usually do not vary our method of data gathering between individuals (e.g., some using questionnaires and others being observed), we better should not speak of method variance.
The things we are really talking about are psychological or psycho-physiological variables, which vary between individuals and impact on the variables as they are measured. These psychological or psycho-physiological variables have been labeled measurement-confounding variables.
When these psychological or psycho-physiological variables would impact on the variables irrespective of how they were measured, they represent congerneric confounding variables (what we usually call third variables).
Sometimes it is difficult to decide whether a confounding variable represents a measurement confounder or a congenenic confounder (e.g. negative affectivity, NA) => let’s go for the next section
Overlap
ChristianDormann22
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Congeneric& Measurement
Confoundedness
ChristianDormann23
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Three Questions Addressed in this Section
1) What does confounding and related terms (spuriousness, third variable-effects) mean?
2) By which mechanisms are relations among IVs and DVs confounded?
3) How to control for confounding?
Overlap
ChristianDormann24
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Confounding (Third) Variables
Several so-called ‘third variables’ can be distinguished.
A confounding variable (CV) is a variable that meets three conditions:- Cond. 1: The CV must cause the independent variable (IV) - Cond. 2: The CV must cause the dependent variable (DV) - Cond. 3: The product of the two effects must be identical to the sign of the causal effect the IV has on the DV.
A suppressor variable (SV) is similar to a confounding variable. Condition 3 is different, however:- Cond. 3: The product of the two effects must be opposite to the sign of the causal effect the IV has on the DV.
A third variable in general terms is a variable, for which Condition 3 does not have to apply, and for which one out of the two first conditions can be relaxed:- Cond. 1/2: The CV is either correlated with the IV or with the DV.
Confoundedness
ChristianDormann25
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Third Variables
StressorStress
Reaction+
Introduction
+ +
Con-founder
-+
+-
Suppres-sor
+-+-
-++-Me-
diator
≠ 0 ≠ 0
ThirdVariable
+ +
Con-founder
Mo-derator
ChristianDormann26
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Confounding (Third) Variabls
“One of the factors that complicates efforts to understand method effect variables is that they may have both
methodological and substantive roles"
Confoundedness (William
s, Gavin, &
William
s , 1996, p
. 88)
ChristianDormann27
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Confounding (Third) Variabls
Measurement Confounding Mechanisms:
- affect priming-model: An individual’s emotional state elicits affect-
congruent memories, which in turn influence his or her questionnaire ratings.
- affect as information-model: Individuals become aware of their current
affect as a reaction elicited by questionnaire items. They regard their affective
state as an indication about how they feel regarding the item content.
Confoundedness (cf. Brie
f, Butch
er and Ro
bertson, 19
95)
ChristianDormann28
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Confounding (Third) Variabls
Congeneric Confounding Mechanisms:
-instrumental/stressor-creation mechanism: Individuals high in NA exhibit
behavior, which leads to situations with more stressors.
-selection-mechanism: Individuals with high NA may select themselves (or are
selected) for jobs with more stressors.
Confoundedness (e.g., Sp
ector, Za
pf, Chen, &
Frese, 2
000)
ChristianDormann29
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Confounding (Third) Variabls
Congeneric Confounding Mechanisms:
- perceptual/hyper-responsivity mechanism: Individuals with high NA
perceive more stressors and more strain. NA enhances stressor perception and
strain perception.
(Strictly speaking, this mechanism applies to neither the measurement-
confounding nor the congeneric model because NA is supposed to affect the
relation between stressors and strains, but neither the constructs themselves
nor their measurements.
Instead, this model implies an interaction between “objective” stressors and
strains on the one hand and NA on the other).
Confoundedness
(Schaubroeck, G
anster, & F
ox, 1992; Spector, Z
apf, Chen, &
Frese,
2000; Watson &
Clark; 1984; W
atson & P
ennebaker, 1989)
ChristianDormann30
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Confounding Congeneric Model
Overlap
StressorStress-reaction
+ .25
Congen. Conf.
Item
1
Item
2
Item
3
Item
1
Item
2
Item
3
+ .50 + .50
r = + .25 + .50 * .50
= + .50 *)
*) when analyzed without controlling for congeneric confounding variable.
(William
s et al., 1996)
ChristianDormann31
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Confounding Measurement Model
Overlap
StressorStress-reaction
Item
1
Item
2
Item
3
Item
1
Item
2
Item
3
Meas. Conf.
+ .25
(William
s et al., 1996)
+ .50
r = + .625 *)
*) when analyzed without controlling for measurement confounding variable.
ChristianDormann32
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Pseudo-Confoundedness
Take Care:
Beware of a logical fallacy: Even if partialling NA would reduce the stressor-
strain relations, this does not prove that NA actually confounded the
relationship.
Explanation: A common confounding variable exists (meta confounder), which
does not only affect stressors and strains, but NA (or any other supposed third
variable), too.
Mechanims:
- mood mechanism
- causality mechanism
Confoundedness
ChristianDormann33
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Pseudo-Confoundedness
Mood Mechanism
A measurement-confounding model of mood: Mood does not only affect
measures of stressors and strains, but also measures of NA.
Therefore, partialling NA will reduce stressor-strain correlations, even though
NA is not the confounding variable.
Causality Mechanism
NA does not affect stressors; it is just the other way round.
Confoundedness
ChristianDormann34
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Meta Confounding Measurement Model
Overlap
StressorStress-reaction
Item
1
Item
2
Item
3
Item
1
Item
2
Item
3
Meta Conf.
Pseudo Conf.
Item
1
Item
2
Item
3
ChristianDormann35
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Controlling for Confounding (Third) Variables
1) Usually, explicit measurement and control of confounding variables is
required to avoid biased estimates of causal relationships.
2) Controlling for congeneric confounding variables is straightforward. Multiple
(hierarchical) regression analysis, ANCOVA, structural equation modeling
etc.
3) Controlling for measurement confounding variables is more demanding:
- avoid measurement overlap
- structural equation modeling
4) Some confounding variables can be explicitly (or are implicitly) controlled
when change is analyzed
=> let’s go for the (quite complex) next section
Overlap
ChristianDormann36
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Interindividual& Intraindividual
Change
Change
ChristianDormann37
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Three Questions Addressed in this Section
1) Should Interindividual Differences be used to substitute for Intraindividual Change?
2) Should difference-scores be used as measures of change?
3) Could change be used to control for confounding variables?
Change
ChristianDormann38
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Cross-sectional analysis is based on some doubtful assumptions:
All men were created equal and any difference is due to change.
All men once had identical job conditions, and any difference …
All men once suffered from identical complaints, and any difference …
From this it follows:
1) Any current value of a variable can be interpreted as a change-score.
2) The relationship between two variables informs about how change in one variable caused a change in the other variable.
When the assumptions do not apply, then:
Potential confounders are not controlled appropriately.
Appropriately addressing change may help controlling confounders.
Change
Why is it worth to look at change?
ChristianDormann39
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
The Seven Commandments of Animal Farm
1) Whatever goes upon two legs is an enemy.
2) Whatever goes upon four legs, or has wings, is a fried
3) No animal shall wear clothes.
Change
ChristianDormann40
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
The Seven Commandments of Animal Farm
4) No animal shall sleep in a bed.
5) No animal shall drink alcohol.
6) No animal shall kill another animal.
Change
ChristianDormann41
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
The Seven Commandments of Animal Farm
7) All animals are created equal.
Change
ChristianDormann42
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
The Seven Commandments of Panel Farm
1) Whatever goes upon two legs is an enemy. 1) Whatever goes not beyond one measurement occasion is an enemy.
2) Whatever goes upon four legs, or has wings, is a fried2) Whatever goes beyond one measurement occasion, or is an experiment, is a friend.
3) No animal shall wear clothes. 3) No student shall get weary of checking results closely.
Change
ChristianDormann43
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
The Seven Commandments of Panel Farm
4) No animal shall sleep in a bed. 4) No student shall sleep at all
5) No animal shall drink alcohol. 5) No student shall drink alcohol
6) No animal shall kill another animal.6) No student shall kill time
- except he/she has to cope with data from more than one measurement occasion.
- except has to wait for Time 2 to come (will be soon).
- except he/she is sleeping over his/her decision to stop doing panel analysis.
Change
ChristianDormann44
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
The Seven Commandments of Panel Farm
7) All animals are created equal
7) All students are created equal
but some are more equal than others.
but some students are more equal than others
and only those, who are more equal than others are allowed to use difference scores.
Change
ChristianDormann45
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Intraindividual Change vs. Interindividual Differences
Year 0
Change
ChristianDormann46
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Year 1
Change
ChristianDormann47
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Year 2
Change
ChristianDormann48
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
Year 3
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann49
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
Year 4
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann50
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
Year 5
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann51
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
Year 6
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann52
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
Year 7
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann53
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
Year 8
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann54
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
Year 9
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann55
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
Year 10
r =-.94
r =-.92
r =-.95
r = .17
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann56
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann57
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
r = 1.00
Change
ChristianDormann58
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
Year 10
r =-.94
r = .95
r =-.95
r = .38
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann59
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
Change
ChristianDormann60
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
95
97
99
101
103
105
94 96 98 100 102 104 106
1
2
3
r = 1.00
Change
ChristianDormann61
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
(A) Differences (mean levels) between subjects: cross-sectional study (T1) + allows controlling for :
- nothing=> see previous slide after data points were collapsed
(B) (Causal) effects among variables: panel study (T1-T2) + allows controlling for :
- differences (mean levels) in T1-variables between subjects => see next slide
(C) Functional differences between subjects: multiple time series (T1-Tn) + allows controlling for :
- differences in variables (mean levels) between subjects - differences in effects among variables between subjects
-> see previous slide before data points were collapsed
Change
ChristianDormann62
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
An idea previously proposed by many writers:
When we control for prior differences (T1) in a variable that we also measured at T2, then we control for those variables that caused these differences, too
For example, when we control for pre-job differences in current health complaints, then we also control for personality factors such as complaining tendencies that may cause complaining.Any relation between health complaints and stressors cannot be confounded by such personality factors if pre-job differences in current health complaints are controlled.
=> Open question: How to control for prior differences in a variable measured at T2 when the same variable was measured at T1?
=> What about the difference between T1 and T2? Let‘s have a look on the next slides …
Change
ChristianDormann63
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intraindividual Change vs. Interindividual Differences
9596979899
100101102103104105
94 96 98 100 102 104 106
1
2
3
r = -.96
rT1 = .40rT2 = .69rT1 = .40rT2 = .69
Change
0 1 2 3 654
-3
-2
-1
0
1
2
ChristianDormann64
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Why not change scores?
The previous slide demonstrated that the relation (correlation) between change scores may give a much better impression of real causal processes than the relation of cross-sectionally obtained variables does give.
However, change scores usually are a biased (i.e., not true) representation of the actual change that occurred.
The next (complex) slides aim at:1. Setting up a simple regression model that has certain properties,
which we would expect most psychological systems to have.2. then re-arranging the equations to show that change should not be
expressed as, e.g. Health2 – Health1. Instead, Health2 – .70*Health1 (or any value >0 and <1) is better.
The final slides of this section then aim at demonstrating, how (and how not), repeated measures could be used to control for confounding variables.
Change
ChristianDormann65
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Change: Should we use difference scores?
Control T1 Control T21.00
-> perfect test-retest correlation: rtt = 1.0
-> invariant variances: Var(Con T1) = Var(Con T2)
=>> no change-model
(unlikely for many biological and social systems: constant change)
Change
ChristianDormann66
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Change: Should we use difference scores?
-> lower test-retest correlation: rtt < 1.0
-> increasing variances: Var(Con T1) < Var(Con T2)
=>> esaclation model (unlikely for many biological and social systems: stationary systems, regression-to-the-mean)
Control T1 Control T2
1.00
Change
ChristianDormann67
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Change: Should we use difference scores?
-> lower test-retest correlation: rtt < 1.0
-> invariant variances: Var(Con T1) = Var(Con T2)
=>> stationarity model
(likely for many biological and social systems: However, how to give meaning to the coefficients?)
Control T1 Control T2
1.00
+
-
Change
ChristianDormann68
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Change: Should we use difference scores?
Rules of Path Analysis:
- There are several rules. We use the tracing rule.
- Tracing rule: to find rxy, take the sum of the product of all the paths obtained from each possible tracing between x and y (ignore paths along variables that are entered and left thorough arrowheads)
-> r = -.30 * 1.00 + 1.00 = .70 (does this help to assign meaning?)
Control T1 Control T2
1.00
Change
ChristianDormann69
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Change: Should we use difference scores?
Lagged Regression (1/3):
- Control T2 = .70 * Control T1 + (Standard type of equation)
Control T1 Control T2
.70
Change
ChristianDormann70
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Change: Should we use difference scores?
Lagged Regression (2/3):Non-standard way of writing the equation. Since:
Control T2 = .70 * Control T1 + | remove (- .70 * Control T1) on both sides
Control T2 - .70 * Control T1 = (Difference/Change in Control = )
-> reflects the actual change. It is always there in regression models.
Control T1 Control T2
.70
Change
ChristianDormann71
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Change: Should we use difference scores?
Lagged Regression (3/3):
Non-standard way of writing the equation. Since:
Control T2 = .70 * Control T1 +
Subtracting (1.0 * Control T1) on both sides of the equation leads to:
Control T2 - 1.0 * Control T1 = .70 * Control T1 – 1.0 * Control T1 +
Control T2 - Control T1 = -.30 * Control T1 +
Thus, when it is aimed at analyzing (causes or consequences of) the non-weighted difference score Control T2 – Control T1, Control T1 has to be used as a predictor because the equation would be not correctly specified otherwise.
ChristianDormann72
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Change: Should we use difference scores?
Control T2 - Control T1 = -.30 * Control T1 +
Okay, perhaps this is unnecessarily complex, but why shouldn‘t we use difference scores?
You could, but you should knowhow (see above)
and when (see next slides)
Change
ChristianDormann73
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intermediate Summary
- Interindividual differences may or may not substitute for intraindividual changes. It may lead to correct estimates, but it sometimes does not.
- When X(T1) is available, it should be included as a predictor of X(T2), and the regression weight should be empirically estimated.
- Omission of a regression path between X(T2) and X(T1) assumes the regression weight to be .00, which is almost always wrong.
- Using difference scores X(T2) – X(T1) assumes the regression weight to be 1.00, which is almost always wrong.
Change
ChristianDormann74
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Using change to control for unknown confounding variables
Remember: The primary aim to analyze change was to overcome the assumption of cross-sectional analysis: All individuals/job conditions/health problems were equal once. All differences are due to change.
The aims of the next three modeling examples are:1. Demonstrating how (and how not) an unmeasured, stable confounding variable (e.g., gender, SES) can be controlled when follow-up measures are available.2. Demonstrating, how unmeasured third variables that may change somehow over time can be ruled out as confounders3. Demonstrating, how unmeasured third variables that may change completely over time can be controlled as confounders
Change
ChristianDormann75
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Change: Should we use difference scores?
H2-H1 = c (C2-C1)
+ s2 H1
+ 2 - 1
1)Good: The difference H2-H1 does
not depend on “something unknown”
2)Bad: H1 (independent Variable) and
1 (part of error term) are correlated.
This violates standard assumption of
regression analysis.
=> Do not use difference
scores!
Change
Health T1 Health T2
Control T1
Control T2
somethingunknown
s
c c
21
b b
Can be modeled
directly using
Structural Equation
Programs
ChristianDormann76
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
What does the CLPC-Technique really tells us?
Proportional Stationarity1)Assumption: The effects of the TV
(something unknown) remains
“relatively” (= proportional) stable:
Change
Health T1 Health T2
Control T1
Control T2
somethingunknown
+.30
+.10
somethingunknown
b e
a d
c
3) The path from Health T1 to
Control T2 implies:
rH1 C2 = b * c * d
a d_ _=b e
2) Solving for d yields: d = a * e / b
d = a * e / b
= c * a * e= b * c * a * e / b
4) The path from Control T1 to
Health T2 implies:
rC1 H2 = a * c * eec *
=a *
Conculsion: When the 2 cross-lagged correlations are equal, control and health might be spuriously related (i.e. confounded by a proportional statonary TV).
When the 2 cross-lagged correlations are different, there might be a causal relations.There is, however, no indication of causal predominance or something alike!
ChristianDormann77
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
How to control for unmeasured confounding variables?
Change
IV1
DV1 DV2
IV1 IV2
DV1 DV2
Allprevious
TVoccasion
factor
ChristianDormann78
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
How to control for unmeasured confounding variables?
Change
IV1
DV1 DV2
IV1
DV1
IV2
DV2
ChristianDormann79
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Intermediate Summary
- Interindividual differences may or may not substitute intraindividual changes. It may lead to correct estimates, but it sometimes does not.
- Hierarchical Linear Models (HLM) are useful for analyzing intraindividual change, but they have shortcomings, too (all points below)
- Using difference scores X(T2) – X(T1) to analyze change assumes the regression weight to be 1.00, which is almost always wrong.
- Omission of a regression path between X(T2) and X(T1) assumes the regression weight to be .00, which is almost always wrong.
- Using Difference scores X(T2) – X(T1) as DV and IV eliminates unmeasured (stationary) background factors, but usually it leads to biased estimates because regression assumptions are violated.
Change
ChristianDormann80
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
How to control for unmeasured confounding variables?
1) 1st Rule: A panel design is required with at least 2 variables measured at least twice (2v2w-Design)
2) 2nd Rule: Do a CLPC-Check. If it fails, stop.
3) 3rd Rule: Use Structural Equation Modeling (SEM). It allows to comply with all other rules.
4) 4th Rule: Within Time 1, do not model causal relations among variables. SEM tempt us doing so, regression analysis does not).=> Prior (unknown) confounding variables and the history of (unknown) mutual influences among variables are controlled!!!!
5) 5th Rule: Within Time 2, do model occasion factors. Occasion factors are transient (i.e., highly fluctuating) confounding variables such as mood. They are the Achilles’ heel of cross-sectional designs, where they have to be explicitly measured in order to control them. This is not necessary in longitudinal designs.
Change
ChristianDormann81
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Reversed& ReciprocalDirectedness
ChristianDormann82
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Two Questions Addressed in this Section
1) What is reversed and reciprocal causation?
2) How should we control for it?
Change
ChristianDormann83
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
How What is reversed and reciprocal causation?
Change
IV1
DV1 DV2
IV1 IV2
DV1 DV2
ChristianDormann84
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Substantive reasons for reversed causaltion
Drift hypothesis: Workers with poor health drift to better jobs.
Refugee hypothesis: Workers under strain seek better job conditions.
Zapf et al. (1996) identified only 16 studies on organizational stress.
which tested for reversed causation.
In six studies, evidence for reversed causation emerged.
Overall, evidence suggests that reversed causation is as likely as effects
of stressors/resources on strain. Hence, reciprocal causation is
probably the most plausible model.
Time
(Frese &
Zapf, 1988; Jam
es & T
etrick, 1986; Kohn &
Schooler, 1983; Leventhal
& T
omarken, 1987; O
’Brien, 1986; S
pector, 1992; Zapf et al., 1996
ChristianDormann85
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
How What is reversed and reciprocal causation?
Change
IV1
DV1 DV2
IV1 IV2
DV1 DV2
IV3
DV3
a ab b
ChristianDormann86
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Summary of reversed and reciprocal causation issues
Reversed Causation is probably more common than usually expected.
Individuals under strain may subsequently perceive more stressors and
less resources (measurement reversed causation) or may actually
become exposed to more stressors and less resources (congeneric
reversed causation).
Individuals with good health subsequently may perceive conditions better
than they are or they may get better conditions.
Time
ChristianDormann87
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
How to control for reversed and reciprocal causation?
With regression analysis: Test an opposite model. When evidence for
hypothesized as well as reversed causation emerges, discuss both
option in the Discussion Section.
With SEM: Include both hypothesized as well as reversed causation in
some models (in those models where it is possible). Interpret your
findings.
Do NOT worry when evidence for reversed but not for hypothesized
causation emerges. Perhaps this is what is actually happening.
Time
ChristianDormann88
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
TimeIt’s
Time for a
Change
ChristianDormann89
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Two Question Addressed in this Section
1) How long is the optimal time lag in between two measurement occasions?
2) Why is an optimal time lag optimal?
Change
ChristianDormann90
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Why longitudinal studies?
A1. Demonstrate cause-effects relations (causality)
A2. Random assignment to IV not possible
A3. Manipulation of IV not possible
A4. Control for unmeasured confounding variables
B1. Demonstrating nonlinear effects (thresholds, phases, disruptions )
B2. Demonstrating age-related effects (growth curves, phases)
C1. Demonstrating sustainability (long term effects)
Time
ChristianDormann91
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
What is Sustainability?
One of researchers’ great desire …
… that we’d better put aside.
(Dormann, today)
Time
Development sustainability: The continuation of benefits after major assistance from the donor has been completed.
(Australian Agency for International Development 2000).
ChristianDormann92
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Example: Effectiveness of Consultancy Sustainability = Long Term Effect?
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 520
1
2
3
4
5
6
7
8
9
10
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52
Time
ChristianDormann93
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
With impoverished theory about issues such as - when events occur, - when they change, - or how quickly they change, the empirical researcher is in a quandary. Decisions about - when to measure and - how frequently to measure critical variables are left to - intuition, - chance, - convenience, - or tradition.
None of these are particularly reliable guides ….
The ProblemWith impoverished theory about issues such as - when events occur, - when they change, - or how quickly they change, the empirical researcher is in a quandary. Decisions about - when to measure and - how frequently to measure critical variables are left to - intuition, - chance, - convenience, - or tradition. - impressions of a good ‘Gestalt’.
None of these are particularly reliable guides ….
Time (Mitchell &
James, 2001, p. 533)
ChristianDormann94
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
not men
tioned
Frequency of Different Time Lags in (almost) all Panel Studies on
Occupational Stress
7-17
3 m
onth
s
6 m
onth
s
1 ye
ar
2 ye
ars
3 ye
ars
4 ye
ars
5 ye
ars
7 ye
ars
10 y
ears
1 da
y
Note: 247 studies; time lags as reported in the abstracts.
Time
ChristianDormann95
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Reasons for Choosing a Particular Time Lag
1. Construct: - change (e.g. change of occupation, mortality)- relative time (e.g. yearly incidence rate)- absolute time (e.g. daily work hours, blue monday)
2. Mechanisms:- strength of effects (e.g., weaker effects require longer time to unfold)
3. Operationalization:- relative time (e.g. counting the frequency of events)- absolute time (e.g. interviews following certain events)
4. Method: - control of seasonal fluctuations (e.g., in workload)- survey feedback research (e.g., feedback phase necessary)- replication study
- demonstration of sustainability (e.g., work re-design on long-term satisfaction)
5. Sample:- long lags increase panel mortality (e.g., participants move)- short lags are too demanding (e.g., 10 diaries within 2 weeks)- participants only accessible at particular points in time (e.g., rotating shifts)
Time
ChristianDormann96
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Astonishing IssuesTime lags seem to be relatively unimportant
Time lags seem to be unbelievably flexible- e.g. job satisfaction: - laboratory experiments using 30min time lags; - passive field studies using 30year time lags; ~ 500.000 times longer)
In panel analyses, synchronous time-2-effects emerge frequently - common explanation: Evidence for short-term „true“ effects, that is, effects
taking less time than 50% of the time lag tested
Effect sizes smaller compared to experimental studies - common explanation: Lack of experimentally-induced maximization of variance
IV1
DV1
IV2
DV2
IV1
DV1 DV2
27
- Dormann’s explanation: Panel-induced minimization of variance
Time
ChristianDormann97
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Effects
(1) Linear Reaction Model
(2) Accumulation Model
(3) Dynamic Accumulation Model
(4) Adjustment Model
(5) Sleeper Effect Model
27
(Frese &
Zapf, 1
988)
(6) Constant Reaction Model
(7) Constant Accumulation Model
=> When is a strong effect of IV Time 1 on DV Time 2 observed?
Time
ChristianDormann98
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Variables and Types of Effects
IV: changing
Effect: linear
IV
DV
Linear Reaction Model(IV onset = -1 * IV offset)
Maximum lagged effect with an optimal time lag!
Time
ChristianDormann99
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Variables and Types of Effects
IV: changing
Effect:
cumulative
IV
DV
Accumulation Model(IV onset <> IV offset = 0)
Maximum lagged effect with a long time lag
?No! Remember consultancy example.
Time
ChristianDormann100
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Variables and Types of Effects
IV: changing
Effect:
cumulative
IV
DV
Dynamic Accumulation Model(IV onset <> IV offset > 0)
Maximum lagged effect with a long time lag
Time
ChristianDormann101
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Variables and Types of Effects
IV: changing
Effect: linear
IV
DV
Adjustment Model(IV onset 1 <> IV onset 2)
Maximum lagged effect with an optimal time lagand optimal time point
Time
ChristianDormann102
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Variables and Types of Effects
IV: changing
Effect: linear
IV
DV
Sleeper Effect Model(= Linear Reaction Model
+ lagged effect)
Maximum lagged effect with an optimal time lag!
Time
ChristianDormann103
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Variables and Types of Effects
IV: stable
Effect: linear
IV
DV
Constant Reaction Model(= Linear Reaction Model
+ stable IV)
Maximum lagged effect with a long time lag?
No! Time lags don‘t matter. A time lag is not even required
Time
ChristianDormann104
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Variables and Types of Effects
IV: stable
Effect:
cumulative
IV
DV
Constant Accumulation Model(= Accumulation Model
+ stable IV)
Again, a time lag is not required. Measure IV & DV late!
Time
ChristianDormann105
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Types of Variables and Types of Effects
Choosing optimal time lags is important in a variety of causal models:
- linear reaction models: YES
- accumulation models: YES
- dynamic accumulation models: no
- adjustment models: YES + optimal time point
- sleeper effects models: YES
- constant reaction models: no – a time lag is not required
- constant accumulation models: no – a time lag is not required
What is an optimal time lag?
Time
ChristianDormann106
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
A ‘True’ Causal Process
IV0
DV0
IV1
DV1
IV2
DV2
IV3
DV3
Time
ChristianDormann107
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Panel Analysis of a Causal Process
IV0
DV0
IV2
DV2
IV3IV1
DV1 DV3
i i
d d
e e e
b
b d w– i w
d – i
= e * d w+ e * i *
IV1
DV1 DV3
What is the size of b?
The optimal Time Lag (Number of ) is defined as the Time Lag, that yields the maximum effect b of the IV on the DV when the data are analyzed.
Time
ChristianDormann108
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
When does b reach ist maximum? When the Time Lag is optimally chosen!
=> = – ( )ln
d ln(d)
i ln(i)
di( )ln
The size of the true causal effect e
is irrelevant!
… some math wizard solves the problem …
Time
ChristianDormann109
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Example 1: i = .990, d =.995; across 1 Day
= 137.01 Days
= 19.58 Weeks
= 4.57 Months
= – ( )ln
d ln(d)
i ln(i)
di( )ln
27
Time
ChristianDormann110
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Example 2: i =.98, d = .97; across 1 Day
= 39.03 Days
= 5.58 Weeks
= 1.30 Months
= – ( )ln
d ln(d)
i ln(i)
d i( )ln
Time
ChristianDormann111
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Effect Sizes as a Function of the Time Lag
IV1
DV1 DV2
IV1
DV1
IV2
DV2
i = .990, d = .995, e = .005i = .980, d = .970, e = .050i = .900, d = .900, e = .100
Time
ChristianDormann112
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
What is Stabiliy?
1. Stability = Effect of a Variable on itself across time
2. Stability > 1: Rarely
3. Stability < 1: Usually (mainly reflecting a regression-to-the-mean)
4. Test-Retest-Correlation
= Stability
+ Effects of Confounding Third Variables
– Measurement Error
.40 .40
.50
.19 .19
5. Stability = .50
6. Retest = .54
7. dissat. Retest = .66
iV0 IV1
TV
IV0 IV0
Time
ChristianDormann113
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Summary
Choosing optimal time lags is of utmost importance for effect sizes.
Do not mix up
- sustainable effects in real life (sustainable WOP is possible) vs.
- cross-lagged long-term effects in panel studies (almost impossible)
For sustainable (i.e., strong) effects we need
- optimal (i.e., usually short) time lags
- or extremely stable variables
Time
ChristianDormann114
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Implications
Stabilities of IV & DV determine the optimal lag. - We need meta-analyses of stabilities of important WOP variables!
Use quasi-trait measures less frequently(„How do you think in general about …?“)
Use state measures more frequently(„ How do you think right now about …?“)
Do more ‘shortitudinal’ research- Diary studies- Event sampling studies- Combining observational & questionnaire studies
Time
ChristianDormann115
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Overall Summary – Christian‘s Personal Opinion
Follow-up studies are much better in controlling for all kinds of
unmeasured confounding variables.
The less confounding is built-in the concepts and measures, the more
good follow-up research approximates experiments.
Shortiduinal designs covering not more than two weeks are most
appropriate.
Dormann‘s Dear Desire: The Dublet Design:
- 2 week gap using measures focusing on the last 2 weeks
- in between: diary design using measures focusing on the immediate
moment (here and now)
Time
ChristianDormann116
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
ChristianDormann117
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Additional Slides
ChristianDormann118
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Further Implications
Most Studies Apply Designs with Time Lags that are much too long
Example Job Satisfaction:
- rtt = .96 (1 month; short-lagged panel studies considered in the
in the meta-analysis by Dormann & Zapf, 2001)
- minus 25% personality dispositions
- Estimated Stability ~ .71
- Assumptions: + IV (e.g. job characteristics) and DV similarly stable+ Job satisfaction may change from day to day
=> Optimal Lag: 12.6 weeks = 2.95 months
Time
ChristianDormann119
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
True Cross-lagged Effects
IV0
DV0
IV1
DV1
IV2
DV2
IV3
DV3
IV4
DV4
IV5
DV5
IV6
DV6
IV7
DV7
Time
ChristianDormann120
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
How Theory and Statistics Combine
= – ( )ln
d ln(d)
i ln(i)
d i( )ln
Statistics
Theory
Minimal Lag: Time Lag to be derived from theoretical Considerations, which is required for a change in the IV to trigger a change in the DV.
– Optimizing Lag: Time Lag to be derived from the stabilities of the IV and the DV in order for maximal effects to emerge from analysis.
Time
ChristianDormann121
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Sexual Intercourse & Birth of a Baby
Assumption:
1) 1-week Stability of the IV: .90 (Christian Dormann‘s estimate)
2) 1-week stability of the DV: .80 (Britannica, 2004).
3) = 9 months (40 weeks, 280 days)
The appropriate equation is:
40 + ln( (.90 * ln(.90) ) / ( .80 * ln(.80) ) )/ ln(.90/.80) = 45.37.
Therefore, the optimal occasion (for the 2nd measurement) is
45.37 weeks (~ 318 Days; ~ 10.59 months) after the first.
Time
ChristianDormann122
Substantive and Methodological Problems in the Study of Stress at Work
European Erasmus Mundus Program
Master on Work, Organizational & Personnel Psychology
Valencia, 15.10.-17.10.2008
Computer
Delta (d) 0,000
Stability of the IV: 0,900Stability of the DV: 0,950
if stabilities are identical: #ZAHL! if stabilities are different: 12,314
Time