Upload
vannga
View
216
Download
1
Embed Size (px)
Citation preview
調查研究調查研究
K th S L (罗胜强)Kenneth S Law (罗胜强)
Department of ManagementThe Chinese University of Hong KongThe Chinese University of Hong Kong
香港中文大学管理系mnlawcuhkeduhk
httpkennethlawblogchinahrdnet
Common Methods in Mgt Research2
Common Methods in Mgt Research1 Qualitative reviews2 Qualitative studies
Case studiesContent analysisContent analysis
3 Experimental laboratory studiesANOVA regression
4 Quasi experiments5 Correlational survey studies
i SEM HLMregression SEM HLM6 Meta analysis7 Field study Content analysis
Kenneth Law 同济大学 2010
7 Field study Content analysis Database helliphellip
1 Qualitative Reviews3
1 Qualitative Reviews
C R iContent ReviewTheory buildingHypotheses developmentHypotheses development
Manor B (2002) International assignmentsManor B (2002) International assignments for career building A model of agency relationships and psychological contracts Academy of Management Review 27(3) 373Academy of Management Review 27(3) 373-391
Kenneth Law 同济大学 2010
Abstract4
Abstract
We present a theoretical model of international assignmentsin which we examine the alignment or nonalignment of the organizationrsquos and individualrsquos expectations of an assignmentorganization s and individual s expectations of an assignment and its effect on assignment success We consider four basic configurations in our model ndash mutual loyalty mutual transaction agent opportunism and principal opportunism ndashtransaction agent opportunism and principal opportunismand present a matrix of organization-individual alignment to help predict varying degrees of success in expatriate assignment and in repatriation Finally we apply this matrixassignment and in repatriation Finally we apply this matrix as a framework for understanding changes in the employment contract over time
Kenneth Law 同济大学 2010
The Model (Agency Theory)5
Relational Transactional
Individual
Cell I Mutual loyalty Cell II Agent opportunism
High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti
Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel
ationa
l
ganiza
tion
Cell III Principal opportunism Cell IV Mutual transaction
High individual success in expatriation and repatriation (P2)
High individual success in expatriation but mixed success in repatriation (P4)
Re
Org
Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in
Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns
action
al
expatriation but failure in repatriation (P6)
Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)
Tra
Kenneth Law 同济大学 2010
2 Qualitative Studies6
2 Qualitative Studies
S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269
Kenneth Law 同济大学 2010
7
AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors
Kenneth Law 同济大学 2010
3 Laboratory designs8
3 Laboratory designs
Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)
Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering
Leadership style Employee performance
Kenneth Law 同济大学 2010
Examples9
Examples
Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Common Methods in Mgt Research2
Common Methods in Mgt Research1 Qualitative reviews2 Qualitative studies
Case studiesContent analysisContent analysis
3 Experimental laboratory studiesANOVA regression
4 Quasi experiments5 Correlational survey studies
i SEM HLMregression SEM HLM6 Meta analysis7 Field study Content analysis
Kenneth Law 同济大学 2010
7 Field study Content analysis Database helliphellip
1 Qualitative Reviews3
1 Qualitative Reviews
C R iContent ReviewTheory buildingHypotheses developmentHypotheses development
Manor B (2002) International assignmentsManor B (2002) International assignments for career building A model of agency relationships and psychological contracts Academy of Management Review 27(3) 373Academy of Management Review 27(3) 373-391
Kenneth Law 同济大学 2010
Abstract4
Abstract
We present a theoretical model of international assignmentsin which we examine the alignment or nonalignment of the organizationrsquos and individualrsquos expectations of an assignmentorganization s and individual s expectations of an assignment and its effect on assignment success We consider four basic configurations in our model ndash mutual loyalty mutual transaction agent opportunism and principal opportunism ndashtransaction agent opportunism and principal opportunismand present a matrix of organization-individual alignment to help predict varying degrees of success in expatriate assignment and in repatriation Finally we apply this matrixassignment and in repatriation Finally we apply this matrix as a framework for understanding changes in the employment contract over time
Kenneth Law 同济大学 2010
The Model (Agency Theory)5
Relational Transactional
Individual
Cell I Mutual loyalty Cell II Agent opportunism
High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti
Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel
ationa
l
ganiza
tion
Cell III Principal opportunism Cell IV Mutual transaction
High individual success in expatriation and repatriation (P2)
High individual success in expatriation but mixed success in repatriation (P4)
Re
Org
Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in
Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns
action
al
expatriation but failure in repatriation (P6)
Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)
Tra
Kenneth Law 同济大学 2010
2 Qualitative Studies6
2 Qualitative Studies
S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269
Kenneth Law 同济大学 2010
7
AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors
Kenneth Law 同济大学 2010
3 Laboratory designs8
3 Laboratory designs
Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)
Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering
Leadership style Employee performance
Kenneth Law 同济大学 2010
Examples9
Examples
Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
1 Qualitative Reviews3
1 Qualitative Reviews
C R iContent ReviewTheory buildingHypotheses developmentHypotheses development
Manor B (2002) International assignmentsManor B (2002) International assignments for career building A model of agency relationships and psychological contracts Academy of Management Review 27(3) 373Academy of Management Review 27(3) 373-391
Kenneth Law 同济大学 2010
Abstract4
Abstract
We present a theoretical model of international assignmentsin which we examine the alignment or nonalignment of the organizationrsquos and individualrsquos expectations of an assignmentorganization s and individual s expectations of an assignment and its effect on assignment success We consider four basic configurations in our model ndash mutual loyalty mutual transaction agent opportunism and principal opportunism ndashtransaction agent opportunism and principal opportunismand present a matrix of organization-individual alignment to help predict varying degrees of success in expatriate assignment and in repatriation Finally we apply this matrixassignment and in repatriation Finally we apply this matrix as a framework for understanding changes in the employment contract over time
Kenneth Law 同济大学 2010
The Model (Agency Theory)5
Relational Transactional
Individual
Cell I Mutual loyalty Cell II Agent opportunism
High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti
Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel
ationa
l
ganiza
tion
Cell III Principal opportunism Cell IV Mutual transaction
High individual success in expatriation and repatriation (P2)
High individual success in expatriation but mixed success in repatriation (P4)
Re
Org
Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in
Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns
action
al
expatriation but failure in repatriation (P6)
Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)
Tra
Kenneth Law 同济大学 2010
2 Qualitative Studies6
2 Qualitative Studies
S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269
Kenneth Law 同济大学 2010
7
AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors
Kenneth Law 同济大学 2010
3 Laboratory designs8
3 Laboratory designs
Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)
Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering
Leadership style Employee performance
Kenneth Law 同济大学 2010
Examples9
Examples
Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Abstract4
Abstract
We present a theoretical model of international assignmentsin which we examine the alignment or nonalignment of the organizationrsquos and individualrsquos expectations of an assignmentorganization s and individual s expectations of an assignment and its effect on assignment success We consider four basic configurations in our model ndash mutual loyalty mutual transaction agent opportunism and principal opportunism ndashtransaction agent opportunism and principal opportunismand present a matrix of organization-individual alignment to help predict varying degrees of success in expatriate assignment and in repatriation Finally we apply this matrixassignment and in repatriation Finally we apply this matrix as a framework for understanding changes in the employment contract over time
Kenneth Law 同济大学 2010
The Model (Agency Theory)5
Relational Transactional
Individual
Cell I Mutual loyalty Cell II Agent opportunism
High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti
Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel
ationa
l
ganiza
tion
Cell III Principal opportunism Cell IV Mutual transaction
High individual success in expatriation and repatriation (P2)
High individual success in expatriation but mixed success in repatriation (P4)
Re
Org
Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in
Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns
action
al
expatriation but failure in repatriation (P6)
Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)
Tra
Kenneth Law 同济大学 2010
2 Qualitative Studies6
2 Qualitative Studies
S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269
Kenneth Law 同济大学 2010
7
AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors
Kenneth Law 同济大学 2010
3 Laboratory designs8
3 Laboratory designs
Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)
Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering
Leadership style Employee performance
Kenneth Law 同济大学 2010
Examples9
Examples
Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
The Model (Agency Theory)5
Relational Transactional
Individual
Cell I Mutual loyalty Cell II Agent opportunism
High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti
Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel
ationa
l
ganiza
tion
Cell III Principal opportunism Cell IV Mutual transaction
High individual success in expatriation and repatriation (P2)
High individual success in expatriation but mixed success in repatriation (P4)
Re
Org
Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in
Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns
action
al
expatriation but failure in repatriation (P6)
Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)
Tra
Kenneth Law 同济大学 2010
2 Qualitative Studies6
2 Qualitative Studies
S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269
Kenneth Law 同济大学 2010
7
AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors
Kenneth Law 同济大学 2010
3 Laboratory designs8
3 Laboratory designs
Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)
Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering
Leadership style Employee performance
Kenneth Law 同济大学 2010
Examples9
Examples
Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
2 Qualitative Studies6
2 Qualitative Studies
S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269
Kenneth Law 同济大学 2010
7
AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors
Kenneth Law 同济大学 2010
3 Laboratory designs8
3 Laboratory designs
Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)
Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering
Leadership style Employee performance
Kenneth Law 同济大学 2010
Examples9
Examples
Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
7
AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors
Kenneth Law 同济大学 2010
3 Laboratory designs8
3 Laboratory designs
Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)
Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering
Leadership style Employee performance
Kenneth Law 同济大学 2010
Examples9
Examples
Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
3 Laboratory designs8
3 Laboratory designs
Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)
Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering
Leadership style Employee performance
Kenneth Law 同济大学 2010
Examples9
Examples
Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Examples9
Examples
Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
10
AbAbstract
The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB
Citizenship Behaviors Overall evaluationBehaviors
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
11
High OCB Low OCB
Manipulation
Results
Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Types of experimental designs12
Types of experimental designs
1 Experimental designs
2Quasi experimentsbull Field experiments bull Time series designs
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Experimental designs13
Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group
Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups
XT1 T2Experimental group 1 2Experimental group
Control group T1 T2
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Quasi experiment14
Quasi-experiment
- Experimental design without real manipulations- No randomization
- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
15
Employees
P ti OCB High OCB Promotion OCB
Antecedents
Low OCB No Promotion OCB
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Correlational surveys16
Correlational surveys
Predictor variables (independent variables)Criterion variables (dependent variables)
Research questionHow would a set of predictor variables affect a set of outcome
variables (causality)
Performance Promotion
time 1 time 1
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Examples17
Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496
Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516
Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744
Kenneth Law 同济大学 2010
Journal 45(4) 735-744
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Survey study18
Survey study
rSafety‐specific
Occupational
rxy
transformational leadership
Occupational safety
Self‐efficacy Job performance
Transformational leadership
Job performance
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Meta Analysis19
Meta Analysis
R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively
Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Meta analysis20
y
Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences
Rxy is the correlation between agreeableness and job satisfaction
Study N rxy
1 150 35 2 247 ‐07
3 386 174 124 485 278 025 76 450 29
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
21
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 22
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 23
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
1 What is your research question24
1 What is your research question Is the research question testableq
改善我国旅游业未来发展的几个意见
Are the constructs well defined 「企业进取性」和企业业绩表现的关系
Do we have validated scales to measure the constructs Existing scales Western scales
Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
What is the research question25
qIs EQ really useful in predicting work outcomes
Would a firm using a strategic human resource management approach be more competitive
What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC
Can supervisors distinguish task performance from contextual performance in the PRC
What are the antecedents of contextual performance
Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
生涯管理要素圖26
生涯情況 信息 計劃 資源目標
探索的動機 候迭人的信息
生活的目標
1個人戰略2時間
1 解決個人問題的技能
2 控制1自我評估價值技術興趣經驗
2組織評估表現潛力分配計劃
生涯發展目標 生涯計劃執行
分配計劃
現有內部勞資市場1工作調整需要2生涯之路結構3內部提升
生涯機會信息1生涯信息系統2生涯咨詢
1未來組織經濟目標
2未來所需職工
1組織人才資源發展戰略
2重要職務分配
1工作機會2贊助人3生涯管理者
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Th di ti l f LMX
27
The mediating role of LMX
OCB
TransformationalLeadership LMX-MDM
OCB
77
3280
TaskPerformance
16
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Contributions28
Contributions1 Theoretical contributions
bull New variables based on theorybull New perspectivesbull New findings
h d l l b2 Methodological contributionsbull New measures (eg new scale development)
N th d ( l l h)bull New methods (eg cross‐level research)
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Identifying Research Questions29
Identifying Research Questions
员 企业的交换契约
公平理论与员工公民行为
Literature
员工企业的交换契约Incremental theoretical
contribution
Research Questions
M h d l l Practitioners公司治理
Methodological rigor
59岁现象
合资企业员工本土化
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
What is a research question30
What is a research question
My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust
How do you like that
Expected returnknowledge sharing
knowledge sharing
Benevolence
Expected return
Trust
s a gtendency
s a gbehaviors
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
A conversation hellip31
A conversation hellip
Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate
ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic
ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
A different view32
A different view
We do research because we want to know We do research because we want to know the answer hellip
Therefore before we start a study we must have a research question
We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
A different view33
A different view
With a research question you would automatically ask for a theoryautomatically ask for a theory
B d h ld Based on your theory you would automatically develop some hypotheses
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
The research process34
The research process
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Trusttendency behaviors
Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
The research process35
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
H d k th t k l d h i i t ti i
Trust
How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g
This is because helliphellip (a theory)
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
The research process36
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
Y d th k l d h i i t ti b
Trust
You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
The research process37
The research process
Benevolence
Expected returnknowledge sharing tendency
knowledge sharing
behaviors
Benevolence
(1) Some people have an in‐born tendency to share with
Trust
(1) Some people have an in‐born tendency to share with others
(2) Some people would expect reciprocal sharing in the future
(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)
(4) helliphellip
Kenneth Law 同济大学 2010
(4) helliphellip
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Some questions38
1 Has anyone done anything on this topic before (literature review)
2 Which one is the most important reason to explain knowledge sharing (theory)
3 Are there established management theories that can explain the phenomenon (theory)
4 Are these antecedents or causes (theory)
5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)
6 What is the most appropriate research design to answer my research questionresearch question
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
When to use which design39
When to use which design
1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Do Not start your research40
Do Not start your research helliphellip
With some hypothesized relationships
With a model
Just because no one has done that before
Only because you see a significant correlation in your datasetyour dataset
Without looking at the literature
Without a research questionq
If you do not find it interesting
If you do not believe it is important
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
A suggested research procedure hellip41
A suggested research procedure hellip
Starts with a research questionq
Think about from what perspective you would dd thi iaddress this issue
Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question
Wh i h b h d h h h What is the best method to test these hypotheses
Research design
Kenneth Law 同济大学 2010
Research design helliphellip
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 42
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
2 What are the hypotheses43
2 What are the hypotheses
A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory
2 Are the hypotheses adding to the current literature yp grelating to the key construct
3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Three possible arguments44
Three possible arguments
1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp
2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip
3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate
d th f h th i d th t
Kenneth Law 同济大学 2010
reward we therefore hypothesized that helliphellip
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 45
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
3 Measure your construct of interest46
3 Measure your construct of interest
a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 47
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
3 l f A l i
48
3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry
levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group
level OCB3 The effect of HRM practices on firm
performance4 The effect of HRM practices on the job
satisfaction of employees
Kenneth Law 同济大学 2010
satisfaction of employees
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Additive model49
Additive model In additive composition models the meaning of the
hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of
th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable
Eg average organizational commitment of employees in an organization
Eg total productivity of employees in a firm
Kenneth Law 同济大学 2010
Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Additive model50
Additive model
Productivity
Employee 1 + Employee 2 + hellip
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Direct consensus model51
Direct consensus model Direct consensus composition uses within-group consensus of
th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level
Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Direct consensus model52
Direct consensus modelLeader characteristics
Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1
Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2
X2Employee 3 group 2
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Reference shift model53
Reference shift modelbull The critical difference between reference shift consensus and
direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct
bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus
Kenneth Law 同济大学 2010
gin such beliefs
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Reference shift model54
Reference shift modelFirm citizenship behaviors
Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1
Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2
X2Manager 3 group 2
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Three examples55
Three examples
Addi i d l1 Additive model我很樂意幫助組內的同事
2 Direct consensus model我認為這是個合作的小組
3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 56
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
3b Data source and CMV57
3b Data source and CMV
Try to solicit data (esp predictor vs criterion variables) from different )sources
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
An example58
An example
HRM practices of
th fi
Degree of social exchange in the
i ti
Individual performance f lthe firm organization of employees
Source of informationSource of information
HR manager Middle managers Top level managersg g p g
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Common Method Variance59
Common Method Variance
The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they
d b h h dare measured by the same method
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
An exampleAn example
I am committed to this organization
I am very satisfied with my current job
Organizational commitment
Job satisfaction
Self-report Self-report
Negative(positive) affectivity
The respondent is a critical judgmental person60
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Another common third variablesAnother common third variables
A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯
2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题
3我有很强地属于「这家公司的人」的感觉
Turnover Intention7 我常想到辞职
8 我很可能在明年另寻新的工作
9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作
61
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
IllustrationIllustration
I am committed to this organization
I often think about leaving this
organizationOrganizational commitment
Turnover Intention
organization
Self-report Self-report
Social Desirability
The respondent likes to give a good impression to others62
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
A Method Factor63
A Method Factor
Oganizational commitment (affective) 不同意 同意
1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5
2这家公司所面临的问题就是我自己的问题 1 2 3 4 5
3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5
Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5
8 我不可能在明年另寻新的工作 1 2 3 4 5
9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
64
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Partialling Venn diagramPartialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
65
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Rotated Factor Matrix in EFA66
Factors
Rotated Factor Matrix in EFA
FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03
Organizational commitment X2 32 81 12 03
X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07
commitment
Job satisfaction5
X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53
Turnover intention
X9 45 26 -13 47intention
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Partial out by a method factor67
Partial out by a method factor
Organizational commitment
Turnover intention
Items 1 2 3 4 5 6 7 8 9
Negative affectivity
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Partialling --Venn diagram
Organizational commitment
Turnover intention
Negativegaffectivity
68
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Include in the model69
Include in the model
Organizational commitment
Job satisfaction
Negativeaffectivity
Other missing method variables
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Different methodssource70
Different methodssource
Organizational commitment
Organizational culture commitmentculture
bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials
bull Self reported by employee
reported by employee reported by supervisorpeer
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 71
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
3c What measure to be used72
3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing
measures do not fit or there is no good measure of the construct
MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Using existing scales73
Using existing scales
MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89
Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item
trimming or not4 If yes we do not know the criteria of item selection
Kenneth Law 同济大学 2010
4 If yes we do not know the criteria of item selection
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Using existing scales74
Using existing scales
bull Follow the proper procedure of scaleFollow the proper procedure of scale translation
bull The minimum requirement is a forward-qbackward translation
bull It is best to pre-test your (translated) scale b fbefore use
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 75
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
3d Developing new scales76
3d Developing new scales
Inductive vs deductive approach for scale Inductive vs deductive approach for scale development
Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe
behaviors of a transformational leaderbull Researcher group all items and sort them into
various dimensions using systematic classificationvarious dimensions using systematic classification techniques
bull Select items to represent each dimension
Kenneth Law 同济大学 2010
bull Pretesting of the scale
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Developing new scales77
Developing new scalesDeductivebull Start with theory to determine the dimensionality
of the constructbull For each and every dimension draft items to y
represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Pros amp Cons78
Pros amp Cons
InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p
domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)
Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly
Kenneth Law 同济大学 2010
May not be respondent friendly
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Scale development process79
Scale development processStep 1 Item Generation (inductive vs deductive)
Step 2 Questionnaire Administration
Step 3 Initial Item Reduction (EFA)
Step 4 Confirmatory Factor Analysis
Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity
Step 6 Replication
Kenneth Law 同济大学 2010
Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in scale development80
Issues in scale development
Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬
Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent
Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)
Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導
他會悉心教導下屬
Kenneth Law 同济大学 2010
‐他會悉心教導下屬
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in scale development81
Issues in scale development
Item selection (EFA IRT)
IRT (item difficulty and item reliability)( y y)
Item difficulty ndash what is the proportion of respondents who will give high ratings on this item
Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Item Response Theory82
Let us use a simply yes (1) or no (0) question to illustration
P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item
B
C
A
B
A
D
Kenneth Law 同济大学 2010
Total score of that construct
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Distribution of total score83
Frequency
хх х
х
х
х
Total score of all items1 2 3 4 5 6
х
Kenneth Law 同济大学 2010
Total score of all items measuring this construct
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 84
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
3e Formative vs Reflective indicators85
Income Relax
Socio-economic
statusParentrsquos income
Life satisfaction Happy
Size of apartment Positive
Formative or causal indicators Reflective or effect indicators
Please give one example of each type of construct
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Measurement models86
Measurement models
For formative indicators
Indicators are purely theory driven
O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA
Content coverage of indicators is crucial Content coverage of indicators is crucial
Measurement model identification is an important issue
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 87
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Multidimensional constructs88
Multidimensional constructs
Quality
Job Performance
y
Quantity
On time
Aggregate Model
On‐time
Math
Mental Ability Verbal
Memory
Latent Model
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 89
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
4 Pilot Test90
4 Pilot Test
Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a
CFA using the same sample Cross validation
C t amp di i i t lidit Convergent amp discriminant validity
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Confirmatory Factor Analysis 91
FactorsVar F1 F2 F3 h2
x1 60 -06 02 36x2 81 12 - 03 67
Exploratory factor analysis
Factors
x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47
19 02 68 0
Does the data ldquofitrdquo our specified model
FactorsVar F1 F2 F3
x1 60 0 081 0 0Guanxi
x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31
x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0
Guanxi
promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53
promotion
Bonus
Kenneth Law 同济大学 2010
x8 0 0 53x9 0 0 47
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Fit Indices in CFA92
Fit Indices in CFA
Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90
SRMR lt 05 RMSEA lt 05
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 93
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
5 Convergent amp Discriminant Validity94
5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead
to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure
DiscriminantValidityy Measures of different attributes should not correlate to
an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
A sample MTMM matrix95
A sample MTMM matrix(Paper amp Pencil self test)
Heterotrait-monomethod
MonotraitMonotrait-monomethodMonotrait-
heteromethod
Heterotrait-heteromethod
Kenneth Law 同济大学 2010
Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm
Note SE self esteem SD self disclosure LC Locus of control
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Elements of a MTMM matrix96
Elements of a MTMM matrix
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
MTMM for construct validity97
f ySelf-rating Parent Rating
Traits EI NEU
EXT OPEN ANT
CON EI NEU
EXT OPEN ANT
CON
EI (78)NEU -39 (77)
Self-rating
( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)
Parent Rating
EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)
Kenneth Law 同济大学 2010
CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Interpreting MTMM98
Interpreting MTMM
R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest
Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high
Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
SEM analysis of MTMM99
SEM analysis of MTMM
Trait 1 Trait 2
T1M1 T1M2 T2M1 T2M2
Method 1 Method 2Method 1 Method 2
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 100
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
6 Questionnaire Design101
Q g1 Question sequencing
bull Dependent variables firstDependent variables firstbull Randomization
2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)
第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈
1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5
2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5
Kenneth Law 同济大学 2010
3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 102
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
7 Data collection103
7 Data collection
1 Minimum N is 15 (one respondent for each item within a construct))
2 Minimum N gt100 for group level gt200 for individual level
3 You should be there during data collection
4 Questionnaire distribution ndash the higher the level the betterbetter
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 104
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
8 Data analysis105
8 Data analysis
1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source
2 Separate measurement model from structural modelU f t 3 Use mean score or factor score
4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with
a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 106
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
8aForming parcels in CFAg p
3)( 3211 xxxg
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
2 4 5 6( ) 3g x x x
3 7 8( ) 2g x x
107
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 108
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
8b Cross level analysis8bCross level analysis
1 Different sources of data1 Different sources of data2 One supervisor rating several
subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM
109
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Individual-group analysis110
Individual group analysis
Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level
Leadership PerformancepSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
=333 =267
Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2
=300 =300
Kenneth Law 同济大学 2010
Sub 4 2 2
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Cross level analysisy1 One supervisor rating several subordiantes
( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling
Job Sat PerformanceSup 1 Sub 1 5 3
Sub 2 2 1Sub 3 3 4
Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2
111
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Hierarchical Linear Modelingg
( )Level Helping b b Mood r 1 0j 1j
2 0j 00 01 0j
( )
(Pr )ij ij ij
j
Level Helping b b Mood r
Level b oximity u
where = Level‐2 intercept
1j 10 1j b u
00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes
112
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Hierarchical Linear Modeling113
Hierarchical Linear Modeling
t l l ti b d b HLM1 not every cross level question can be answered by HLM
2 HLM does not consider measurement model
3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM
4 Only group level variables affect individual level intercept and slope in HLM
5 The exact procedure of testing mediators using HLM is not established yet
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 114
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
8 M d i d M di i
115
8cModeration and Mediation
Moderator Interaction effectsMediatorMediator
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Moderator116
Moderator
A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)
Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)
Kenneth Law 同济大学 2010
Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Moderation and Interaction117
EmotionalLabor
EI Performance
Labor
EI
PerformanceThe total effects is greater than the
Mental Ability
summed effects
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
MediatorsMediators
Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)
A mediator is a variable which two variables
118
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Mediated Regression119
g
OCB OCB OCB OBSEFull or partial mediation
OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36
R2
Similarity
OBSE
OCB
Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit
Kenneth Law 同济大学 2010
2 The importance of causality
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Mediating and indirect effects120
Mediating and indirect effects
J b A i tJob Assignment
f
LMX
29
19
44
Performance ratings
Guanxi
29
2983 5271
Bonus allocation
Commitmentto supervisor 21
Chances of promotion
p
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010
Issues in survey design 121
1 What is the research question
2 What are the hypotheses
3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs
4 Pilot test
5 Convergent amp Discriminant Validity
6 Issues in Questionnaire design (2) Explanation
(3) Model amp Theory(4) prediction (5) Application
Y
X2
X1
6 Issues in Questionnaire design
7 How to collect data
8 Data analysis(1) Observation
(2) Explanation
H1H2helliphellip
Kenneth Law 同济大学 2010
a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators
Theory Building Theory Testing Theory Application
122
E dEnd
Kenneth Law 同济大学 2010