Upload
taylor-oconnor
View
225
Download
2
Tags:
Embed Size (px)
Citation preview
2
Welcome
• Today’s course• Your facilitator• The EQ-360• The facilities
Logistics
• Schedule• Meals • Parking• Phone/computer access• Restrooms• Questions• Workbook• Feedback practice• Networking contact list
4
Objectives
By the end of today’s workshop, you should:
1. Be familiar with the EQ-3602. Know its scales and their definitions3. Understand report components4. Understand the science behind EQ-3605. Know how to interpret the results and
provide feedback to respondents6. Be qualified to purchase the EQ-360
5
Objectives
What are your goals for today?
Agenda
1. 360° assessments2. Understanding
the EQ-3603. Psychometrics4. Planning and
Ethics 5. Administration
6. Scoring and reports
7. Interpretation and feedback
8. Coaching9. Evaluation
7
Evaluation Process
1. Written exam
• Interpret a sample report
• 60 minutes in-class today
2. After the workshop:
• Administer, interpret, and provide feedback for one real-life EQ-360 assessment on a person of your choice
Introductions
Let’s get to know each other…
Part One
An Introduction to 360° Assessments
10
Part One Objectives
• Applications• Advantages• History • Misuses
By the end of this section, you should be familiar with 360° assessments and their
What are 360° Assessments?
Applications• Self-development • Leadership development • Teambuilding• Coaching• Performance management• Change management• Stress management• Career planning• Predicting performance • Retaining high performers
12
What are 360° Assessments?
Advantages of multiple perspectives1. Objective measurement of performance2. Provides perspective from different
angles 3. Increases awareness of work-
related behaviors4. Improves communication
between self, raters, and the organization
13
What are 360° Assessments?
Advantages of multiple perspectives5. Taps many sources of feedback,
e.g., customers, suppliers, clients6. Clarifies differences of opinions 7. Identifies performance gaps 8. Helps with goal setting and
development planning
14
History of 360° Assessment
• Traditionally, lack of direct feedback hindered organizations.
• Authoritarian leadership was the norm, with limited supervision given to employees.
• After World War II, worker satisfaction surveys introduced a more democratic ideology.
15
History of 360° Assessment
• The 1990s saw 360° feedback processes incorporated into corporate strategies and goals
• Now, most organizations believe that 360° assessments are crucial to selecting and developing the best talent for success
Misuses of 360° Assessment
• Misuses include personnel decisions such as promotions, compensation, demotions, and termination
• The use of 360° assessments for purposes other than individual development has often failed
17
Misuses of 360° Assessment
Lack of feedback…• Removes follow-up or
development opportunities• Regularly guarantees failure• May discourage individuals from
providing honest feedback • May influence participation in
future assessments
18
360° Assessments
• What experience do you have with multi-rater assessments?
• How could the assessment have been improved?
• What were the strengths of the tool?
Discuss your experience using 360° (multi-rater) feedback assessments:
19
Questions?
Part Two
Understanding the EQ-360
21
Part Two Objectives
• Development• Participants• Rater definitions• Basic assessment process
By the end of this section, you should be familiar with the EQ-360’s:
22
Development of EQ-360
• First version consisted of 46 items• Detailed data analysis led to
improvements• Recommendations were
implemented (see page 27 of the Technical Manual) and a new version was created
23
Development of EQ-360
• Current version consists of 88 items• Rating scale from 1 to 5
(“Very seldom true” to “Very often true”)
• EQ-360 factor structure is identical to EQ-i
- Total EQ- 5 Composite scales- 15 Subscales
24
EQ-360 Scales
Intrapersonal• Emotional Self-Awareness• Assertiveness• Self-Regard• Self-Actualization• Independence
Interpersonal• Interpersonal Relationship• Empathy• Social Responsibility
Adaptability• Problem Solving• Flexibility• Reality Testing
Stress Management• Stress Tolerance• Impulse Control
General Mood• Optimism• Happiness
25
Participation
26
Participation
Rater confidentiality• Anonymous responses • Grouped by relationship to facilitate
interpretation• If fewer than 3, raters go into a
“mixed” category (except manager)Rater familiarity
• Raters are asked how well they know the ratee
27
Rater Familiarity
Rater familiarity information is found on the first page of the Coach’s Summary in the Multirater Feedback Report
28
EQ-360 Rater Items
29
How the EQ-360 Works• Scoring converts raw scores to standard scores• 1-5-15 factor structure• Interpretive guidelines
Standard Score
Guideline
130 and over Atypically well developed emotional and social functioning
120 to 129 Extremely well developed emotional and social functioning
110 to 119 Well developed emotional and social functioning
90 to 109 Effective emotional and social functioning
80 to 89 Consider developing
Under 80 Develop
How the EQ-360 Works
• Who participated?• Impressions of your EQ-360 experience?• Subscale consistencies or discrepancies?• Thoughts on feedback?• Were there any surprises?
Discuss your experience as a ratee in the EQ-360 process:
31
Questions?
Part Three
An Introduction to Psychometrics
33
Part Three Objectives
• What the EQ-i and EQ-360 measure
• EQ-360 normative data• Internal consistency • Connections between the EQ-i
and EQ-360 response patterns
By the end of this section, you should be familiar with:
34
EQ-i Refresher
35
EQ-i Refresher
• Validity• Reliability• Norms• Norm Groups
36
EQ-360 Normative Data
7
385353
Males
Females
Not reported
745 Ratees…• Age range from 2170• Mean age of 40.4 years
37
EQ-360 Normative Data
331
869
289
411 Managers
Peers
Direct Report
Others
1900 Raters…• 909 male & 969 female • 22 did not report gender
38
Internal Consistency
• Reliability of Total EQ is .98 • Inter-scale reliability for Composite
Scales ranged from .91 to .95• Inter-scale item reliability for
Subscales ranged from .77 to .91
39
EQ-i and EQ-360 Connections
Ratee and Rater Group CorrelationsScales with significant* relationships:
*Magnitude was moderate
Emotional Self-Awareness
Empathy ProblemSolving
Manager Peer
Direct Report
40
EQ-i and EQ-360 Connections
Ratee and Rater Group DifferencesRatee was significantly higher than raters on four scales only:
Emotional Self-Awareness
Empathy
ProblemSolving
Social Responsibility
Manager Peer Direct Report
Other
41
EQ-i and EQ-360 Connections
Managers Q Q Q
Peers Q Q Q
Direct Reports
Independence
Self-Actualization
Intrapersonal Composite
Age DifferencesScales with significant* relationships:
*Magnitude was very small
42
EQ-i and EQ-360 Connections
Empathy Social Responsibilit
y
Interpersonal Composite
Managers F>M F>M Q
Peers F>M F>M F>M
Direct Reports
Q Q Q
Gender DifferencesScales with significant* relationships:
*Magnitude was very small
43
EQ-i and EQ-360 Connections
What do the differences mean?• Ratee may be inclined to inflate responses• Context of assessment may affect
responses• Supervisor unfamiliarity results in
inaccurate responses• Differing definitions of effective emotional
functioning between groups
44
Questions?
Part Four
Planning and Ethics
46
Part Four Objectives
• How to plan an EQ-360 assessment
• Getting organizational buy-in• Ethical issues in administration• Selecting Raters• How to debrief participants
By the end of this section, you should be familiar with:
47
Purposeful Planning
• What is the purpose of assessment?
• How will the results be used?• What does the ratee know about
or expect from the process?
48
Purposeful PlanningIssues to Consider:
•Timing•Extenuating circumstances•Personnel involvement•How will the results be used?•Methods available to confirm
confidentiality•Feedback•Who will be facilitating
development and action planning?
49
Purposeful Planning
Before you begin, establish:1. What the organization hopes to
gain from the assessment2. Connections between purpose and
specific business needs or goals3. Clear participant expectations for
accountability
Organizational Buy-In
• Senior Management support lends credibility to the process
• Prepare an EQ business case • Educate management team
on the value and relevance of the process (ROI)
OrganizationalBuy-In
Convey the importance of getting senior management involved:
– Consider creating a steering committee responsible for guiding the 360 process
– Show them how they will benefit
52
Selecting RatersIdentifying who to involve
• Supervisor, colleagues, direct reports, clients, and family members round out the mix
• Determine who gets input in the rater selection process– HR?
– The ratee?
– The ratee’s manager?
53
Selecting RatersDefining roles
• Will rater nominations be submitted to manager(s) and/or HR for approval or adjustment?
• The ratee’s manager:– Will his/her responses be singled out?
– Will he/she be involved in skill development and coaching?
54
Selecting Raters
•Are credible and trustworthy
•Work closely with the ratee
•Know the ratee well
Choose raters who:
For a fresh perspective, select some raters who have worked with ratee for only a short period of time.
55
Selecting Raters
Tips for defining raters:1.There must be at least three raters
per category.
2.If fewer than three, put raters in the Other/Mixed category to ensure confidentiality.
3.Anyone with a supervisory influence on the Ratee is considered a Manager. If there is only one manager, discuss confidentiality issues.
56
Ethical Issues
Who is the client?
• Individual respondent • Group• Organization
57
Ethical Issues
Confidentiality• EQ-360 ratees own their data• Sharing report/results and
determining action steps are decided by the ratee
• Breaches of confidentiality and rater anonymity can jeopardize feedback and compromise the integrity of the administrator
58
Ethical Issues
Ensuring Rater Anonymity• Raters feel more comfortable giving
honest responses when they are anonymous
• If they feel their identity is unprotected, raters may respond overly favorably or be reluctant to participate
59
Ethical Issues
Ensuring Rater Anonymity• Each rater category is scored
together as an aggregate• If there is only one rater in a
category, those results will be placed in the Other/Mixed group
• All online responses are stored ina secure MHS scoring database
60
Ethical Issues
Informed Consent• Obtain before the administration• Ensure voluntary participation and
freedom to withdraw from participating without punishment
• Provide contact information of administrator
• Check with your State/Provincial or Country psychology board regarding ethical issues
61
Ethical Issues
Debriefing Raters• Explain purpose of the debriefing
session• How did they find the EQ-360
process?• What were their expectations and
were they met?• How could the process be improved?
62
Ethical ScenariosFor each of the scenarios provided, consider the following questions:
1. What is/are the ethical issues with the scenario?
2. What steps should be taken to ensure the privacy of participants?
3. What are the appropriate steps that should be followed in order to rectify the issue?
4. What resources are available in order to rectify the issue?
63
#1: Acquisition
• An I/O psychologist has collected extensive developmental planning data on the top 100 executives of a very large corporation. The data includes the results from the EQ-360.
• Participants completing the EQ-360 were promised by the Board of Directors that the results would be used for development purposes only.
• Now, the organization is being acquired, and the team making staffing decisions for the merged organization has requested the 360 data.
64
#2: Succession Planning
• A management consultant has decided to relocate to another city and must leave behind data from a recent EQ-360 assessment used for development purposes with various managers from an Engineering company.
• The Engineering company indicated that it will not replace the management consultant. Instead, a HR manager will assume responsibility for the EQ-360 feedbacks, as well as the development initiatives.
65
#3: Scope Planning
• A consultant was hired by a firm to conduct
EQ-360 assessments.
• The organization has not yet decided whether the assessment results will be used for selection in addition to development, or who will have access to the data.
• The consultant is concerned and wants to tell the ratees and raters that the firm has not yet decided how the data will be used.
Part Five
Administering the EQ-360
67
Part Five Objectives
• Prepare EQ-360 participants• Use your Scoring Organizer
account• Select an administration format
By the end of this section, you should understand how to:
Preparing Participants
• Speak with the ratee and raters before administration
• Discuss with the participants their right to confidentiality, feedback, and the development of skills after the assessment
• Participants need to know how they will benefit from the assessment
69
Preparing Participants
• Brief participants on – What emotional intelligence is– What a 360° assessment is – How the EQ-360 will be used– What are the rating scales– Definitions of scales
70
Preparing Participants
• Discuss the importance of providing honest answers that reflect current functioning
• Discuss how long it will take to complete the inventory
• Clarify ratee and rater expectations
ExerciseCreate a correspondence to raters
Invite a group of raters to complete the EQ-360 on their coworker, Jim Smith.Your invitation should contain the following information:
• What the EQ-360 is
• Why they are being asked to complete it
• Reasons for participation
• That they will complete the assessment online
• What will be done with the results
72
Administration Options
• Online:– Link to survey
website in an email
– Web scoring
• Paper– Packets shipped
to participants– Mail/fax scoring
73
Scoring Organizer• You will receive an MHS Scoring
Organizer account that stores your rater information online
• Full instructions in the online Help and Getting Started Guide
74
Scoring Organizer
75
Online Administration
Setting up the EQ-360
• Each ratee must belong to a different Scoring Organizer Group
• Decide whether you wish to add open-ended items to the EQ-360
• Raters’ short responses will appear on the Coach Report
76
Online Administration
Add a new group for each individual ratee
77
Online AdministrationAdd up to 7 open-ended items:
78
Mail/Fax Administration
1. Use the item booklets, scanable response sheets, and scoring service processing form
2. Mail or fax forms to MHS for scoring
3. Reports can be scored from your Scoring Organizer account
79
Administration Service
No time to handle administration and scoring?MHS can contact raters for you, track completion, and send you scored reports
80
Questions?
Part Six
Scoring and Reports
82
Part Six Objectives
• Score EQ-360 reports• Differentiate the EQ-360
Client and Coach reports• Decide what reports to use• Find information in the reports
By the end of this section, you should know how to:
83
EQ-i RefresherReport Narrative
Information
Interpretive Guidelines
Bar Graphs Best for…
Development Report
Clinical with suggestions for development for lowest 3 subscales
Numeric 5 areas:“Very Much Below Average” to “Very Much Above Average”
Counseling and life coaching(Counselor’s Section stays with clinician)
Resource Report
Corporate with suggestions for development for lowest 3 subscales
Numeric in Counselor’s Section only
3 areas:Enrichment, Effective Functioning, Enhanced Skills; divided by approx 1 SD
Human capital development in organizations(Counselor’s Section stays with coach/HR)
BusinessReport
Corporate with suggestions for development for all 15 subscales
Numeric in Coach’s Section only
3 areas:Enrichment, Effective Functioning, Enhanced Skills; divided by approx 1 SD
Development planning with SMART goal setting and activity sheets(Coach’s Section stays with coach/HR)
EQ-360 Reports come with your choice of one of the above reports. Individual Summary Reports are generated separately.
85
Scoring ReportsScoring generates 2 EQ-i and 2 EQ-360 reports:
See your Getting Started with EQ-360 Guide for instructions
86
Scoring Reports
• Under EQ-360, select the EQ-i report type to determine the self-rated report output
• There is no need to score the EQ-i separately
87
Report Comparison
Coach Report Client Report• Provides the
administrator, coach, or employer* with in-depth response analysis, including validity and rater information
• Numeric cut-offs may be distracting to the ratee
• Not recommended for raters or the ratee
• Provides the ratee or employer with general response analysis and development information
• Sensitive language encourages positive change
• Not recommended for raters
*Access to reports requires the consent and permission of the ratee.
88
Report Comparison
Coach Report Client Report
Interpretive guidelines
< 8080-8990-109110-129>130
Develop Consider Developing Effective Well-Developed Enhanced
Coach’s summary section
Yes No
Open-ended items Yes No
89
Report Comparison
Coach Report Client Report
Numeric Non-numeric
90
Report Components
91
Report ComponentsDevelopment Strategies• Shown for up to three subscales with the
highest discrepancy between ratee and rater
92
Report Components
Coach’s Report Summary• Rater familiarity • Scale scores by rater group
(significant differences flagged)• Profile graph• Item response frequencies
(with mean score for each item)
93
Report Components
Open-Ended Items (Coach Report)
Respondents’ short descriptions give qualitative information about the ratee:
94
Questions?
Part Seven
Interpretation and Feedback
96
Part Seven Objectives
• Gauge validity of responses• Understand report results• Provide feedback to the ratee• Recommend strategies for
development• Deal with common challenges
By the end of this section, you should know how to:
97
Assessing Validity
Examine the overall validity of the EQ-360 assessment with regards to:
- EQ-i validity- Number of raters- Rater familiarity- Selection of raters
98
Assessing Validity
EQ-i Validity• Omission Rate• Positive Impression Scale• Negative Impression Scale (if
applicable)• Inconsistency Index• Ratee response style• Time taken to complete the EQ-i
99
Assessing Validity
Group Size• Ensure that there are enough
raters in each rater group (3 rater minimum)
• Ensure that there is a similar number of peers and direct reports
100
Assessing Validity
Rater Familiarity• Review rater familiarity chart
(in the Coach’s Summary)• Responses may not be accurate if
raters attest to knowing the rater “Not Very Well” or “Under 1 year”
101
Assessing Validity
If raters are selected by…• The ratee:
– Make sure that raters are objective and work with the rater in some current capacity
• The ratee’s manager or HR:– Verify rater relationships with the ratee– Manager and HR may not know all of
the people with whom the ratee works
102
Understanding the Results
• EQ-360 raw scores converted to standard scores
• Each EQ-360 scale has same mean and SD as EQ-i scales
Mean = 100SD = 15
103
• Review scores for scales and items:
•Total EQ•Composite scales•Subscales•Item responses
Understanding the Results
104
What are score discrepancies?
• Compared to EQ-i scores, are the EQ-360 scores significantly (> 10 points) higher or lower?
• Do the ratings of one rater group differ substantially from others?
Understanding the Results
105
• If ratee scores lower than rater – Indicative of underestimating own EI ability– May also reflect low self-awareness or self-
esteem
• If ratee scores higher than raters – Indicative of overestimating own EI ability– May reflect inability to view self critically,
ego centrism/narcissism
Understanding the Results
106
• Significant score differences between rater groups – Indicative that ratee behaves
differently with each rater group
• Consensus of scores between rater groups– Indicative of accurate picture of
ratee’s EI functioning most of the time and across situations
Understanding the Results
107
• EQ-360 results should be integrated with other data and information:– Interview data from ratee– Additional psychological and behavioral
assessments– Qualitative feedback of
colleagues/raters– Direct observation of ratee
Understanding the Results
108
Feedback Challenges
• Potentially vulnerable time• Defense mechanisms• Areas that need development are
for strategic correction, not blame or guilt
• Talk about strengths• Establish your role in the
development process
109
Providing Feedback
1. Set the Stage • Verify the client’s understanding of
the purpose of the assessment• Discuss how the results will be
used• Assure the client of confidentiality
110
Providing Feedback
2. Explain the Feedback Process• Ask when and where the client
completed the assessment• Point out that scores are a basis
for discussion• Discuss the reasons why there are
no numbers on the report
111
Providing Feedback
3. Discuss the report• Validity: explore what motivated
the client’s answers• Select subscales you want to
discuss• Ask the client what he or she thinks• Draw connections between
subscales• Identify patterns between rater
groups
112
Case Studies
• Feedback Report practice interpretation
Refer to case studies in the appendix of your workbook
Case Study #1Justin
• 33-year-old Customer Service manager
• EQ-360 to aid in professional development
• Referred by manager, who is grooming Justin for a middle management position
Case Study #2Linda
• 34-year-old Project Manager
• EQ-360 to aid in transition to new role
• Problems with staff management
• Referred by manager to HR
Case Study #3Amalie
• 49-year-old NFP CEO
• EQ-360 given to all executives to identify leadership gaps
• Amalie’s task-oriented work style differs from her sociable coworkers
116
Case Study #4Christopher
• 48-year-old commercial airline pilot
• Referred by HR due to co-pilot complaints
• Concern over communication and following logbook protocol
117
Case Study #5Bill
• 49-year-old sales manager
• Referred by VP due to interpersonal problems
• Concern over lack of staff commitment
118
Case Study #6Joan
• 55-year-old university department chair
• Directing a new department committee
• Instigated committee-wide EQ-360 for teambuilding and performance management
• Wishes to rekindle dedication and teamwork
119
Questions?
Part Eight
Coaching and Development
121
Part Eight Objectives
• Help the ratee create a personal development plan
• Assess the organizational factors that affect the ratee’s progress
• Reassess for success
By the end of this section, you should know how to:
122
Analyzing Development Strategies
• Strategies appear for the three subscales with the greatest discrepancy between Ratee and the All Rater Group
• Choose the strategies that are most appropriate to improving work performance
123
Analyzing Development Strategies
• Discuss accessible resources and support systems to help leverage client strategies
• Encourage the use of development strategies on the job
124
Creating a Personal Development Plan
• Develop a plan of action with clear goals and timelines
• The goal is lasting behavioral change
• Who, when, how: – Who will be involved– When will it begin– How it will happen
125
Creating a Personal Development Plan
• Pick one or two subscales that are relevant to job performance
• Record the current level of behavior in these subscales
• List how the attainment of goals will be beneficial to the organization
126
Creating a Personal Development Plan
• List internal and external factors that will – Enhance your EQ
performance– Hinder your EQ
performance
• Schedule regular feedback from peers, direct reports, and manager
ExerciseCreate a personal development plan
Jim Smith’s results show that he is low on the Interpersonal Relationship scale, which is important to his job as an engineering supervisor.His plan should contain the following information:
• The end goal
• How development will benefit the organization
• Who, when, how
• Timelines
• Factors that will help or hinder progress
128
Follow-up Coaching
• Summarize prior meeting
• Check whether progress towards goals has been made
• When a goal is met…– Congratulate!– Choose another EQ-i
area to focus on– Set new goals
129
Follow-up Coaching
• When a goal is not met… – Look to see if goals were
unrealistic– Set new goals with new
timelines and action steps– Find additional resources
to achieve revised goals
130
• Begin with organizational objectives • Identify a job-specific competency
model with detailed skills and behaviors
• Quantify performance outcomes such as customer satisfaction, leadership, sales, profit, or deadline adherence
• Establish cut-offs for poor performance
Measuring Outcomes
131
• What EQ-i areas are most important to organizational and individual success?
• What competencies will you investigating?– Individual?– Organizational?
• How do the EQ-360 results map to the strategy or needs of the organization?
Measuring Outcomes
132
Reassessment to Gauge Progress
• Repeat assessment as a measurement of outcomes
• Annual assessment is the norm but more frequent assessments may be needed
• Compare effectiveness over time• Do performance measures need to change?
133
Questions?
Wrap Up
Review
Goals
1. 360° assessments2. Using the EQ-3603. Psychometrics4. Planning and
ethics5. Administration
6. Scoring and reports
7. Interpretation and feedback
8. Coaching9. Exam
Exam
Interpret a sample report (60 minutes)
137
EQ-360 Assessment
• Complete a EQ-360 administration, interpretation, and feedback on a client of your choice
• Provide a detailed written description of the following (include copies of forms and letters used if any):– How you addressed informed consent– How you set up the administration– How you conducted your initial interpretation– Describe the feedback session(s)– Describe any additional plans discussed
with the client
138
EQ-360 Assessment
• Required for qualification• Format & length• Due date• Submission
Thank You• Please fill out an evaluation form
• How to contact me