Envisia Learning

Envisia Learning

360° Feedback Tools and Interpretation Guide

Envisia Learning is a leader in providing innovative assessment products, services, and other Internet based resources that are responsive to the unique needs of consultants and coaches and the individuals they serve throughout the world.

We are committed to building strong, mutually beneficial, and enduring relationships with a focus on providing superior customer service, high-quality products, and excellent price value to our Customers.

Who we are…

Envisia Learning has developed an approach to customer assignments, which is characterized by:

A thorough understanding of the current needs of our customers before proceeding at each stage of the assignment

A mutual understanding between the customer and the consultants of the objectives, work processes, timing and results

Providing appropriate opportunities for participation by those affected by the project

A thorough review of project plans with the customer

A commitment to customer satisfaction that the mutually agreed upon desired results have been met

Our Approach…

Toll-free customer support line to talk to our customer service support staff or Industrial/Organizational Psychologist at (800) 335-0779

Coach/Consultant online administration of their own 360 feedback projects

Branding our tools with your logo - regardless of how much or how often you buy for online administration and reporting

Free trials to your potential clients to support your marketing efforts!

No requirement of certification programs or fees as a requirement to purchase any product in the ViewSuite360

Free resources to help you utilize our tools using the best practices

Ability to monitor rater activity and generate progress reports for all ViewSuite360 projects

About Customer Support…

Long-term research, knowledge and experience of published specialists

Web-Based Performance Learning Solutions

360° Feedback

Personality, Career & Stress Assessment

Survey Solutions

Proven Range of Products based upon…

Multi-Rater Feedback Definition

A systematic process of data collection and feedback

from relevant observers (e.g., Supervisors, Direct

Reports, Peers, Team Members, Customers)

targeting job-relevant competencies required for

effective performance on the job.

MULTI-RATER / 360° FEEDBACK IS DEFINED AS:MULTI-RATER / 360° FEEDBACK IS DEFINED AS:

360° 360° FeedbackFeedback

InterventionsInterventions

SuccessionPlanning

Performance Management

Leadership Development

Executive Coaching

Integrated 360° Feedback Systems

Introducing ViewSuite360

ViewSuite360 Comprises:

ExecutiveView360

LeaderView360

ManagerView360

PerformanceView360

TeamView360

Emotional IntelligenceView360

Purpose

Assessment of 22 senior executive competencies in the areas of Performance Leadership, Change Leadership, Interpersonal Leadership and Personal Leadership based on succession planning executive profiling

Outcome

Development of senior executives

Enhanced awarness of board members, peers and direct reports

Improve communication and relationships with others

Audience

Senior executives and senior management

ExecutiveView360

ExecutiveView360

Performance Leadership Visionary Leadership Drive for Results Technological Leadership Financial Leadership Cross Functional Versatility Depth of Industry Knowledge Political Leadership Strategic Problem Analysis Decision Making

Change Leadership Entrepreneurial Leadership Driving Strategic Direction Driving Change

Interpersonal Leadership Build Strategic Relationships Empowering Others Team Building Interpersonal Effectiveness Oral Communication/Presentation Influence/Negotiation Coaching/Talent Development

Personal Leadership Self-Development Adaptability/Flexibility Engenders Trust

Respondents:Self, Board, Peers, Reports, Team Members

Purpose

Analysis of managerial strengths and developmental needs measuring 20 Leadership, Interpersonal, Communication and Problem Solving competency areas based on job analyses of managers in diverse industries

Outcome

Improved communication & performance

Enhanced supervisory and management skills

Increased productivity

Audience

Supervisors up to senior managers

ManagerView360

ManagerView360

Communication Listening Written Communication Vision/Goal Setting Two-Way Feedback Oral Presentation Oral Communication

Task Management Planning/Organizing Follow-Up Performance Management Delegation Performance Review Recognizing Performance

Interpersonal Team Development Interpersonal Sensitivity Negotiation/Conflict

Management Coaching/Employee

Development Leadership/Influence Employee Involvement

Problem Solving Strategic Problem Analysis Decisiveness/Judgment

Respondents:Self, Boss, Reports, Peers

PurposeAnalysis of individual strengths and developmental needs measuring 14 Task Management, Interpersonal/Team, and Communication competencies

OutcomeImproved communication & performance

Enhanced interpersonal and task/project management skills

Increased productivity

AudienceIndependent Contributors and Non-Managers

PerformanceView360

PerformanceView360

Communication Listening Two-Way Feedback Written Communication Oral Communication Oral Presentation

Task Management Planning/Organizing Task/Project Management Problem Solving Decisiveness/Judgment

Interpersonal/Team Collaboration Interpersonal Sensitivity Conflict

Management/Negotiation Team Support Leadership/Influence

Respondents:Self, Supervisor, Peers, Team Members

PurposeAssessment of 7 critical leadership competencies

OutcomeDevelopment of senior staff

Enable a self development strategy

Refine work processes

Improve communication and relationships with others

AudienceMiddle and senior management

LeaderView360

LeaderView360

Competencies Planning Problem-Solving Controlling Self Management Managing Relationships Communicating Leading

Participants:Managers

Respondents:Self, Boss, Reports, Peers

PurposeMeasures each individual's scores compared to the team members and the teams' overall average scores within 7 major competencies

OutcomeUnderstanding and enhancing team effectiveness and performance

AudienceTeams

TeamView360

TeamView360

Participants:Team Members

Respondents:Self, Team Members

Competencies Planning Problem-Solving Controlling Self Management Managing Relationships Communicating Leading

Purpose

Assessment of 17 Emotional Intelligence Competencies based on the

latest emotional intelligence model

Outcome

Increase effectiveness in self management, relationship management and communication competencies

To be used with

All Employees


Relationship Intelligence Model Competencies

• Empathy• Organizational Awareness• Service Orientation

• Self-Development

• Adaptability/Stress

Tolerance

• Self-Control

• Trustworthiness

• Strategic Problem

Solving

• Achievement

Orientation


Self Management Self-Development Adaptability/Stress Tolerance Self-Control Trustworthiness Strategic Problem Solving Achievement Orientation/Drive for

Results

Communication Listening Oral Communication Two-Way Feedback Oral Presentation Written Communication

Relationship Management Building Strategic Relationships Conflict Management Leadership/Influence Interpersonal Sensitivity/Empathy Team/Interpersonal Support Collaboration

360° Feedback/Online Process

ParticipantsAdded

E-Mail InvitationNominate

Respondents

Self QuestionnaireCompleted

Others QuestionnaireCompleted

Report Processing FeedbackChasing-up

Questionnaire Invite sent to Respondents

Adding Participants

E-Mail Invitation

Rater Nomination

Rater Nomination

Rater Nomination

Rater E-Mail Invitation

Rater Online Questionnaire: Introduction

Rater Online Questionnaire: Instructions

Rater Online Questionnaire

Rater Online Questionnaire

Rater Online Questionnaire: Written Comments

Rater Online Questionnaire: Review Responses

Rater Online Questionnaire: Edit Responses

Rater Online Questionnaire: Completion

Rater Online Questionnaire: Thank You

Online Administration: Reminder

E-Mail Reminder

1. Introduce the 360 tools to be used in the coaching or development program and their competencies

2. Briefly describe what will be included in the feedback reports

3. Briefly review confidentiality in the process and reports

4. Distribute the reports and discuss each of the sections separately

5. Review the number and type of raters who provided feedback

6. The proceed through the different sections of the report emphasising some “key points”

Steps in Feedback Report Interpretation

1. Group (composite) reports to highlight team strengths and development areas

2. Time series reports to compare and contrast changes when the ViewSuite360 questionnaires are used over time

3. Company specific norms, ConsultingTools national norms or average scores

4. Line or bar graphs comparing self to other ratings5. Statistical measure of rater agreement to assist in the interpretation6. Option of “Johari Window” summary graphs (online reports only)7. Option of rater distribution for most/least frequent tables (online

reports only)8. Options of how free response items (open ended questions) are

summarised9. Customised narrative for ManagerView360 and PerformanceView36010. Branding of consultant/client logo on the report

ViewSuite360 Report Features

To facilitate the implementation of a

professional development action plan

To identify skill strengths to enhance

and leverage job performance

To provide participants with feedback about developmental

opportunities

ViewSuite360 Feedback Process Objectives

Graphical Comparisons Most and Least Frequently Observed Behaviours Narrative Interpretation with Developmental

Recommendations for each competency measured Summary of Average Scores by Rater Category Statistical Measure of Rater Agreement Written Comments Developmental Action Plan

Feedback Report Components

Key Points No raters are identified except for the boss in the feedback report Online administration uses passwords to protect confidentiality (Internet

administration) No graphs are shown unless at least two raters respond in a rater

category (e.g., peers) The summary feedback report is shared only with the respondent and is

intended for development purposes only The respondent decides how much of the summary feedback report

he/she wants to share with others

Confidentiality of the Feedback Process

Key Points Graphs (line or bar) will utilise either average scores or ConsultingTools

norms (standardised scores) The first graph is an overview of the major performance factors. The

others that follow summarise self and other perceptions on each of the separate competencies

The legend at the bottom of the graph will summarise rater groups and the number of raters

The symbol “AP” is used for anonymity protection and indicates that less than 2 raters have completed the questionnaires in this rater category

The normalised (standardised) scores reported are z-scores with “0” being the average score, or t-scores with “50” being the average score for those in the ConsultingTools database

68% of all participants will score between a z-score of +1 and -1 (one standard deviation above and below the mean)

Percentile scores are shown on the opposite axis for ease of interpretation Normative score differences on a competency between raters of .50 or

more might be considered as meaningful perceptual differences

Graph Section

Self-Other Perceptions (Line Graph – z-scores)

Self Management

0.610.43

0.98

1.38

0.26

1.91

2.30

1.16 1.35

1.161.08

0.16

- 0.21

0.85

1.501.45

1.02

0.64

- 2.87

- 2.12

- 0.23

- 1.56

- 1.02

- 0.47

1.65

- 0.16

- 0.47

- 1.09

- 0.31

- 0.59

-3

-2

-1

0

1

2

3

Self-Development Adaptability/StressTolerance

Self-C ontrol Trustworthiness Strategic ProblemSolving

AchievementOrientation

Sta

nd

ard

ized

Sco

res

Se lf (1) Manager (1) Peer (3) Direct Report (3) Team Mem ber (2)

(99%)

(98%)

(93%)

(84%)

(70%)

(50%)

(30%)

(26%)

(17%)

(2%)

(1%)

Very High

High

Mod High

Average

Mod Low

Low

Very Low

Self-Other Perceptions (Line Graph – t-scores)

Self-Development97.29

97.29

25.6525.65

40.14

54.43

83.00

68.71

49.57

59.19

63.95

54.43

19.57

54.43

21.10

49.57

61.57

54.43

25.65

20

25

30

35

40

45

50

55

60

65

70

75

80

(1) (18) (35) (52)

Sta

nd

ard

ized

Sco

res

Se lf (1) Manager (1) Peer (3) Direct Report (3) Team Mem ber (2)

(99%)

(98%)

(93%)

(84%)

(70%)

(50%)

(30%)

(26%)

(17%)

(2%)

(1%)

Very High

High

Mod High

Average

Mod Low

Low

Very Low

Self-Other Perceptions (Line Graph – avg-scores)

Self Management

4.00

4.50

4.805.50

6.00

4.50

5.005.20

4.504.50

3.75 3.75

4.93

4.25

5.09

5.27

4.50

4.17

2.50

2.71

3.38

2.50

2.80

3.73

4.63

3.303.13

2.60

3.223.29

1

2

3

4

5

6

7

Self-Development Adaptability/StressTolerance

Self-C ontrol Trustworthiness Strategic ProblemSolving

AchievementOrientation

Self (1) Manager (1) Peer (3) Direct Report (3) Team Mem ber (2)

Self-Other Perceptions (Bar Graph – z-scores)

-3 -2 -1 0 1 2 3

Direct reportTeam Member

Average

Peer

Manager

Self

Building Strategic RelationshipsZ N

5.54 1

4.69 14.42 3

-3.35 3

1.64 2

1.27 9

-3 -2 -1 0 1 2 3


Average

Peer

Manager

Self

24. Effectively builds relationships and partnerships with others outside the organizationZ N

4.99 1

7.48 15.51 3

-2.83 3

2.48 2

2.48 9

-3 -2 -1 0 1 2 3


Average

Peer

Manager

Self

7. Effectively initiates and cultivates strategic internal alliances with key senior managers and others within the organizationZ N

7.48 1

7.48 12.48 3

-1.69 3

-1.25 2

0.61 9

20 25 30 35 40 45 50 55 60 65 70 75 80

Self-Other Perceptions (Bar Graph – t-scores)

20 25 30 35 40 45 50 55 60 65 70 75 80


Average

Peer

Manager

Self

Building Strategic RelationshipsT N

116.42 1

80.75 194.19 3

16.42 3

68.42 2

52.71 9


Average

Peer

Manager

Self

24. Effectively builds relationships and partnerships with others outside the organizationT N

80.75 1

124.75 1

105.05 3

24.75 3

74.75 2

74.75 9

20 25 30 35 40 45 50 55 60 65 70 75 80


Average

Peer

Manager

Self

7. Effectively initiates and cultivates strategic internal alliances with key senior managers and others within the organizationT N

124.75 1

124.75 1

74.75 3

33.05 3

37.25 2

58.08 9

Self-Other Perceptions (Bar Graph – avg-scores)

1 2 3 4 5 6 7


Average

Peer

Manager

Self

Building Strategic RelationshipsAvg N

6.57 1

6.00 1

5.78 3

2.57 3

4.67 2

4.52 9

1 2 3 4 5 6 7


Average

Peer

Manager

Self

24. Effectively builds relationships and partnerships with others outside the organizationAvg N

6.00 1

7.00 1

6.33 3

3.00 3

5.00 2

5.00 9

1 2 3 4 5 6 7


Average

Peer

Manager

Self

7. Effectively initiates and cultivates strategic internal alliances with key senior managers and others within the organizationAvg N

7.00 1

7.00 1

5.00 3

3.33 3

3.50 2

4.33 9

Self-Awareness Report

Key Points The “Most Frequent” section and “Least Frequent” section summarises

those competencies and behaviours that were most frequently/least frequently observed by various rater groups

The number in the first column corresponds to the average score for all raters providing feedback (1 to 7 scale or 1 to 5 scale)

The “Most Frequent” should be considered as perceived strengths to leverage and build on

The “Least Frequent” should be considered as possible behaviours to practice more frequently

Most/Least Frequent Report Section

Most/Least Frequent Report - Sample

Most Frequent Behaviors – All Raters

Key Points Each question is summarized and categorised in its appropriate

competency Average scores across all raters are reported for each competency and

question A statistical measure of rater agreement based on the standard deviation

is reported as a percentage - a score less than 50% suggests that the raters providing feedback had enough disagreement to warrant a cautious interpretation of the average score reported (e.g., raters had diverse perceptions and rated the participant quite differently on that question or competency)

An overall average of all raters is calculated in the last column

Item Behavior Report

Behavior Summary Report

Key Points Comments are randomly listed by all raters who volunteered to share

written perceptions Comments are provided verbatim from the online questionnaire - no

editing Some comments are specific, behavioural and constructive - others may

be less useful or hard to understand It is important to focus on themes that emerge, rather than, to dwell on any

one individual comment

Written Comments Section

Sample – Comments Section

Development Plan Section

Key Points The ViewSuite360 tools provide developmental planning worksheets to

summarize strengths and development areas to initiate a development plan

Participants should be encouraged to complete these pages and commit to enhancing at least one strength and focusing on one developmental area as a result of this 360 degree feedback process

Research suggests that when these developmental plans are based on SMART objectives (specific, measurable, achievable, realistic, time based) and in writing they are more likely to achieve desired results

Development Plan Section

“Our life transformation is in exactproportion to the amount of truth we can take without running away.”

-Vernon Howard

The Talent Accelerator is a web-based professional development tool to help you understand your online assessments and assist you in implementing a plan for your professional development

The Talent Accelerator will provide you with a guided process for developmental planning based on “Best Practices” of how people change

The online tool is designed to help translate awareness from all of our assessments into lasting behavior change

Description of the Talent Accelerator

Development Resource Library: The Talent Accelerator resource library provides a comprehensive source of readings, websites, media, and suggestions to facilitate your development.

Feedback Reports: The Talent Accelerator provides you with an electronic copy of your assessment summary report.

Development Suggestions: For each assessment tool, specific developmental suggestions or tips are provided for you to try on the job to enhance your effectiveness

Development Planning “Wizard”: Our development “wizard” will walk you through your assessment report and provide a structured way to allowing you to focus on those behaviors that are most important

Automated Reminders: Talent Accelerator allows you to set select how often you want the system to send you reminders about due dates on your development plan (Preference Tab)

Components of the Talent Accelerator

Login to the Talent Accelerator

Viewing Assessments & Selecting Development Areas

Creating Your Development Plan

Monitoring Your Development Plan

Do I understand my ViewSuite360 summary feedback report?

Does it seem accurate/valid?

Is the feedback similar or different for the different rater groups?

What are my perceived strengths to leverage for future success? Are the areas perceived by others for development relevant to my

current or future position?

Am I motivated to change?

Feedback Report Questions to Consider

Review your summary feedback report

Thank those who have provided you with feedback Obtain additional feedback from your manager, direct

reports, peers and team members

Identify specific developmental goals

Draft a development plan

Meet with your manager to finalise your plan

Implement your development plan

Track and monitor progress

Re-assess ViewSuite360 tools in 10-12 months

Next Steps for Participants Using Our Products

Dr. Kenneth M. Nowack is a licensed psychologist and Chief Research Officer of Envisia Learning. He has over twenty years of experience as an executive coach and in the development and validation of assessment instruments, organizational climate surveys, questionnaires, simulations, and tests.

He received his B.S. and M.S. degrees in Educational Psychology at the University of California, Davis, and his Ph.D. in Counseling Psychology from the University of California, Los Angeles. Dr. Nowack has conducted research and published extensively in the areas of behavioral medicine, 360-degree feedback, executive coaching, assessment centers, training needs assessment, and evaluation.

Dr. Nowack is a member in Daniel Goleman's Consortium for Research on Emotional Intelligence in Organizations: http://www.eiconsortium.org/members/membership_consortium.htm

About Our Co-Founder…

Nowack, K. (2005). Longitudinal evaluation of a 360 degree feedback program: Implications for best practices. Paper presented at the 20th Annual Conference of the Society for Industrial and Organizational Psychology, Los Angeles, March 2005

Nowack, K., (2002). Does 360 degree feedback negatively effect company performance: Feedback varies with your point of view. HR Magazine, Volume 47 (6), June 2002. Work. Training Magazine

Nowack, K. and Heller, B. (December, 2001). Making Executive Coaching Work. Training Magazine www.trainingmag.com

Nowack, K., Hartley, J. & Bradley, W. (1999). Evaluating results of your 360-degree feedback intervention. Training and Development, 53, 48-53.

Nowack, K. (1999). 360-Degree feedback. In DG Langdon, KS Whiteside, & MM McKenna (Eds.), Intervention: 50 Performance Technology Tools, San Francisco, Jossey-Bass, Inc., pp.34-46.

Wimer, S. and Nowack, K. (1998). How to benefit from 360-degree feedback. Executive Excellence, Volume 15, No. 10, 16.

Wimer, S. and Nowack, K. (1988). Thirteen common mistakes in implementing multi-rater feedback systems. Training and Development, 52, 69-80.

Selected References

Nowack, K.M. (1997). Self-ratings as a predictor of assessment center performance. Journal of Social Behavior and Personality, 12 (5), 145-166.

Nowack, K. (1997). Manager View/360. In Fleenor, J. & Leslie, J. (Eds.). Feedback to managers: A review and comparison of sixteen multi-rater feedback instruments (3rd edition). Center for Creative Leadership, Greensboro, NC.

Nowack, K. and Wimer, S. (1997). Coaching for human performance. Training and Development. Volume 51, No.10, 28-32.

Nowack, K. (1994). The secrets of succession: Focusing on development in succession planning systems. Training & Development Journal, 48, 49-54.

Nowack, K. (1992). Self-assessment and rater-assessment as a dimension of management development. Human Resources Development Quarterly, 3, 141-155.

Nowack, K. (1993). 360-Degree feedback: It takes two to know one. Training & Development Journal, 47, 69-72.

Selected References (Continued)

Please visit our website at …

www.envisialearning.com

Further Information

Documents

Envisia Learning