Designing the Model for Evaluation

Model: Svenson and Rinderer, 1992

Without measurement,there is no possibility of

continuous improvement.

OBJECTIVES:

1. Introduce Svenson.

2. Review Evaluation Types, Traditions, and Perceptions.

3. Learn the Role of Evaluation in the Strategic Planning Process (according to Svenson).

4. Discuss Svenson’s Nine-Step Results-Measurement Model

5. Determine the Applications of Svenson’s Model to Evaluation Traditions and Perceptions.

Who is Svenson?

Co-author (with Monica Rinderer) of:

The Training and Development Strategic Plan Workbook

More than twenty-five years experience in training

One of the five models for Evaluation

E v a l u a t i o n D e f i n i t i o n s :

1 . G e n e r a l T h e s y s t e m a t i c p r o c e s s o f c o l l e c t i o n ,a n a l y s i s , a n d i n t e r p r e t a t i o n o f i n f o r m a t i o nt o d e t e r m i n e a c h i e v e m e n t o f g o a l s a n do b j e c t i v e s .

2 . S u m m a t i v e C o l l e c t i n g d a t a t o d e t e r m i n e i f t h e t r a i n i n g p r o g r a m h a s a c h i e v e d i t s i n t e n d e d g o a l s i n c l u d i n g u n a n t i c i p a t e d o u t c o m e s a n d m a k i n g c o m p a r i s o n s w i t h a l t e r n a t e p r o g r a m s .

3 . F o r m a t i v e P e r p e t u a l c o l l e c t i o n a n d a n a l y s i s o f d a t a f o r t h e p u r p o s e o f r e v i s i n g a n d i m p r o v i n g t h e t r a i n i n g p r o g r a m .

Evaluation Traditions:

1. Scientific Using control of quantitative variables.

2. Systems Using flow of order (i.e. sequential modules).

3. Qualitative Based on value.

4. Eclectic Collected from various sources.

Evaluation Perceptions(Viewpoints):

1. Discrepancy Comparison to known benchmarks.

2. Democratic Changes made mostly from a governing board.

3. Analytical Based on empirical data (taken over time).

4. Diagnostic Research-based examination of program by sections.5. Other

Strategic Planning Process Phases (Svenson and Rinderer, 1992)

StrategicVision &Goals

AlternativeStrategies &

ResourceRequirements

OrganizationManagementAdministrative

Strategies

Implementation

Organizationof

Training

Governance &AdvisoryStructure

MeasuringResults

Supervisor & Management

Support

Finance&

Accounting

Primary Results

Secondary Results

Results Data

Sources

SystemDesignProcess

Svenson’s Nine-Step ModelFor Measuring Results

1. Identify Decisions.

2. Define Results to be Measured.

3. Identify Required Data.

4. Define Data Sources and Measurement Means.

5. Specify Reports.

6. Specify Storage/Retrieval Database.

7. Design Overall Information Flow.

8. Design Reports-Handling Administrative System.

9. Evaluate Cost of Evaluation Compared to Benefits.

Svenson’s Results Measurement Model for Evaluation

2. Define Resultsto be Measured

4. Define DataSources andMeasurementMeans

PrimaryResults

SecondaryResults

Training Needs MetBenefits from Training Needs MetCost of Training Needs MetQuality/Effectiveness of TrainingProvidedProductivity of Training ResourcesTraining Needs Not MetCost of Training Needs Not Met

Needs Analysis and CurriculumProcessInstructional MaterialsDevelopmentInstructional DeliveryTraining Facilities Utilization

Registration and Scheduling DataPersonnel DataAccounting DataTrainee Feedback QuestionnaireInstructor Feedback QuestionnaireSupervisor Feedback QuestionnaireMastery Test ResultsEvaluation Team Data

ResultsDataSources

1. IdentifyDecisions

3. IdentifyRequired data

IdentifyDefine

continued

9. Evaluate Cost ofEvaluation Compared toBenefits

8. DesignReports-HandlingAdministrativeSystem

6. SpecifyStorage/RetrievalDatabase

7. Design OverallInformation Flow

5. SpecifyReports

DesignSpecify

Svenson’s Results Measurement Model for Evaluation(continued)

Measuring Results

1. Primary Training Results

2. Secondary Results

3. Results Data Sources

4. System Design Process

Primary Training Results Training needs met. Organizational benefits from training needs met. Cost of training needs met. Quality/effectiveness of training provided. Productivity of training resources. Training needs not met. Organizational cost of training needs not met.

Secondary Results

Needs analysis & curriculum architecture process results.

Instructional materials development results.

Instructional delivery results.Training facilities utilization.

Company Annual Training Results This Year Last Year Deviation

Number of Employees 3730 3925 5.0%

Hourly 3000 3200 -6.25% Professional 350 325 -6.25% Management 380 400 -5.00%

Number of Training per Employee

Hourly 5.1 4.6 +10.9 Professional 10.3 7.2 +43.1 Management 7.4 6.5 +13.4

Results Data Sources

Registration and scheduling data. Personnel data. Accounting data. Trainee feedback questionnaire. Instructor feedback questionnaire. Supervisor feedback questionnaire. Mastery test results. Evaluation team data.

“It’s Results That Matter”

George Bergeron,

ALCOA RPD President

Evaluation (Review)

1. Types:GeneralSummativeFormative

2. Traditions:ScientificSystemsQualitativeEclectic

3. Perceptions (Viewpoints):DiscrepancyDemocraticAnalyticalDiagnosticOther

Questions from Svenson Presentation

1. What evaluation type, tradition, and perception does Svenson’s system fit into best and why?

“Statistical” Perception: The “Other” Evaluation Perception

Employing the principles of statistics this perception deals with the process of the collection, analysis, interpretation, and presentation of masses of data from the training program’s results sources.

Questions from Svenson Presentation

2. Give an example of how Svenson’s model could work in your industry.

Questions from Svenson Presentation

3. Differentiate Svenson’s model from any of the other models of evaluation.

References:

Svenson, R. & Rinderer, M. (1992). The Training and Development Strategic Plan Workbook. Prentice-Hall, Englewood Cliffs, N.J.

Rae, L. (1993). Evaluating Trainer Effectiveness. Business One Irwin, Homewood, Ill.