Analysis PhaseTECM 5180

Dr. Lam

Agenda

• Outline the purpose of analysis• Describe three models of needs analysis• Walk through learner analysis • Wrap up and tie into design phase, which

we’ll discuss next week

Types of analyses

• There are several reasons an instructional designer must conduct analyses.

1. To determine if we need instruction (Needs assessment)

2. To determine (specifically) what performance gaps there are (performance assessment)

3. To determine exactly what the scope of the training should include (training needs assessment)

4. To determine who our learner is and what they need (audience analysis/ learner assessment)

Needs Assessment

• When do we conduct a needs assessment? • Condition A: There is a problem: Clients are

not satisfied; students are dropping out; parents are complaining; test scores are down; etc.

• Condition B: There is or may be something new that learners need to learn: New guidelines for healthcare; etc.

• Condition C: No big problem is apparent but the organization wishes to or is required to engage in evaluation of its learning/training: Accreditation; etc.

Models of Needs Assessment

• Problem Model – Determine what the problem is and if training is the solution.

• Innovation Model – Determine changes/innovations in the organization and determines whether new learning goals should be added.

• Discrepancy Model – Determines whether existing instruction adequately meets learners’ needs (summative evaluation model)

The Problem Model1. Determine whether there really is a problem• Who says there is a problem?• Why do they say there is a problem?• do others perceive it to be a problem• Who disagrees?• When was the problem first noticed?• Who is affected?; How pervasive is the problem• How seriously does the problem affect the mission of the

organization?

Goal: Identify and clarify the problem and determine how serious it actually is.

The Problem Model2. Determine whether the cause of the problem is related to employees’ performance in training environments or learners’ achievements in educational environments. • How does employees’ performance related to the

problem?• Is this relationship correlational?• Does performance appear to be impacted by another

factor that causes both the problem and the performance deficits?

• What evidence is there that suggests that performance or achievement deficits cause the problem or affect it?

The Problem Model

3. Determine whether the solution to the problem is learning • Is there evidence that performance problems

may be caused by motivation, incentives, facilities design, tools design, policies, or other nonlearning factors?

The Problem Model

4. Determine whether instruction is currently offered.• If no, proceed to innovation model. • If yes, proceed to discrepancy model.

Innovation Model

1. Determine the nature of the innovation or change• Has there been a change in the composition of

the learner population?• Has there been a significant change in the

tools, policies, or organization?• Has there been a significant change in the

training philosophy of the organization? • How will the innovation affect the mission of

the organization?

Innovation Model

2. Determine the learning goals that accompany this innovation.• How will the innovation or change affect what

is expected in performance?• Does this effect significantly change what

employees must understand, know, or do?• Can these new understandings, knowledge, or

actions be taught?

Innovation Model3. If there is a choice, determine whether these goals are appropriate and high priority in the learning system.• Are the resources available to support this instruction?

Are they adequate to design and develop this instruction?

• Do these goals conflict with existing goals?• How will these goals be interpreted by affected groups?• Do these goals represent partisan positions or vested

interest groups?• Are there groups that may object to these new goals?

Innovation Model

4. Begin design activities

Discrepancy Model (Gap analysis)

1. List the goals of the instructional system • Learning goals are “what ought to be”• Goals can be for a lesson, unit, course, semester,

year, or several years• In business, goals are usually defined with the help

of SMEs

"Each learner can recognize and analyze ethical problems and choose and defend resolutions for practical situations that occur in accounting, human resource management, and marketing."

Discrepancy Model (Gap analysis)

2. Determine how well the goals are already being achieved. • Paper and pencil tests• Observations of individuals completing tasks on the

job or in simulated environments• Error rates in quality control/assurance systems

"Each learner can recognize and analyze ethical problems and choose and defend resolutions for practical situations that occur in accounting, human resource management, and marketing."

Discrepancy Model (Gap analysis)

3. Determine the gaps between “what is” and “what should be”• Gaps can be very precise; e.g., employees collectively

scored an average of 64% on a paper or pencil exam• They can be less precise, but still rooted in evidence.

E.g., “6/10 employees were not able to recognize ethical problems when provided four different scenarios”.

"Each learner can recognize and analyze ethical problems and choose and defend resolutions for practical situations that occur in accounting, human resource management, and marketing."

Discrepancy Model (Gap analysis)

4. Prioritize gaps according to agreed-upon criteria. Consider the following:• The biggest gaps first• The most critical goals • The number of students affected• Consequences of not meeting the goal• Probability of reducing the gap

Discrepancy Model (Gap analysis)

5. Determine which gaps are instructional needs and which are most appropriate for design and development of instruction.

Data Collection Methods

• Analysis of extant data• Analysis of the subject matter• Interviewing • Observing • Focus groups• Questionnaires • Surveys

Weigh the Pros and Cons of each Method

• Analysis of extant data• Analysis of the subject matter and subject matter

experts• Interviewing • Observing • Focus groups• Questionnaires • Surveys

Learner Analysis

• Once we’ve determined a need for training and before we can start designing instruction, we should start to think about our learners.

• Understanding our learners can help us when making design decisions like what to include in the training, how to deliver it, and how much time to spend on a given topic.

• Sometimes done in the analysis phase, other times it’s done at the beginning of the design phase

A few problems to avoid…

1. Ethnocentrism 2. Describing characteristics you hope the

learners include, and not their actual characteristics

3. Relying on assumptions and/or using only extant data

A(nother) Model for Learner Analysis

Similarities DifferencesStable • Sensory capacities

• Information processing

• Types and conditions of learning

• Aptitudes • Cognitive styles• Psychosocial traits• Gender, ethnicity, and

racial groupings

Changing Development processes • Intellectual • Language• Psychosocial • Moral• Other

Development State• Intellectual • Other

Prior Learning• General• Specific

Definitions

• Stable similarities: similarities among people that are relatively unchanging over time.

• Stable differences: differences among people that are relatively unchanging over time.

• Changing similarities: similarities among people that change over time.

• Changing differences: differences among people that change over time.

Stable SimilaritiesSimilarities Differences

Stable • Sensory capacities• Information

processing• Types and conditions

of learning

• Aptitudes • Cognitive styles• Psychosocial traits• Gender, ethnicity, and

racial groupings

Changing Development processes • Intellectual • Language• Psychosocial • Moral• Other

Development State• Intellectual • Other

Prior Learning• General• Specific

Stable Similarities

• Sensory capabilities- While they differ in degree, they are similar among most humans.

• Practical Implications: As designers, it may seem obvious, but it’s important to consider things like adequate lighting, appropriate font size, loud enough audio elements, etc.

Stable Similarities• Information Processing- The way humans

process information has several practical implications for instructional designers. For instance, Mayer’s cognitive theory of multimedia learning outlines specific limitations on human processing when processing audio, video, and animation at the same time (cognitive overload).

• Practical Implications: Deciding on delivery systems should rely heavily on what we know (or can learn/research) about human information processing.

Stable Similarities• Types of learning- Fundamental ways in which humans

acquire different types of learning (declarative knowledge, concepts, rules, and problem solving—See Gagne (1985).

• Conditions of learning – internal and external conditions that must be present to learn a specific type of learning. For example, to learn an attitude, you much be exposed to a credible role model or persuasive arguments.

• Practical Implications: Your instructional strategy should thoughtfully keep in mind types of learning and provide adequate conditions for learning.

Stable Differences

• Aptitudes- ability related to readiness or facility to learn or achieve• Single factor aptitude- Intelligence • Multiple factor aptitude- Tests like SAT,

ACT, etc. • Multiple Intelligences- Gardner, 1993 is

most famous for this (verbal, logical/mathematical, interpersonal, intrapersonal, and bodily.

Stable Differences

• Cognitive Styles- Styles like “visual learner” vs. “auditory learner” vs. “horizontal learner”.

• Remember, though, these are differences between learners. We cannot cater our instruction to address every style.

Stable Differences

• Pyschosocial traits- trait anxiety, trait locus of control, and academic self-concept

• Gender, Ethnicity, and Racial Group

General implications for stable differences

We can either:1. Make sure that a single instructional

treatment or approach can accommodate learners across the range of differences

2. Create several instructional treatments, each of which is adjusted to a narrowed range of characteristics

Changing Similarities

Development processes that include: • Intellectual development (See Piaget

and Vygotsky)• Language development (See Smith and

Ragan, 1999)• Psychosocial and Personality

development

Changing Differences

Least useful to the instructional designer• Values, beliefs, motivations, and

interests• Intellectual development• Pscyhosocial development• Moral development• Specific Prior Learning**

Summary of Learner Analysis

• Similarities among learners (both stable and changing) provide designers with general perspectives about our audience. We can use research-based principles to design our instruction.

• Changing differences (especially prior learning) provide very specific guidance to the designer on organizational strategy decisions.

• It is not your responsibility to know every theory from every researcher. It is, however, your responsibility to have a fundamental understanding about how human beings learn (we will discuss more next week).

Want to Read more?• http://www.instructionaldesign.org/theories/conditions-

learning.html• http://www.csulb.edu/~dkumrow/conference/

learning_theory.html• http://www.angelfire.com/la2/learners/learners.html• Hilgard & Bower, 1965• Miller, 1956• Broadbent, 1958• Paivio, 1971• Ragan & Smith, 2003; Smith & Ragan, 2000