Upload
blake-craig
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
To be sure of hitting the target,
shoot first and whatever you hit,
call it the target.
Ashleigh Brilliant
Performance Feedback SystemsPerformance Feedback Systems
relyrely on Staff Behavior on Staff Behavior
In order for a performance feedback system to be effective, staff must:
implement it
compliancecompliance
correctly over time
competencecompetence sustainabilitysustainability
Performance Feedback SystemsPerformance Feedback Systems
compliance / competencecompliance / competence
• ask the right questions
• identify appropriate data to collect (validity)
• implement interventions according to plan (treatment integrity)
• collect data accurately (reliability)
• display / analyze data
• interpret data / draw correct conclusions
• give / receive feedback based on the data
• modify interventions based on data
Performance Feedback ObstaclesPerformance Feedback Obstacles
staff resistance to a performance feedback:staff resistance to a performance feedback:
long standing mistrust of the purpose of data
educator autonomy, implicit power relationships
cynicism about fads, new ideas, education reform
resistance to performance feedback
data collection is too difficult
data collection causes too much change
desired outcomes take too long to materialize
perceived costs exceed perceived benefits
Performance Feedback SystemsPerformance Feedback Systems
sustainability sustainability
• implemented with procedural fidelity and desired outcomes (effectiveness) at the consumer level
• maintains over time
• maintains over generations of practitioners and decision-makers
• operates within existing resources (financial, staff, materials) and existing mandates
• becomes institutionalized, routine…
National Implementation Research Network (NIRN)
“the way we do business”
SystemSystem’’s Change Track Records Change Track Record
average life of an education innovation is 18-48 months (Latham, 1988)
initial data on comprehensive school reform models initiated in 2000:
1 in 5 maintained reforms through 2002
1 in 10 maintained reforms through 2004
(American Institute for Research)
Performance Feedback SolutionsPerformance Feedback Solutions
Sustainable implementation requires a Sustainable implementation requires a CULTURE CHANGECULTURE CHANGE across all levels of an organization:across all levels of an organization:
changes in adult professional behavior (all stakeholders)
changes in organizational structures and cultures, both formal and informal (systems, policies, contingencies, values, procedures)
changes in relationships to consumers, stakeholders, and systems partners (metacontingencies)
National Implementation Research Network (NIRN)
2004 - present2004 - present
independent, non-profit operating foundation
promote evidence-based education policies and practices
act as a catalyst to facilitate communication, cooperation and collaboration between individuals and organizations currently engaged in evidence based education
1978 - 20041978 - 2004
Operated a large non-profit organization in SF Bay AreaOperated a large non-profit organization in SF Bay Areasix spec. ed schools adult programs
residential programs employment supportive services
public school consultation teacher training campus
Operated comprehensive performance feedback Operated comprehensive performance feedback systemssystems
student performance
staff performance
organizational performance
METRICS
Program Classrooms Students
Schools (6) 60 450
Satellites (2) 2 16
Public School Consultation 300 300
Contracting school districts 95
Total Staff 450
Total New Staff Each Year 150
“real world” challenges
Operating within direct service funding (no grants, research, university students)
Perpetual growth mode (services, programs, technology)
Serving extremely “high risk” kids w/ challenging behaviors
(requiring high treatment integrity implementing sophisticated programs)
High profile (regulatory oversight, parents, districts, community)
Constant shortage of trained staff (staff turnover, failure of Universities to train in effective teaching strategies)
A Buddhist View of CultureA Buddhist View of Culture
Kōan
A Kōan consists of a story, dialogue, question, or statement, the meaning of which cannot be understood by rational thinking but may be accessible through intuition.
VERBAL BEHAVIOR
AUTOMATIC BEHAVIOR
Kōan of the DayKōan of the Day
To implement and sustain a performance feedback culture….
you need a performance feedback culture.
A Behavioral View of CultureA Behavioral View of Culture
“As a set of contingencies of reinforcement maintained by a group…
it has…
a continuing existence beyond the lives of members of the group,
a changing pattern as practices are added, discarded, or modified…
A culture so defined controls the behavior of the members of the group that practice it.”
B.F. Skinner
cognitive dissonance
Culture change requires an expanded unit of analysis
organizations, systems, cultures…in addition to individual behavior
new analytic tools
utilization of expanded forms of research (group designs, quasi-experimental, qualitative)
the uncomfortable feeling caused by holding conflicting ideas simultaneously
Stakeholderspolicy makersparentsschool administratorsclassroom staffstudents
What is an Organizational Culture?What is an Organizational Culture? The complex interaction of formal and informal contingencies governing the behavior of all stakeholders, embodied in:
External Contingencieslaws & regulationsfundingjob markettraining and ideology
Internal Contingenciespoliciespracticesvaluesresource allocationsdata systemsfeedback systemsreporting requirements
program evaluationrecruitment & hiringinitiativesjob expectationscompensationstaff trainingstaff coachingstaff feedback
Overcoming Baseline Cultural Obstacles:Overcoming Baseline Cultural Obstacles: Calibration, Process and EngagementCalibration, Process and Engagement
a “learner centered” culture (calibration)focus on student learning and educational practicesestablishing consensus on standards, definitions, goals
a culture of “inquiry” rather than “compliance” (process)use of data to answer questions, problem solveuse of data-based decision making at all levels of the organization not having all of the answers
a culture of “universal participation” (engagement)wide-spread involvement (ownership, pride, participation)collaboration across disciplinesgiving, receiving, and using feedback data analysis as positive, non-threatening experience
Overcoming Baseline Cultural Obstacles:Overcoming Baseline Cultural Obstacles: AlignmentAlignment
Alignment of all organizational cultural components so that contingencies consistently support performance feedback
policiespracticesvaluesresource allocationsdata systemsfeedback systemsreporting requirements
program evaluationrecruitment & hiringinitiativesjob expectationscompensationstaff trainingstaff coachingstaff feedback
Using Using ““cultural alignmentcultural alignment”” to increase performance feedbackto increase performance feedback
Goals:
Definitions:
Outcomes:
Increase the number of staff using effective performance feedback
staff share common values about data, accountability, feedback and problem solving
staff have technical skills in instruction, data analysis, problem solving
staff positions filled by qualified staffstaff retentionperformance feedback systems implemented
Using Using ““cultural alignmentcultural alignment”” to increase performance feedbackto increase performance feedback
X X Xstaff training
Process Measures
Outcome MeasuresStrategies
X X Xstaff feedbackX X XhiringX X Xjob expectationsX X XrecruitmentX X XselectionX X X
X X X
X X X
X X X
X X X
X X X
policies
practices
resource allocations
staff evaluation
compensation
initiatives
Evolution is chaos with feedback.
John Ford, In Clifford A. Pickover, Computers, Pattern, Chaos, and Beauty (), 203.
Aubrey Daniels International© 2008
Five-Step Behavior Management Process
5. Evaluate
4. Reinforce
3. Feedback
2. Measure
1. Pinpoint
Technical skills: Technical skills: specialized skills required to be successful specialized skills required to be successful in a in a
specific positionspecific position
teaching skills organization formative assessment
data collection reinforcement direct instruction
behavior analysis reading data writing reports
Core skills:Core skills: foundation skills required to be successful in foundation skills required to be successful in thethe organizational cultureorganizational culture
assertiveness outcome oriented optimism
effective communication takes initiative motivation / buy-in
conflict resolution established priorities data driven
feedback (give & receive) leadership scientific
flexibility high tolerance for ambiguity nice
1. Pinpoint
Keeper of the Organizational Culture: HUMAN RESOURCES
job descriptions orientation
recruitment employee contracts
hiring orientation
offer sheets
employee contracts
1. Pinpoint
SupervisorsTeachers
Process
Outcomes
treatmentfidelity
proceduralfidelity
studentteacher
performance
2. Measure
direct observation, behavior products, checklists, etc.
Keeper of the Organizational Culture: HUMAN RESOURCES
“incidental” feedback
“formative” feedback
“formal” evaluations
3. Feedback
Keeper of the Organizational Culture: HUMAN RESOURCES
compensation
recognition
bonuses
promotions
n
4. Reinforce
Keeper of the Organizational Culture: HUMAN RESOURCES
consumers measures
staff measures
organizational measures
n
5. Evaluate
Performance Feedback SystemsPerformance Feedback Systems
relyrely on Staff Behavior on Staff Behavior
antecedent
need to know need to know what to dowhat to do
behavior consequence
need to know need to know how to do ithow to do it
need to be need to be motivated motivated to do itto do it
Aubrey Daniels International © 2009
A B CTeacher
A B CStudent
Principal A B C
Administrators A B C
Cascaded ABCs