Upload
sabrina-briggs
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
School-wide Evaluation Tool (SET) Developed by
•George Sugai
•Terri Lewis-Palmer
•Anne Todd
•Rob Horner University of Oregon (2001,
revised 2005)
SET Purposes Research validated instrument
• Results are primarily used for research and reporting as a way to validate SWPBS practices
Designed to determine• The extent to which schools are using SW-PBS• If training and technical assistance efforts
result in change when using SW-PBS• If use of SW-PBS procedures is related to
valued change in the safety, social culture, and violent behaviors in schools
SET can be used to Assess features of SWPBS that are
in place Evaluate on-going efforts Design and evaluate procedures Determine annual goals and build
action plans Compare year to year efforts in
SWPBS
Do Schools Already Use School-Wide PBS?
SET Scores Oregon and Hawaii
0
20
40
60
80
100
a b c d e f g h I j k l m n o p q r
Schools
SE
T T
ota
l S
core
Pre
Can Schools Adopt School-Wide PBS Systems?
SET Scores Oregon and Hawaii
0
20
40
60
80
100
a b c d e f g h I j k l m n o p q r
Schools
SE
T T
ota
l S
core
Pre
Post 1
Post 2
LHS SET Scores
25%
33%
16%
62.50%
37.50%
55%
67%
100% 100%
83%
67%63%
100%
83%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Defined Taught Rew ards Violations Evaluation Leadership District
LHS 1/01
LHS 12/02
SETElementary School K pre/post
features
Expec
t. de
fined
Expec
t tau
ght
Ackno
wledgm
ent
Corre
ction
s
Evalua
tion
Lead
ersh
ip
Distric
t Sup
port
mea
n0
20
40
60
80
100
% o
f fe
atur
es im
plem
ente
d
fall 98
fall 99
Linwood Middle SchoolCaddo Parish
Linwood Middle SET Scores 4/05
100
60
6763
100 100
50
77
0
10
20
30
40
50
60
70
80
90
100
(A)Expectations
Defined
(B)Expectations
Taught
(C) RewardSystem
(D) ViolationsSystem
(E) Monitoring (F)Management
(G) DistrictSupport
Mean Score
% o
f fe
atu
res
imp
lem
ente
d
Henry V. Adams ElementaryMorehouse Parish
Henry V. Adams Elementary School SET Scores 5/05
84%
50%
100%100%
88%
83%
70%
100%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
(A)Expectations
Defined
(B)Expectations
Taught
(C) RewardSystem
(D) ViolationsSystem
(E) Monitoring (F)Management
(G) DistrictSupport
Mean Score
% o
f fe
atu
res
imp
lem
ente
d
SWPBS Evaluation SET is only one means of evaluating a
school’s level of implementation on SWPBS
It should be used in conjunction with• Office discipline referral patterns
• Staff survey results
• Safety surveys
• Team Checklist information
• Etc.
Louisiana Plans for SET To identify schools that have
exemplary SWPBS practices in place •Validation of self-report surveys for
schools reporting 80% or higher on the Self-Assessment Survey
Conducted by personnel who have met reliability criteria
It is not a test or a way to pass judgment on schools
Features of SETThe SET has 28 items focused on 7
areas:A. Expectations definedB. Behavioral expectations taughtC. Acknowledgement proceduresD. Correction proceduresE. Monitoring and evaluationF. ManagementG. District level support
Conducting the SET (1 of 3)
Interviews (avoid any disruption to instruction)
•Administrator - 30 minutes
•Staff – at least 10 randomly selected
•Students - at least 15 randomly selected
Observation - tour school building•School rules posted in 10 specified
locations
Conducting the SET (2 of 3)
Product Review •Discipline Handbook•Office discipline referral forms, records
and summary reports•Instructional materials for teaching and
correcting behavioral expectations•School Improvement Plan•Annual action plan for SWPBS activities•Other related written products
Conducting the SET (3 of 3)
Approximately 2 hours to complete•30 minute interview with administrator
•Interviewing staff and students who are not actively engaged in instruction
•Tour of the school building
•Review school-wide discipline records, related materials, plans, etc.
Preparing School for SET Send confirmation letter to principal
one week before scheduled meeting date•Include the SET Overview
•Include a list of products to review Begin every SET with the
Administrator interview•Requires 30 minutes of uninterrupted
time!
SET Administrator Interview Items
Areas Items Discipline System 1-5 School Expectations 6-11 Leadership Team 12-21
The responses provided to these questions will be the basis for the rest of the interviews
Attempt to begin interview 45 minutes before lunch/recess
Student Interviews Interview 15 random students
•During administrator interview find out the lunch and recess times
Avoid interrupting instruction time
•On the scoring sheet, note the student grade level and gender
Get a representative sample of the student population (different grade levels, genders, ethnicity, etc.)
Remember small, medium, large
Student Interview Questions What are the school expectations?
•Count how many the students can name that match administrator’s responses
(i.e., 5/5, 2/3,…)
Have you received a reinforcer in the last two months?•Try to equate to a holiday or special event
(e.g., since Mardi Gras, since Christmas, etc.)
Staff Interview Procedures Request the administrator
introduce you to the SWPBS team members
Interview staff members individually•Avoid group responses
Staff Interview Questions (1
of 3)
What are the school expectations? • Count how many the students can name
that match administrator’s responses (i.e., 5/5, 2/3,…)
Have you taught the school expectations/rules to students this year? yes/no
Have you given out any reinforcers in the last 2 months? yes/no
Staff Interview Questions (2
of 3)
What type of student behaviors do/would you refer to the office?• Write exactly what behaviors they name
Yes (or +) if response matches administrator’s No (or –) if response does not match admin.
What is the procedure for dealing with a stranger?• Write exactly what they describe
Yes (or +) if response matches administrator’s No (or –) if response does not match admin.
Staff Interview Questions (3
of 3)
Is there a team in your school to address school-wide behavior support systems?•Yes or no
Are you on that team?•Yes or no
If no, then they are done If yes, move to 3 team member questions
Team Member Interview Questions Does your team use discipline data to
make decisions? Has your team taught the School-wide
program to staff this year? Who is your team leader?
• Name the person
Scoring items• Yes (or +) if response matches admin• No (or -) if response does not match admin
Product Reviews It helps if the administrator stays
around in case you have questions on the products. Often the notebooks lack organization
Ensure you have all the products you need before the administrator leaves the room
Observation During the interviews, tour the
school and observe school rules and crisis plan posted in 10 locations
Do not enter classrooms while teaching is in progress
If empty rooms are locked, request the administrator to allow you to quickly observe
Scoring the SET Write administrator answers directly
on the Interview Sheet Match administrator responses with
staff and student responses Staff and student responses and
observations are written directly on the Interview and Observation Form
Product Review is scored directly on the SET Scoring Guide
What does the SET score mean? Overall SET score
•General index of SWPBS implementation
Criteria for SWPBS implementation•80% total score AND
•80% “Expectations Taught”
Interview and Observation Score Calculation
Example #1 Scoring the Set Tab (pg. 19 A-D)
•Calculate score of interview and observation items
•Omit items requiring a product (P) Example #2 Scoring the Set Tab (pg. 21 A-D) Example # 3 Scoring the Set Tab
(pg. 23 A-D)
Product Score Evaluation (p.27) Teaching Behavioral Expectations
• Examples 1-3 Rewarding Student Behavior
• Examples 4-5 Dealing with and reporting behavioral
violations• Examples 6-8
Current Action plan with specific goals• Examples 9-12