Running Head: SELF-MONITORING TEACHERS’ USE OF SPECIFIC ...cdci/best/readings/Readings2011/strandD2.pdf · Running Head: SELF-MONITORING TEACHERS’ USE OF SPECIFIC PRAISE 1

Running Head: SELF-MONITORING TEACHERS’ USE OF SPECIFIC PRAISE 1

The Effects of Self-Monitoring on Teachers’ Use of Specific Praise

SELF-MONITORING TEACHERS’ USE OF SPECIFIC PRAISE 2

Abstract

Teachers typically enter the field with limited training in classroom management, and research

demonstrates that training alone does not result in improved practice. Typically, researchers

have relied on time-intensive training packages that include performance feedback to improve

teachers’ use of classroom management practices; however, initial evidence suggests that self-

management may be an effective and efficient alternative. In this study, we directly compared

the effects of three different self-monitoring conditions (tally, count, and rate) and no self-

monitoring on five middle school teachers’ rate of specific praise using an alternating treatments

design. We also included baseline and follow-up phases to descriptively explore the effects of

self-monitoring across time. Results indicated that noting each instance of specific praise by

either tallying or using a counter resulted in optimal performance, and teachers preferred using a

counter. Additional study results, limitations, and implications are discussed.

Keywords: classroom management, specific praise, teacher training,

teacher self-management, teacher self-monitoring


The Effects of Self-Monitoring on Teachers’ Use of Specific Praise

Teachers typically enter the field with inadequate training in behavioral instruction and

classroom management (Bergeny & Martens, 2006). Lack of self-efficacy and training in

classroom management has been linked to differences in teachers’ self-reported burnout

(Browers & Tomic, 2000) and reactions to student aggression scenarios (Alvarez, 2007),

respectively. Therefore, school leaders and specialized support staff members (e.g.,

administrators, school psychologists, special educators) need to identify effective and efficient

ways to support teachers’ use of evidence-based classroom management practices. Multiple

strategies have been explored for teacher training, including didactic training, prompting,

modeling, role playing, feedback, and reinforcement (Allen & Forman, 1984). Across studies,

the consensus is that training alone does not result in changes in teacher behavior (Allen &

Forman; Fixen, Naoom, Blasé, Friedman, & Wallace, 2005).

Instead, research suggests that training, in combination with performance feedback,

results in desired increases in teachers’ use of classroom management practices and concomitant

increases in desired student behaviors (e.g., Abbott et al., 1998; Jeffrey, McCurdy, Ewing, &

Polis, 2009; Simonsen, Myers, & DeLuca, 2010). Although effective, performance feedback is

time intensive, and typical school resources often limit its feasibility. Instead of relying on

another individual to observe, collect data, and provide feedback, it may be possible to train

teachers to monitor, record data, and provide feedback on their own behavior. Thus, self-

management may be a potential solution to the training to practice gap.

According to Skinner (1953), individuals manage their own behavior in the same manner

as they manage anyone else’s—“through the manipulation of variables of which behavior is a

function” (p. 228). That is, individuals manipulate the antecedents and consequences of their


own behavior, and they engage in other (self-management) behaviors to make target behaviors

more or less likely. Self-monitoring is noting the presence, absence, or level of a specific

behavior, and is one example of self-management (Cooper, Heron, & Heward, 2007).

Over the last 10 years, researchers have studied self-management in various populations

of adults, including adults who are obese (Donaldson & Norman, 2009), have asthma (Caplin &

Creer, 2001; Creer, Caplin, & Holroyd, 2005; Ngamvitroj & Kang, 2007), have depression

(Rokke, Tomhave, & Jocic, 2000), and are experiencing insomnia (Creti, Libman, Bailes, &

Frichman, 2005). Generally, studies have found that self-monitoring and other self-management

interventions are related to desired behavior changes in adults. In addition, researchers have

begun to explore the use of self-management with teachers. For example, Browder, Liberty,

Heller, and D’Huyvetters (1986) found that teachers made better instructional decisions based on

data trends when they were trained to self-monitor. Similarly, Allinder, Bolling, Oats, and

Gagnon (2000) found that teachers who self-monitored data-based instructional decisions made

better decisions that resulted in better student performance than teachers who did not self-

monitor.

More recently, researchers have started to examine the effects of self-management on

teachers’ use of praise. Praise is an empirically-supported classroom management practice

(Simonsen, Fairbanks, Briesch, Myers, & Sugai, 2008) that can be effective if contingent,

credible, and specific (Brophy, 1981). That is, saying “Thank you for raising your hand”

immediately after a student raised her hand would be more effective than providing general

feedback, like “good job,” minutes (or longer) after the desired behavior. Research has

demonstrated, for example, that increases in teachers’ use of specific praise is associated with


increases in students’ on-task behavior (Chalk & Bizo, 2004; Sutherland, Wehby, & Copeland,

2000).

Sutherland and Wehby (2001) investigated the effects of teachers’ self-evaluation on

their use of specific praise with students with emotional and behavioral disorders. Using a

repeated measures control group design with stratified random assignment, they compared the

praise rates of teachers in the self-evaluation group with those in a control group before

(pretreatment), during (treatment), and after (maintenance) self-evaluation. Specifically, they

delivered training in (a) praise, (b) use of a micro-cassette recorder, (c) procedures to review the

tape, (d) calculation of praise rates, (d) goal setting, (e) self-praise, and (f) graphing strategies to

teachers in the self-evaluation group. Teachers in this group were further instructed to audio

record a short segment of instruction, and review the tape later in the day to calculate praise

rates. They found that teachers in the self-evaluation group demonstrated higher levels of praise

and lower levels of reprimands than teachers in the control group during self-evaluation.

Further, students of teachers in the self-evaluation group gave higher levels of correct responses

than students of control teachers. These differences were smaller but still apparent after self-

evaluation (during maintenance).

Keller, Brady, and Taylor (2005) conducted a similar study (using self-evaluation of

audio-tapes) with student teaching interns. They also found increases in pre-service teachers’

specific praise rates as a result of self-evaluation. Together, these studies demonstrated the

positive effects of self-evaluation on teachers’ use of praise; however, the self-evaluation

methods required additional time outside of instruction (i.e., daily review of taped instruction),

and researchers did not explore additional or alternative methods for self-evaluation.


In the present study, we directly compared the effectiveness of three simple self-

monitoring conditions (tallying, counting, and rating) and no self-monitoring on teachers’ use of

specific praise. Specifically, this study addressed the following research question: Which self-

monitoring strategy is associated with the highest rate of specific praise during teacher directed

instruction for individual middle school teachers? In addition, we explored which strategy (a)

resulted in the highest fidelity of implementation (including measures of both adherence and

accuracy) and (b) was preferred by each teacher.

This study contributes to the literature base in three important ways. First, our findings

add to the limited literature on teachers’ use of self-management strategies to increase their use

of evidence-based classroom management strategies. Second, implementation by general

education teachers from a large urban middle school demonstrates the feasibility and

effectiveness of these strategies with larger class sizes that include students with diverse abilities

and backgrounds. Finally, the use of simple and efficient self-monitoring strategies are explored

that may be employed while teaching.

Method

Setting and Participants

This study took place in an urban middle school in New England that shared many of the

common challenges identified for urban schools. For example, the year before this study,

approximately half (range 44%- 66%) of the student body (N=926 students, grades 5-8) scored

below proficient in reading, writing, math, or science as measured by state-wide tests

(http://www.greatschools.net); and more than half (75%) of the student body was eligible for free

and reduced lunch (http://nces.ed.gov). Student ethnicity of students was described as 62.7%


Hispanic, 29.9% White, 5.6% Black, 1.5% Asian, and 0.4% American Indian

(http://nces.ed.gov).

The staff at this school (including 87.8 FTE teachers) had been implementing tier 1 of

school-wide positive behavior support with fidelity for 4 years, tier 2 with varying fidelity for 3

years, and piloting tier 3 for approximately 1.5 years. Therefore, staff at this school had been

exposed to positive behavior support strategies across a variety of staff in-service and

professional development events. Despite this training, some teachers continued to struggle with

implementing evidence-based classroom management strategies with fidelity. The primary

researchers (the first two authors) approached teachers during team meetings, and presented this

study as an opportunity to receive feedback and support with classroom management. Thus, the

study was presented as both an opportunity for (a) skilled classroom managers to refine their

practices and (b) for other teachers to gain additional support to improve their practices.

Interested teachers provided researchers with a preferred mode of contact (e.g., email), and

researchers scheduled individual meetings to explain the scope of the study and obtain written

informed consent. Ultimately, five female teachers volunteered to participate in this study.

Demographic characteristics of each teacher are summarized in Table 1.

Each participating teacher selected the class period in which she experienced the greatest

challenges with classroom management. In addition, to document the effects of changes in

strategies on student behavior, each teacher identified three students who attended regularly and

engaged in disruptive behavior. No additional or identifying information was collected on these

students. All observations were scheduled during the time teachers indicated that they would be

providing direct instruction in the selected period, and observers recorded data on the behavior of

the teacher and the identified students, as described subsequently.


<Insert Table 1 about here>

Dependent Measures

Teachers’ use of empirically-supported classroom management skills (i.e., specific

praise) was the primary dependent variable of this study. In addition, we explored the collateral

impacts on student behavior, teachers’ fidelity of implementation, and the social validity for each

self-monitoring strategy, which are described in the next subsections.

Indicators of implementation fidelity. To determine whether teachers were self-

monitoring with fidelity, we collected data on both the adherence to and accuracy of teachers’

self-monitoring across conditions. To assess adherence, trained observers noted whether the

teacher was implementing the assigned self-monitoring strategy (fully, partially, or not at all) or

the incorrect strategy by checking the appropriate box on the data sheet for each observation.

Specifically, observers recorded that the teacher was implementing (a) fully if she consistently

used the assigned self-monitoring strategy throughout the observation, (b) partially if she

implemented the assigned strategy for part of the time or implemented some (but not all) of the

features of the strategy, (c) not at all if she did not use any self-monitoring strategy for any

period of time, or (d) the incorrect strategy if she implemented a different strategy than assigned

for that observation (e.g., tallied when she should have rated).

To assess accuracy, observers recorded the self-monitoring data each teacher collected

during the observation. That is, observers looked at the total number of praise statements

recorded by the teacher using the assigned self-monitoring strategy (i.e., total tallies, total count,

or rating) at the end of each observation, and wrote that number in the appropriate place on the

data sheet. The agreement between to the teacher and observer was calculated by dividing the


smaller number of praise statements recorded (agreements) by the larger number of praise

statements recorded (opportunities for agreement) and multiplying by 100%.

Systematic direct observation measurement system. Systematic direct observation

(SDO) data were collected during 15-min observations of teacher-directed instruction during the

class period identified by each teacher as their biggest challenge for classroom management.

Trained data collectors recorded the frequency with which teachers delivered specific praise by

making a tally mark any time the teacher provided specific positive verbal feedback to one or

more students contingent on behavior (e.g., “Thank you for raising your hand”). For praise to be

recorded, it had to be audible. The frequency of specific praise was converted to rate (i.e.,

divided by minutes observed). In addition, to explore the impact of changes in teacher behavior

on student behavior, observers recorded the on-task behavior of three students (identified by the

teacher) using momentary time sampling. That is, at the end of each minute, the observer

quickly scanned the three students’ behavior and recorded whether the student was on task (i.e.,

performing the appropriate task for the context) or off task (i.e., oriented toward anything other

than the task at hand).

Four data collectors (two PhD and two MA students) were trained to collect SDO data

across a series of training activities. First, data collectors met with the lead researcher (first

author) to review operational definitions and procedures for data collection. Then, the graduate

student project coordinator and lead data collector (second author) provided additional practice

using video segments and in-vivo observations in each teacher’s classroom. Training activities

continued until all data collectors met or exceeded 85% inter-observer agreement (IOA) for

frequency data (teacher behavior) and 95% IOA for momentary time sampling data (student


behavior). In addition, re-training meetings (to review operational definitions and discuss areas

of disagreement) were held in the event of decreases in IOA (below criterion levels).

IOA was assessed during 40% of (100 of 249) sessions. Agreement for teachers’ use of

specific praise (frequency count data) was calculated by dividing the smaller number of praise

statements recorded (agreements) by the larger number of praise statements recorded

(opportunities for agreement) and multiplying by 100%. IOA was high across the study (M =

85.2%, SD = 8.8%) and for each teacher (Teacher 1: M = 82.7%, SD = 11.5%; Teacher 2: M =

85.5%, SD = 8.8%; Teacher 3: M = 85.74%, SD = 7.7%; Teacher 4: M = 86.5%, SD = 5.5%;

Teacher 5: M = 86.2%, SD = 9.0%). Agreement for students’ on-task behavior (momentary time

sampling data) was calculated by dividing the number of intervals with agreements by the total

number of intervals per session and multiplying by 100%. IOA remained high throughout the

study (M = 95.4%, SD = 7.4%).

Social validity measures. The first author adapted questions on the Intervention Rating

Profile-15 (IRP-15; Witt & Elliott, 1985) to collect descriptive data on the acceptability of each

self-monitoring strategy from the teachers’ perspectives. The IRP-15 consists of five questions

that prompt responses on a 1 (strongly disagree) to 6 (strongly agree) point scale, and a sixth

open ended question prompts teachers to share any comments or concerns. Scores on the IRP-15

have been found to be reliable indicators of intervention acceptability (Martens, Witt, Elliot, &

Darveaux, 1985), and the IRP-15 has been used to assess the acceptability of academic and

behavioral interventions by teachers (e.g., Harris, Preller, & Graham, 1990; Reynolds & Kelley,

1997).


Design and Procedures

We used a modified alternating treatments design, with baseline, alternating treatments,

optimal treatment, and follow-up phases, to explore the relative effectiveness of different self-

monitoring strategies on five teachers’ use of specific praise during teacher-directed instruction.

All five teachers progressed through baseline, alternating treatments, and indicated treatment

phases. Teachers whose data were either stable or demonstrated clear increasing or decreasing

trends progressed to one of two possible follow-up phases: (a) maintenance (for teachers who

demonstrated high stable levels or increasing trends during the indicated treatment phase) or (b)

performance feedback (for teachers who demonstrated low levels or decreasing trends during the

indicated treatment phase). Each phase is described in the following sections.

Baseline phase. During baseline, we observed and recorded each teacher’s rate of

specific praise before any training was delivered. After a stable pattern was documented, each

teacher received a brief scripted training on how to provide specific praise. The training

comprised the following components: discussion (definition, rationale, examples, and critical

features of specific praise), application activity (scripting contextually appropriate specific praise

statements), introduction to self-monitoring (definition of self-management, explanation of three

self-monitoring strategies, and instructions on how to use each), and recap of study goals. All

training components were scripted and delivered with 100% fidelity to all teachers.

Implementation fidelity was measured by an observer completing a rating of the extent to which

each component was delivered with no, partial, or full fidelity during the training of each teacher.

Alternating treatments phase. Following the brief training, the alternating treatments

phase of the study began. During this phase, the teacher’s behavior was observed during the

following four self-monitoring “treatments” or conditions.


1. Tally of Specific Praise Statements. Teachers were instructed to make a tally each time

they gave one (or more) students specific praise during teacher directed instruction (i.e.,

during observation). Most teachers chose to use a post-it or clipboard so that they could

carry the tally sheet around the classroom with them.

2. Count of Specific Praise Statements (using counter). Teachers were instructed to press a

button to advance a small yellow golf counter each time they gave one (or more) students

specific praise during teacher directed instruction.

3. Rating of Specific Praise Statements (using brief rating scale). Teachers were instructed

to rate their use of praise statements during teacher directed instruction. That is, we asked

teachers to estimate the number of specific praise statements provided per minute on a 0-4

times per minute scale.

4. Day off. To directly evaluate the effects of the three conditions relative to the absence of

self-monitoring, teachers were also given “days off” (i.e., no self-monitoring).

Teachers were also instructed to record their data daily. For tally and count conditions,

teachers (a) recorded the total number of specific praise statements, (b) recorded the number of

minutes of data collection (typically 15), (c) calculated their rate of specific praise (number of

specific praise statements divided by number of minutes on summary sheet), and (d) graphed

their specific praise rate on the summary sheet we provided. For the rate condition, teachers

recorded and graphed their rating of specific praise on the summary sheet we provided.

Each condition was implemented once a day, during the same 15-min period of teacher-

directed instruction observed during baseline. Condition order was randomly scheduled by

drawing condition names without replacement, such that each condition was implemented once

every 4 days. Condition order was communicated to teachers via a written schedule, which was


located on the summary sheet where they recorded their self-monitoring data. If additional days

needed to be scheduled (e.g., if data were unstable and the teacher needed to remain in the

alternating treatments phase), additional conditions were communicated in writing via email and

with an updated summary sheet. In addition, teachers were offered an email reminder of

condition order, and all teachers were emailed if schedule changes (e.g., half days, snow days)

altered the planned schedule.

Data collection continued until a stable pattern of behavior and separation among the

conditions was documented for each teacher. In addition, observers collected data on the fidelity

with which each teacher implemented the selected self-management strategy each day. If a

teacher did not implement at all or implemented the incorrect strategy, they received a reminder

either in person from the data collector or via email about condition order.

At the end of this phase, primary researchers (first two authors) met with each teacher to

(a) review the components of the initial training (i.e., each competent was quickly reintroduced

and teachers were given the opportunity to ask questions), (b) inform each teacher which

condition was considered optimal for her, and (c) give each teacher the opportunity to complete

the social validity questionnaires (based on the IRP-15) for each self-monitoring strategy.

Optimal treatment phase. During this phase, each teacher continued to implement the

self-management strategy associated with her best performance. The optimal self-monitoring

strategy was selected using the following decision rules (in order of preference): the strategy

associated with the highest (a) level or increasing trend of specific praise, (b) mean rate, (c) mean

accuracy (agreement between teacher and data collector), or (d) mean adherence (rated by the

data collector) during the observed teacher-direction instruction activities. During this phase,


observers continued to collect SDO data on teacher and student behavior and record the fidelity

(accuracy and adherence) with which each teacher used the self-management strategy.

Teachers remained in the optimal treatment phase until a stable pattern of responding or

clear trend emerged. If highly variable performance was observed, a teacher remained in this

phase through the end of the study. If stable data or a clear trend emerged, a teacher was moved

to a follow-up phase.

Follow-up phases. Based on data, teachers were moved into one of two follow-up

phases: maintenance (weekly data probes) or performance feedback (daily data updates and

suggestions for using specific praise). Specifically, if a teacher demonstrated either a high stable

level or clearly increasing trend in specific praise rate, she was moved into maintenance. That is,

the lead data collector (second author) informed her that her performance indicated that she was

ready to move into maintenance (weekly data probes). During each probe, the teacher self-

monitored with the optimal strategy and observers continued to collect data on teacher and

student behavior and fidelity of implementation.

If a teacher demonstrated either a low stable or clearly decreasing trend in specific praise

rate during the optimal treatment phase, she was moved into performance feedback. That is,

researchers met with her and provided verbal and graphic performance feedback using a 1-page

sheet that summarized the critical features of specific praise, shared contextually appropriate

examples of specific praise statements for her classroom, and presented summary data (bullet

points summarizing means and a graph of specific praise rates across conditions and phases).

Throughout this phase, observers continued to collect daily data, and the lead data collector

emailed the teacher an updated performance feedback sheet, which included each additional day

of data, other examples, and summary statements, for example, “You increased/decreased your


rate of specific, contingent praise to X today. Remember your goal is to meet a rate of at least Y

times per minute.” Teachers were asked to respond via email that they had received and

reviewed the performance feedback sheet.

At the end of the study, researchers met with each teacher to (a) provide feedback about

their performance throughout the study, which included 1 page data summary with suggestions

for on-going improvement in classroom management after the study; (b) share data on the three

identified students, which was included in the data summary; and (c) thank them for their

participation with a $50 gift card.

Results

In the following sections, results are summarized for (a) implementation fidelity, (b) SDO

of teachers’ specific praise rates and students’ on-task behavior, and (c) social validity.

Fidelity of Self-monitoring

In general, teachers adhered to the self-monitoring conditions across phases (Table 2).

All teachers fully refrained from self-monitoring during the “day off” (no intervention)

condition, and all teachers rated their specific praise during the rating condition. The count and

tally conditions required a greater response effort throughout the 15-min observation, and were

associated with higher variability across teachers during the alternating treatments, optimal

condition, and follow-up phases. The accuracy of teachers’ self-monitoring varied among the

teachers and across conditions (Table 3).

<Insert Tables 2 and 3 about here>

SDO Data (Teacher Praise Rates and Student Behavior)

SDO data for teachers’ specific praise rates were graphed and analyzed visually within

and across phases for each teacher (Figure 1). In addition, means and standard deviations were


calculated for teachers’ specific praise rates (Table 4) and students’ percentage of intervals on-

task (Table 5). Given the concerns with these measures of central tendency and spread for auto-

correlated data (i.e., repeated measures), these data should be interpreted with caution. Results

are summarized for each teacher by phase.

<Insert Figure 1 and Tables 3 and 4 about here>

Teacher 1. During baseline, Teacher 1 demonstrated low and generally stable levels of

specific praise. The three identified students were on-task less than half of observed intervals.

During the alternating treatments phase, Teacher 1 demonstrated an increase in level of specific

praise across all conditions, and the count condition was associated with the greatest level of

specific praise, with the exception of the final data point. Interestingly, the highest level of on-

task student behavior was associated with the day off (no intervention) condition. During the

optimal treatment phase, Teacher 1 demonstrated a clear increasing trend, resulting in a high

level of specific praise. In addition, this phase was associated with the highest level of student

on-task behavior. Given the high level and increasing trend of specific praise during the optimal

treatment phase, Teacher 1 was moved into maintenance (weekly probes), and she maintained a

high and stable level of specific praise. Even though this phase extended to the last weeks of

school, the identified students were also on-task during a majority of intervals during the

maintenance phase.

Teacher 2. During baseline, Teacher 2 demonstrated a low and stable specific praise

rate, and the identified students were on-task during fewer than half of observed intervals on

average. During the alternating treatments phase, Teacher 2 demonstrated variable specific

praise rates across conditions, and the identified students were on-task during a majority of

observed intervals, on average. Because count, tally, and rate conditions were associated with


similarly high levels of specific praise, the count condition was selected as her optimal condition

as it was associated with the highest level of accuracy. During the optimal treatment phase,

Teacher 2 demonstrated highly variable specific praise rates, which were generally higher than

her rates in previous phases and increased in trend throughout the phase. Due to high variability,

data collection continued in this phase, and she was not moved to a follow up phase.

Teacher 3. Teacher 3’s specific praise rates increased throughout the baseline phase.

Unlike other teachers, she received training part way through baseline (as indicated by the arrow

in Figure 1); however, she asked not to begin the alternating treatments phase until a later date.

During the alternating treatments phase, an immediate jump in level was observed for all self-

monitoring conditions. Throughout this phase, specific praise rates were variable and generally

decreased in trend across all conditions. Tally condition was associated with the highest average

specific praise rate, and was selected as her optimal condition. During the optimal treatment

phase, Teacher 3 maintained a relatively high specific praise rate, and students were on-task for

more than half of observed intervals. However, her data were variable and generally lower than

data for the same condition (tally) during the alternating treatments phase. Therefore, we

provided performance feedback during the follow-up phase. When receiving performance

feedback, Teacher 3’s specific praise rates were more variable and slightly lower, and her

identified students exhibited lower levels of on-task behavior, on average, than in the optimal

treatment phase. In other words, daily performance feedback was not associated with greater

improvements in praise rate than self-monitoring with the optimal strategy (tally) for Teacher 3.

Teacher 4. Teacher 4 engaged in low and stable rates of specific praise throughout the

baseline phase. During the alternating treatments phase, Teacher 4’s specific praise rates

remained relatively low with variability across conditions, and students also varied in their levels


of on-task behavior across conditions. Tally and count conditions were both associated with

similar specific praise rates; therefore, tally was selected as the optimal condition based on her

accuracy. During the optimal treatment phase, Teacher 4 demonstrated a slight increase in

specific praise rate, but her data were variable and demonstrated a slight decreasing trend. As a

result, performance feedback was provided during the follow-up phase, and her specific praise

rates increased in level and trend. This phase continued into the last weeks of school, and was

associated with the lowest average on-task behavior for identified students.

Teacher 5. Teacher 5 provided low and stable rates of specific praise during baseline.

Her specific praise rates were highly variable, and overlap was noted among conditions

throughout the alternating treatments phase. In addition, students engaged in differing levels of

on-task behavior across the conditions. Both tally and count conditions were associated with the

highest average rate, and she implemented both with a similar level of accuracy. Therefore, the

count strategy was selected because it was associated with the highest level of adherence.

During the optimal treatment phase, Teacher 5 increased her average specific praise rate, but her

data were still variable and relatively low in comparison with other teachers. Therefore,

performance feedback was provided during the follow-up phase. The introduction of daily

performance feedback was associated with a slight increase in level and trend of specific praise

rate, and on average, identified students were on-task for a majority of observed intervals.

Social Validity

In general, teachers found self-monitoring strategies acceptable (Figure 2). Relative to

other strategies, teachers indicated that self-monitoring with the counter resulted in greater

decreases in students’ inappropriate behavior, greater increases in students’ appropriate behavior,


and was easier and less effortful. They also indicated that they were more likely to recommend it

to others.

<Insert Figure 2 about here>

Discussion

In this study, we examined the effects of three self-monitoring strategies (tally, count, and

rate) and no self-monitoring on five middle school teachers’ use of specific praise during

teacher-directed instruction. In general, we found that (a) teachers generally adhered to all self-

monitoring conditions, but recorded their praise rates with varying levels of accuracy across

conditions; (b) teachers’ specific praise rates were higher during self-monitoring conditions than

baseline or the no self-monitoring condition, with either count or tally considered optimal; and

(c) teachers preferred the count strategy. Therefore, self-monitoring may be a promising strategy

for increasing teachers’ use of specific praise. In the following sections, we discuss study

results, limitations, and implications in more detail.

Discussion of Study Results

All teachers engaged in low and stable rates of specific praise during baseline, and the

introduction of the various self-monitoring strategies during the alternating treatments phase was

associated with an increase in level, trend, or both across teachers, with the exception of the

rating strategy for Teacher 3. Count and tally conditions were found to be optimal because they

were associated with the highest levels of specific praise, accuracy of recording, or adherence to

the strategy for participating teachers. Both conditions required teachers to note each time they

delivered specific praise and differed only in the mode of recording (counter vs. paper and

pencil). Interestingly, teachers preferred the counter condition to the other self-monitoring

strategies, and indicated that the counter was the easiest to implement, resulted in most desired


outcomes, and required an acceptable level of effort. For example, Teacher 1 commented that

the counter may have served as a prompt because holding it “reminded [her] to make praise

statements,” whereas she “forgot to tally and praise.” Similarly, Teacher 2 commented that the

counter was “easier to have with her for the whole 15 min,” unlike the tally sheet, which she

often left on her desk. In other words, teachers considered the counter the most efficient and

effective strategy.

Following the alternating treatment phase, two additional phases were implemented:

optimal treatment and follow-up. Because neither phase was implemented in a staggered fashion

across teachers, experimental control was not achieved and the following results are descriptive

in nature. During the optimal treatment phase, Teacher 1 clearly increased the level and trend of

specific praise, Teacher 2 increased the level of specific praise but her performance remained

variable, and Teachers 3-5 engaged in inconsistent levels of specific praise.

When Teacher 1 was moved into the maintenance phase, she maintained her level of

praise across three weekly probes. She commented that she appreciated learning how to

effectively praise, and used this skill throughout the selected period and across her other classes.

Due to high variability, Teacher 2 was not moved into a follow-up phase. However, her average

praise rate was higher during the optimal treatment phase than any of the conditions during the

previous alternating treatments phase.

When performance feedback was introduced for Teachers 3-5, findings were inconsistent.

Teacher 3 did not appear to respond to performance feedback in that her specific praise rate

initially decreased and then returned to previous levels. She commented that it was difficult to

tally on various days during this phase. Teacher 4 did not appear respond initially, but her use of

specific praise increased toward the end of the phase. Interestingly, this increase in praise


corresponded to a decrease in adherence to self-monitoring; therefore, another variable (e.g., end

of school year pressure to widely distribute school-wide behavior coupons paired with praise)

may better explain changes in her specific praise rate. Teacher 5 gradually increased her specific

praise during the performance feedback, but all data points overlapped with those from the

previous optimal treatment phase. In sum, most teachers engaged in their optimal specific praise

rates during the optimal treatment phase, and performance feedback did not result in substantial

gains over self-monitoring.

Therefore, self-monitoring appears to be an effective tool to increase teachers’ use of

specific praise. This finding adds to existing literature demonstrating that self-monitoring

(Allinder et al., 2000; Browder et al., 1986) and self-evaluation (Keller et al., 2005; Sutherland &

Wehby, 2001) are associated with increases in teachers’ use of evidence-based practices (i.e.,

data-based decision making and specific praise, respectively). In addition, results of this study

suggest that simple and efficient self-monitoring strategies employed while teaching may be

effective, and may reduce the need for more time-intensive performance feedback and self-

management procedures, such as reviewing and evaluating audio recordings of instruction.

In this study, a clear relationship between teachers’ specific praise rates and on-task

behavior for identified students was not demonstrated. That is, the conditions associated with the

highest average specific praise rates were not consistently associated with the highest average

on-task student behavior. This finding contradicts previous research indicating that increases in

specific praise were associated with increases in students’ on-task behavior (Chalk & Bizo, 2004;

Sutherland et al., 2000). Three explanations for the inconsistent relationship between specific

praise and student behavior in this study are plausible. First, praise was not specifically directed

at the three identified students and, therefore, would not be expected to systematically reinforce


and increase on-task behavior. In the previous studies, praise was provided to participating

students. Second, student-specific data were not gathered for students in this study, and adult

attention may not have functioned as a reinforcer for the identified students. Third, specific

praise is considered a universal (tier 1) classroom management strategy that may not be of

sufficient intensity for students identified by their teacher as engaging in high levels of disruptive

behavior, and these students may require Tier 2 or 3 supports to be successful. Future research

should continue to explore the relationship between specific praise and student behavior in

general education classroom settings.

Limitations

Study results should be viewed in light of the following limitations related to the scope,

context, measurement, and design of this study. First, we explored the effects of various self-

monitoring strategies on five middle school teachers’ use of one classroom management practice:

specific praise. Generalizations of findings to other populations of teachers and to other

classroom management practices (e.g., use of prompts, opportunities to respond) are premature.

The effects of self-monitoring on other populations of teachers and on teachers’ use of other

classroom management practices should be systematically studied.

Second, although we scheduled direct observations during the times each teacher

identified as direct instruction, variability existed among instructional conditions within and

across teachers. In addition to providing direct instruction, teachers worked with individual

students, facilitated independent seatwork, and delivered other types of instruction. Also, four of

the classrooms were typical general education classrooms, and one classroom (Teacher 3) was a

small-group special education setting. This variability in instructional practices and class


composition may have influenced teachers’ specific praise rates and students’ levels of on-task

behavior.

Third, observers may not have captured all instances of specific praise delivered by

teachers. Because instructional conditions varied, observers may have had difficulty hearing and

recording teachers’ specific praise statements during certain instructional conditions. For

example, if a teacher walked around the room and quietly provided feedback to students,

observers may not have had heard all instances of praise. Similarly, Teacher 3 delivered

instruction in both English and Spanish. Although her style was to repeat all information in both

languages, some instances of praise in Spanish may have been missed by primarily English-

speaking observers.

Finally, although the study design allowed direct comparison of self-monitoring

strategies and no self-monitoring during the alternating treatments phase, the introduction of the

optimal treatment and follow-up phases were not staggered; therefore, a functional relationship

between the optimal strategy or follow-up (e.g., performance feedback) and teacher behavior was

not documented. In future studies, researchers should use different designs (e.g., multiple-

baseline or withdrawal) to clearly document a functional relationship between self-monitoring

and teacher behavior, relative to baseline. In addition, researchers were directly involved in

providing brief trainings between phases and delivering feedback during the performance

feedback phase. Therefore, future research should also explore the use of self-monitoring in the

context of typical school routines, like school-based training and consultation activities.

Implications

Although preliminary, the results of this study suggest that simple and efficient self-

monitoring strategies may be related to increases in teachers’ use of specific praise. In


particular, recording each instance of specific praise using either a counter or tally resulted in

optimal praise rates for all teachers, and all teachers preferred using the counter. Therefore,

school administrators, consultants, school psychologists, and others involved in supporting

teachers may consider asking teachers to use a simple self-monitoring counter strategy to record

their use of specific practices, like specific praise, to increase their implementation of that

practice.

In addition, this study clearly highlights a need for additional research in the use of

simple strategies to increase teachers’ use of evidence-based classroom management practices.

First, researchers should continue to study the effects of self-monitoring on teachers’ use of

specific praise by using experimental designs (e.g., multiple baseline, withdrawal, group

experimental) that document clear functional relationships between self-monitoring and teachers’

specific praise rates. Second, if self-monitoring is functionally related to increases in specific

praise rates, researchers should study the use of self-monitoring with other evidence-based

classroom management practices, such as delivering prompts and providing opportunities to

respond across other elementary, secondary, and special educators.

Third, if self-monitoring is effective across classroom management behaviors and teacher

populations, researchers should explore the conditions under which self-monitoring may be used.

For example, it would be useful to examine (a) how many behaviors teachers can effectively and

efficiently monitor at one time; (b) whether self-monitoring effectiveness is similar under other

instructional contexts, such as transitions, student led activities, and teacher-lecture; and (c) how

“dose,” or length and intensity, of self-monitoring required to sustain the desired level of teacher

behavior after self-monitoring can be faded.


Fourth, given the variability in teacher characteristics with respect to years of experience,

prior training, skill fluency, etc., we would expect general strategies, like self-monitoring, to be

effective with most teachers, but not all. Therefore, future researchers should examine what

additional supports might be needed if simple self-monitoring is ineffective.

Finally, researchers should explore the effectiveness of self-monitoring under typical

school conditions. In this study, researchers provided training, feedback, and prompting. An

important question is whether similar teacher implementation fidelity can be achieved when

support is provided by school administrators, school psychologists, and peer mentors under

typical work conditions.

In sum, if teachers are to benefit from the use of effective practices, they must be able to

implement that practice with fidelity. Performance feedback can enhance implementation

fidelity; however, obtaining useful and meaningful feedback may be difficult when resources and

personnel are limited. The findings from this study suggest that self-monitoring may be strategy

for teachers to obtain information about their implementation in a relevant, efficient, and

effective manner.


References

Abbott, R. D., O’Donnell, J., Hawkins, J. D., Hill, K. G., Kosterman, R., & Catalano, R. F. (1998).

Changing teaching practices to promote achievement and bonding to school. American Journal

of Orthopsychiatry, 68, 542-552.

Allen, C. T., & Forman, S. G. (1984). Efficacy of methods of training teachers in behavior

modification. School Psychology Review, 13, 26-32.

Allinder, R., Bolling, R., Oats, R., & Gagnon, W. (2000). Effects of teacher self-monitoring

implementation of curriculum-based measurement and mathematics computation achievement

of students with disabilities. Remedial and Special Education, 21, 219-226.

Alvarez, H. K. (2007). The impact of teacher preparation on responses to student aggression in the

classroom. Teaching and Teacher Education, 23, 1113-1126.

Begeny, J. C., & Martens, B. K. (2006). Assessing pre-service teachers’ training in empirically-

validated behavioral instruction practices. School Psychology Quarterly, 21, 262-285.

Brophy, J. (1981). Teacher praise: A functional analysis. Review of Educational Research, 51, 5-

32.

Browder, D., Liberty, K., Heller, M., & D’Huyvetters, K. K. (1986). Self-management by teachers:

Improving instructional decision making. Professional School Psychology, 1, 165-175.

Browers, A. & Tomic, W. (2000). A longitudinal study of teacher burnout and perceived self-

efficacy in classroom management. Teaching and Teacher Education, 16, 239-253.

Caplin, D., & Creer, T. (2001). A self-management program for adult asthma, Part III:

Maintenance and relapse of skills. Journal of Asthma, 38, 343-356.

Chalk, K. & Bizo, L. A. (2004). Specific praise improves on-task behavior and numeracy

enjoyment: A study of year four pupils engaged in numeracy hour. Educational Psychology in


Practice, 20, 335-351.

Cooper, J., Heron, T., Heward, W. (2007). Applied Behavior Analysis (2nd ed). Upper Saddle

River: Pearson Education, Limited.

Creer, T., Caplin, D., & Holroyd, K. (2005). A self-management program for adult asthma, Part

IV: Analysis of context and patient behaviors. Journal of Asthma, 42, 455-462.

Creti, L., Libman, E., Bailes, S., & Fichten, C. (2005). Effectiveness of cognitive-behavioral

insomnia treatment in a community sample of older individuals: More questions than

conclusions. Journal of Clinical Psychology in Medical Settings, 12, 153-164.

Donaldson, J., & Normand, M. (2009). Using goal setting, self-monitoring, and feedback to

increase calorie expenditure in obese adults. Behavioral Interventions, 24, 73-83.

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation

Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la

Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI

Publication #231).

Harris, K. R., Preller, D. M., & Graham, S. (1990). Acceptability of cognitive-behavioral and

behavioral interventions among teachers. Cognitive Therapy and Research, 14, 573-587.

Jeffrey, J. L., McCurdy, B. L., Ewing, S., & Polis, D. (2009). Classwide PBIS for students with

EBD: Initial evaluation of an integrity tool. Education and Treatment of Children, 32, 537-550.

Keller, C. L., Brady, M. P., & Taylor, R. L. (2005). Using self-evaluation to improve student

teacher interns’ use of specific praise. Education and Training in Developmental Disabilities,

40, 368-376.

Martens, B. K., Witt, J. C., Elliott, S. M., & Darveaux, D. X. (1985). Teacher judgments

concerning the acceptability of school-based interventions. Professional Psychology: Research


and Practice, 16, 191-198.

Ngamvitroj, A., & Kang, D. (2007). Effects of self-efficacy, social support and knowledge on

adherence to PEFR self-monitoring among adults with asthma: A prospective repeated

measures study. International Journal of Nursing Studies, 44, 882-892.

Reynolds, L. K., & Kelley, M. L. (1997). The efficacy of a response cost-based treatment package

for managing aggressive behavior in preschoolers. Behavior Modification, 21, 216-230.

Rokke, P., Tomhave, J., & Jocic, Z. (2000). Self-management therapy and educational group

therapy for depressed elders. Cognitive Therapy and Research, 24, 99-119.

Simonsen, B., Fairbanks, S., Briesch, A., Myers, D., & Sugai, G. (2008). A review of evidence

based practices in classroom management: Considerations for research to practice. Education

and Treatment of Children, 31, 351-380.

Simonsen, B., Myers, D., & DeLuca, C. (2010). Providing teachers with training and performance

feedback to increase use of three classroom management skills: Prompts, opportunities to

respond, and reinforcement. Teacher Education in Special Education, 33, 300-318. doi:

10.1177/0888406409359905

Sutherland, K. S., Wehby, J. H. (2001). The effect of self-evaluation on teaching behavior in

classrooms for students with emotional and behavioral disorders. The Journal of Special

Education, 35, 161-171.

Sutherland, K. S., Wehby, J. H., & Copeland, S. R. (2000). Effects of varying rates of behavior-

specific praise on the on-task behavior of students with EBD. Journal of Emotional and

Behavioral Disorders, 8, 2-8.


Table 1

Demographic characteristics for each teacher.

Highest Degree

Earned Years Teaching

Area (Grades) of Certification

Grade(s) Taught Subject(s) Taught Population (General/Special)

1 BA 2 Math (7-12) 8 Math General Educationa 2 BA 3 Elementary (K-6) 5 Language Arts General Education

3 MA 13 Special Education 5-8 Reading, Language

Arts, Math Special Education

(Bilingual)

4 MA+15 28 Elementary (K-8),

Science (4-8) 7 Science General Education

5 MS 4 Elementary (K-6) 5 Math General Education a This school includes students with disabilities (e.g., Specific Learning Disabilities, Attention-Deficit Hyperactivity Disorder,

Intellectual Disabilities) in the general education population.


Table 2

M (and SD) rating of adherence to self-monitoring (0-not at all, 1-partially, 2-fully) for each condition (alternating conditions,

optimal condition, and follow-up phases) across teachers (1-5).

Alternating Condition Optimal Follow-up

No Intervention Count Tally Rate Count or Tallya

Maintenance or Feedbackb

1 2.00 (0.00) 2.00 (0.00) 1.80 (0.45) 2.00 (0.00) 2.00 (0.00) 2.00 (0.00) 2 2.00 (0.00) 2.00 (0.00) 1.75 (0.50) 2.00 (0.00) 1.95 (0.22) - 3 2.00 (0.00) 1.60 (0.55) 1.67 (0.82) 2.00 (0.00) 1.94 (0.25) 1.75 (0.71) 4 2.00 (0.00) 2.00 (0.00) 1.83 (0.41) 2.00 (0.00) 2.00 (0.00) 1.29 (0.95) 5 2.00 (0.00) 1.50 (0.84) 1.20 (0.84) 2.00 (0.00) 2.00 (0.00) 2.00 (0.00) aCount was the optimal condition for Teachers 1, 2, and 5; Tally was the optimal condition for Teachers 3 and 4.

bTeacher 1 was moved into maintenance, and Teachers 3, 4, and 5 received performance feedback during follow up phases.


Table 3

M (and SD) accuracy of self-monitoring (agreement between teacher and observer) for each condition (alternating conditions,


Alternating Conditiona Optimal Follow-up

Count Tally Rate Count or Tallyb

Maintenance or Feedbackc

1 0.69 (0.19) 0.62 (0.20) 0.62 (0.26) 0.77 (0.16) 0.74 (0.07) 2 0.73 (0.14) 0.63 (0.23) 0.48 (0.28) 0.59 (0.15) - 3 0.58 (0.38) 0.76 (0.17) 0.57 (0.40) 0.76 (0.18) 0.66 (0.36) 4 0.24 (0.16) 0.35 (0.27) 0.20 (0.20) 0.45 (0.20) 0.23 (0.25) 5 0.50 (0.32) 0.52 (0.37) 0.15 (0.12) 0.49 (0.19) 0.79 (0.10) aAccuracy data were not collected during the no intervention condition as teachers did not record data.

bCount was the optimal condition for Teachers 1, 2, and 5; Tally was the optimal condition for Teachers 3 and 4.

cTeacher 1 was moved into maintenance, and Teachers 3, 4, and 5 received performance feedback during follow up phases.


Table 4

M (and SD) rate of specific praise statements per minute for each condition (baseline, alternating conditions, optimal condition, and

follow-up phases) across teachers (1-5).

Baseline Alternating Condition Optimal Follow-up

No Intervention No Intervention Count Tally Rate Count or Tallya


1 0.15 (0.21) 0.53 (0.20) 1.11 (0.46) 0.55 (0.32) 0.75 (0.34) 1.23 (0.58) 1.51 (0.38) 2 0.02 (0.04) 0.19 (0.11) 0.69 (0.28) 0.62 (0.19) 0.67 (0.54) 1.08 (0.59) - 3 0.62 (0.36) 1.00 (0.83) 1.34 (0.57) 1.53 (0.91) 0.61 (0.54) 1.07 (0.35) 0.79 (0.49) 4 0.01 (0.03) 0.20 (0.19) 0.32 (0.20) 0.32 (0.19) 0.23 (0.21) 0.38 (0.20) 0.74 (0.60) 5 0.15 (0.04) 0.18 (0.18) 0.40 (0.26) 0.47 (0.23) 0.31 (0.18) 0.52 (0.15) 0.66 (0.35) aCount was the optimal condition for Teachers 1, 2, and 5; Tally was the optimal condition for Teachers 3 and 4.



Table 5

M (and SD) average percentage of intervals on-task for identified students for each condition (baseline, alternating conditions,


Baseline (%) Alternating Condition (%) Optimal (%) Follow-up (%)

No Intervention No Intervention Count Tally Rate Count or Tallya


1 44.89 (34.74) 62.78 (26.25) 57.46 (30.17) 59.21 (20.45) 52.70 (12.65) 76.26 (10.15) 60.00 (34.71) 2 45.56 (29.86) 69.73 (25.06) 73.96 (15.48) 83.70 (17.19) 78.79 (9.14) 80.11 (16.15) - 3 69.12 (28.28) 61.11 (8.22) 57.80 (18.83) 60.83 (26.72) 71.84 (23.71) 65.11 (22.92) 56.11 (37.86) 4 65.27 (26.60) 59.07 (31.99) 43.74 (33.31) 46.00 (34.46) 61.70 (32.72) 59.74 (33.63) 35.56 (27.25) 5 64.57 (23.02) 88.06 (11.74) 70.21 (5.48) 72.22 (21.72) 67.52 (21.64) 76.30 (13.75) 74.01 (13.86) aCount was the optimal condition for Teachers 1, 2, and 5; Tally was the optimal condition for Teachers 3 and 4.



Figure 1. Specific praise rate (per minute) across phases and conditions for Teachers 1-5.


Figure 2. Average ratings of acceptability on the IRP-15. Note: high scores are desired on items

1, 2, 3, and 5; low scores are desired on question 4.

Documents

Running Head: SELF-MONITORING TEACHERS’ USE OF SPECIFIC ...cdci/best/readings/Readings2011/strandD2.pdf · Running Head: SELF-MONITORING TEACHERS’ USE OF SPECIFIC PRAISE 1