The Creation, Validation, and Reliability Associated with the EQUIP: A Measure of Inquiry-

Based InstructionJeff C. Marshall

Clemson University

NARST 2009 Conference

Challenge• K-12 teachers report 39% of instructional time is

spent on inquiry (Marshall, Horton, Igo, & Switzer, 2009).– Is this true?– Regardless of amount, what is the quality of

the inquiry?• Current protocols were too broad, too limiting, or

not valid for our purpose.• Goal: create and validate a more useful protocol

Inquiry Defined• NSES definition (NRC, 1996, p.23):

– multifaceted activity that involves making observations; posing questions; examining books and other sources of information to see what is already known; planning investigations; reviewing what is already known in light of experimental evidence; using tools to gather, analyze, and interpret data; proposing answers, explanations and predictions; and communicating the results. Inquiry requires identification of assumptions, use of critical and logical thinking, and consideration of alternative explanations.

Existing Protocols• Inside the Classroom Observational Protocol (Horizon

Research, 2002)

• Reformed Teaching Observation Protocol (RTOP) (Sawada et al., 2000)

• The Science Teacher Inquiry Rubric (STIR) (Beerer & Bodzin, 2003)

• The Science Management Observation Protocol (SMOP) (Sampson, 2004)

• Various teacher efficacy scales (Riggs & Enochs, 1990) have been used as a measure to predict whether reform is likely to occur. This approach is often used because self-reports of efficacy have been closely tied to outcome expectancy (Saam, Boone, & Chase, 2000).

Need for New ProtocolIn our search for a protocol, we found several instruments that all have

significant value. None fully matched our needs. • Inside the Classroom Observational Protocol –too global• The Reformed Teaching Observation Protocol (RTOP) —focuses on

constructivist classroom issues; uses Likert scale instead of descriptive rubric—we desired specific incremental targets to guide reformed practice; micro view not reliable, only macro is limiting  

• The Science Teacher Inquiry Rubric (STIR) –aligned with NSES definition but lacks specifics necessary to facilitate each aspect of inquiry.  

• The Science Management Observation Protocol (SMOP)—emphasizes classroom management issues but not key components of inquiry-based instruction.

• Various teacher efficacy scales—possible predictors but not instructional focused

Need for New ProtocolIn our search for a protocol, we found several instruments that all have

significant value. None fully matched our needs. • Inside the Classroom Observational Protocol (Horizon Research, 2002) –too global

lacks granular look at inquiry instruction• The Reformed Teaching Observation Protocol (RTOP) (Sawada et al., 2000) focuses on

constructivist classroom issues, but goes beyond a look at inquiry-based instruction to more of an evaluation of teaching. Furthermore, the use of a Likert scale to assess classroom instruction was a limiting factor for our needs. We ultimately sought an instrument with a descriptive rubric that can be used to guide teachers and help them set specific incremental targets as they seek to improve their inquiry-based instruction.  

• The Science Teacher Inquiry Rubric (STIR) (Beerer & Bodzin, 2003) aligned with the NSES definition but lacks insight into the specifics of inquiry that teachers must facilitate with each aspect of inquiry.  

• The Science Management Observation Protocol (SMOP) (Sampson, 2004) emphasizes classroom management issues and does not assess key components of inquiry-based instruction.

• Various teacher efficacy scales (Riggs & Enochs, 1990)

Instrument Development

• Protocol requirements:– measure the quantity and quality of inquiry

instruction (validation focuses on quality)– work with various inquiry instructional models

(e.g., 4E x 2, 5E, Learning Cycle)– appropriate for both researchers and teachers– view of instruction from micro and macro

Flowchart of Design

Instrument Validity• Face Validity

– 11 Researchers (4 internal and 7 external)• Internal Consistency

– Cronbach’s Alpha (.880-.889)• Inter-Rater Reliability

– Cohen’s Kappa (.55-62)– R2 = .856

• Content and Construct Validity– Literature– SEM

Structural Equation Modeling (SEM)

• Confirmatory Factor Analysis conducted

• Model Trimming (26-14 indicators)– 3 Constructs– 4th Construct justified from remaining

indicators– Resulting in 4 constructs

• Instruction, Discourse, Assessment, Curriculum• 19 total indicators

Moving from 3-4 constructs

Summary of SEM Data

All parameters within acceptable boundaries. 2 is significant p < .001, 2//df 2 indicates reasonable fit (Kline, 2005), root mean square error of approximation, RMSEA of .1 is on the threshold of reasonable fit (Browne & Cudeck, 1993), standardized root mean squared residual, SRMR < .1 is considered favorable (Kline, 2005), and the computerized fit index, CFI, of > .90 is considered a good fit (Hu & Bentler, 1999). The four-construct model, 19-indicator model, though not quite as parsimonious as a 14-indicator model, provides a good-fitting model that also is solidly supported by the literature base regarding effective inquiry instruction.

Discussion

• EQUIP provides valid macro (construct and lesson level) and micro (indicator level) view of inquiry.

• EQUIP’s descriptive rubric can be used formatively by teachers and researchers.

• Valid for both math and science.

• Likely effective for both pre- and in-service teacher training.

Further Information

• Jeff C. Marshall, Clemson University

• email: marsha9@clemson.edu

• website: www.clemson.edu/iim (select research and evaluation tab for EQUIP; select outreach and dissemination for paper and PowerPoint)