Krista E. Wood

Department of Math, Physics

& Computer Science

May 16, 2015

Kathy Koenig

Department of Physics &

Department of Science Education

Ohio-Project Kaleidoscope (OH-PKAL)

Scientific Reasoning

Scientific Reasoning (SR) is a set of skills

necessary in carrying out scientific practices

related to collection and analysis of evidence, and

generation of evidence-based arguments6

SR skills positively correlate to:

• ability to learn concepts1,2

• develop higher levels of problem solving skills3,4

K.E. Wood & K. Koenig 2 May 16, 2015

How to Develop SR Skills Curriculum should:

• Explicitly target SR skills

• Use a process of inquiry1,5,6

3

Guided Inquiry • Engage & Explore with simulations and physical equipment

• Design experiment to answer research question

• Conduct experiment and analyze data to develop a mathematical model

• Synthesize & Evaluate to develop evidence-based claim.

K.E. Wood & K. Koenig May 16, 2015

How to Assess SR Skills

• Lawson Classroom Test of Scientific

Reasoning7 is commonly used 2,5,6,8

has ceiling effect with college students9

• Inquiry for Scientific Thinking and

Reasoning (iSTAR)10 used for finer

grain analysis of Control of Variables

(COV) sub skill11

K.E. Wood & K. Koenig 4 May 16, 2015

Purpose of Study

•Redesign General Physics II labs to use process of guided inquiry

target specific SR skills

•Evaluate effect on student development

of SR skills.

•Evaluate level of COV sub skills to

further refine lab curriculum.

K.E. Wood & K. Koenig 5 May 16, 2015

Study Context

•Spring 2014 semester at UC Blue Ash, an open

access two-year regional college of UC

•20 students in algebra/trig-based General

Physics II Lab

Lab Curriculum Redesign

–Original labs traditional, prescriptive labs

–Redesigned to be guided, inquiry-based labs

–Focus: Design controlled experiments &

make evidence-based claims

K.E. Wood & K. Koenig 6 May 16, 2015

Objective Tasks Task Goals,

SR Skills Targeted Engagement

&

Exploration

Guided inquiry activities using PhET

simulations and lab equipment.

Explore concepts and

potential factors involved

Design

Experiments

Identify possible factors. Narrow to

Independent Variable (IV), Dependent Variable

(DV), and Control Variables (CVs) involved to

address research question.

Control of Variables

Hypothetical Deductive

Reasoning

Implement

Experiments

Collect and analyze data. Develop conceptual

or mathematical model.

Quantitative Linear

Correlation

Elaborate Analyze “student reasoning” scenario. Reveal common naïve

conceptions.

Extend Apply model to another scenario. Integrated Hypothesis

Explanation Students explain during instructor checkpoints

interspersed throughout lab.

Check for

understanding.

Communicate & explain

results.

Synthesis &

Evaluation

Whole class Whiteboard meetings to present

Claims, Evidence, and Reasoning and

collaboratively evaluate results.

Develop team work and

evaluation skills.

Methods Lab Curriculum Redesign Instructional Framework12

Methods Lab Curriculum Redesign for General Physics II

K.E. Wood & K. Koenig 8 May 16, 2015

Lab Content Focus Simulations Used

Labs 1, 2 – Properties of

Ideal Gas

Explore properties of ideal gas using PhET Gas

Properties. http://phet.colorado.edu/ *

Labs 3, 4, 5 – Behavior of

Light (Snell’s Law, Lens

ray optics, Diffraction)

Explore behavior of light using PhET Bending

Light, Wave Interference, and Optics Simulation.

http://physics.bu.edu/~duffy/java/Opticsa1.html

Labs 6, 7 – Electric Fields

and Electric Potential

Explore electric fields and equipotential surfaces

using PhET Charges and Fields.

Labs 8, 9, 10, 11 – DC

Circuits

Explore using PhET Circuit Construction Kit (DC

only)

Lab 12 – Magnetic Force

on a Wire

No simulation used for this topic.

* All PhET simulations available at http://phet.colorado.edu

Methods Data Collection

iSTAR administered pre-test and post-test

• 29-question multiple choice test

• 5 paired questions. Students chose a statement to

explain their reasoning for a previous question

• Assessed 9 SR sub skills:

quantitative linear basic probability

COV statistical probabilistic

integrated hypothesis causation correlation

hypothetical deductive conditional logic reasoning

correlation

K.E. Wood & K. Koenig 9 May 16, 2015

iSTAR Sample Questions – Quantitative Linear

10

1. A twelfth grade class has 9 students. The teacher brings in 6 bottles

of water that fully fill all students’ glasses (no water is left). How

many glasses could be filled with 8 bottles of water?

a. 5 ⅓ glasses c. 8 glasses e. 11 glasses g. 14 glasses

b. 6 ¾ glasses d. 9 glasses f. 12 glasses h. 16 glasses

i. none of the above

2. From the previous question, how many bottles of water are needed

to fully fill 6 glasses?

a. 2 bottles c. 4 bottles e. 8 bottles g. 12 bottles

b. 3 bottles d. 7 bottles f. 9 bottles h. 18 bottles

i. none of the above

May 16, 2015 K.E. Wood & K. Koenig

iSTAR Question – Control of Variables (Basic w/o data)

4. A group of students completed a project which involved making soy milk

ice cream. In the process, the soy milk had to be heated to boiling first. Most of

the students cooled their hot soy milk to room temperature before placing it in

Type of Soy

Milk

Amount of Soy

Milk

Temperature of Soy

Milk before Placing in

Freezer

Experiment 1 sugar-free soy

milk (2) 300C

Experiment 2 (1) half cup of soy milk 700C

a. sugar-free soy milk; (2) half cup of soy milk

b. soy milk with sugar in it; (2) half cup of soy milk

c. soy milk with sugar in it; (2) one cup of soy milk

d. sugar-free soy milk; (2) one cup of soy milk

11

the freezer. However, Jessy placed her hot soy milk directly in the freezer

before it was cooled. Later it was observed that Jessy’s soy milk took less time

to completely freeze compared to the others. The students are puzzled and

wonder “do hot liquids freeze faster than cold liquids?” The table below

provides the conditions for various experiments that enable this question to be

answered. However, there are two items missing from the table (labeled (1) and

(2)). Determine what these items (1) and (2) need to be in order to address the

students’ question.

iSTAR Question – Control of Variables (Basic w/ data) 5. As shown below, a string hangs from a bar and has a small ball attached to its end.

The string (and the attached ball) can be made to swing back and forth, and the number

of complete swings during a certain time interval can be counted. A student wants to

know whether or not the number of swings in 10 seconds is affected by the length

of the string, the mass of the ball, and/or the angle the string is pulled away from the

vertical at the time of release.

The student carried out several experiments to investigate what factors affected the

number of swings in 10 seconds. The conditions and results are shown in the table below.

Trial 1 Trial 2 Trial 3

Variables

length of string 10cm 10cm 40cm

mass of ball 20g 30g 30g

angle at release 15° 30° 15°

Number of swings in 10 seconds 16 16 8

Ignoring all other variables, which variable or variables do you think

can be tested using the information shown in the table above?

a. the length of the string

b. the mass of the ball

c. the angle at release

d. “a” and “b”

e. “b” and “c”

f. “a” and “c”

g. “a”, “b”, and “c”

h. No variable can be tested using the

information provided in the table.

Methods Data Analysis

Analyzed iSTAR class averages for pre-

test, post-test, and change in SR skills for:

• Overall SR

• SR sub skills

• COV sub skills11

K.E. Wood & K. Koenig 13 May 16, 2015

Results and Discussion Overall Scientific Reasoning Skills

Shift in overall

iSTAR scores

from

48.6% to 55.0%

for a change of

6.4%

K.E. Wood & K. Koenig 14 May 16, 2015

Overall

Avg Pre 48.6%

Avg Post 55.0%

Change 6.4%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

% C

orr

ec

t

SS14 iSTAR Overall SR Skills (n=20)

Results and Discussion Scientific Reasoning Sub Skills

15

QuantitativeLinear

Control ofVariables

BasicProbability

IntegratedHypothesis

HypotheticalDeductive

CorrelationStatistical

ProbabilisticCausationCorrelation

ConditionalLogic

Pre SS14 63.5% 38.6% 71.4% 61.0% 85.0% 76.2% 28.6% 31.0% 19.0%

Post SS14 72.7% 48.6% 75.0% 67.3% 86.4% 81.8% 29.5% 34.1% 18.2%

Change 9.2% 10.1% 3.6% 6.3% 1.4% 5.6% 1.0% 3.1% -0.9%

-10.0%

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

SS14 iSTAR SR Sub Skill Class Means (n=20)

• 5-10% increase in most targeted SR sub skills: quantitative linear, COV,

integrated hypothesis, and correlation, but not in HD reasoning

• Below 5% for SR sub skills not targeted: basic probability , statistical

probabilistic, causation correlation and conditional logic.

Results and Discussion Control of Variables Sub Skills

K.E. Wood & K. Koenig 16 May 16, 2015

COV(Basic,withoutdata)

COV(Basic, with

data)

COV(Causation)

COV(HiddenRelation)

Pre SS14 58.7% 42.9% 16.7% 23.8%

Post SS14 68.2% 59.1% 18.2% 34.1%

Change 9.5% 16.2% 1.5% 10.3%

0.0%10.0%20.0%30.0%40.0%50.0%60.0%70.0%80.0%

SS14 iSTAR COV Class Means (n=20)

• Student

post-test was

highest on

COV, basic

without data11

• Increase was

greatest for

COV, basic

with data.

Conclusion

• Increase of 5-10% considered reasonable for a

15-week course13 that explicitly targets scientific

reasoning.

•Little change occurred in SR sub skills not

targeted.

Next Steps Make improvements based on results.

Increase sample to all UCBA lab students SS15.

K.E. Wood & K. Koenig 17 May 16, 2015

1. H. She and Y. Liao, J. Res. Sci. Teach. 47(1), 91-119 (2010).

2. V.P. Coletta and J. A. Phillips, Am. J. Phys. 73(12), 1172-1182 (2005).

3. C. Fabby and K. Koenig, Physics Education Research Conference Proceedings, Portland, OR (2013).

4. S. Ates and E. Cataloglu, European J. Phys. 28(6), 1161-1171 (2007).

5. K. Koenig, M. Schen, M. Edwards, and L. Bao, J. College Sci. Teach. 41(4), 23-29 (2012).

6. K. Koenig, M. Schen, and L. Bao, Sci. Educator. 21(2), 1-9 (2012).

7. A.E. Lawson, J. Res. Sci. Teach. 15(1), 11-24 (2000).

8. J.C. Moore, and L.J. Rubbo, Phys. Rev. ST – Phys. Educ. Res. 8, 010106 (2012).

9. Bao, L., et al., Science, 323, 586-587 (2009).

10. L. Bao and K. Koenig, Inquiry into Scientific Thinking and Reasoning (private communication, 2013).

11. S. Zhou, L. Bao, K. Koenig, A. Raplinger, J. Han, Y. Pi, and H. Xiao, Assessment of student reasoning in

control of variables. Am. J. Phys. (under review).

12. L. Bao and K. Koenig, (2012). TI21: A Technology Enhanced Inquiry Framework for Developing and

Assessing 21st Century Skills, iSTARAssessment.org

13. L. Bao (private communication, 2014). 18

References

Thank You!

For more information, feel free to contact

Krista Wood Kathy Koenig Krista.Wood@uc.edu Kathy.Koenig@uc.edu