Using a Concept Inventory to Inform the Design of Instruction and Software

ASEE Rocky Mountain 2009 - Doug HoltonApril 10, 2009 1

Utah State UniversityUtah State University

Using a Concept Inventory to Inform the Design of

Instruction and SoftwareDoug Holton

[email protected] of Instructional Technology & Learning Sciences - http://itls.usu.edu/


About MeAbout Me

• Instructional Technology & Learning Sciences @ USU

• Background in engineering and science education research and development:

• Learning by Design middle school design challenges @ GaTech

• VaNTH bioengineering education @ Vanderbilt

• ONR funded project to investigate the difficulties students have understanding electrical circuits.


The ProblemThe Problem

The problem we worked on in the past:

Identifying difficulties and misconceptions students have when learning about electrical circuits.

Much pre-existing research, but it was only on the simplest of circuits (bulbs/batteries).

We extended it to AC, capacitors, inductors, RLC, etc. Undergraduate level.


InterviewsInterviews

• We interviewed instructors – what concepts did students have the most difficulty with?

• Interviewed students – Probed their thinking while they worked on circuit problems.

• Identified core invariant principles, and a list of misconceptions.


InvariantsInvariants

• Ohm's Law• Kirchoff's Current and Voltage Laws• Effective resistance – series/parallel• Charge & impedance of capacitor• Inductor and impedance, flux• Power


Misconceptions, DifficultiesMisconceptions, Difficulties

• “empty pipe” - wires are empty when circuit is “off”

• “current consumption” - current is consumed as it travels through wires

• more resistance means more power• AC is spatial rather than temporal• ignore negative part of AC• confuse high and low pass filters• don't distinguish caps & inductors


AC/DC Concept InventoryAC/DC Concept Inventory

• Designed a multiple choice test• boiled it down to 20 items• Correct choices target invariants• Incorrect choices target misconceptions• Improved its reliability - test-wiseness• See ASEE 2008 Paper – Holton,

Verma, & Biswas• “Assessing Student Difficulties in

Understanding the Behavior of AC and DC Circuits”


Example QuestionsExample Questions

• (DEMO) Concept Inventory

• Why do the lights come on so fast when you flip the switch?

• What happens in a basic battery, bulb, capacitor circuit?

• Understanding a basic filter circuit


Other Concept InventoriesOther Concept Inventories

• Signals & Systems• Electronics• DIRECT – high school circuits• Statistics, Biology, Mechanics• Statics, Dynamics, Materials• Thermodynamics• Force Concept Inventory

• Google “concept inventory”


AC/DC Inventory ResultsAC/DC Inventory Results

• Across many schools, students were only getting about 55%-65% correct on these 20 test questions even after they had completed 1-2 undergraduatecircuits classes.

• Did worse on questions involving AC, capacitors, inductors – the dynamic components and circuits


Using a Concept Inventory to Using a Concept Inventory to Improve InstructionImprove Instruction

• ”We believe concept inventories can be critical change agents for transformation of pedagogy and learning in STEM disciplines” (Concept Inventory Central)

• Goal is to see how we can improve student performance on these questions.


Dynamic AssessmentDynamic Assessment

• First strategy was to give students help while they took the test.

• This is called dynamic assessment• Students selected the answer, and

then selected the principle/invariant.• If incorrect – received access to

resources for learning:– video explanations– links to website resources


Results from Inductor ToolResults from Inductor Tool

• Our web-based dynamic assessment tool: Inductor

• Students learned while taking this dynamic assessment

• But they were not motivated, saw declining performance

• Problems:1) It still was just a test2) The resources were not that good

or consistent


Animated Circuit SimulationAnimated Circuit Simulation

• Because of #2, I looked into finding/creating better resources with which students could understand the behavior of circuits.

• Found an animated circuit simulation which I modified for use in a study.


MBL ResearchMBL Research

• Modifications inspired by MBL (microcomputer-based labs) research.

• Example: Using distance sensor, a student moves a car back and forth, and see its position/speed/acceleration graphed in real-time

• Very effective – students better understood motion graphs in a matter of minutes

• Key aspects 1) real-time 2) haptic


Real-time Reactive ControlReal-time Reactive Control

• Tried the same thing by modifying the circuit simulation.

• Added real-time reactive control (enactive interface)

• You can “wiggle” the voltage • and see its effects immediately.

• (DEMO)


Results from SimulationResults from Simulation

• The results were similar to MBL.• Students were given concept

inventory as pre-test, tutored with simulation, then given post-test.

• Students showed significant gains after just a half hour tutoring session.

• With 'turning on lights' question, students went from 20% correct to 65%, even though nothing about lights was mentioned during tutoring.


CaveatCaveat

• BUT, the gains seen were only on half of the test questions.

• What was in common with those 10?• They forced you to consider the

behavior of current over time• (Examples of temporal/non-temporal

questions)• Students improved from 53% to 73%

on temporal questions.• Only 68% to 71% on non-temporal.


Other 10 questionsOther 10 questions

• The other 10 questions often involved students' lack of distinctions between different things, such as voltage vs. current, power vs. resistance, capacitors vs. inductors, high-pass vs. low-pass filters, etc.

• (Show example questions)


Contrasting CasesContrasting Cases

• The third strategy we are looking into is based on perceptual theory about how people learn to make distinctions and notice differences.

• Researchers found that if you show one item, people don't notice many features, but if you show 2 side by side, people attend to the differences between the two.

• (DEMO – simulation & contrasts)


Contrasting Cases examplesContrasting Cases examples

• In the next slides, ask yourself – what do you notice in the left box?


What do you see in left box?What do you see in left box?


What did you notice?What did you notice?

A Circle

Now try it again

What do you notice in the left box?





A Smaller Circle

You noticed the size nowAnd perhaps the white color too

Try It Again





A Circle in the Middle

Now the position of the circleIs more salient

(Demo circuit examples)


Future PlansFuture Plans

• Software – release new circuit simulation this summer called nodicity

• Research – test contrasting cases strategy, combine other strategies with simulation in the context of a class

• Pilot work in high schools• Mainly collaborating with Amit Verma

(Texas), Oenardi Lawanto, and Paul Schreuders


Take Away MessagesTake Away Messages

• Recommend you try giving your own pre-tests and post-tests to see gains for yourself. Works with regular tests

• Try incorporating animated simulations and visualizations – they can have great effects on student learning and help them understand the behavior that underlies the equations.

• Check out the book How People Learn for more on other strategies.


Thank You, Questions?Thank You, Questions?

[email protected]://edtechdev.blogspot.com

Doug HoltonDepartment of Instructional Technology

& Learning Sciences http://itls.usu.edu/

Education

Using a Concept Inventory to Inform the Design of Instruction and Software