Classification of Introductory-Level Physics Problems

Pedagogy in Introductory Physics: Working Toward a “Taxonomical”

Classification of Introductory-Level Physics Problems

Jeff Schueler

Phys 486

1/31/2013 1

Motivation

• Recent Physics Education Research (PER) has made progress in adopting important findings from expert-novice and cognitive science research

1/31/2013 2

Motivation

• Recent Physics Education Research (PER) has made progress in adopting important findings from expert-novice and cognitive science research – Findings show that asking students to solve high-level thinking

problems can help them attain a more “expert-like” status as a problem solver

– As a result, physics problem developers have begun to move down this route

1/31/2013 3

Motivation

• Recent Physics Education Research (PER) has made progress in adopting important findings from expert-novice and cognitive science research – Findings show that asking students to solve high-level thinking

problems can help them attain a more “expert-like” status as a problem solver

– As a result, physics problem developers have begun to move down this route

What does expert-like mean?

1/31/2013 4

Motivation

• Recent Physics Education Research (PER) has made progress in adopting important findings from expert-novice and cognitive science research – Findings show that asking students to solve high-level thinking

problems can help them attain a more “expert-like” status as a problem solver

– As a result, physics problem developers have begun to move down this route

• In order to successfully use these problems, an understanding of the relationship between the problems themselves and the cognitive processes involved in the problems needs to be established

1/31/2013 5

GOAL

To come up with an adaptation of a taxonomy (an orderly classification

of items according to their presumed natural relationship) of educational

objectives specifically for physics problem solving

1/31/2013 6

GOAL

Answer the following questions: •Can physics problems be categorized according to cognitive processes and knowledge domains? •Is there any relationship between physics problems, knowledge domains, and cognitive processes? •Are there relevant cognitive processes that are not activated by existing physics problems?

1/31/2013 7

GOAL

Answer the following questions: •Can physics problems be categorized according to cognitive processes and knowledge domains? •Is there any relationship between physics problems, knowledge domains, and cognitive processes? •Are there relevant cognitive processes that are not activated by existing physics problems?

1/31/2013 8

GOAL

• Choose an already existing taxonomy of educational objectives that satisfy the following criteria: 1. Addresses problem solving

2. Involves not only knowledge domains, but also cognitive processes that have been identified by PER as relevant for problem solving in physics

3. Makes a clear distinction between the knowledge involved in problem solving.

1/31/2013 9

NTEO

• Choose an already existing taxonomy of educational objectives that satisfy the following criteria: 1. Addresses problem solving

2. Involves not only knowledge domains, but also cognitive processes that have been identified by PER as relevant for problem solving in physics

3. Makes a clear distinction between the knowledge involved in problem solving.

• Only one existing TEO (called the New Taxonomy of Educational Objectives) met each of the criteria, so it was adopted.

1/31/2013 10

NTEO

• A two-dimensional framework: – Three systems of thinking (self, metacognitive, and cognitive)

– Three domains of knowledge (information, mental procedures, and psychomotor procedures)

1/31/2013 11

NTEO

• A two-dimensional framework: – Three systems of thinking (self, metacognitive, and cognitive)

– Three domains of knowledge (information, mental procedures, and psychomotor procedures)

• Actions of the systems upon the knowledge domains are driven by 6 levels arranged in hierarchical order: Level 6: Self-system

Level 5: Metacognitive system

Level 4: Knowledge utilization

Level 3: Analysis

Level 2: Comprehension

Level 1: Retrieval

1/31/2013 12

Self

Metacognitive

Cognitive

Information

Mental Procedures

Psychomotor Procedures

NTEO

• A two-dimensional framework: – Three systems of thinking (self, metacognitive, and cognitive)

– Three domains of knowledge (information, mental procedures, and psychomotor procedures)

• Actions of the systems upon the knowledge domains are driven by 6 levels arranged in hierarchical order: Level 6: Self-system

Level 5: Metacognitive system

Level 4: Knowledge utilization

Level 3: Analysis

Level 2: Comprehension

Level 1: Retrieval

Focus on these levels, as they are the cognitive levels!

1/31/2013 13

NTEO

Schematic diagram of the levels of model behavior used in NTEO.

1/31/2013 14

Classification of Physics Problems in TIPP

• Two criteria:

1/31/2013 15

Classification of Physics Problems in TIPP

• Two criteria:

1. Type of knowledge in a problem

1/31/2013 16

Classification of Physics Problems in TIPP

• Two criteria:

1. Type of knowledge in a problem

2. Highest complex cognitive process necessary for solving it

1/31/2013 17

Level 1: Retrieval

Information:

(1a). Recalling and recognizing of

physics information.

Mental Procedures:

1/31/2013 18

Level 1: Retrieval

Information:

(1a). Recalling and recognizing of

physics information.

Mental Procedures: (1a). Recalling and recognizing of physics mental procedures.

1/31/2013 19

Level 1: Retrieval

Information:

(1a). Recalling and recognizing of

physics information.

Mental Procedures: (1a). Recalling and recognizing of physics mental procedures. (1b). Executing physics mental procedures.

1/31/2013 20

Level 2: Comprehension

Information:

(2a). Integrating

Mental Procedures: (2a). Integrating

1/31/2013 21

Level 2: Comprehension

Information:

(2a). Integrating

(2b). Symbolizing

Mental Procedures: (2a). Integrating (2b). Symbolizing

1/31/2013 22

Example

1/31/2013 23

Classification of Physics Problems in TIPP

This problem would be classified as:

(I:2, MP:1)

Highest cognitive process with regards to information (I) is symbolizing (level 2)

Highest cognitive process with regards to mental procedures (MP) is executing (level 1) 1/31/2013 24

(I:2, MP:1)

1/31/2013 25

Level 3: Analysis

Information: (3a) Matching

Mental Procedures: (3a) Matching

1/31/2013 26

Level 3: Analysis

Information: (3a) Matching

(3b) Classifying

Mental Procedures: (3a) Matching (3b) Classifying

1/31/2013 27

Level 3: Analysis

Information: (3a) Matching

(3b) Classifying

(3c) Analyzing errors

Mental Procedures: (3a) Matching (3b) Classifying (3c) Analyzing errors

1/31/2013 28

Level 3: Analysis

Information: (3a) Matching

(3b) Classifying

(3c) Analyzing errors

(3d) Generalizing

Mental Procedures: (3a) Matching (3b) Classifying (3c) Analyzing errors (3d) Generalizing

1/31/2013 29

Level 3: Analysis

Information: (3a) Matching

(3b) Classifying

(3c) Analyzing errors

(3d) Generalizing

(3e) Specifying

Mental Procedures: (3a) Matching (3b) Classifying (3c) Analyzing errors (3d) Generalizing (3e) Specifying

1/31/2013 30

Example 2

1/31/2013 31

1/31/2013 32

(I:3, MP:3)

1/31/2013 33

Level 4: Knowledge Utilization

Information: (4a) Decision making

Mental Procedures: (4a) Decision making

1/31/2013 34

Level 4: Knowledge Utilization

Information: (4a) Decision making

(4b) Overcoming obstacles

Mental Procedures: (4a) Decision making (4b) Overcoming obstacles

1/31/2013 35

Level 4: Knowledge Utilization

Information: (4a) Decision making

(4b) Overcoming obstacles

(4c) Experimenting/Investigating

Mental Procedures: (4a) Decision making (4b) Overcoming obstacles (4c) Experimenting/Investigating

1/31/2013 36

Assessment

• Can physics problems be categorized according to cognitive processes and knowledge domains?

• Is there any relationship between physics problems, knowledge domains, and cognitive processes?

1/31/2013 37

Concluding Remarks

• Reliability of rating problems – Among professors and graduate students who have administered the

TIPP, the inter-rater coefficient of assessing reliability of rating the problems was > 0.7, provided a clear statement of assumptions in the problem.

– Problems without a clear statement of assumptions had a kappa coefficient of < .5, likely due to different professors assuming different difficulty levels in the problems

• Rich database of problems allowing instructors to freely choose problems based off of the shift in thinking they want their students to achieve.

1/31/2013 38

References

• http://prst-per.aps.org/abstract/PRSTPER/v9/i1/e010103

• http://arxiv.org/ftp/physics/papers/0508/0508131.pdf

1/31/2013 39

Questions?

1/31/2013 40