id@id@

 S&T Student Course Evaluations:Beyond Question 7 – Part 2

Dan Cernusca, Ph.D.Instructional Design Specialist

Missouri University of Science and Technology

Active Learning FLC - March 24, 2010

id@id@

Why student evaluations?

Classroom observations

Instructor’s perspective

Students’ perspectives

GAP

?

id@id@

Focal Point:

Analysis of S&T evaluation questions;

Additional evaluation questions tested;

Classroom strategies to enhance learning

id@id@

Current Evaluation[In]Famous 7:

1. Course educational value2. Use of assignments and tests3. Concern for students’ understanding 

of the materials4. Preparedness for class5. Ability to communicate6. Ability to motivate7. Overall teaching effectiveness

id@id@

Some challenging evaluationsWhat is the value of Q7?

Q1- course value 2.1 3.2Q2- use of tests 2.1 1.8Q3 – concern for understanding 2.6 1.4Q4 – preparation for class 2.6 1.4Q5 - communication 1.6 1.5Q6 - motivation 1.3 1.4

Means of Q2 – Q6 2.0 1.5

Required coursessurvey specialty

id@id@What is the value of Q7?

Q1- course value 2.1 3.2Q2- use of tests 2.1 1.8Q3 – concern for understanding 2.6 1.4Q4 – preparation for class 2.6 1.4Q5 - communication 1.6 1.5Q6 - motivation 1.3 1.4

Q7 - overall 1.8 1.3

Means of Q2 – Q6 2.0 1.5

Required coursessurvey specialty

id@id@

Unfortunately not that much…

0(poor)

1 2 3 4(excellent)

Qi 1 3 8 10 6

Qi+1 4 9 7 6 2

Meaning beyond the S&T means?

id@id@

How about written comments?

“Dr. _____ knows the material but lacks the ability to communicate that knowledge to us.”

“Poor communication. Looks down on everyone. Never Positive.”

The tone of comments …

The exception…“…I thought the classes were entertaining. It was easy to pay attention. … Thanks for always being prepared for class.”

Q5 1.6 1.5

Q6 1.3 1.4

id@id@

Focal Points:

Analysis of S&T evaluation questions;

Additional evaluation questions tested

Classroom strategies to enhance learning

id@id@

Focus on communication & motivation

Q1- course valueQ2- use of tests

Q3 – concern for understanding 0.49* 0.72**Q4 – preparation for classQ5 - communication 0.80** 0.76**Q6 - motivation 0.82** 0.68**

Q7 - overall 1.8 2.9

** p < 0.01;    * p < 0.05

(Pearson Correlations)

id@id@

Additional Evaluation Questions(15 initial items)

Instructor Mastery Goal Orientation (IMGO)• perceived interest in student learning;

Teacher Immediacy (TI)• perceived personal closeness;

Communication Competence (CC)• perceived communication skills

id@id@Testing and RefiningThe15 items first tested in small classes• encouraging trends;• some items appeared more powerful than others

Refined in a large introductory course• adapted items for S&T evaluation scale; • exploratory factor analysis conducted;• 7 items loaded in one major scale;

id@id@Additional Evaluation Questions

The “chosen” 7:1. Instructor’s willingness to have you enjoy 

new things2. Instructor’s focus on understanding not 

memorizing3. Mistakes are okay if you are learning4. Instructor recognizes you for trying hard5. Gesture, movement, and dynamic delivery6. Instructor’s good eye contact in classroom 7. Instructor’s ability to express ideas clearly

IMGO

TI

CC

id@id@

Used at the midterm & finals• good indicators for major problem zones (typically “clarity of ideas” and “recognizing us for trying hard” mirrored Q7);

• as a scale showed significant strong correlations with Q7 (> 0.70);

• as a scale the 7 additional questions predicted more than 60% of variance in Q7 (R2 in regression analysis);

Additional Evaluation Questions

id@id@

Can offer:• insightful opinions related to the motivation and communication issues;

• additional information for each major instructional activity (lecture, lab, test)

• support for more informed instructional interventions; 

Essay Questions for Midterm

id@id@

Warnings!• If you use midterm surveys, then analyze students’ input in a timely manner;

• Select the critical but constructive suggestions and ignore individual/empty/negative ones; 

• [If available] choose suggestions that you consider valid, inform students, and act on them;

Essay Questions for Midterm

id@id@

Focal Points:

Analysis of S&T evaluation questions;

Additional evaluation questions tested

Classroom strategies to enhance learning

id@id@

Levels of classroom interventions 

Visible behavioral & presentation issues

Classroom dynamics and engagement

Partial restructuring/redesigning of instructional materials

id@id@

Visible behavioral & presentation issues

• verbal annoyances (“yes” / “OK”/ “that is”)

• talking to the board or the slide;

• being “glued” to the podium;

• deleting parts of existing information on the board & writing over;

• using “look down” terms (Students - Class)

id@id@

Balance slides and board work:• slides  for outlines and summaries ; 

• board for detailed information/examples;

Classroom dynamics & engagement

Engage students in the classroom• ask questions and insist on an answer;

• listen to & answer students’ questions;

• use in-class transfer problems for your worked-examples;

id@id@

Partial instructional restructuring

Topic 1

Topic 2

Issue 1.1

Issue 1.2

…… has example Mj

integrates Mj

has part Mj

has part Mjhas element Mj

has element Mjhas element Mj

precedes Mj precedes Mj

has part Mj

requires Mj

has type Mj

has type Mj

has type Mj

has part Mj

has part Mj

has part Mjhas part Mj

has part Mj

summarized as Mj

introduced in Mj

introduced in Mj

expanded in Mj

expanded in Mjintroduced in Mj

reinforced in Mj

expanded in Mj

integrated in Mj

used in Mj

used in Mj

used in Mj

Design of MjHorizontal Alignment Mj

learning Mjoutcomes Mj

this course Mj prior courses MjCE001 Mj

(Surveying) Mj

networked supporting Mjconcepts Mj

SSD Mj(stoping sight distance) Mj

Elements of circular Mjcurve Mj

perception Mjreaction time Mj friction factor Mj rate of Mj

deceleration Mj

fundamental Mjsupporting concepts Mj

basic math Mj

algebra Mj

geometry Mj

trigonometry Mj

minimum Mjradius Mj

degree of Mjcurve Mj point of Mj

intersection Mj

deflection Mjangles Mj

Station MjHorizontal Alignment Mj

…you do this …you expect  that

id@id@

Partial instructional restructuring

Potential issues to look into:

• content and assessment are not aligned; 

•  misconceptions are not identified and addressed before the major exams;

• not enough instructional support for term/team projects;

• good instructional tools used in a wrong way;

id@id@

Contact Information

Dan Cernusca, Ph.D.

207 Norwood Hall (CERTI office)

(573) 341 4632

dcernusca@mst.edu