Evaluating Learning Using Online Homework

Andrew BennettCenter for Quantitative Education

Kansas State University

The Big Question

• Does Online Homework Improve Learning?

The Big Question

• Does Online Homework Improve Learning?• This is the wrong question!

The Big Question

• Does Online Homework Improve Learning?• This is the wrong question!

• Better questions:– Which students does it help?– Under what circumstances does it help?– Why?

Knowledge Discovery in Databases

• Classification– Different groups of students react differently.

What are the groups and how do they differ?– What problems do students do similarly on?

• Modeling– Can we track changes through time?– Can we predict success (or failure) for students?

Classification

• Clustering techniques– Hierarchical clustering (agnes)– K-means (pam)

• Projections– Singular Value Decomposition– Non-linear methods

Clustering points

Clustering points

Oranges vs. Peaches

• Suppose an orange is fruit for r < 0.9 and peel for r>0.9

• And a peach is fruit for r>0.5 and pit for r<0.5• Which has more fruit?

The Peach

• The orange is 72.9% fruit• The peach is 87.5% fruit

37 Dimensions

• Outer 10% of radius encompasses 98% of total 37D volume.

• Any change in the center of the clusterdrastically changesits contents.

Project onto lower dimension using Singular Value Decomposition

Modeling

• Regression– Linear– Logistic

• Hidden Markov Models– Lets you model through time

Qualitative Methods

• Student evaluations• Interviews– Asking students to solve problems while you

watch is often useful.– $10 or Extra Credit or Help going over an exam

can induce students to participate.• Key to answering WHY questions.• IRB

Use Online Homework to Evaluate Offline Instruction

• Can handle large numbers easily• Maintain standards for scoring• “It’s not all about technology” – Don’t ignore the context

Useful Measurements

• Perfect vs. Non-perfect scores– Sometimes 90% works better than perfect

• Inverse time to perfect (speed)• Acceleration (differences of speeds)

• Don’t neglect non-online data– Problem by problem exam scores– Attendance data

Useful Software

• R– Free– Can find packages to do almost anything– Very useful in teaching stats

• Matlab– Need to purchase appropriate toolboxes– I find it easier to use than R– Very useful in teaching numerical methods

Useful Software

• SQL Database– Allows you to slice and dice the data to form the

measures you find most useful.• Spreadsheet– Basic statistics – Goal seek for logistic regression– Easy Graphs

Research Nuggets

• Student ideas about what it means to learn mathematics are very important.

• Students start interested in learning, but shift to being interested in finishing.

• After about 4 attempts, outside intervention is appropriate.

• Timing of deadlines matters.• Being mobile-friendly is useful.