Using CATs in Online Using CATs in Online CoursesCourses

Dr. Eli Collins-BrownDr. Eli Collins-Brown

ecollins-brown@mmci.orgecollins-brown@mmci.org

Classroom Assessment Classroom Assessment TechniquesTechniques

CATsCATs

Work of Angelo and Cross, 1993Work of Angelo and Cross, 1993

Condensed from Condensed from LTAs To Make Student LTAs To Make Student Learning Visible in Web-Based Learning Visible in Web-Based Classes: CATs - A TLT Group Online Classes: CATs - A TLT Group Online WorkshopWorkshop. B. Mills, R. Purdom, D. Eder. B. Mills, R. Purdom, D. Eder

Formative AssessmentFormative Assessment

Assessing student learning during the Assessing student learning during the coursecourse

F2F – non-verbal cues, facial expression, F2F – non-verbal cues, facial expression, look in their eyeslook in their eyes

Online, no non-verbal cuesOnline, no non-verbal cues

CATs gives us a way to ‘look into the eyes’ CATs gives us a way to ‘look into the eyes’ of our online studentsof our online students

CATs are..CATs are..

Learner-centeredLearner-centeredTeacher-directedTeacher-directedMutually beneficialMutually beneficialFormativeFormativeContext specificContext specificOn-goingOn-goingRooted in best practicesRooted in best practicesEasy to administer, assess and report onEasy to administer, assess and report onTypically low-threshold, perfect for onlineTypically low-threshold, perfect for online

How People LearnHow People Learn

Bransford’s workBransford’s work

Three key learning principles:Three key learning principles:1.1. Prior knowledge – students construct new Prior knowledge – students construct new

knowledge and understanding on what they already knowledge and understanding on what they already know or don’t knowknow or don’t know

2.2. Deep foundational knowledge – students need a Deep foundational knowledge – students need a deep knowledge base and conceptual frameworksdeep knowledge base and conceptual frameworks

3.3. Metacognition: students must identify learning goals Metacognition: students must identify learning goals and monitor their progress toward them.and monitor their progress toward them.

#1 – Prior Knowledge#1 – Prior Knowledge

What Do They Know?What Do They Know?

New learning is based on what students New learning is based on what students already knowalready know

Imperative that we learn where our Imperative that we learn where our students are and what they already know students are and what they already know of don’t know, including misperceptionsof don’t know, including misperceptions

CATsCATs

Background Knowledge Probe (BKP)Background Knowledge Probe (BKP)

Focused ListingFocused Listing

Applications CardApplications Card

Directed ParaphrasingDirected Paraphrasing

Minute PaperMinute Paper

BKPBKP

Preview of material to come and what they Preview of material to come and what they know. know.

1.1. Classroom Assessment Techniques (CATs) are a pedagogical Classroom Assessment Techniques (CATs) are a pedagogical way to monitor student learning as it is taking place.way to monitor student learning as it is taking place.

2.2. Assessment monitors student learning; it does not evaluate Assessment monitors student learning; it does not evaluate faculty teaching.faculty teaching.

3.3. The principles for using CATs in an online environment are much The principles for using CATs in an online environment are much the same as thos for using CATs in a F2F environmentthe same as thos for using CATs in a F2F environment

4.4. Because CATs engage students in their own learning, they do not Because CATs engage students in their own learning, they do not add to the teaching workload but rather, reduce it by making add to the teaching workload but rather, reduce it by making teaching easier.teaching easier.

a.a. I've never encountered this idea beforeI've never encountered this idea before

b.b. I've encountered this idea but wouldn't want to have to explain it I've encountered this idea but wouldn't want to have to explain it to a naive audienceto a naive audience

c.c. I've encountered this idea and can explain it with examplesI've encountered this idea and can explain it with examples

d.d. I've encountered this idea and already adopted it for my teaching I've encountered this idea and already adopted it for my teaching environment.environment.

Best way to do online?

Focused ListingFocused Listing

Ask students to write the key word or idea at the top of a Ask students to write the key word or idea at the top of a page and within a set time limit (usually 2 - 3 minutes) page and within a set time limit (usually 2 - 3 minutes) job down related terms important to understanding that job down related terms important to understanding that topic.   topic.   Helps determine what students recall about a specific Helps determine what students recall about a specific topic, including concepts they associate with the central topic, including concepts they associate with the central point. Can be done in pairs and can be used before, point. Can be done in pairs and can be used before, during or after a lesson. during or after a lesson.

Compare with a master list you created and categorize Compare with a master list you created and categorize as related, unrelated, appropriate, in appropriate.  as related, unrelated, appropriate, in appropriate.  Or compile a master list of all responses and have Or compile a master list of all responses and have students categorize. students categorize. Best way to do online?Best way to do online?

Application CardsApplication Cards

Students give one or more real-world Students give one or more real-world applications for an important principle, applications for an important principle, generalization, theory or procedure.  generalization, theory or procedure.  Give two specific examples of Stephen Covey's Give two specific examples of Stephen Covey's Win-Win performance contracts: one in current Win-Win performance contracts: one in current news and one in your own life. Or give a news and one in your own life. Or give a concrete example of formative assessment in concrete example of formative assessment in your course.   your course.   Sort responses as unacceptable, marginal, Sort responses as unacceptable, marginal, adequate, or excellent.  adequate, or excellent.  Best way to do online?Best way to do online?

Directed ParaphrasingDirected Paraphrasing

rephrase the definition to explain to rephrase the definition to explain to different audiences. different audiences.

ei. explain 'corporation' to high school ei. explain 'corporation' to high school students or ‘irrevocable trust’ to retirees.  students or ‘irrevocable trust’ to retirees. 

Sort responses as confused, minimal Sort responses as confused, minimal adequate, or excellentadequate, or excellent

How to do online? How to do online?

Minute PapersMinute Papers

What was the most important thing you What was the most important thing you learned during this session? What learned during this session? What important questions remain unanswered? important questions remain unanswered?

How to do online?How to do online?

#2 - Deep Foundational Knowledge#2 - Deep Foundational Knowledge

Student must have a deep foundation of Student must have a deep foundation of factual knowledgefactual knowledge

Understand facts and ideas in the context Understand facts and ideas in the context of a conceptual frameworkof a conceptual framework

Organize knowledge in ways that facilitate Organize knowledge in ways that facilitate retrieval and applicationretrieval and application

Sequence and StructureSequence and Structure

Sequence and structure activities so that Sequence and structure activities so that students approach the same materials students approach the same materials from different perspectivesfrom different perspectives

Students learn by processing information: Students learn by processing information: they do not typically learn at a first they do not typically learn at a first exposure unless there is a great deal of exposure unless there is a great deal of relevant prior knowledgerelevant prior knowledge

CATs accomplish bothCATs accomplish both

Think-Pair-ShareThink-Pair-Share

Ask a question and have them write down some Ask a question and have them write down some answers. answers. Then share with another person (pair) and share Then share with another person (pair) and share responses.  responses.  Then pairs share responses with the rest of the Then pairs share responses with the rest of the class. class. WebCT allows you to put students into groups.  WebCT allows you to put students into groups.  Each group has their own discussion board. Each group has their own discussion board. Low Risk, promote active student learning, Low Risk, promote active student learning, promotes discussionpromotes discussion

Structured Problem-SolvingStructured Problem-SolvingF2FF2F

Divide students into group and each member of the group is assigned an Divide students into group and each member of the group is assigned an identity (colors, playing cards, roles) identity (colors, playing cards, roles) Students complete a task together,Students complete a task together,Group responds to the whole class with each member taking a turn as Group responds to the whole class with each member taking a turn as spokesperson.spokesperson.Responses occur by identities (colors, suit, or roles)Responses occur by identities (colors, suit, or roles)

Positive Results:Positive Results:students individual accountable; all must know the materialsstudents individual accountable; all must know the materialspeer coaching because the team is only as strong as its weakest memberpeer coaching because the team is only as strong as its weakest memberstudents are giving a group answer not individual answer - less stressfulstudents are giving a group answer not individual answer - less stressfulstudents who normally never speak up in class with participate through their students who normally never speak up in class with participate through their response. response.

Working in BB or WebCT groups, complete the task together, then randomly Working in BB or WebCT groups, complete the task together, then randomly select the person in each group to respond based on their identity. select the person in each group to respond based on their identity.

Send a ProblemSend a Problem

Each group identifies a problem or issue to solve or Each group identifies a problem or issue to solve or discussion. This is written on the front of a folder.discussion. This is written on the front of a folder.

1.1. The group prepares a response to the problem/issues, The group prepares a response to the problem/issues, writes it on a sheet of paper and puts it in the folder.  writes it on a sheet of paper and puts it in the folder. 

2.2. The Folder is forwarded to a second group.  The Folder is forwarded to a second group.  3.3. The second group does NOT look at the first group’s The second group does NOT look at the first group’s

response,  write their own response, puts in folder and response,  write their own response, puts in folder and passes to the 3rd group.passes to the 3rd group.

4.4. Third group takes responses from first two groups and Third group takes responses from first two groups and identifies the two best responses. They then formulate identifies the two best responses. They then formulate their response by using best two responses, their response by using best two responses, consolidating ideas or coming up with their own consolidating ideas or coming up with their own response. Report back to group.response. Report back to group.

Send a Problem OnlineSend a Problem Online

1.1. Online - Each group gets a problem to solve, clearly articulated.  Online - Each group gets a problem to solve, clearly articulated.  Work in groups in Bb or WebCT, each groups comes up with as Work in groups in Bb or WebCT, each groups comes up with as many viable solutions as possible for this problem.many viable solutions as possible for this problem.

2.2. Each group submits their problem and solution to instructor.  Each group submits their problem and solution to instructor. 

3.3. Instructor sends only problem to 2Instructor sends only problem to 2ndnd group.  group. 

4.4. 2nd group solves and sends back answer to instructor. 2nd group solves and sends back answer to instructor.

5.5. Instructor forwards to third group include the responses from the Instructor forwards to third group include the responses from the first two groups. first two groups.

6.6. Each group reads and synthesizes the other two groups Each group reads and synthesizes the other two groups responses and theirs, and shares with the entire group by posting responses and theirs, and shares with the entire group by posting to the discussion boardto the discussion board

7.7. Allow opportunities for comments and discussion. Allow opportunities for comments and discussion.

ExamplesExamples

History; Support the territorial claims of 1) History; Support the territorial claims of 1) ranchers 2) farmers 3) native americansranchers 2) farmers 3) native americans

Premed: What would a clinician need to Premed: What would a clinician need to know for a disgnosis of 1: attention-deficit know for a disgnosis of 1: attention-deficit disorder 2) AIDS 3) Alzheimer'sdisorder 2) AIDS 3) Alzheimer's

Literature: in Antigone, explain the chief Literature: in Antigone, explain the chief character traits and motivations of 1) character traits and motivations of 1) Antigone 2) Creon 3) Haemon 4) IsmeneAntigone 2) Creon 3) Haemon 4) Ismene

#3 - Metacognition#3 - Metacognition

A “metacognitive” approach to A “metacognitive” approach to instruction can help students learn to instruction can help students learn to take control of their own learning by take control of their own learning by defining learning goals and monitoring defining learning goals and monitoring their progress in achieving them.their progress in achieving them.

Goal-oriented learningGoal-oriented learning

What roles do goals play in learning?What roles do goals play in learning? Focus attention and effortsFocus attention and efforts Identify purpose for learningIdentify purpose for learning Tracks progressTracks progress

Monitoring Own LearningMonitoring Own Learning

MotivationMotivation

ResponsibilityResponsibility

AwarenessAwareness

Self-CorrectionSelf-Correction

LTAs for Learning Principle #3LTAs for Learning Principle #3

•Punctuated Lectures •Classroom Opinion Polls•Minute Paper

Possible Questions for Possible Questions for Punctuated LecturesPunctuated Lectures

““How fully and consistently were you concentrating How fully and consistently were you concentrating on the discussion/lecture during these few minutes? on the discussion/lecture during these few minutes? Did you get distracted at any point? If so, how did Did you get distracted at any point? If so, how did you bring your attention back into focus?you bring your attention back into focus?What were you doing to record/process the What were you doing to record/process the information you were receiving? How successful information you were receiving? How successful were you?were you?What were you doing to make connections between What were you doing to make connections between this “new” information and what you already know? this “new” information and what you already know?

What did you expect to come next in the What did you expect to come next in the discussion/lecture and why?discussion/lecture and why?””

Mills, Purdom, Eder, 2007Mills, Purdom, Eder, 2007How best to do online?

Classroom Opinion PollsClassroom Opinion Pollson Course-related Issueson Course-related Issues

Quick Poll:Quick Poll:

How many believe groupwork improves How many believe groupwork improves understanding of main concepts?understanding of main concepts?

How many have learned something useful How many have learned something useful during this activity?during this activity?

How best to do online?How best to do online?

CATs for Improving CourseCATs for Improving Course

Use the feedback to determine if you Use the feedback to determine if you instruction is effectiveinstruction is effective

One-minute Paper formatOne-minute Paper format

Ask what workedAsk what worked

What didn’t workWhat didn’t work

Suggestions for improvementsSuggestions for improvements

Anonymous or not?

Final Considerations for Final Considerations for Using CATsUsing CATs

Reporting back to students is a mustReporting back to students is a must

Don’t ask if you don’t want to knowDon’t ask if you don’t want to know

Use CATs creatively!Use CATs creatively!

ReferencesReferences

Classroom Assessment TechniquesClassroom Assessment Techniques, Angelo & , Angelo & Cross, ISBN: 1555425003Cross, ISBN: 1555425003Welcome to the online workshop: Welcome to the online workshop: LTAsLTAs To Make Student Learning Visible in Web-Base To Make Student Learning Visible in Web-Based Classes: d Classes: CATsCATs - A TLT Group Online Workshop - A TLT Group Online Workshop, B. Mills, , B. Mills, R. Purdom, D. EderR. Purdom, D. EderEngaging the Online LearnerEngaging the Online Learner, Conrad and , Conrad and Donaldson, ISBN 0787966673 Donaldson, ISBN 0787966673 Discussion-Based Online Teaching to Enhance Discussion-Based Online Teaching to Enhance Student LearningStudent Learning, Tisha Bender, ISBN 1579220657 , Tisha Bender, ISBN 1579220657