Transcript - ceisys.ust.hkceisys.ust.hk/magna/...Classroom-Assessment-Techniques-mos-tran… · Getting Started with Classroom Assessment Techniques A Magna Online Seminar was presented

Getting Started with Classroom Assessment TechniquesA Magna Online Seminar was presented on September 23, 2014 by Tom Pusateri, Ph.D.

Getting Started with Classroom Assessment Techniques teaches participants to:• Articulate your primary teaching goals for one of your courses

• Distinguish between formative assessment and summative assessment of student learning in that course

• Describe how formative assessment of student learning can be used to improve teaching and aid documentation of teaching effectiveness

• Plan how to implement one or more classroom assessment techniques in that course

• Locate additional resources, both online and print, on implementing classroom assessment techniques

MAGNA ONLINE SEMINARSTranscriptEditor’s note:This is a written transcript of an audio recording. Our policy is to edit only the occasional unintelligible phrase. Everything else appears exactly as it was spoken.

© 2014 Magna Publications Inc.All rights reserved. It is unlawful to duplicate, transfer, or transmit this program in any

manner without written consent from Magna Publications.The information contained in this online seminar is for professional development purposes

but does not substitute for legal advice. Specific legal advice should be discussed with a professional attorney.

To make this program available to all your faculty and staff, contact Magna’s Customer Service department at 1-800-433-0499 Ext. 2 and ask about our Campus Access License.

Magna Online Seminar Getting Started with 1 of 22 Classroom Assessment Techniques

Tom Pusateri

©2014 Magna Publications 1 of 22

Rob Kelly: Hello, and welcome to Magna's online seminar Getting Started with

Classroom Assessment Techniques, cosponsored by Magna Publications

and The Teaching Professor. I'm Rob Kelly, editor of The Teaching

Professor, and I'll be the moderator today. I'm pleased you could join us.

If you haven't already printed the handouts, select the file you wish to

print from the file share box on the left of your screen, and then click the

save to my computer button to download, open, and print it.

You may listen to this seminar through your computer, or you may choose

to listen through your telephone. To listen through your phone, dial the

number, and use the access code shown in the box at the bottom of your

screen. We like to have our seminars be interactive, so there are

opportunities throughout for your input through polling and the chat, so

you can send in your questions and comments in to the chat as well as

respond to the presenter's prompts at designated times.

And now I'm pleased to introduce our presenter, Thomas Pusateri. He's the

associate director for the Scholarship of Teaching and Learning and

professor of psychology at Kennesaw State University. He received the

2012 Felton Jenkins, Jr. Hall of Fame Faculty Award from the Board of

Regents of the University System of Georgia. On behalf of KSU, he

submitted successful applications for the 2008 Council of Higher

Education Accreditation's Award for Institutional Progress in Student

Learning Outcomes. Welcome, Tom Pusateri.

Tom Pusateri: Thank you, Rob. Welcome to everyone at this seminar. What I'd like you

to do is I know that some of you are in rooms with multiple rooms. I'd like

you to select one faculty member to represent the group to answer the

questions for their course only. And those of you in larger groups, if you

could just take down your own comments on those questions, and maybe

you can share them with your colleagues after the session, but to save

time, if one of you would be the representative for your group.

And we're going to start off with a question, so if you could have someone

go to the keyboard. And what I want you to do is think about a course that

is a very important course that you teach. What is the main outcome that

you want students to develop in that course? Are you focused on higher-

ordered thinking skills like the higher levels of Bloom's taxonomy,

analysis, synthesis, creation? Are you trying to help the students with

basic academic skills that will help them in success in college like writing,

reading, early mathematic skills, etc.?

Are you focused on trying to help students in the discipline develop

specific knowledge and skills that will help them within their major,

academic values and lifelong learning that you're attempting to promote


Tom Pusateri


those core learning outcomes for their futures, and specifically, work or

career preparation and/or personal growth and development?

Okay. Our next question, how many of the following acronyms are

familiar to you? These are acronyms that we're going to be talking about

in this session. Okay. So we're seeing that some of you are familiar, at

least I'm hoping that you're familiar, with one of them because it is the

focus of this session, classroom assessment techniques. But we will be

discussing some of the others later.

Okay. That first question that I asked you on higher-order thinking skills is

actually a shortened version, very shortened version, of a teaching goals

inventory that appears in the book Classroom Assessment Techniques,

which is going to be the focus of my presentation. This book was

produced by Angelo and Cross in 1993.

There is actually an earlier version of the book that is available online via

the ERIC website. If you go to the supplemental materials, you'll get the

URL to download the earlier version, but I would strongly recommend

that you get a copy of this version. It is a very strong resource for 50

different methods. I'm going to be just focusing primarily on five or six of

those methods today.

That teaching goals inventory that we just took, you can actually go also to

the University of Iowa website, the URL is in the supplemental materials,

to take the full version of that. And it gives you some really useful

information that might help you to plan your own assessment of what core

classroom assessment techniques you might want to use. And it might give

you a good idea of how to develop a syllabus for your course.

We will be going through these acronyms, and we already talked about the

first acronym, classroom assessment techniques, which is the focus of this.

The second acronym is a online website that was developed through a

grant from the National Science Foundation primarily for science,

technology, engineering, and mathematics instructors, but it is a very

useful website for any discipline. It's called the Field-tested Learning

Assessment Guide, or the FLAG.

So I would recommend all of you, after this session, look at that, and it

gives you some strategies that you can download for many of the

classroom assessment techniques that I'm going to be discussing today as

well as additional ones.

During this session, I'm going to show some strategy slides. And this first

strategy slide, I want to consider why should we use classroom assessment


Tom Pusateri


techniques? And there's both formative and summative reasons to do this.

The formative ones are just to help ourselves become better teachers.

We all have had experiences in a classroom where we see the results of an

examination, a paper, or other assignment, and we're just unhappy with the

quality of student product that we're seeing, how well our students are

learning in our course. And it's, might suggest some strategy for adapting

our teaching.

One of the difficulties with looking at examinations and papers is it's too

late to do much intervention. If you are on a 15-week schedule, it's very

difficult to give an exam and then to try to redo that material for classes in

which it's unsuccessful. Otherwise, you might get behind. And so the

classroom assessment technique is an attempt to get you some information

about how well prepared students are for doing well on future

examinations and future papers.

I won't be talking specifically, I see a question about formal course

evaluations. I'm going to be talking instead about summative assessment

in a different way. How we might use the data from classroom assessment

techniques to demonstrate our teaching effectiveness above and beyond

the types of ratings that we get from students, like student readers of

teaching forums.

So the, what I find is a very useful strategy for classroom assessment

techniques is to document changes in your own teaching skills and

changes that might influence what students are, how students are

performing in this class or future classes. This is also a very rich area in

which you might be able to share your observations and your successes

with other colleagues.

The supplemental materials include several articles in which faculty have

published some research on their use of classroom assessment techniques

that document how their classes have changed as a result of the use of

those classroom assessment techniques.

So for those of you who go through annual review processes or

applications for promotion and tenure that require demonstration of

teaching effectiveness above and beyond the student ratings of teaching at

the end of the semester, I would suggest this is very useful supplemental

material that you might want to include demonstrating what you've done

in the classroom using classroom assessment techniques.

So to give you an example of the classroom assessment techniques that

we've already demonstrated, it's called the background knowledge probe. I

had asked you questions about the acronyms, and we have two of the


Tom Pusateri


acronyms on this slide right now. The CAT is the classroom assessment

technique. That's referring to the book by Angelo and Cross.

Their first classroom assessment technique is this background knowledge

probe. If you go to the Field-tested Learning and Assessment site, the

name for that is the conceptual diagnostic test. It's similar to the

background knowledge probe. And when you see a slide like this, you'll

see the similarity between the classroom assessment technique and the

FLAG site, similar types of processes that they use.

And so the, here's a variation that I actually use in some of my classes. I

ask students to, at the beginning of a semester, to rate their familiarity with

various tomes that we are going to be discussing within the class during

that semester. You filled out this for the workshop. Here's an example of

what I do in my theories of personality course.

If you go to the supplemental materials, you'll see that I provided you

several examples of the classroom assessment techniques that I have

incorporated in my theories of personality course, which is a junior-level

course in psychology.

I ask students, first of all, whether they've ever been familiar with the

names of those theorists, have they ever heard about them in other classes

or in the news, and also the concepts that we're going to be covering. And

I also do a matching test where I ask students can they identify which of

the theorists came up with that concept or who was most associated with

that particular concept.

So what I want you to do, whoever is at the keyboard, think about a course

that you regularly teach. What background knowledge might be helpful to

collect from students that might give you some insight as to their

preparation for your course or demonstrate whether they have already

been, have been familiar with some of the content of your course?

And as we're talking, as people are typing, give you some ideas of, in the

supplemental materials, you'll see some examples of how other faculty

members have used these classroom assessment techniques in their course.

Okay. Yes, so how to use a graphic organizer, the literature they've read is

a very important area, obviously, in the humanities. Yeah, what prior

coursework have they taken? Why are they taking the class is a very good

question that might give you some idea of motivation, the personal

development or the work or career preparation.

Okay. Yeah, we're going to be getting, one of you had written identifying

personal values and biases. I'm going to suggest another classroom


Tom Pusateri


assessment technique a little bit later that is probably even going to be a

better example of how to collect information about student opinions on

important topics. Yeah, student expectations, a very useful question at the

beginning of the semester.

Let me give you some examples of some strategies that you could use the

background knowledge probe for. I don't only give this at the first day of

class. I also return to this, give the same questions at the very end of the

semester. I ask them now are they familiar with the theorists, are they

familiar with the concepts, and can they match the concepts of the

theorists?

And what I found was a little bit surprising and dismaying for me.

Although students were doing pretty well on examinations during the

semester, they tended to recall and match the names of the theorists near

the end of the semester more than they could recall and match, especially

the names of the theorists and their concepts, in the middle of the

semester.

I was giving out questions that were or exams that were not cumulative,

and I think what happens is students retain the information enough just to

get by with the first exam, which suggests maybe I need to consider a

different strategy for teaching that course in the future. And I have done

that.

So this is very useful information. If you are expecting students to take

some information away with them at the end of the semester, first of all,

see whether they have that information at the start of the semester, but

then probe them at the end of the semester for that information, ungraded

but still useful type of way of collecting this.

If you compare the pre-test and post-test scores, you get some sort of idea,

so I, you know, students were demonstrating some familiarity with those

names, so I was able to demonstrate in my annual reviews that students

were gaining information from my class. And it gave me some sort of

formative information that I might have to rethink how I present the

information in the future so that students will retain it throughout the

entire semester.

Next question, again, no help from anyone in the audience. What I want

the person at the keyboard to do is answer this true/false question. Now

some of you may have gotten a little bit of assistance from people in the

audience, or some of you may have looked forward in this. When I've

done this in, while using classroom response systems in sessions, I

oftentimes get about a 50% split in faculty who say true and false.


Tom Pusateri


The answer to this question is that it's false. Most of us . . . or

misconception about that we gain from our own personal experiences with

how heat sources, the closer you are to a heat source, like a furnace, the

hotter you feel, the further away, the colder you feel. That's a

misconception when it comes to earth sciences and physics. In fact, the

earth is further away from the sun during our summer, which also happens

to be Australia's winter.

And so, you know, at the same time, no matter where the earth is, half of

the earth is in summer, half of the earth is in winter. And the same thing

happens with fall and spring. We're closer to the earth in fall and spring.

What really matters in terms of, at least in terms of the seasons, is the tilt

of the earth. We're getting more sunlight because we're closer, we're tilted

to the earth during the summer, whereas Australia is tilted away from the

earth.

That's a common misconception that seems to hold very strongly, even

though most of us have taken science courses in which we learned that

fact. Half of the faculty in my workshop here at Kennesaw were unable to

answer that question correctly. So there are misconceptions that occur and

are very difficult to overcome.

Preconceptions are similar. I teach psychology, and a common, a

preconception about schizophrenia is that we hear about schizophrenia a

lot in the news. And oftentimes happens when we hear about a person with

schizophrenia who commits a violent crime, people associate

schizophrenia and violence. They also associate the term schizophrenia

because the term means split mind. They think that also then means

multiple personality, which it doesn't.

The actual symptoms of schizophrenia are delusions and hallucinations,

but it's a common preconception that students enter my psychology classes

with that we attempt to dispel through the course. Multiple personality

disorder, which is now called dissociative identity disorder, is a disorder,

but it's not the equivalent of schizophrenia. It's a different disorder.

So what are some common misconceptions or preconceptions that students

might have in your course? For some background information about the

misconception test too, you may want to look at the Manson article that I

have in the supplemental materials. Manson was teaching a junior-level

intercultural communication course and used the misconception check to

assess students' awareness of the content of an intercultural

communication class.

Yeah, students oftentimes, I see that some of you have talked about

evolution. Students oftentimes have strong preconceptions, especially


Tom Pusateri


about controversial topics like evolution and other biological concepts.

And students, when you talk about being not good at mathematics, I'm

thinking that you're talking about students' own levels of anxiety and their

belief that they cannot do that. Again, I'm going to be talking about

another classroom assessment technique a little bit later that will address

that more directly. Yeah, nature is about the environment and trees.

Okay. Let's move on. When we look at the misconception and

preconception test, one of the strategies that is recommended is that you

might present some information within your class and then ask a question

about that information to check on student misconceptions. If there are

situations in which most of the students are incorrect, you get about 30%

or fewer students who get the correct answer, you may want to ask, you

may need to revisit that lesson. Students may not be getting what you're

attempting to accomplish.

If you get more than 70%, then I don't think there's really much of a

reason to address that misconception during that class. Just confirm that

the correct answer was provided by most of the students and move on.

What's most interesting is the situations in which about half of the students

get the misconception inaccurate and the other half get it correct because

that's a situation in which you could perhaps have pairs of your students

talk with each other and see if they can come to some reasonable

agreement as to the correct answer.

This is called the Think-Pair-Share method, and it does give a lot of value

to the students because the students who have the correct answer now are

able to articulate and defend their correct answer to the students who are

incorrect. If they're persuasive enough, the students who are incorrect will

be able to understand that in a language that you as a professor may not be

able to communicate as successfully to students who are less expert in

your area.

The student may be communicating it, the who knows the answer may be

communicating in a way that the other student will be able to comprehend

much more readily. And so it's a very powerful strategy that Eric Mazur

and others, we'll talk about Eric Mazur a little bit later in this session, and

others have developed in the class to get student interaction, especially on

these topics in which there is disagreement and especially where there's a

correct answer that students may not understand.

This then leads to the classroom opinion poll as another source. In the

FLAG guide online, you can look at similar ideas under the attitude survey

and/or the multiple-choice test site as you look at that. And basically, what

you do in a classroom opinion poll is you simply ask students questions

that may not have right or wrong answers. They could.


Tom Pusateri


What I typically do in my theories of personality class, if you look at the

supplemental materials, is I ask students at the start of the semester what

are their perspectives currently on some of the controversial topics in

which personality theorists disagree?

Someone mentioned the nature/nurture issue a little bit earlier. I ask

students how much of our personality comes from environmental

influences versus our biology and genetics. Someone mentioned evolution.

One of the big concerns, again in theories of personality, are these issues

of biology. Does biology play a role in our personality? Another basic

issue in theories of personality is the free will determinism area. To what

extent do we control our personality, or is our personality a result of forces

like biology and environment that are beyond our control?

So I poll students with this at the beginning of a semester, and I use the

classroom response systems in my class. These are often known as

clickers. It's a way of getting input from students almost instantaneously.

You get immediate display of the information. And students really do find

these fun and engaging, especially if you're asking questions that have

correct answers. Students who get the correct answer oftentimes high-five

each other. So it's sort of an engaging way of getting students involved in

the class.

But for these attitude polls in which there may not be a correct answer or

there are differences of opinion, it gives you a good indication of the

extent to which students are, in your class have differing perspectives that

they then may be able to discuss. Angelo and Cross were suggesting that

classroom opinion polls in this context are very helpful for personal

development of students, giving them some sort of indication of the

diversity of opinion and how they stand on various issues.

If you are interested in using the classroom assessment techniques, one of

the supplemental materials that I would recommend you look at is Sue

Frantz's website. It's called Technology for Academics, and I have links in

the supplemental materials to three of the blogs that Sue Frantz has

developed for that site.

One for the Turning Point Technology clickers, which are the ones that I

use. You need to purchase them, we actually purchase them through my

department, and I bring them to class with me. Some faculty members

actually have students purchase them. Some colleges or universities use

the same systems in multiple classes, and so it might be helpful for

students to do that.


Tom Pusateri


Another system that Sue Frantz recommends is a system that does not

require students to purchase as long as they have a smartphone or a

computer that they bring to class. That may be prohibited in some of your

classes, but if all of the students in a class have access to a smartphone or

a computer during class, the system known as Socrative, it's like the word

Socrates with an -ive ending, is a useful system where students don't have

to remember to bring the clickers every time, and you don't have to be

dragging them in and out of your class.

There's also another system that's a relatively recent system that I think is

just fascinating. It only requires the faculty member to have a smartphone.

It's called the Plicker, clicker with a p instead of a c at the beginning. And

basically, what you do is you distribute to the class cards. They're like five

by five, or if you have a very large class, they're going to be eight by eight.

And they look like the QR codes that you see that students, that when

you're out shopping, you can scan it with your smartphone and take

information, get information about a product.

Well, you can ask students to raise those codes up with their answers. And

depending on the orientation, there's four different orientations that they

can hold the card up that would be an A, B, C, or D answer. And then you

take your smartphone, and you just scan the entire class, and students can

see when their code, they each have unique cards, and so they can see that

their code has been registered on the, on your display system. And so they

can just put their code down.

So it requires just you having a smartphone and the ability to scan a room.

And Sue Frantz says that she's been in very large classes where this

technology works pretty successfully.

Some of the strategies, you can assess student understanding for these

questions. Those are like the misconception checks. You can do that

relatively easily using the clickers. I talked about surveying student

opinions in theories of personality class.

I use it also for preparing or reviewing for exams. We play a Jeopardy!

game near the end of the semester, and I do ask for students to, three of the

students to respond to the Jeopardy!-like environment, but all the other

students can answer the questions with their clickers, and we can see the

extent to which students are prepared for the exam before we distribute the

exam.

I also have used it for collecting attendance data. Every student has a

clicker, and so I can tell if they answer the very first question, then I know

that they were in class at the beginning. If they answer the last question, I


Tom Pusateri


know that they stayed throughout the class. And I collect all the clickers at

the end, so I know that students have been in the class.

Some people also use, there's a Turning Point Technology I know has this

function. You can actually administer a quiz or exam using the clickers.

You display the questions on your display software, and students can, or

you can distribute the exam, and students fill out the questions on their

clickers. You don't have to worry about scanning in the results. You get

the results as part of the clicker technology.

Rob Kelly: Tom, there's a question about the class size you were referring to before.

Just wanted some . . .

Tom Pusateri: Yes, oh, I'm sorry. I wasn't scrolling down, so I didn't see that. A large

class, how large a class can you use? The Turning Point Technology, I

have done this in seminars where I have over 100 people. So I just have to

make sure I bring them. The Plickers, I'm not certain. The eight-by-eight

card is the size that you need so you'll be able to scan it, so I think it really

has to depend on the size of the room more so than the number of students.

You just have to be able to do that. And with the smartphone technology

with Socrative, I would guess there's no upper limit.

Some of these products do have different pricing structures depending

upon the number of individuals. And so a large class can be 300, 400,

1000, and you can be doing this type of technology. But I think the

Turning Point Technology, it depends on how many receiver, you have to

connect a receiver to your computer. I know it works with at least 100.

Yeah, and someone has just said that they do 300 people on two campuses

with Turning Point. It's wonderful technology.

I want to stop here for a second and ask a question. Those of you at the

keyboard, what's the most important point you heard so far in this

presentation? What's your takeaway message, is what I'm guessing?

And as they're answering, those of you in the room, if you go to the very

last page of the supplemental materials, I had given you the teacher

behavior checklist, the teacher goal inventory at the beginning of the

session. You go to the University of Iowa, you can take it. That last page

will give you some suggestions as to what, which of these techniques that

I'm talking about today best reflect your goals.

So, for example, the background knowledge probe is especially useful for

basic academic skills and discipline-specific knowledge. The

misconception check really can be used very well for discipline

knowledge.


Tom Pusateri


Okay. So let's look at some of these responses. Socrative sounds like an

interesting resource. Plickers are cool. I'm just amazed at that technology.

I haven't tried it yet. Sue Frantz has tried it, and she's just amazed by it.

And also, you will love her website. If you haven't gone to the website,

she has some really neat strategies that any faculty member can adapt for

using multiple forms of technology for class management, for course

delivery.

Okay. And some of you are talking about the importance of formative

assessment and different ways to conduct it. Using assessment results to

supplement student evaluations, that's a message that I really would like

you to take home. That most of us rely heavily on student ratings of

teaching. That's a good measure but not the only measure of teaching.

The end of this session, I want to talk about how you might be able to use

what you're finding in classroom assessment techniques to document

changes in your teaching and hopefully changes in what students are doing

within your class.

Okay. Moving on. The most common or the one that Angelo and Cross

suggest that faculty start with is the minute paper, and I just demonstrated

that minute paper with your answers here, the questions that you ask, like

what are the most important points so far. There's a similar strategy that

Angelo and Cross discuss called the muddiest point. The FLAG guide

does not really differentiate, calls this the minute paper.

And basically, what you simply do in the minute paper, usually, you can

do this at the end of a semester or end of a class, but you can also do it at

any time during the class if you want to collect some data. You simply ask

students what was the most important point that you heard today or what

was the most confusing point. That's the muddiest point. What questions

remain unanswered in your mind?

You can distribute that question, I'll give you some sort of strategies for

doing this. You can ask students to just fill out a piece of paper. Takes

about a minute or two for them to provide you an answer. You collect

these at the end of the class.

You can do this online as well using an online discussion board. In fact,

one of the resources that I suggested in the supplemental materials, the one

by Vonderwell and Boboc, shows how you can adapt many of these

strategies, including the minute paper, to an online course. They're focused

on an online graduate course. You can see that this can be used at any

level of course.


Tom Pusateri


Manson's article also used the minute paper in a junior-level intercultural

communication course, and Goldstein used the minute paper as part of an

assessment within a statistics course taught in the psychology department.

So what you do with the minute paper is collect these during the last few

minutes of class mostly. You can collect them, you can read them, and

then as you read them, it doesn't take very long, even in a class of several

hundred, as you're reading them, they're usually relatively brief, so it may

take you 10, 15 minutes at most.

For my classes of about 50, it takes me about 4 or 5 minutes to read them.

And it gives me some sort of ideas of what students are taking away from

the class. It oftentimes reflects some of the misconceptions or

preconceptions that students might have that I may want to address in the

next class. So when I use this technique, I do make sure at the beginning

of the next class to provide some summary data for students so that they

know that I'm using this information and I'm responding to it.

You can also do this in the middle of class where you simply ask students

to write down something and then to share what they've just written down

with their colleagues. You can't read them all during the middle of class,

but if you do a Think-Pair-Share and then you ask students to report back

some of their observations, you can do a minute paper in the middle of

class to see if there are any misconceptions or other difficulties with the

course material that you have been just covering.

Although Angelo and Cross strongly recommend this as the first

classroom assessment technique that most people use, we don't want to

necessarily overdo this. Don't do this every single class unless you really

think that this is something that you need to collect within the class to

make sure students' progress is going successfully. But do this in those

classes in which you think the topics are particularly difficult for students

to comprehend. You've seen poor performance in students on exams or in

papers or in other assignments.

But as I said, you could adapt this very easily to large classes. You don't,

because it's ungraded, you don't have to read everything. You can get

basically the gist of what the comments are by reading a sampling of the

responses. As I said, you can do this in an online environment, so with a

simple chat or discussion board.

So there are multiple ways of adapting the minute paper, and it is a very

powerful strategy is, one of you suggested the muddiest point is, a very

effective strategy to see if students really are having difficulty with a

concept. Isn't it better to know that before they demonstrate their lack of

understanding on an exam? If you know that prior, you might be able to

address it through some additional intervention in the class.


Tom Pusateri


Okay. A little bit earlier, I had discussed the fact that there are some better

strategies for collecting data about student preparation for a class. These

next two strategies, Angelo and Cross's course . . . self-confidence survey

and the teacher-designed feedback form are probably better strategies than

the background knowledge probe for collecting this information.

The FLAG site also refers to another website called the Student

Assessment of Learning Gains, which is a website that you go to, and they

do want you to register it. It was a National Science Foundation-developed

website where they just simply want to assess how many faculty are using

it, so they will ask you some information at the beginning about where you

are located.

And when you go to that site, you'll see several examples of forms that

were developed for various classes. Most of the forms are focused on the

science, technology, engineering, and mathematics disciplines, but again,

they are relatively easy to adapt to any discipline.

And basically, at the SALG site, that's where, one of the acronyms, by the

way, that we had at the beginning, the SALG, that's what that acronym

refers to. Students can provide feedback about the following at any time

during the semester. You don't have to wait for the final course evaluation.

You can ask them simple questions about what aspects of the course do

they believe are helping them with their learning.

And you can select things like, for example, did the syllabus help? Did the

instructions for this assignment, did that aid in your preparation for the

assignment? Did our discussion in class before you wrote that paper help?

Did actually producing the paper give you some insight? So there's

multiple questions you can ask students. And then you also can ask them

questions about their own self-perceived gains in learning. So how

successful do you think you are right now in terms of conducting statistics

in this class?

The one paper that I mentioned earlier, the Goldstein article, the statistics

in psychology class used a variation of one of Angelo and Cross's

classroom assessment techniques to collect information about student self-

confidence in conducting statistics and then did frequent minute papers

during the semester to track how students were succeeding in the class.

They gave out this self-assessment of self-confidence twice during the

semester.

And then Goldstein looked at classes in which he had not used it versus

classes in which he had mandated students complete these forms and

found that students were doing better on the final exam if he had mandated


Tom Pusateri


the use of classroom assessment techniques like the minute paper and the

student self-confidence inventory. It gave students some sort of insight

into their own difficulties with the course that seemed to help them

perform better. And students themselves rated the use of those very highly

at the end of the semester.

So I would recommend that you might want to look at that article to see

what Goldstein had done. Nice thing also about the articles that I have,

they're formal examples of how you may be able to document your own

use of classroom assessment techniques. There's a potential here, if you

are doing this successfully, this may not just become some supplemental

material for your annual review or a promotion and tenure document, but

it might be publishable research in the Scholarship of Teaching and

Learning.

The article on the use of the course-related confidence surveys is by

Goldstein, and it's in the supplemental materials. When you go to the

supplemental materials, you'll see a description of each of the articles. I

have an annotated bibliography.

Okay. But the SALG site, one other use of the SALG site, students can

provide baseline data at the start of the semester, and then you can see

whether students are telling you that they're gaining information, so

whether their confidence on performing these various activities at the

beginning of the semester, how do they rate themselves in terms of

confidence at the end of the semester?

It's an indirect measure of their learning. You'd still want to do exams,

papers, assignments, or other types of authentic assessments, but you give

some sort of idea of the self-confidence. At this point, before we move on,

what I would like to ask, are there any questions or comments from the

room?

Rob Kelly: I don't see any in there right now, but I encourage participants to send in

your questions, and there will be some time at the end to address those.

Tom Pusateri: Okay. Then let's move on. The next classroom assessment technique is

one that I strongly recommend, especially for faculty who want to do

something, not wait for the course evaluation at the end of semester, but

do some really helpful intervention in the middle of the semester. It's

called the group instructional feedback technique or the GIFT. That's

another acronym that I had at the beginning of the session here.

Sometimes it was referred to in some more recent literature as small group

instructional diagnosis or the SGID, S-G-I-D.


Tom Pusateri


If you go to the Berridge, Penney and Wells article, you'll see an online

variation of this technique that they refer to as eFACT, electronic

formative assessment of classroom teaching. And it's a simple but

effective strategy.

What you simply do is this. You find a colleague who is willing to do this

with you. And you switch classes for part of a day. You can do this in the

last 15-20 minutes of the session. You ask, basically, a colleague will

enter your class, and you should prepare students in advance for this visit.

You should let them know that you are going to be collecting information

from the group. You ask the students, basically, the person who is in your

room will ask the students three questions.

First, you ask them to write their answers down, and then you can have a

group discussion so you can get some feedback about whether students are

in agreement or in disagreement about any of these. What's helping you

learn in this class, what's hindering your learning, and what suggestions do

you have?

The faculty members then summarize that data. It's basically anonymous

for the students. And you can then share that together. That's why it's

called a GIFT. You're basically sharing with a colleague what feedback

they're getting from your class. This, the article that you may find more

about this on is the Berridge, B-e-r-r-i-d-g-e. He's the, or Berridge is the

first author. And this article is really helpful because it's a variation that is

used in online classes.

So you can, what they basically do in the article is set up a website in

Blackboard. They have a class that students go to and, although they're not

sharing information. They're just providing feedback directly. Again, a

faculty member has access to that website and can summarize the data for

you so the students are anonymous and you're getting information that you

can use.

Especially in classes, I would suggest using this in any class, but

especially if you're finding that one of your classes is not working as

successfully this semester as it might have in the past, this might give you

some information that can help you recover that class before the end of the

semester. This is done oftentimes sometime near the midterm, and it's a

very effective strategy.

Again, this is one way of documenting that you are concerned about your

teaching. The other faculty member is getting the same benefit. You can

share this information in your annual review or your promotion and tenure

and especially document how you use the information for changes in your

class, which I think is the most important strategy.


Tom Pusateri


And as Mary suggested in her comment, most of the learning management

system sites can make the discussion anonymous so you can do this within

your own class. But sometimes it's very helpful to get the feedback from

another faculty member who's interpreting this so that you, oftentimes we

get defensive about our own class. So if we're getting negative feedback

from students, we may not respond as successfully as if one of our trusted

colleagues can provide that feedback for us.

Okay. And the last technique that I want to present is a much larger

technique. It's a change in the philosophy of how we teach our classes. It's

actually not in the Angelo and Cross book, but it uses a lot of the strategies

that Angelo and Cross have used and that I've discussed so far.

It was developed by Eric Mazur at Harvard University, a physics professor

who was concerned about the lack of performance of students in the class,

and it's called Just-In-Time Teaching, our last acronym, J-I-T-T. You may

have heard of Just-In-Time Technology, which is the business version of

this. This is the teaching version. Today we oftentimes refer to this as a

flipped classroom.

Basically, in Just-In-Time Teaching, what you're going to ask students to

do is some type of assignment outside of class that you are not going to

present in the class. You're going to expect them to get the information

outside of the class. And then they're going to produce some type of

assignment prior to your class that enables you to assess whether students

got the assignment. And then you can devote your time in the class to

problems associated with the students' understanding of the material.

The Just-In-Time Teaching site is on that slide, so you're basically going

to see submitting some type of assignment prior to class. You review those

responses, and you focus your class time on difficult concepts. I do this in

my theories of personality class. I want students to read Freud and Jung

and Adler and Skinner in English. I do have translations of Freud. I'm not

expecting them to read them in German.

But we, I give them original articles that were written by those theorists,

and I ask the students questions about, and you'll see this in my

supplemental materials, questions about what did they agree or disagree

with, what did they have difficulty understanding, have they heard of these

concepts in the past in other classes, and did they find that there were

differences in the way that the theorists presented their own concept

compared to what they heard in the other class. This gives me rich

information.


Tom Pusateri


I ask the students to do this prior to the class, 6:00 p.m. before my 8:00

a.m. class. And what I then do is read them at night. Again, it doesn't take

me very long to read them, maybe about half an hour to an hour,

depending on the size of my, I have relatively small classes. But it's, you

know, it takes about half an hour to an hour. I sometimes comment, if I

know that a student is making a factual error, I can comment to the student

in my, in the discussion board. And I do allow other students to see each

other's comments.

I give them a little bit of extra credit for doing the work prior. And this

gives me rich data that I can then bring in to class the next day. I can use

some of the examples that students provided. I can address some of the

misconceptions that students have. And, yes, some of you, as Bruce

McCluggage at Pikes Peak suggested, this is the Socratic method. So it is

a very helpful strategy for giving students some feedback or getting

feedback for yourself from the students before the class.

That's where I’m going to end, and we do have some time for questions.

So if any of you do have additional questions, please do type them in. And

again, the discussion, the supplemental materials have a lot of the

information that I suggested. I tried to annotate it. I know I do talk a little

bit fast. You should be able to find relatively quickly the information that I

provided.

And I gave some examples of the classroom assessment technique that I've

used in theories of personality. You're welcome to borrow them. The last

page shows how you can adapt some of these classroom techniques, which

of those teaching goals is most relevant. Okay.

Oh, would I consider a webinar for faculty group working on

accreditation? I've done some of that for the Georgia System, so if that's

something of interest to you, please do call. Yeah, and the idea is, yeah,

the Socratic method, all before discussing something in class, it's such a

useful strategy.

Oh, someone asked how often I use the flipped classroom. Well, I do the

readings for every theorist. We cover 16 theorists, and so at least 16

classes out of about 30 that I have. I do, I teach twice a semester, or twice

a week.

And so students are submitting those assignments at least, I would like

them to do it every time, but I give them extra credit for the first eight or

nine times that they do it. That helps also to reduce the amount of reading

that I do, but it does motivate the students to do this. And it's just very

helpful to get the information from students struggling with some very

difficult readings, to know what they're having difficulties with.


Tom Pusateri


I very much thank you for all your assistance, and I'll wait for additional

questions here. And I should suggest, when you're doing the documenting

of the teaching effectiveness, I guess I do have one more slide I forgot.

I mentioned this already. Some of the documentation that you can show, if

you're using classroom assessment techniques. If you're changing your

syllabi as a result of that data, you can show the syllabus prior to and after.

If you're changing any of your assignments or the pedagogical approaches

that you're taking, like the PowerPoint slides or the instructions that you

provide to students, as a result of the classroom assessment, provide an

indication of here's what I did prior to the use of the classroom

assessment. Here's the classroom assessment technique. Here's what I've

done after.

Simple couple of documents with a short narrative, a paragraph in your

annual review or your promotion and tenure can really show that you are

taking seriously the student feedback that you're getting. You can also do

this, of course, with the end-of-course evaluations, but I think there's a

richer data source here from the classroom assessment techniques.

So you can document the student learning gains in your class as well. If

students are performing better as a result of this, why not indicate that

you've used this data, and here's the results of actual student performance

as a result of the use of classroom assessment techniques that differs from

how you've done it in the past. At some point, you might have a rich

enough data source that you should be able to share this with colleagues

outside of your department in a publication or a presentation at a teaching

conference.

So I would certainly suggest at least doing some sharing of this within a

brown-bag session within your own department or among your colleagues

at your institution. But really do consider how you might be able to

contribute to those difficult questions that all of us face in our own

disciplines.

Rob Kelly: And I see a question in here. Is there a software program that will pick up

the most repeated words from the one-minute paper?

Tom Pusateri: That's an interesting question. I am not certain, but I would think that if

you were able to do a, you know, if you could put them into a Word

document. I don't know whether the classroom management systems have

that, but I would suppose if you were able to move it into a Word

document, there might be a word, some type like of a word count system

that actually not counts, just counts the words but the frequency of the use

of words.


Tom Pusateri


Rob Kelly: Word clouds.

Tom Pusateri: Yeah, like a word cloud. Yeah, but I don't know if word clouds are built

in. Are they built into the discussion boards that you get online?

Rob Kelly: I don't know.

Tom Pusateri: If they are, that would be great.

Rob Kelly: Well, I see some people are still typing. Anyway, it's, I guess, it's about all

the time we have. There is a survey that we'd like you participants to fill

out. It's located at the URL on the slide, and we're sending out an e-mail

with this URL in it. Please let us know what you think of today's program

and what programs you'd like to see in the future.

And I don't think there are any leftover questions, but if there are, we will

forward them to our presenter, and we will send out the responses. And

thank you, Tom Pusateri, and thank you, participants, for joining us today.

Have a great day.

Adobe Connect chat Bruce McCluggage- Pikes Peak CC: Will the presenter be addressing end-of-semester formal Course Evaluations? Truman State University: how to use a graphic organizer Community College of Baltimore County: Literature they've read Luzerne County Community College: Prior course in H.S. GPC_CTL: why taking class? guest: basic physiology, basic grammar; Community College of Baltimore County: History of the development of the field Jeffrey: Have the students experienced clinical documentation in the past Bruce McCluggage- Pikes Peak CC: As philosophy instructor, I ask students whether they have read any philosophy-type books before or attended ANY classes related to it. Truman State University: (KWL - graphic organizer) what do you know, what do you want to know... Carroll University 2: experience with mathematical models, scientific writing experience Mary: Knowledge on Transformational Learning - Theorists Guest 2: identifying personal values and biases Bruce McCluggage- Pikes Peak CC: Expectation s for the class on a scale of 1 to 5 and why? guest: student expectations Sanda: What is their major. Bruce McCluggage- Pikes Peak CC: Any major philosophical questions or dilemmas that student may currently hold? joyce.langenegger: student goals


Tom Pusateri


Bruce McCluggage- Pikes Peak CC: future major or profession student may currently hold? Guest 2: false joyce.langenegger: false Sanda: false Concordia University Portland: b Community College of Baltimore County: false Jeffrey: True Bruce McCluggage- Pikes Peak CC: tilt the earth baby! Truman State University: LOL :-D Bruce McCluggage- Pikes Peak CC: it's what philo majors do... Jeffrey: Delusions and hallucinations Bruce McCluggage- Pikes Peak CC: that the teacher knows all... Carroll University: course purpose Jeffrey: I'll never use this material in real life. Community College of Baltimore County: Preconceptions about addictions treatment GPC_CTL: evolution...we come from monkeys guest: The course is easy because it is entry-level. Sanda: Not good at mathematics Luzerne County Community College: Psych...Nature is about environment, trees Concordia University Portland: that they really don't need the information...it's common sense Charles & Dan -- ACPHS: that the course is hard Truman State University: That they already know the subject Community College of Baltimore County: That college is just like high school, just 13th grade joyce.langenegger: that it's easy because it's online guest: I don't have to study, I've learned this before/ Mary: The topic is not a necessary learning tool guest: You don't need to spend time outside of class. Bruce McCluggage- Pikes Peak CC: with Philosophy, most students think they know nothing about it...FALSE Community College of Baltimore County: that they don't have to have/read the textbook guest: Overestimate their skills and abilities. Truman State University: That they just need to come to class - Prof will spoon-feed Bruce McCluggage- Pikes Peak CC: Neighbor nudges (pairs) are one of the best tools I use in class...also helps in students' social dev as well. Truman State University: The pairs also really help to facilitate community building in larger classes Bruce McCluggage- Pikes Peak CC: there is also a I[phone app for students to use as 'clickers' guest: Poll anywhere is a great APP guest: Socrative is also a good APP. Mary: I use Socrative - they use their cell phones (all kinds) to contribute and for polls Truman State University: We are trialing TopHat here this semester - it's a multi-device clicker app Truman State University: What do you mean by "large" classes? Truman State University: How many kids? Jeffrey: We have done 300 on two campuses in two states with Turning Point Carroll University 2: plickers are cool Guest 2: get students involved Cheryl S: various applications for formative assessment techniques Truman State University: Incorporating JITT - really interested in Plickers


Tom Pusateri


Community College of Baltimore County: Presurveys are a good thing to do, and I am already doing that! GPC_CTL: Socrative and TopHat Carroll University: Socrative sounds like an interesting resource Chadron State College: getting students involved with technology - clickers joyce.langenegger: Using assessment results to supplement student evaluations of teaching. guest: The importance of formative assessment and the different ways to conduct it. Luzerne County Community College: Monitoring student learning during course with CAT and Ability to make adjustments. Jeffrey: See First to Understand, then to be understood (S. Covey) Know who your students are Bruce McCluggage- Pikes Peak CC: Most important (unique) point: Resource of CATs and FLAGs will be helpful long-term. (already utilize stated techniques) Luise: socrative and plicker sound fun Bruce McCluggage- Pikes Peak CC: Covey was quoting St. Francis of Assisi Bruce McCluggage- Pikes Peak CC: very powerful point! Jeffrey: Thanks Bruce. Makes sense. Carroll University: I love muddiest point - very effective for achieving objectives Bruce McCluggage- Pikes Peak CC: Minute paper...fantastic idea...thank you! Bruce McCluggage- Pikes Peak CC: name of article again? Mary: In most LMS course sites you can make the discussion anonymous and the students will give you relevant data. Bruce McCluggage- Pikes Peak CC: IMO another great idea...thank you! Bruce McCluggage- Pikes Peak CC: JITT ...one of the most helpful approaches for creating a Socratic (Discussion-based) class...literally 'flipping the Class'. Luzerne County Community College 2: Thank you. ~ J.Menapace LCCC Bruce McCluggage- Pikes Peak CC:I give a quick quiz on the reading and then d2L and blackboard can give you immediate results and you can then focus on the troublesome questions Bruce McCluggage- Pikes Peak CC: all before discussing it in class Mary: Would you consider doing a Webinar for a faculty group working on accreditation? Sanda: Thank you, very good ideas.- (from GPC) Community College of Baltimore County: How often in a semester do you use the flipped classroom Carroll University: Thank you so much. Georgia Perimeter College: Thank you. Mary: Thank you very much for conducting this Webinar. :) Bruce McCluggage- Pikes Peak CC: Thank you, Tom, for your affirming webinar today as well as direct to-the-point suggestions for class assessment :) Chadron State College: great introduction that puts CATs in context guest: Is there a software program that will pick up the most repeated words from the one minute paper? Bruce McCluggage- Pikes Peak CC: thank you to my fellow colleagues who joined this webinar today and for some of the ideas that YOU have shared! Mary: Most of us need to do class evaluations and modify our classes to Universal Design for Learning. Bruce McCluggage- Pikes Peak CC: UDL, Mary? T Garren: Word clouds will collect most common words Mary: Yes, U of Washington site has great ideas on how to help you redesign your courses. Bruce McCluggage- Pikes Peak CC: thank you, Mary! Ever walked on Red Square there at the U-Dub campus?


Tom Pusateri


Mary: Yep. ;) Bruce McCluggage- Pikes Peak CC: ;)

Documents

Transcript - ceisys.ust.hkceisys.ust.hk/magna/...Classroom-Assessment-Techniques-mos-tran… · Getting Started with Classroom Assessment Techniques A Magna Online Seminar was presented