This article was downloaded by: [University of Kiel]On: 25 October 2014, At: 16:29Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number:1072954 Registered office: Mortimer House, 37-41 Mortimer Street,London W1T 3JH, UK

Computer Science EducationPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/ncse20

Improving Learning WithoutImproving TeachingPeter Kugel aa Boston CollegePublished online: 03 Aug 2006.

To cite this article: Peter Kugel (1989) Improving Learning WithoutImproving Teaching, Computer Science Education, 1:2, 145-152, DOI:10.1080/0899340890010206

To link to this article: http://dx.doi.org/10.1080/0899340890010206

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of allthe information (the “Content”) contained in the publications on ourplatform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy,completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views ofthe authors, and are not the views of or endorsed by Taylor & Francis.The accuracy of the Content should not be relied upon and should beindependently verified with primary sources of information. Taylor andFrancis shall not be liable for any losses, actions, claims, proceedings,demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, inrelation to or arising out of the use of the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,

reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of accessand use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

ity o

f K

iel]

at 1

6:29

25

Oct

ober

201

4

COMPUTER SCIENCE EDUCATION 1, 145-152, (1989)

Improving LearningWithout Improving Teaching

Peter KugelBoston College

I have discovered that, by giving my students quizzes at the end of eachclass, I can noticeably improve their learning without having to improvemy teaching. In this paper, I want to discuss how this "discovery" mightbe used by others, how it might be studied scientifically, and how it mightbe developed into a well engineered teaching tool. I want to suggest thatdeveloping simple, well engineered, educational tools might be a worth-while goal for those who want to try to improve computer science educa-tion.

1. THE IDEA

For years now, I've been trying to improve the quality of my courses byimproving the quality of my teaching. Recently, I've found I can domore toward this end by working on my students' learning.

One thing that works, at least for me, is giving my students dailyquizzes. These quizzes seem to serve a diagnostic purpose. They seemto tell students what they do not understand, soon enough for them todo something about it. Perhaps that is why they work. I am not sure.But, I do know this: My students do better in my courses when I givethem daily quizzes than when I don't.

In this paper I want to describe this idea so that others can considerusing it and so that some can think about trying to study it further. But Iwant to do more than that. I think that the idea of a daily quiz is an

Acknowledgements: I want to thank Stan Dmohowski for discussing his use of dailyquizzes with me, Jim Gips for telling me why he always spends the last 15 minutes of his3-hour classes having his students do things in class, and Jeff Parker for his very helpfulreview of an earlier version of this article. My last (IV2 - word) paragraph is borrowed fromJohn Barth.

Correspondence and requests for reprints should be sent to the author at the ComputerScience Department, Boston College, Chestnut Hill, MA 02167.

145

Dow

nloa

ded

by [

Uni

vers

ity o

f K

iel]

at 1

6:29

25

Oct

ober

201

4

146 P. KUGEL

example of a class of techniques that might be worth paying moreattention to. What these techniques have in common is that, rather thantrying to help the teacher do something for the student, they try to helpthe students do things for themselves. And the development of suchtechniques seems like a good way to try what I like to think of as anengineering approach to the improvement of education. The aim of suchan approach is to develop a set of thoroughly tested and well engineeredtools that educators can use.

2. ITS HISTORY

I first considered the use of daily quizzes when I was teaching anintroductory course in Artificial Intelligence (AI). My students alreadyknew how to program, but I wanted to teach the course in LISP, a goodlanguage for expressing AI ideas. Since they did not know LISP, I spentthe first two weeks teaching it to them. I thought I did a pretty good job,but when I gave them a test, I was appalled at the results. They hadn'tlearned what I thought I was teaching. I was worried and annoyed. Ineeded to teach them LISP in order to get on with the course. I felt thatdrastic measures were called for.

Later that day, I overheard a student thanking her accounting pro-fessor for giving daily quizzes. The student said she didn't like thequizzes. She lived in terror of them since they were graded and countedtoward her final grade. But she admitted that they had helped her learnthe subject. That seemed like an idea worth trying, so I spent anotherweek on LISP, this time giving short quizzes at the end of each class.The students learned LISP, and I learned something, too. I discovered atool that worked for me then and that has worked for me since.

3. THE TECHNIQUE

The injunction to "Give a daily quiz!" does not define a technique. Thedetails need to be filled in, and how they are filled in can make animportant difference. For one thing, I make a big point of telling mystudents that the quizzes are for their benefit and not for mine. I tellthem that what they are learning in my classes is like a program thattheir mind uses to do things, and that the quizzes are like debuggingruns that they can use to see if the "programs" they have developed intheir minds from what we have done in class are working properly. Ifthey are not, the quiz results give them early warning and enough time

Dow

nloa

ded

by [

Uni

vers

ity o

f K

iel]

at 1

6:29

25

Oct

ober

201

4

IMPROVING LEARNING 147

to develop their mental programs before they have to use them in teststhat really count toward their grades.

My quizzes are not like the exams they dread. I make it as clear as Ican that the quizzes are for their information rather than for mine. (Idon't count them in determining their final grades.) I let my studentsuse their notes, so they don't have to memorize the material; and I letthem take their time, so they don't have to rush, and the students gradetheir own quizzes. That helps not only to reduce their anxiety, but alsomy work. (If I had to grade exams all the time, I probably wouldn't givethem.)

I make it clear that I am not going to use their performance to evaluatethem. Since they grade their own quizzes, they don't even have to showthem to me unless they want to (which they often do). Letting themgrade the quizzes in class has another advantage: Students see theirmistakes right away so that they can correct today's misunderstandingsbefore they go on to tomorrow's material. I welcome their questions(and I get a lot more with quizzes than without), and I tell them otherways to work on the problems that the quizzes reveal. Sometimes justthinking about their difficulties is enough. When it is not, there is thetext and there are the other students in the class. Sometimes, I ask themto study in specific groups and I am surprised at how often the idea ofstudying with others seems new to them.

If only one or two students have a particular problem with the mate-rial, I help them with it individually. If many students share a similarproblem, I deal with it in the next class. Misunderstandings can beidentified early and individually so that they can be dealt with effec-tively, before they pile up and the students go, as the author of onecomputer science textbook puts it, "into autopilot".

Done properly, daily quizzes can change students' attitudes towardmistakes. I find that students often look at mistakes as something toavoid. I encourage my students to think of mistakes as opportunities tocorrect misunderstandings. And I discuss interesting mistakes in class.(At a neighboring institution, two well-regarded professors give anaward for the best student mistake at the end of each term.)

Another thing that helps make my quizzes less unpleasant for mystudents is that I never test their memory for facts or their speed. I amconcerned with understanding. My tests are always about what I cov-ered in class that day. I usually ask them to apply what we've beentalking about in class to a situation different from any that I used asexamples in class. For example, I may write a simple program in front ofthem to illustrate an idea and then ask them to write a program to usethat idea to do something different. Sometimes, asking them to apply

Dow

nloa

ded

by [

Uni

vers

ity o

f K

iel]

at 1

6:29

25

Oct

ober

201

4

148 P. KUGEL

something we have covered in class is not appropriate. Sometimes, Ican't think of a good problem. When that happens, I may ask them tosummarize, in a few of their own words, something I have said in manywords of my own. That makes them put it all together and, if it doesnothing else, makes them look over their notes and try to make sense ofthem.

My quizzes are short. I try to focus on the wheat, and avoid the chaff.At first, I made up my questions before class, but I often found myselfracing ahead to get far enough to ask that question. Now I make up afew possible questions and choose the one I am going to ask when Iknow how far I am going to get. (If none of my preplanned questions fit,I go into my emergency mode and ask them to summarize.) That's whatI do. Others might want to handle it differently.

Letting students grade their own quizzes has worked well for me, butthat may not always be the best thing to do. If you want to be sure thatyour students have it right, you don't trust them to determine that, youmay want to do your own grading. You may want to do your owngrading if you teach a subject in which the correctness of an answer ishard to determine or requires judgment. You may want to have yourstudents grade each other's quizzes. In some situations, it may be betterto give the quiz at the start of the next class, forbid the use of notes, askthem to hand in their quizzes after they have graded them, or to varyother details.

The basic idea seems, to me, to be sound and for those of us who havedone about as much as we can to improve our teaching, the idea of thedaily quiz, which seems to improve student learning, may be a soundstep toward improving the effectiveness of our classes.

4. EVALUATION

That is where this article would stop if all I wanted to do was to pass onan interesting idea. But, I think that the development of educationalideas should involve more than dreaming them up, trying them out inone's classes, and passing them on. I want to suggest that they couldprofit from further development. Think of an educational idea like aprogramming idea or algorithm. After we dream up an algorithm, andtry it out on a few examples to make sure it works, we typically try to domore. We may try to determine the conditions under which it will workwell so that others can decide when it is appropriate to use it and whenit is not. We may try to prove that it will always work correctly and try todetermine its efficiency. We may try to understand it. And we may try to

Dow

nloa

ded

by [

Uni

vers

ity o

f K

iel]

at 1

6:29

25

Oct

ober

201

4

IMPROVING LEARNING 149

refine it and explain it clearly to others. I think that we ought to do manyof these things with new educational techniques.

Look, for example, at how the idea of a daily quiz might be developed.First, we might do some experiments to determine (i) whether it worksin general, (ii) the conditions under which it works well, and (iii) why itworks. After we have done those things we might try to sharpen it up inmuch the same way as a computer scientist tries to sharpen up analgorithm or an engineer tries to sharpen up a new technique.

i. Does It Work?

An idea, like that of the daily quiz, can be said to work if it makes asuitable difference. When we think about experiments to show that atechnique makes a difference, it quickly becomes apparent that there aremany extraneous variables to worry about in any experiment. Suppose,for example, that we decide to test the idea by trying it out on a largeclass with many sections and a single common final. We will give dailyquizzes to some sections, but not to others. Then, we will compare theperformance on the final exam of sections that had the quizzes to theperformance of those that did not and see if they really did better.

If we find a difference, and it goes in the right direction, how do weknow that the quizzes made the difference and that it wasn't somethingelse? The students in different sections will differ in ability. They willmeet in different rooms and at different times of day. They may havedifferent instructors, and they will differ in a variety of other ways.There are familiar techniques for handling these problems, but it isn'tgoing to be easy and it's going to take a lot of time to get enough data tobe significant. Still, such experiments might be worth conducting.

That is the scientific way to determine the effectiveness of this tech-nique, but there are two less scientific techniques that can both besummarized by saying "try it." One is the one I suggested earlier. Try itin your classes and see if it works for you. The other you can do is theproverbial "privacy of your own home." When you get to the end of thisarticle, spend five more minutes writing a summary of it, as though youhad read it in a class and you had been asked to take a quiz on it. If youdo that (or even if you only think about doing that), I think you will seethat the idea of a quiz at the end of a lecture might have some merit.That's not very scientific, but it can be quite convincing.

ii. When Does It Work Well?

Once we establish that a technique works in general, the next thing todo seems to be to try to characterize the conditions under which it works

Dow

nloa

ded

by [

Uni

vers

ity o

f K

iel]

at 1

6:29

25

Oct

ober

201

4

150 P. KUGEL

well. Does the effectiveness of the daily quiz depend on the subjectmatter? On the teacher? On the nature of the students in the class? Is itbetter if the students grade their own quizzes? Their neighbors' quizzes?Are weekly quizzes better than daily ones? Should the quizzes be givenat the end of one class or the beginning of the next? Are quizzes that askfor summaries better or worse than quizzes that ask for an idea to beapplied to an example? And the like.

We might study teaching techniques much as we now study algo-rithms. We might try to determine the conditions under which theywork well and those under which they do not work well. That providesinformation to the users (or, in this case, the teacher) who can use it tochoose the technique appropriate to his or her situation since, presum-ably, no single teaching technique is "best" for all possible situations.

iii. Why Does It Work?

Once we know that a technique works and and the conditions underwhich it works well, we might try to determine why it works. There areseveral reasons why daily quizzes might work. Here are some that cometo my mind:

• Chunking: Perhaps daily quizzes work because they help studentsbreak up the material into manageable-sized chunks that they canthen try to master, one at a time. In elementary education, the ideaof presenting material in bite-sized chunks and letting studentsdigest each chunk before going on to the next is called "masterylearning." (In computer science, breaking a job into parts and test-ing the parts is called "modularization," and it has been shown tobe an effective technique for developing large programs.) Quizzesmark the edges of chunks and tell a student whether or not morework is needed on the current chunk before going on to the next.

• Doing: Perhaps daily quizzes work because they force students to dosomething in class rather than just sitting and listening to others.Remember the Chinese proverb that goes "I hear and I forget. I seeand I remember. I do and I understand."? Learning to program, andlearning much of the rest of computer science, is learning to dosomething and not learning to talk about doing something. And, asAristotle put it, "what we learn to do, we learn by doing."

• Stimulation: Perhaps daily quizzes may simply help students stayawake. The quiz coming in a few minutes gives students somethingin the near future to listen for. The anticipation of a quiz coming in afew minutes provides a more convincing motive for listening care-fully than the much bigger and more important quiz coming next

Dow

nloa

ded

by [

Uni

vers

ity o

f K

iel]

at 1

6:29

25

Oct

ober

201

4

IMPROVING LEARNING 151

month. Students often "tune out" at the end of a class. It's hard totune out at the end of a class if you are taking a quiz. I have a friendwho teaches night school courses that run for three hours an eve-ning. He makes a point of spending the last 15 minutes of those longclasses having students write something. As he points out, it keepsthem awake during the time when it's hardest to stay awake.

• Confidence: Perhaps daily quizzes help students overcome whatsome psychologists call "learned helplessness," the feeling thatthey just can't do what they are being asked to do. By asking themto do little things every day, right after they have been explained,students may be learning that they can do things. They may feelbetter about themselves. I would imagine that this could be partic-ularly important for students brought up in those parts of oursociety (typically the very lowest and highest classes in our society)where learned helplessness seems common.

• Metacognition: Perhaps daily quizzes help students think about theirown thinking, which some people call "metacognition". There issome evidence that thinking about your own thinking improves thatthinking.

• Feedback: Perhaps daily quizzes really do teach students something.Perhaps they teach each student what he or she does not under-stand. Perhaps students use the information they get from quizzesto correct their understanding of the subject much as artillery guid-ance systems use feedback to correct their aim in mid-course. (Thatpun is, alas, intended.)

• The Banal Reason: We should not overlook the possibility that dailyquizzes may help students do better on quizzes simply because theyteach students how to take quizzes. They may not help the studentlearn the subject matter at all.

• The Holistic Reason: Nor should we overlook the possibility that dailyquizzes work for all these reasons, or many of them.

5. ENGINEERING

Once we have an educational technique in hand, we know that it works,when it works and why it works, there remains one more thing to do:refine that technique into an elegant and robust tool. Developing spe-cific techniques that teachers can use in class and refining them intoelegant tools is only one possible goal for educational research, but itstrikes me as a rather promising one.

Ideas that serve to improve learning may lend themselves more readi-ly to such development than ideas that try to improve teaching. They

Dow

nloa

ded

by [

Uni

vers

ity o

f K

iel]

at 1

6:29

25

Oct

ober

201

4

152 P. KUGEL

seem somehow to be more independent of each other and less depen-dent on the subject matter being taught. Other ideas that might beworth working on include: teaching students how to take better notesand forcing them to review their notes right after class, teaching stu-dents to consciously think about their own thinking, forming studentstudy teams for mutual help, and the like.

But this kind of development is not limited to tools for improvinglearning only. One teaching tool that might be worth developing in thisspirit is the idea of speaking more slowly and pausing more often. It hasbeen shown that the teacher who speaks slowly, and pauses often, getsbetter results than the teacher who speaks more quickly, and pausesinfrequently, even though it seems as though the latter is able to presentmore material. Another idea that seems worth investigating is the valueof summarizing at the beginning and end of class.

Such tools are much more modest than computerized instructionalsystems that teach a whole subject or tutoring systems that diagnosestudent errors as they arise. But they are much easier to hone into sharptools. In the development of such techniques, the sensibilities of anengineering mind seem to be the right ones to bring to bear. Engineersoften try to develop simple, but powerful, tools that have been thor-oughly explored and tested.

The idea of daily quizzes may not help us understand the educationalprocess, but it might help us to improve it. There are undoubtedly manyother ideas out there that can do likewise. To improve education ingeneral, such ideas probably ought to be studied experimentally andthen engineered into robust educational tools.

6. SUMMARY

I have suggested that daily quizzes can improve student learning. And Ihave suggested three things that we might to with the idea of such dailyquizzes:

• Use it in our classes to improve student learning.• Study it to determine when it works and why it works.• Refine it, or engineer it, into a more elegant educational tool.

The idea of the daily quiz is, in some sense, a "little" idea. Much oftoday's educational research seems to be devoted to developing "big"ideas: new curricula, new computer systems, or new theories of instruc-tion. But, it seems to me that there may be some merit in trying todevelop the "little" ideas.

Let's.

Dow

nloa

ded

by [

Uni

vers

ity o

f K

iel]

at 1

6:29

25

Oct

ober

201

4