12 L I B E R A L ED U C A T I O N SP R I N G 2008

ARE NEW APPROACHES to transforming under-graduate learning in science, technology, engineering, and mathematics (STEM) makinga difference? If so, how? How do we know? Andwhat next? These are the questions explored ina 1999 report from Project Kaleidoscope, whichconcluded by making predictions and recom-mendations for the coming decade (Rothmanand Narum 1999). Now that that “coming

decade” is here, it istimely to ask how ac-

curate those predictions were and to offer somenew recommendations for the next decade.

The predictions made in 1999 addressed abroad range of issues, from faculty to facilitiesand more. In each of the scenarios for the fu-ture that were developed then, the underlyingtheme was that attention to learning and as-sessment would be pervasive in the undergrad-uate STEM learning environment on campusesacross the country. One reason we thoughtthis would be the case had to do with the an-ticipated impact of How People Learn: Brain,Mind, Experience and School, a seminal reportpublished that year by the National ResearchCouncil. The report called for the developmentof academic cultures where deep understandingabout how students learn determines howcourses and curricula are planned, technologiesselected, spaces designed, and faculty recog-nized and rewarded. Further, it was a report thatcould be used as a resource for shaping and sus-taining such cultures.

There were several other compelling reasonsfor basing our future scenarios upon the ex-pected emergence a new kind of learning cul-ture. In 1999, there was growing externalpressure—from public agencies, accrediting

agencies, funding agencies, and the businesscommunity—for greater transparency with re-gard to student learning outcomes. New ac-creditation practices for engineering educationprograms were challenging that community ofprofessionals, and many other STEM communi-ties were giving new or renewed attention tostudent learning outcomes in their specific dis-ciplines. Moreover, the National Survey ofStudent Engagement was piloted in 1999.

Equally important was the increasing visi-bility and maturity of the work of pedagogicalpioneers—agents of change whose efforts hadbeen supported by the National Science Foun-dation (NSF) since the late 1980s. Their expe-riences and expertise were beginning to informa generation of what dissemination literaturecalls “early adapters.” There was a growing bodyof research-based theory and practice aboutwhat works in the iterative cycle of exploring,examining, addressing, and assessing under-graduate student learning goals.

So, where are we today? Where and whenare conversations about students and studentlearning taking place within institutions orscholarly communities? How widespread amongSTEM faculty and their administrative colleaguesis awareness of the work of pedagogical pioneersand of the growing body of research on learn-ing and cognitive science? Is it now possible toarticulate a general set of goals for studentlearning in STEM fields on which local effortscan be built and against which they can becompared? And if so, what recommendationscan be made for the next decade?

Current conversations covering the rangeof issues related to student learning are dra-matically different from those of a decade ago.There is a growing national consensus aboutwhat students should know and be able to do

FE

AT

UR

ED

T

OP

IC

JEANNE NARUM is director of Project Kaleidoscope.

J E A N N E N A R U M

Is it now possible to articulate a general set of goalsfor student learningin STEM fields?

Looking Back and Looking Ahead

Transforming Undergraduate Programs

inScience,Technology,

Albion College

&Engineering, Mathematics

FE

AT

UR

ED

T

OP

IC as an outcome of their careers

as undergraduate learners.The work of the Associationof American Colleges andUniversities (AAC&U)—most importantly through itsLiberal Education and Amer-ica’s Promise initiative—has been a significantcatalyst in engaging communities in discus-sions about the kinds of learning needed for acomplex and volatile world. Explicit and re-markably consistent goals for student learninghave been articulated and promulgated bygreatly diverse groups both within and beyondthe academy.

The Council on Competitiveness (2005, 76),for example, has called for preparing “a wholegeneration with the capacities for creativethinking and for thriving in a competitive cul-ture, able to work in multidisciplinary teams, . . .

be comfortable with ambiguity,recognize new patterns withindisparate data, . . . [and] tobe inquisitive and analytical.”The undergraduate neuro-science community has out-lined learning goals, including

critical thinking and independent thought;communicating effectively in written and oralform as well as with figures, graphs, and throughpresentation software; and an appreciation ofthe value of diversity and the ability to workwith colleagues from a variety of backgroundsand perspectives (see Wiertelak 2003). TheAmerican Chemical Society’s Committee onProfessional Training has stated that theoutcome of laboratory experiences should“give students hands-on experience withchemistry and the self-confidence and compe-tence to . . . interpret experimental results and

14 L I B E R A L ED U C A T I O N SP R I N G 2008

Attention to how people learn is beginning to transform the

undergraduate STEMlearning environment

Penn State

draw reasonable conclusions, analyze data sta-tistically and assess reliability of results . . . andcommunicate effectively in small groups andteams. . . ” (2003, 10).

Are the conversations taking place withinnational groups changing the dialogue oncampuses? One indication that they are is thefact that colleges and universities across thecountry are beginning to make public their ex-plicit visions of student learning by publishingthem on their Web sites. The public announce-ment of the specific learning outcomes estab-lished by Miami Dade College is one of themost recent and most visible examples of themainstreaming of attention to setting and as-sessing student learning goals both within andbeyond STEM fields (see the article by EduardoJ. Padrón in this issue of Liberal Education).

This brings me to the first of my recommen-dations for the coming decade, namely that

leadership teams on campuses gather and distilllists of learning outcomes articulated by leader-ship associations such as AAC&U, as well as byother professional and disciplinary societies andcorporate leaders, and translate those statementsinto a coherent and institutionally-appropriateset of goals for student learning that servestheir vision for the future. Make those goals,and the actions that advance them, publicand transparent.

Pedagogies of engagementThe most compelling evidence of the main-streaming of conversations about how peoplelearn comes from the field. Just-in-Time Teach-ing, Problem-Based Learning, and Student-Centered Activities for Large EnrollmentUndergraduate Programs are three examplesof what Russell Edgerton (2001) has describedas “pedagogies of engagement,” and theydemonstrate the ways in which attention tohow people learn is beginning to transform theundergraduate STEM learning environment.

Just-in-Time Teaching (JiTT) involves dia-logue between student and student as well asbetween student and instructor, much of whichoccurs outside of the classroom—thanks, inpart, to the maturation of electronic technolo-gies. Gregor Novak, one of the JiTT pioneers,says that at the heart of the JiTT pedagogy arepre-instruction, Web-based assignments called“warm-ups.” These are short, thought-provokingquestions that, when fully discussed, often havecomplex answers. Students are expected to de-velop the answers as far as they can on theirown, and then the job is finished by workingtogether in the classroom. These warm-upsare submitted electronically just a few hoursbefore class, giving the instructor (just)enough time to incorporate into the upcominglesson the insights gained from student sub-missions. Exactly how the classroom time isspent depends upon a variety of issues such asclass size, classroom facilities, and student andinstructor personalities.

In a JiTT classroom, students construct thesame knowledge as in a passive lecture, butwith two important added benefits. First, havingcompleted the Web assignment very recently,they enter the classroom ready to engage ac-tively in their learning. Second, they have afeeling of ownership of their learning becausethe interactive lesson is based on their own

SP R I N G 2008 L I B E R A L ED U C A T I O N 15

FE

AT

UR

ED

T

OP

IC

FE

AT

UR

ED

T

OP

IC wording and understanding of the relevant is-

sues. “Our goal,” Novak explains (pers. comm.),“is to create and sustain team spirit. We all,students and faculty, work together toward thesame objective, that students pass the coursewith the maximum amount of enduringknowledge, skills, and habits of the mind thatare critical for success in STEM communitiesof learners and practitioners.” Does JiTTwork? Consider the following testimony froma student:

It is easy to feel disconnected from a sciencecourse as a student. Each day can seem as anew set of notes to take from the instructor’smonologue, another chapter to read, andanother problem set to work on, but eachunrelated to the previous day—that is, untilthe exam. The situation changes if the as-signments are designed to pose questionsthat require some real effort and interactionwith other students ahead of class, but pro-viding the assurance that the toughest pointswill be cleared up in the class makes thatwork worthwhile. Just-in-Time Teachingoffers the kind of day-to-day motivation thatdrives the course forward for me. (ProjectKaleidoscope 2007, 2) Problem-Based Learning (PBL) simulates

workplace projects that require mastery of arange of content knowledge as well as the de-velopment and application of process skills inan integrative and interesting format. Facultyof the biomedical engineering program at

Georgia Institute of Technology, for example,arrived at PBL as the foundation for planning aprogram from scratch. Theirs was a disciplinewithout a history of pedagogies or a traditionof textbooks. (Admittedly, this can be seen asa luxury when trying to incorporate researchon learning into the process of curricularchange!) The inherent interdisciplinarity ofbiomedical engineering drew them into re-search on cognitive flexibility—that is, theability to look at problems from a variety ofperspectives. Their program was designed tochallenge students with the right kind ofproblems, and the goal was to produce integra-tive problem-solvers who have the cognitiveflexibility to apply engineering analysis andsynthesis to problems in the biosciences.

On the impact on student learning, WendyNewstetter, one of the founding members ofthe PBL team at Georgia Tech, reports that

solving problems on the frontiers of sciencethat other experts are trying to solve at thesame time does two things: it motivatesstudents tremendously, and has a very inter-esting impact on identity. A major problemwith students going into the sciences andbeing sustained is that they don’t identifywith the kind of activity they are being askedto do. They don’t see their own personalidentities or lives aligned with science.Whereas, when you give them complicated,multi-dimensional, interdisciplinary prob-lems from the real world, their imaginationis sparked. They begin to say, “I can seemyself doing this.” So problem solving isabout motivation and identity, about en-gaging students through the excitementand fantasy of trying to solve those problems.(Project Kaleidoscope 2006)Student-Centered Activities for Large En-

rollment Undergraduate Programs (SCALE-UP)is based on the conviction that, in order to un-derstand the science, students must actually dothe science as a central activity in studyingthe science. SCALE-UP overcomes the manybarriers to “doing” science in the traditionallarge lecture class: the isolation of individualstudents in the crowd of strangers, the compet-itive atmosphere, and the little one-on-onecontact with the instructor. SCALE-UP usescooperative learning pedagogical techniqueswith classes of approximately one hundredstudents, with lecture and lab integrated intechnology-rich settings. Active group learning

16 L I B E R A L ED U C A T I O N SP R I N G 2008

Albion College

is promoted within consistentgroups, and the grading sys-tem requires teamwork to en-sure that each student in thegroup—even the really brightones—benefits from workingtogether.

SCALE-UP pioneer RobertBeichner reports that substan-tive evaluation—video andaudio recordings, interviews, focus groups, pre-and post-tests, and student profiles—has re-vealed improvements in students’ ability tosolve problems and their attitudes towardscience and learning, as well as increases stu-dents’ conceptual understanding. And SCALE-UP reduces failure rates. When asked whatimpression a visitor would get from walkinginto a SCALE-UP classroom, Beichner says,

their impression would be that the learningspace looks more like a restaurant than aclassroom, or perhaps more like a banquethall, because there is much noise from thevisibly engaged students. They would seethe realization of our idea that social inter-actions between students and their teachersis the “active” ingredient that works for us.From our own experiences and from re-search on learning, we knew that as stu-dents collaborate on interesting tasks theybecome deeply and personally involvedwith what they are learning. The doors ofthe closets and the walls of the classroomare covered with whiteboards—publicthinking spaces—to help them share theirlearning with each other and their instruc-tor. (Project Kaleidoscope 2005)What we find in examining the core princi-

ples of these and other pedagogies of engage-ment is that each of them can be seen, in someway, as a grandchild or great-grandchild of un-dergraduate research, which is surely the epit-ome of a pedagogy of engagement. Each ofthese contemporary pedagogies is designed tointroduce students to the community of prac-tice in STEM fields and to enhance their per-sonal understanding of how scientists,engineers, and mathematicians make sense ofour world. In other words, each invites stu-dents to assume the identity of the STEM pro-fessional. Each seeks to give studentsconfidence in their ability to pursue study inSTEM fields, based on successive (and presum-ably successful) social interactions within a

collaborating community.These are communities inwhich expert and novicelearners come together in thesame spaces—sometimesphysical, sometimes virtual—to engage in the process of dis-covery as they move fromwhat is known to what mightbe known. These are pedago-

gies designed to help students understand theimportance of being a part of a collaboratingcommunity with a sense of shared purpose.

Another similarity is that, although mostbegan with support from the National ScienceFoundation, these pedagogies work for all disci-plines, serve all institutional types, strengthenthe learning of all students, and reflect societaland disciplinary goals for undergraduate learn-ing. They all call upon instructors to reflectdeeply on how to get students to take owner-ship of their learning, how to transform thelearning environment from one of passivetransmission of information to active construc-tion of knowledge and skills, and how to moni-tor the progress of their students’ learning.

BarriersDespite the recognized national need for thekind of problem-solving, critical-thinkingrisk-takers formed through these pedagogiesof engagement, their adoption is not yet

SP R I N G 2008 L I B E R A L ED U C A T I O N 17

FE

AT

UR

ED

T

OP

IC

Each of these contemporary pedagogies is designed to

introduce students to the community

of practice in STEM fields

Project Kaleidoscope (PKAL) is a leadingadvocate for what works in building and sustaining strong undergraduate programs inthe fields of science, technology, engineering,and mathematics (STEM). An informalalliance, PKAL takes responsibility for shapingundergraduate STEM learning environmentsthat attract students to STEM fields and inspirethem to persist and succeed. Such environmentsgive students personal experience with the joy ofdiscovery and an awareness of the influence ofscience and technology in the world. Resourcesderived from the work of the extensive PKAL community are available for adaptation by leaders on campuses across the country. Visit www.pkal.org for more information.

Project Kaleidoscope

FE

AT

UR

ED

T

OP

IC

widespread. One significant barrier to thespread of contemporary pedagogies of engage-ment is institutional culture. The compellingresearch conducted by Charles Hendersonand Melissa H. Dancy (2007, 1) examineswhy “proven strategies are slow to integrateinto mainstream instruction” even though, asthey document, STEM faculty understandsome of the problems with traditional ways ofteaching. Henderson and Dancy concludethat situational characteristics consistent withtraditional instruction account for the majorimpediments. Their suggestion, directed atthe broader STEM research community, isreally for us all: identify both the situationalbarriers faculty face—bolted-down chairs,large enrollment classes, the need to “cover”content, student lack of experience with active learning, etc.—and the means to re-moving those barriers. As they say, “after all,getting the chairs unbolted is often a non-trivial task.”

This brings me to the second of my recom-mendations for the coming decade, namelythat leadership teams gather critical informa-tion about interest and expertise relative topedagogies of engagement within their cam-pus community and about local barriers topromoting and adapting those approaches;design and implement an action plan to over-come barriers in the process of adapting peda-gogical innovations to serve learning goals forthe students for whom they are responsible.

An oft-cited barrier to the mainstreamingof pedagogies of engagement is the lack ofevidence of their efficacy. Over the past decade,however, this barrier has been largely sur-mounted by building on efforts to link researchon STEM learning to STEM teaching—effortsthat go back many years, at least to the workat the University of Washington by ArnoldArons and Lillian McDermott (see Narum andRothman 1999). Although these efforts werepresent and becoming visible in 1999, the pastten years have seen a significant increase inefforts like those of Nobel Laureate CarlWeiman (2007) to take a scientific approachto pedagogical and institutional transformation.

Among those leading these efforts are a cadreof assessment pioneers, who, with support fromthe NSF’s Assessing Student Achievement(ASA) program, have been deeply involved inresearch-based initiatives designed to get insidethe learning process and gather evidence todocument what works and for which students.Some start from what disciplinary content stu-dents should know (the Calculus Concept In-ventory, the Geoscience Concept Inventory,and Measuring What Students Know aboutHow to Learn Chemistry). Others approachassessment directly from what skills studentsshould acquire (Assessing Problem-SolvingStrategies in Chemistry and Assessing CriticalThinking Skills) or from the STEM literacyperspective (Assessing Students’ Value forScience and Math Literacy).

At a meeting of the ASA community in2006, project principal investigators wereinvited to share insights from their experi-ences with colleagues across the country. Theyurged STEM faculty to recognize that studentsknow and understand less when they emergefrom courses than most faculty think they do;that what we teach, despite our best efforts, isnot what students learn or how they learn;that student achievement can be increasedwith effective assessment; and that you canteach better and enjoy it more if your studentsare demonstrably learning better. This groupof experienced practitioners also thought thatthe situational barriers that keep others fromexploring, adapting, and extending their workcould be overcome if three conditions weremet. First, STEM faculty and their administra-tive colleagues would need to realize there isno need to reinvent the assessment wheel—aneffort costly in both time and energy. Second,

18 L I B E R A L ED U C A T I O N SP R I N G 2008

Drury University

faculty would need to identify themselves asmembers of the community of STEM pedagogi-cal and assessment practitioners in the sameway they feel an identity as a member of aSTEM research community. And third, formalopportunities would need to be available atthe local level—campuswide, within depart-ments or programs—for conversations abouthow difficult it is to teach certain concepts orstudents with different learning styles, conver-sations that capitalize on the expertise and in-terest resident in their community by engagingthat broader community in asking, what worksfor our students, and how do we know?

I hesitate to predict what the STEM worldwill look like in 2019, especially given howdifferent it is today from the way the 1999 Pro-ject Kaleidoscope report predicted it would be.And so I will end instead with the third of myrecommendations for the coming decade,namely that those with a stake in a robusttwenty-first-century undergraduate STEMlearning environment step back, take time towork with the communities of which they area part to agree on outcomes from undergradu-ate engagement in STEM learning, determinetheir individual and collective responsibilityfor ensuring their students achieve those goals,and make it happen. ■

To respond to this article, e-mail liberaled@aacu.org,with the author’s name on the subject line.

REFERENCESAmerican Chemical Society Committee on Professional

Training. 2003. Undergraduate professional educationin chemistry: Guidelines and evaluation procedures.Washington, DC: American Chemical Society.

Council on Competitiveness. 2005. National innova-tion initiative summit and report: Thriving in aworld of challenge and change. Washington, DC:Council on Competitiveness.

Edgerton, R. 2001. Education white paper.http://www.pewundergradforum.org/wp1.html.

Henderson, C. and M. Dancy. 2007. Barriers to theuse of research-based instructional strategies: The influence of both individual and situationalcharacteristics. Physical Review Special Topics:Physics Education Research 3 (2): 1–14.

Narum, J. L., and F. G. Rothman. 1999. Then, nowand in the next decade. Washington, DC: ProjectKaleidoscope.

National Research Council. 1999. How People Learn:Brain, Mind, Experience and School. Washington,DC: National Academies Press.

Project Kaleidoscope. 2005. Student-centered activitiesfor large enrollment undergraduate programs(SCALE-UP): An interview with Robert Beichner.www.pkal.org/documents/Beichner_SCALEUP_Interview.cfm.

———. 2006. Problem-based learning at GeorgiaTech: An interview with Wendy Newstetter.www.pkal.org/documents/PBL_GeorgiaTech.cfm.

———. 2007. Just-in-time teaching. What Works:Reports from the Community 4: 2. www.pkal.org/documents/All%20Pedagogies%20in%20one%20document%20for%20posting.pdf.

Rothman, F. G., and J. L. Narum. 1999. Then, nowand in the next decade: A commentary on strengtheningundergraduate science, mathematics, engineering andtechnology education. Washington, DC: ProjectKaleidoscope.

Weiman, C. 2007. Why not try a scientific approachto science education? Change 39 (5): 9–15.

Wiertelak, E. 2003. Neuroscience education: Goalsfor the undergraduate program. www.pkal.org/ documents/GoalsUndergradNeuroscience.cfm.

SP R I N G 2008 L I B E R A L ED U C A T I O N 19

FE

AT

UR

ED

T

OP

IC

Engaging Science, Advancing Learning: General Education, Majors, and the New Global Century

November 6 - 8, 2008 | Providence, Rhode Island

There is strong agreement that the United States must make science achievement a top priority. We need more suc-cessful science graduates, and we need many more graduates— whatever their

in making decisions. With decades of innova-tion in science teaching behind us, what are the most promising avenues for raising the quality of students’ engagement and achievement in science? This conference will focus on how colleges and universities—working together—can create a climate for engaging science for far-reaching educational change.

Association of American Colleges and Universities

Network for Academic Renewal Conference

For more information or to register:www.aacu.org | 202-387-3760 | network@aacu.org