This article was downloaded by: ["University at Buffalo Libraries"]On: 08 October 2014, At: 13:53Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

Journal of GeographyPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/rjog20

Concept Mapping Strategies: Content, Tools andAssessment for Human GeographyStephanie Wehry a , Heather Monroe-Ossi a , Sharon Cobb b & Cheryl Fountain aa Florida Institute of Education , University of North Florida , Jacksonville , Florida , USAb Department of Economics and Geography, Coggin College of Business , University of NorthFlorida , Jacksonville , Florida , USAPublished online: 22 Mar 2012.

To cite this article: Stephanie Wehry , Heather Monroe-Ossi , Sharon Cobb & Cheryl Fountain (2012) ConceptMapping Strategies: Content, Tools and Assessment for Human Geography, Journal of Geography, 111:3, 83-92, DOI:10.1080/00221341.2011.604094

To link to this article: http://dx.doi.org/10.1080/00221341.2011.604094

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Concept Mapping Strategies: Content, Tools and Assessment forHuman Geography

Stephanie Wehry, Heather Monroe-Ossi, Sharon Cobb, and Cheryl Fountain

ABSTRACTThis article examines the use of conceptmapping for formative and summativeassessment of northeast Florida middleschool students’ knowledge of humangeography. The students were participantsin an afterschool, academic, college reach-outprogram that provided opportunities to testconcept mapping strategies that supportspatial thinking and stimulate interest inhuman geography. The study documents theuse of concept mapping for assessment ofseventh graders’ achievement of a specificlesson and for students’ human geographyachievement across all implementation grades.Concept mapping results provided insightinto aspects of the curriculum and instructionwhere appropriate modifications could betterfacilitate meaningful learning.

Key Words: AP Human Geography,assessment, geographic education, conceptmapping, urban education

Stephanie Wehry, Ph.D., is Assistant Director forResearch and Evaluation at the Florida Institute ofEducation located at the University of North Florida,Jacksonville, Florida, USA. Her research interestsinclude concept mapping, research methodology,evaluation, and measurement.

Heather Monroe-Ossi, doctoral student, is a cur-riculum research associate at the Florida Institute ofEducation located at the University of North Florida,Jacksonville, Florida, USA. Her research interestsinclude concept mapping, instructional technology,curriculum development, and evaluation.

Sharon Cobb, Ph.D., is an associate professor in theDepartment of Economics and Geography, CogginCollege of Business, University of North Florida,Jacksonville, Florida, USA. Her research interestsinclude financial geographies, business GIS andgeography education.

Cheryl Fountain, Ed.D, is Executive Director ofthe Florida Institute of Education located at theUniversity of North Florida, Jacksonville, Florida,USA. Her research interests include underservedpopulations, instructional design, evaluation re-search, and concept mapping applications.

INTRODUCTIONGraphic representations such as charts, maps, still photographic images, videos,

and GIS (geographic information system) output are some of the major tools thatgeographers use to document and analyze the distribution of human activities.We suggest that concept maps, used to visually represent knowledge, are toolsthat teachers can use to assess student learning. Learners build knowledge bymaking a growing network of connections among concepts, and concept maps(nonlinguistic, hierarchical diagrams) systematically show these connections.

The overall purpose of this project was to explore the use of concept mappingfor formative and summative assessments of the human geography knowledge ofmiddle school students participating in an afterschool, college reach-out program,PreCollegiate Connections: College Reach-Out Program (CROP). College reach-out programs are a response to the national focus on increasing the number ofeconomically disadvantaged students who enroll in and complete postsecondaryeducation. These students frequently become the first generation in their familiesto do so. The academic content of CROP is focused on human geography—a choiceprompted by the offering of Advance Placement Human Geography (APHG)1

beginning in the ninth grade in district high schools. A CROP goal is to stimulateparticipating students’ interest in human geography in hopes of motivating themto enroll and to be successful in APHG.

We believed that concept mapping could be used to explicitly teach connectionsamong topics and to measure students’ understandings. Because we believecurriculum design is an evolving process, we hoped to use results from theconcept mapping assessments to revise the geography curriculum in ways thatbetter prepare middle school students for higher level thinking activities, increasetheir problem solving skills, and quantify their human geography achievement.In the next section, we document previous literature addressing the state of K–12geography in the United States, describe the context of CROP, and delineatethe uses of concept mapping in educational settings. We then describe andanalyze the process for and design of concept maps used to assess middle schoolgeography knowledge. We conclude with a discussion of using concept mappingfor assessment and present additional thoughts for future work.

BACKGROUNDGeography is “the art/science that deals with where things are located, why

they are located there, and what difference their location makes” (Gersmehl2008, 3). Unfortunately, many students are geographically illiterate (Grosvenor1985, 1995). Several efforts to reverse this state have been met with varyingdegrees of success; these efforts would include the work of the GeographyEducation National Implementation Project (a consortium including the NationalCouncil for Geographic Education, the Association of American Geographers,the National Geographic Society, and the American Geographical Society) amongothers. While the development of national geography standards (GeographyEducation Standards Project 1994) were a positive part of this project, local andstate implementation of the standards is voluntary and often perfunctory.

In Florida, Department of Education (FLDOE) staff established detailedbenchmarks for geography in 1998, and updated them as Next Generation SunshineState Standards in 2008 (FLDOE 2008). Geography instruction and curriculumdevelopment in Florida is now linked in grades K–12 using a database of 139

Journal of Geography 111: 83–92C©2012 National Council for Geographic Education 83

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4

Wehry et al.

benchmarks. At the local level, course offerings includeFlorida Studies in the fourth grade and a geography coursein the seventh grade. Over the past decade, the devel-opment and implementation of APHG has breathed newlife into teaching geography for many students, teachers,schools, and school districts. Florida students, like othersacross the country, are able to take the course as an elective.

CROPCROP design and implementation began in the 2007–

2008 academic year and, by the third year of imple-mentation, researchers completed the design work andimplemented CROP across all three middle school grades.CROP participants attended two low-performing middleschools in the urban core of a large Florida city. Acombination of several factors including below grade-level achievement on either or both of the state readingand mathematics accountability measures, poverty-status(eligibility for the free or reduced-priced lunch), and thepotential for being a member of the first generation in theirfamily to complete college determines whether studentsare eligible to participate in CROP. During the 2009–2010academic year seventy-four, seventy, and fifty-eight CROPstudents participated in the sixth, seventh, and eighthgrades, respectively.

More often than not, CROP students have social oracademic barriers or both to overcome and as a resultare reluctant learners—often lacking the motivation tosustain sufficient effort necessary for academic success.Therefore, afterschool weekly learning sessions focusedon building students’ human geography knowledge andimproving their self-esteem. Learning excursions and afive-day nonresidential summer ecological camp extendedthe afterschool curriculum, which included technology-richactivities and college-student mentors. The CROP initiativeincorporated academic and social support for studentsand occurred weekly in three-hour sessions led by middleschool teachers and undergraduate college students. Thepurpose of CROP was to build a community of studentswho see themselves as academically able, emotionallyready, and active in their pursuit of positive futures inmiddle school, high school, and postsecondary education.

Weekly learning sessions included a teacher-led ninety-minute lesson that provided the human geography contentnecessary to complete the day’s activities (small-group andindividual work) facilitated by the college students. In thelearning sessions, teachers used concept mapping as aninstructional strategy and students used concept mappingas graphic organizers. Concept mapping provided aninnovative strategy supportive of CROP goals becauseteachers rarely used concept mapping strategies during thetraditional school day.

Concept MappingNovak characterized learning as an active process

that involves acquiring, creating, and using knowledge

(e.g., Novak and Gowin 1984; Novak 1998). Furthermore,the dual-coding theory of knowledge storage positsthat humans store knowledge in two ways—linguisticand nonlinguistic (imagery) forms (Paivio 1990; Sadoskiand Paivio 2001). Marzano, Pickering, and Pollock(2001) found that students who used both ways to storeinformation were better able to recall and apply knowledge.

Concept maps are nonlinguistic, two-dimensional, hier-archical diagrams that result from systematically mappingthe relationships among concepts, and their use helpsindividuals visualize the structure of the mapped knowl-edge. Novak and Canas (2008) noted that on concept mapsdirectional lines with words articulating the relationshipbetween the concepts connect related concepts. Linkedconcepts form propositions and, when read, propositionsform meaningful statements. Cross-linked propositionslink concepts across different map segments. Novak andCanas contended that the two features of concept mapsthat indicate leaps of creative thinking are the hierarchicalstructure of the map and the use of cross-links (see Fig. 1for an example of a concept map).

Education researchers have long seen concept mappingas a powerful tool to promote meaningful learning (e.g.,Novak and Gowin 1984; Novak 1990, 1998; Kinchin andHay 2000; Novak and Canas 2008). In their 2003 report tothe Chief of Naval Education and Training, researchers atthe Institute of Human and Machine Cognition (IHMC)broadly summarized the uses of concept mapping ineducational settings as support for learning, assessmentof learning, and for the organization and presentationof knowledge. Support of learning applications includedschematic summaries of what students know, displaysof students’ prior knowledge, summaries of what hasbeen learned, and detection of misconceptions. Assessmentapplications of concept mapping included formative andsummative assessments and the documentation of changesin students’ conceptual knowledge. Teachers use conceptmaps as advance organizers to scaffold learning and presentknowledge for course and curriculum development. Teach-ers use classroom concept maps, constructed by teachers orother experts, to present a global overview at the beginningof a unit and also to scaffold learning throughout the unit(IHMC 2003).

Connect to Knowledge (Conectate al Conocimiento) is theprimary component of a Panamanian strategy, IntelligentPanama (Panama Inteligente), implemented in 2005 inPanamanian schools. Through Connect to Knowledge thegovernment plans to facilitate a change in school focus fromrote memorization of facts to a system in which childrendevelop the necessary skills to construct knowledge contex-tualized by the interconnectedness of ideas (United NationsDevelopment Program (UNDP) 2010). In an address atthe Third International Conference on Concept Mapping,Secretary Tarte (2006) described the professional learningthat teachers received to facilitate meaningful learningthrough the use of concept mapping—the main focus ofConnect to Knowledge. Tarte reported that the private

84

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4

Concept Mapping Strategies: Content, Tools and Assessment for Human Geography

Figure 1. The master concept map for the seventh grade landmarks lesson. Theconcept map also provides an example of the scoring system.

sector, the Panamanian government, and the GabrielLewis Galindo Foundation supported the initiative fromits inception. UNDP supported the operational structure.Other countries implemented similar educational strategiesinvolving a focus on meaningful learning and conceptmapping, but Panama is the first country to implementconcept mapping at the national level. Between 2005 and2008, 6,000 teachers in 734 schools received training inconcept mapping strategies (UNDP 2010).

The National Assessment Governing Board of the U.S.Department of Education (NAGB) in Framework for the2009 National Assessment of Educational Progress [NAEP],identified concept mapping as a format for constructedresponse items and as one of four possible item formats foruse in interactive computer tasks. NAGB members stronglyrecommended that at least one concept mapping item beincluded in the eighth and twelfth grade NAEP sciencemeasures (NAGB 2008).

Uses of concept mapping in geography classroomsinclude support for student learning and for assessment.For instance, as reported by Ahlberg and Ahoranta (2002),Rikkinen, in her Finnish geography methods text, offersconcept mapping as a method for teaching geographicalcontent. Witherick et al. (1998), in a study guide for A-levelgeography, suggested that concept mapping encouragesclarification of complex interconnected content and allowsteachers to monitor students’ understanding. The authorsprovided examples of concept-mapped geography content.Furthermore, Bednarz (2011), in her essay linking maps,spatial thinking skills, and APHG, stated that geographerstraditionally have used maps to display and communicate

information, but that over time thegeneral population has come toconsider the use of maps an im-portant skill. Skills essential for thetwenty-first century include spatialthinking, the skill underlying theconstruction and use of maps. Bed-narz further suggested that graph-ics such as concept maps facilitatethinking with space. The construc-tion of concept maps forces map-pers to move nonspatial informa-tion into spatial forms and theiruse requires interpretation of spa-tial representations. Both actionsfacilitate spatial thinking.

Trygestad (1997) investigatedhow students at the sixth, ninth,and twelfth grades conceptualizedgeography. She analyzed the rela-tionships between prior academicachievement, grade level, the iden-tification and use of geographyconcepts, and the complexity ofconstructed concept maps. Stu-dents sorted a list of twenty-five

salient geography concepts as familiar and unfamiliar andthen created concept maps using the familiar concepts.Study results indicated that the complexity of the conceptmaps increased with grade level and achievement. Ofparticular interest is Trygestad’s summarizing statement,in which she said,

Concept maps were considered valuableby most students. They felt concept mapswere visual models that organized andreplicated their thinking. As a tool forclarification, concept mapping was a help-ful metacognitive strategy. Through theirbrief encounter with concepts and conceptmaps, students responded positively andsuggested continued emphasis on them inthe classroom. Although students may havebeen challenged by the process of conceptidentification and use, they tended to findthe conceptual orientation helpful and mapconstruction valuable to understand geog-raphy. (Trygestad 1997, 15–16)

Ahlberg and Ahoranta (2002) reported the results of amultiple case study of concept mapping in a geographylearning project using ten- to twelve-year-old Finnish stu-dents. The students, three high-, three average-, and threelow-achievers, constructed concept maps at the beginningand end of the project. Their results included two types ofconcept map scores: one calculated by adding the numberof relevant concepts and the other by adding the number ofrelevant propositions. Almost all of the students obtained

85

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4

Wehry et al.

higher scores at the end of the project. Additionally,students’ maps began to show a hierarchy suggesting thatstudent learning was becoming increasingly connected andordered.

ASSESSMENT DECISIONS AND RESULTS

Concept Mapping and AssessmentNovak (1990) suggested four primary uses for concept

mapping: learning, instruction, planning, and assessmentof student learning. However, the primary focus of this arti-cle lies in using concept mapping for assessment of studentlearning. When developing concept mapping strategiesfor assessments, teachers and researchers make decisionsabout three assessment-related facets of concept mapping:the task for the respondent, the format of the response,and a scoring system that produces reliable and validresults. Concept mapping tasks that do not constrain theresponder have the highest validity for measuring studentknowledge, but at the same time take longer to constructand more cognitive effort to produce. Thus, when usingconcept mapping for assessment, developers must weighthe potential tradeoffs between efficiency (the time requiredto produce a concept map and the cognitive demandplaced on responders) and validity (the degree to whichthe assessment measures the intended construct) resultingfrom their decisions (Ruiz-Primo and Shavelson 1996).

Ruiz-Primo and Shavelson (1996) provided three waysthat concept mapping tasks draw out students’ knowledge:task demands, constraints, and structure. Task demandsfor students range from the heavy cognitive demand ofstudent-generated concept maps, the gold standard, to themore moderate demands of fill-in type concept maps. Taskconstraints involve restrictions imposed on the task. Forinstance, students may be supplied a list of concepts touse, asked to construct an hierarchical map, supplied a listof linking phrases, or provided a map structure to fill in.The structure of the task results from combining the taskdemands and the constraints. Researchers have proposedand studied numerous concept map scoring systems, withmost systems building on the work of Novak and Gowin(1984) in which propositions, cross-links, and the conceptmap’s hierarchical structure receive scores. However, at thispoint there is no universally recommended scoring system,and researchers continue to develop and study scoringsystems in efforts to quantify the structure of knowledgeand to automate the use of scoring systems.

The CROP investigations involved the use of conceptmapping for formative and summative assessments of thestudents’ geography knowledge. The formative assessmentinvolved the use of a master concept map, developed by theteacher/researcher, to scaffold the learning of the seventhgraders (Fig. 1). At the end of the lesson the students’ taskwas to generate concept maps by incorporating conceptsfrom the master concept map and adding the newly learnedconcepts. The design of the summative assessment wasto assess the human geography learning across all three

grades at the end of the academic year. We present thedecisions involved in designing the summative assessmentand results of a field test of the draft instrument.

Formative Assessment StudyThe formative assessment study occurred in spring

2010, following eighteen weeks of afterschool sessions. Aresearcher (familiar with students), teachers, and collegestudents led all study activities. The human geographycurriculum varied by grade and the following exampleoutlines the instructional processes used with seventhgraders. The lesson, a content summary of a previoussession about landmarks, included mostly fact-based,declarative knowledge. The teacher/researcher reviewedconcept mapping vocabulary and activated students’ priorgeography knowledge by reviewing the previous lesson.The teacher/researcher then modeled the creation of theconcept map shown in Figure 1; however, most studentshad prior experience with concept mapping.

The teacher/researcher provided students with chartpaper containing the focus question “What have I learnedabout landmarks?”, the sixteen concepts used on theseventh-grade master map (see Fig. 1), four new concepts(Mount Rushmore, Mount St. Helens, the Great SmokeyMountains, and the Statue of Liberty) from the landmarksunit, six blank Post-It Notes for additional concepts, twentyindex cards for linking phrases, a glue stick, and a pencil.Students worked independently on their concept mapsfor about forty minutes with college students monitoringthe activity and probing middle school students about theconnections on their maps.

The research team scored the student-generated maps(n = 26) using the scoring system developed by Wehry et al.(2008), which is an adaptation of a system developed by No-vak and Gowin (1984). The scoring system provides scoresfor the three concept map components and a compositeconcept map score that is the sum of the component scores.To address the complexity of the knowledge structuremapped, we differentially scored the quality of the propo-sitions (e.g., McClure and Bell 1990; Yin et al. 2005). In doingso, we increased the threshold for awarding cross-link andhierarchy level scores—the concept map components thatoften reflect creative thinking. Propositions received zeropoints if incorrect or irrelevant; one point if correct butsomewhat relevant; two points if correct and relevant;and three points if causative. Cross-linked propositionsreceived five points when they linked concepts that are partof a proposition awarded at least two points; otherwise,cross-linked propositions scored as any other proposition.Hierarchy level one, the focal concept, received no points.Hierarchy level two received five points when three or moreconcepts form quality propositions with the focal concept.(See the master concept map in Fig. 1, which was scoredusing the described scoring system.) In Fig. 1, propositionsscored one are correct but provide examples that score onepoint. Because the lesson involved declarative knowledge,many of the propositions are this type.

86

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4

Concept Mapping Strategies: Content, Tools and Assessment for Human Geography

Table 1. Comparing the attributes of the student-generated andmaster landmarks lesson concept maps.

Number Score

Con Prop Level New Cons Prop CL Level Total

Master 16.0 20.0 3.0 – 26.0 5.0 0.0 31.0Student 14.2 14.3 2.8 2.2 20.0 1.9 0.8 22.6

Note: Con is concept, Prop is propositions, and CL is cross-links.

The seventh graders’ average concept maps scores as wellas the average number of concepts, propositions, levels,and new concepts used on the students’ concept mapsallow a comparison to the master concept map (Table 1).The analyses indicated that participants were able to createconcept maps with a hierarchal structure and to organizepropositions in a connected way. Seventy-three percent ofthe participants included concepts not found on the mastermap. On average, students were able to organize contentusing a hierarchical order of at least two levels, and 62percent used three or more levels. The average cross-linkscore was 1.9 resulting from 23 percent of the concept mapsexhibiting at least one scored cross-link.

Summative Assessment StudyCROP students have had numerous experiences with

high-stakes, accountability assessments, and the studentshave not necessarily enjoyed positive outcomes. Further-more, CROP participation was voluntary, and studentsfrequently chose CROP participation over competitive,less academically focused afterschool programs. Thus itwas in everyone’s best interest that we evaluated theCROP students’ human geography knowledge using novelmethods that they saw as pleasant activities rather than asanother high-stakes test. We planned to use the developedinstrument across all three grades; therefore, we expecteddifferentiated outcomes. Our first step was to select a con-cept mapping format among numerous available formats.

Ruiz-Primo, Schultz et al. (2001) studied the validity ofthree types of concept maps frequently used for assessingscience achievement: construct-a-map with a concept listprovided and two types of fill-in concept maps. Thecreation of select-and-fill-in (SAFI) formats involves twosteps; the first is the creation of a master map by anexpert, often the teacher. Then, the expert omits someor all of the concepts or linking phrases. The student’stask is to fill in what is omitted from the concept mapby making appropriate selections from a list of conceptsor linking phrases. Both types of fill-in concept mapsstudied by Ruis-Primo, Schultz et al. began with expertconcept maps. In the first type, concept map developersremoved some of the concepts from the expert map andplaced them in a concept bank for students to use when

completing the fill-in task. In the second type, conceptmap developers removed some of the linking phrases andplaced them in a linking-phrase bank. The provision oftwenty concepts was all that constrained the construct-a-map task. Ruiz-Primo, Shavelson et al. (2001) used think-aloud techniques (activities in which respondents think outloud as they perform specified concept mapping tasks) togain insight into the cognitive activities that students usedto complete the concept mapping tasks. Researchers con-cluded that low-constrained construct-a-map tasks allowstudents more opportunity to use conceptual knowledgeto complete the task while more-constrained fill-in mapsencourage students to closely monitor the accuracy of theirresponses.

Ruiz-Primo, Shavelson et al. (2001, 125) also evaluatedconcept mapping tasks across proficiency groups. Theyreported that a higher percentage of low-performing stu-dents used monitoring activities than did high-performingstudents, and when low-performing students completedconstruct-a-map tasks, “The low-performing student readall the concepts, selected one with no apparent justificationfor doing so, and started drawing the map.”

Schau et al. (2001) provided detailed research on thedevelopment and use of fill-in concept map formats. Theydeveloped and evaluated twenty-two different fill-in-typemaps. In the final part of the study, they used a SAFI conceptmap format with up to 50 percent of the nonconsecutiveconcepts removed. Field testing resulted in a final versionwith 36 percent of the 105 concepts removed. Scores onthis SAFI concept map, reflecting the percent of correctresponses, resulted in a Cronbach’s alpha of .94.

After reviewing the work of Ruiz-Primo and her col-leagues (Ruiz-Primo and Shavelson 1996; Ruiz-Primo,Schultz et al. 2001; Ruiz-Primo, Shavelson et al. 2001) andSchau and her colleagues (Schau and Mattern 1997; Schauet al. 2001), we developed a SAFI concept map similar tothe one developed and tested by Schau et al. We agreedwith Schau and Mattern that the task in student-generatedconcept maps could potentially place a high cognitivedemand on low-achieving students. Ruiz-Primo, Shavelsonet al. also reported think aloud findings indicating that low-performing students did not develop strategies to completeconstruct-a-map tasks; rather they began the task seeminglyat random. We thought that the use of monitoring activitiesin performing fill-in concept mapping tasks was a positiveoutcome especially when the activities lead to effectivereflection, inspection, and development of task completionstrategies.

The design of the CROP curriculum resulted in thecreation of a table of curriculum specification for each grade.The curriculum specification tables included detailed infor-mation concerning what students should know and be ableto do after experiencing the curriculum. The first step in thecreation of the master concept map was to use the tables ofspecification to identify the concepts that students shouldknow at each grade. In looking at the identified concepts,we recognized that the curriculum was designed from the

87

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4

Wehry et al.

Table 2. SAFI concept map item means and Cronbach’s alpha for the total score and bygrade.

Grade All 6th 7th 8th

ItemsCompass Rose 6 .88 .96 .77 .86

Outside Influences 6 .28 .22 .31 .43Behaviors 6 .49 .52 .31 .71Family Traditions 6 .30 .22 .46 .29Rivers 7 .86 .87 .85 .86Beaches 7 .84 .78 .92 .86Flora 7 .93 1.00 .77 1.00Fauna 7 .91 .91 .85 1.00St. Augustine 7 .61 .65 .39 .86Mayors 7 .67 .65 .77 .57Revenue 7 .67 .65 .62 .86Congressional Representative 7 .35 .30 .46 .29Migrant Workers 7 .63 .70 .46 .71Landmarks 7 & 8 .61 .65 .46 .71Flags 8 .65 .74 .46 .71International Peace 8 .58 .57 .54 .71New York City 8 .63 .65 .46 .86Underdeveloped Country 8 .49 .39 .54 .71Death Rate 8 .93 .96 .85 1.00GDP 8 .47 .39 .31 1.00

Total Score All 63.84 63.91 57.69 75.00Coefficient Alpha .84 .36∗ .83† .73‡

Note: Bolded values represent very easy concepts (mean > .90) and italicized underlined valuesrepresent difficult concepts (mean < 30). ∗ indicates that only omitted concepts specified for the sixth-grade curriculum (4) were used in calculations; † indicates that only omitted concept. specified for theseventh-grade curriculum (10) were used in calculations; and ‡ indicates that only omitted conceptsspecified for the eighth-grade curriculum (7) were used in calculations.

bottom-up starting at the sixth grade with self, family, andneighborhood. By the eighth grade, the curriculum hadexpanded to the more global concepts of nation states. Theidentified concepts led to internal discussions of the bigideas of human geography—the big ideas that form thefirst levels of a top-down hierarchical map and that weremissing from the list of concepts.

A university research team (comprised of educationalresearchers and a colleague from the Department ofEconomics and Geography) worked together to create theexpert map. The first step in constructing the expert mapwas the incorporation of the big ideas (places, culture,economic production, and change) using terms that theCROP students would understand. The draft of an expertSAFI geography map contained seventy-two concepts and,in keeping with the concept map developed by Schau et al.(2001), consecutive concepts were not removed. Most of theomitted concepts were lower in the concept hierarchy, thuslocated at the bottom of the map. Figure 2 is the field-testedversion of the SAFI map.

During the first week of May 2010 forty-three CROPstudents agreed to field test the SAFI concept map bycompleting the SAFI map as part of their routine afterschool

activities. The sample of twenty-nine girls and fourteen boys in-cluded twenty-three, thirteen, andseven sixth, seventh and eighthgraders, respectively. We analyzedstudent responses, coded 1 for cor-rect responses and 0 for incorrectresponses to determine item (omit-ted concept) difficulty, item differ-entiation by grade, and Cronbach’salpha (Table 2).

Across all grades, the SAFI mapexhibited an acceptable coefficientof internal consistency, .84, andonly one concept (outside influencesfrom the sixth grade) was difficultwith an estimated mean of .28.(Because the mean of dichotomousitems represents the proportion ofcorrect responses it simultaneouslyrepresents item difficulty.) Cron-bach’s alpha by the grade level ofthe items ranged from an unac-ceptable .36 for the sixth grade toan acceptable high of .83 for theseventh grade.

More than 90 percent of the stu-dents correctly used three concepts(two from the seventh-grade cur-riculum and one from the eighth-grade curriculum). Analysis ofSAFI items by grade indicatedthat the sixth graders struggledwith concepts from the sixth-grade

curriculum while they found concepts from the seventh-and eighth-grade curricula only moderately difficult. Theseventh graders somewhat struggled with the sixth-gradeconcepts but found most of the seventh-grade conceptsmoderately difficult. All eighth graders correctly used fourconcepts, two from the seventh-grade curriculum and twofrom the eighth-grade curriculum.

DISCUSSION AND CONCLUSIONThe growth of the program, in part, documents the

success of CROP. Eighty-seven eighth graders participatedduring the 2010–2011 academic year in contrast to fifty-eight during the 2009–2010 academic year. Additionally,of the twenty-two CROP students enrolled in district highschools in fall 2010, fifteen attended high schools offeringAPHG. Among these students, nine enrolled in APHG.

Our discussion centers on what we learned from thestudents’ concept maps and how that knowledge caninform program improvement. In both studies, conceptmapping provided insights into curricular and instruc-tional deficits and gaps in student learning. The conceptmaps developed by the seventh graders, on average,were not as complex as we expected. However, some

88

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4

Concept Mapping Strategies: Content, Tools and Assessment for Human Geography

Figure2.

SAFIhum

angeography

conceptmap

andm

issingconceptbank.

89

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4

Wehry et al.

student-constructed maps were more complex than theexpert map. Two possible explanations provide insight intothe students’ maps and our instructional procedures. Inprevious learning sessions, the students generated con-cept maps using a computer-based program, CmapTools(available for download at http://cmap.ihmc.us/), whichautomatically provides the linking structure of the conceptmap as new concepts are added and linked. (Conceptmaps displayed in Figs. 1 and 2 were generated usingCmapTools.) Student use of pencil, paper, and glue ratherthan CmapTools in the formative assessment may haveconstrained map generation—some students ran out ofpaper space to complete their maps, some forgot to usedirectional arrows on the connecting lines, some forgot touse linking phrases on the directional lines, and others didany combination of the listed difficulties. If students hadused CmapTools, it may have been more difficult for themto overlook these concept map features.

Additionally, the CROP nontraditional, academicenvironment—a less-constrained environment that oftenpromotes creativity—may, in part, explain the lower-than-expected student concept map scores. Many students, inconstructing their maps, did not use the master map asa guide in placing concepts on their maps. Instead, theyfreely mapped the concepts relative to their experiencesand understanding. This outcome was also unexpected aswe thought the students would simply add new concepts tothe existing structure. However, while the students’ conceptmap scores were lower because of their approach to thetask, their concept maps provided information about theirconceptual understanding of the landmarks lesson.

Grade-specific analysis of the students’ responses to theSAFI concept map field test (Table 2) point to the needto rethink the selection of which sixth-grade concepts toomit from the map and place in the concept bank. Of thefour concepts selected for omission, two were difficult forthe sixth graders; furthermore, the four missing conceptsrepresented some of the most difficult items across allgrades. In contrast, the omitted seventh-grade conceptspresented little difficulty for the sixth graders. Thus, theresulting SAFI scores did not differentiate between the sixthand seventh graders as expected.

Because we have evidence indicating growth in CROPparticipation and our first group of participants enteringhigh school enrolled in APHG, we want to carefullyrevise some aspects of CROP without negatively impactingaspects contributing to program success. In general, thesixth-grade concepts are the most abstract of the SAFI con-cepts. However, the sixth-grade curriculum was specificallydesigned to help students understand that they control theirown path to academic success. To achieve the CROP goalof building a community of students who see themselvesas academically able, emotionally ready, and active intheir pursuit of positive futures in middle school, highschool, and postsecondary education, students must firstsee themselves in charge of their futures. To that end, theCROP geography curriculum begins in the sixth grade at

the small, local end of the scale continuum with self, family,friends, and community.

In the future, the researchers will revise the existing sixth-grade curriculum to more explicitly connect the conceptssurrounding self-concept to the foundational concepts ofhuman geography. Additionally, teachers could reinforcesixth-grade students’ learning by using a concept mappingformative assessment lesson similar to that described forthe seventh-grade implementation. SAFI results indicatedthat the seventh grade concepts were among the easiestconcepts across the three grades. This finding suggeststhat we revisit the seventh grade curriculum to providemore focus on application of the content in ways thatbuild connections to the foundational concepts of humangeography. In fact, during summer 2011, guided in partby these assessment results, we revised the seventh gradecurriculum by incorporating a wider variety of technologyand by strengthening the alignment of the curriculum withhuman-environment interaction, region, and movement.This new curriculum draws more from human geographyrather than physical geography, which is offered to seventh-grade students during the school day.

In summary, our investigation indicated that teachers andresearchers can use concept mapping for formative andsummative assessment in an urban, afterschool program.The students willingly completed the student-generatedand SAFI concept maps. Their concept maps providedinsight into the structure of their geography knowledge andhelped us identify strengths and weaknesses of the CROPhuman geography curriculum.

ACKNOWLEDGMENTSThe PreCollegiate Connections: College Reach-Out Pro-

gram is made possible by funding provided by the Uni-versity of North Florida, Division of Academic Affairs; theFlorida Institute of Education at the University of NorthFlorida; and the Florida Department of Education Bureauof School Improvement.

NOTE1. The Advanced Placement (AP) program is a widely

recognized program of the College Board. AdvancedPlacement Human Geography (APHG) is one of thecourses offered to high school students through theAP program and, through participation, studentshave the opportunity to earn college credit.

REFERENCESAhlberg, M., and V. Ahoranta. 2002. Two improved educa-

tional theory based tools to monitor and promote qual-ity of geographical education in learning. InternationalResearch in Geographical and Environmental Education 11(2): 119–137.

90

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4

Concept Mapping Strategies: Content, Tools and Assessment for Human Geography

Bednarz, S. W. 2011. Maps and Spatial Thinking Skills inthe AP Human Geography Classroom. New York, NewYork: College Board AP Central. http://apcentral.collegeboard.com/apc/public/courses/teachers corner/151317.html?type=print (accessed April 2011).

FLDOE. 2008. Next Generation Sunshine State Standards.Tallahassee, Florida: Florida Department of Edu-cation. http://www.floridastandards.org/Standards/FLStandardSearch.aspx (accessed April 2011).

Geography Education Standards Project. 1994. Geographyfor Life: National Geography Standards. Washington, D.C.:National Geographic Society.

Gersmehl, P. 2008. Teaching Geography. 2nd ed. New York:Guilford Press.

Grosvenor, G. M. 1995. In sight of the tunnel: The renais-sance of geography education. Annals of the Associationof American Geographers 85 (3): 409–420.

———. 1985. President’s column. National Geographic Mag-azine 167 (6).

IHMC. 2003. A Summary of Literature Pertaining to theUse of Concept Mapping Techniques and Technologies forEducation and Performance Support. Pensacola, Florida:Institute of Human and Machine Cognition. http://www.ihmc.us/users/acanas/Publications/ConceptMapLitReview/IHMC%20Literature%20Review%20on%20Concept%20Mapping.pdf (accessed April 2011).

Kinchin, I. M., and D. B. Hay. 2000. How a qualitativeapproach to concept map analysis can be used toaid learning by illustrating patterns of conceptualdevelopment. Educational Research 42 (1): 43–57.

Marzano, R. J., D. J. Pickering, and J. E. Pollock. 2001.Classroom Instruction That Works: Research-Based Strate-gies for Increasing Student Achievement. Alexandria,Virginia: Association for Supervision and CurriculumDevelopment.

McClure, J. R., and P. E. Bell. 1990. Effects of an EnvironmentalEducation-Related STS Approach Instruction on CognitiveStructures of Preservice Science Teachers. ERIC DocumentReproduction Service No. ED 341 582. http://eric.ed.gov/ERICWebPortal/search/detailmini.jsp? nfpb=true& &ERICExtSearch-SearchValue 0=ED341582&ERICEstSearch SearchType 0=no&accno=ED341582(accessed April 2011).

NAGB. 2008. Science Framework for the 2009 NationalAssessment of Educational Progress. Washington, D.C.:National Assessment Governing Board. http://www.nagb.org/publications/frameworks/science-09.pdf(accessed April 2011).

Novak, J. D. 1998. Learning, Creating, and Using Knowledge:Concept Maps as Facilitative Tools in Schools and Corpora-tions. Mahwah, New Jersey: Erlbaum.

———. 1990. Concept mapping: A useful tool for scienceeducation. Journal of Research in Science Teaching 27 (10):937–949.

Novak, J. D., and A. J. Canas. 2008. The TheoryUnderlying Concept Maps and How to Construct andUse Them. Technical Report IHMC CmapTools 2006-01 Rev 01-2008. Pensacola, Florida: Institute ofHuman and Machine Cognition. http://cmap.ihmc.us/Publications/ResearchPapers/TheoryUnderlyingConceptMaps.pdf (accessed April 2011).

Novak, J. D., and D. B. Gowin. 1984. Learning How to Learn.New York: Cambridge University Press.

Paivio, A. 1990. Mental Representations: A Dual CodingApproach. New York: Oxford University Press.

Ruiz-Primo, M. A., S. E. Schultz, M. Li, and R. J. Shavelson.2001. Comparison of the reliability and validity ofscores from two concept-mapping techniques. Journalof Research in Science Teaching 38 (2): 260–378.

Ruiz-Primo, M. A., and R. J. Shavelson. 1996. Problems andissues in the use of concept maps in science assessment.Journal of Research in Science Teaching 33 (6): 569–600.

Ruiz-Primo, M. A., R. J. Shavelson, M. Li, and S. E. Schultz.2001. On the validity of cognitive interpretations ofscores from alternative concept-mapping techniques.Educational Assessment 7 (2): 99–141.

Sadoski, M., and A. Paivio. 2001. Imagery and Text: A DualCoding Theory of Reading and Writing. Mahwah, NewJersey: Erlbaum.

Schau, C., and N. Mattern. 1997. Use of map techniquesin teaching applied statistic courses. The AmericanStatistician 51 (2): 171–175.

Schau, C., N. Mattern, M. Zeilik, W. Teague, and R. J.Weber. 2001. Select-and fill-in concept maps scores asa measure of students’ connected understanding ofscience. Educational and Psychological Measurement 61(1): 136–154.

Tarte, G. 2006. Conectate al conocimiento. Una estrategianacional de Panama basada en mapas conceptuates.In Concept Maps: Theory, Methodology, Technology. Pro-ceedings of the Second International Conference on ConceptMapping, ed. A. J. Canas and J. D. Novak, pp. 1:144–161.San Jose, Costa Rica: Universidad de Costa Rica.

Trygestad, J. 1997. Students’ Conceptual Thinking inGeography. Paper presented at the annual meeting of theAmerican Educational Research Association (Chicago,Illinois). ERIC Document Reproduction Service No. ED422 231. http://eric.ed.gov/ERICWebPortal/search/simpleSearch.jsp?newSearch=true&eric sortField=&searchtype=keyword&pageSize=10&ERICExtSearchSearchValue 0=trygestad&eric displayStartCount=I&pageLabel=ERICSearchResult&ERICExtSearch SearchType 0=au (accessed April 2011).

91

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4

Wehry et al.

UNDP. 2010. UNDP Success Story Leads. New York,New York: United Nations Development Program.http://www.undp.org/comtoolkit/success-stories/LAC-Panama-povred.shtml (accessed April 2011).

Wehry, S., J. Algina, J. Hunter, and H. Monroe-Ossi. 2008.Using concept maps transcribed from interviews toquantify the structure of preschool children’s knowl-edge about plants. In Concept Maps: Connecting Educa-tors. Proceedings of the Third International Conference onConcept Mapping, ed. A. J. Canas, P. Reiske, M. Ahlberg,and J. D. Novak, pp. 2:732–739. Tallinn, Estonia andHelsinki, Finland: University of Finland.

Witherick, M., N. Law, D. Elcome, G. Nagle, M.Thomas, and S. Warn. 1998. Study Advice for Ge-ography A-Level. Cheltenham, UK: Stanley ThornesLtd.

Yin, Y., J. Vanides, M. A. Ruiz-Primo, C. C. Ayala, and R. JShavelson. 2005. Comparison of two concept-mappingtechniques: Implications for scoring, interpretation,and use. Journal of Research in Science Teaching 42 (2):166–184.

92

Dow

nloa

ded

by [

"Uni

vers

ity a

t Buf

falo

Lib

rari

es"]

at 1

3:53

08

Oct

ober

201

4