Journal of Educational Psychology1992, Vol. 84. No. 1,76-84

Copyright 1992 by the American Psychological Association. Inc.0022-0663/92/$},0<>

Comprehension of Arithmetic Word Problems:Evidence From Students' Eye FixationsMary Hegarty, Richard E. Mayer, and Carolyn E. Green

University of California, Santa Barbara

Students have difficulty solving arithmetic word problems containing a relational term that isinconsistent with the required arithmetic operation (e.g., containing the term less, yet requiringaddition) rather than consistent. To investigate this consistency effect, students' eye fixationswere recorded as they read arithmetic word problems on a computer monitor and stated asolution plan for each problem. As predicted, low-accuracy students made more reversal errorson inconsistent than consistent problems, students took more time for inconsistent than consis-tent problems, this additional time was localized in the integration/planning stages of problemsolving rather than in the initial reading of the problem, these response-time patterns wereobtained for high-accuracy but not for low-accuracy students, and high-accuracy students requiredmore rereadings of previously fixated words for inconsistent than for consistent problems.

Arithmetic word problems, such as those presented in Table1, can be viewed as assays of students' problem-solving skillsin elementary mathematics. Unfortunately, students performparticularly poorly on arithmetic word problems even whenthey perform well on corresponding arithmetic computation(Cummins, Kintsch, Reusser, & Weimer, 1988; Dossey, Mul-lis, Linquist, & Chambers, 1988; Robitaille & Garden, 1989),suggesting that problem comprehension is a source of stu-dents' difficulties. In this study we attempted to identify thelocus of these comprehension difficulties more precisely bymonitoring students' eye fixations as they read and preparedto solve arithmetic word problems.

Problems such as those in Table 1 have been called compareproblems because they contain a relational statement com-paring the values of two variables (such as "Gas at Chevronis 5 cents more per gallon than gas at ARCO"). Students havemore difficulty with compare problems than with other typesof word problems (Mayer, 1982; Morales, Shute, & Pellegrino,1985; Riley & Greeno, 1988; Riley, Greeno, & Heller, 1983).In particular, students have difficulty solving compare prob-lems in which the relational term in the problem is inconsis-tent with the required arithmetic operation (e.g., the relationalterm is less, and the required operation is addition) than whenthe relational term is consistent with the required operation(Lewis, 1989; Lewis & Mayer, 1987). We refer to this findingas the consistency effect.

Whereas previous research on the consistency effect hasfocused on analyses of students' problem-solving errors, inthis study we also monitored students' solution times andtheir eye fixations as they read and planned solutions toproblems. The solution-time analysis allowed us to assesswhether inconsistent problems require more processing forstudents. The eye-fixation analysis allowed us to identify

This research was supported by a grant from the Academic Senateof the University of California, Santa Barbara.

We thank Ted Bicknell for his assistance with data coding.Correspondence concerning this article should be addressed to

Mary Hegarty or Richard E. Mayer, Department of Psychology,University of California, Santa Barbara, California 93106.

phases in the solution of arithmetic word problems and tofind the locus of students' difficulties with inconsistent prob-lems within these phases.

Consistency Effect

Students' errors in solving inconsistent compare problemscan be traced to difficulties in representing relational state-ments (Lewis & Mayer, 1987; Mayer, 1982). Understandinga relational statement involves converting a declarative state-ment about the relative values of two variables to a mathe-matical function in which a value of one variable is derivedby applying an operator to the other variable. The translationof a relational statement into a solution plan is particularlydifficult in inconsistent problems, such as the inconsistent/marked1 problem in Table 1. To solve this problem, a studentmust translate the relation in the second sentence into thefollowing equation: (cost of gas per gallon at Chevron) = (costof gas per gallon at ARCO) + $0.05.

There are several reasons why this translation might bemore difficult for inconsistent than for consistent problems,such as those shown in Table 1. First, the relational term isinconsistent with the operation to be performed in inconsis-tent problems (e.g., the relation less primes the subtractionoperation, whereas the correct operation is addition). Second,subjects have an expectation for the unknown term in therelational statement to be the grammatical subject of thesentence (Lewis & Mayer, 1987), but in inconsistent problemsit is the grammatical object of the sentence (see Huttenlocher& Strauss, 1968, and Clark, 1969, for a similar result withlogic problems). Third, in inconsistent problems, the rela-tional statement involves a pronominal reference to the sub-ject of the first sentence, necessitating the search for thereferent of the pronoun (Carpenter & Just, 1977; Corbett &Chang, 1983; Ehrlich & Rayner, 1983).

1 In the present study, negative relational terms such as less aredefined as marked, and positive relational terms such as more aredefined as unmarked (cf. Clark, 1969).

76

ARITHMETIC WORD PROBLEMS 77

Table 1Four Types of Arithmetic Word Problems

Relational term/line no. Consistent language Inconsistent language

Unmarked ("more")Line 1Line 2Line 3Line 4

Marked ("less")Line 1Line 2Line 3Line 4

At ARCO gas sells for $ 1.13 per gallon.Gas at Chevron is 5 cents more per gallon than gas at ARCO.If you want to buy 5 gallons of gas,how much will you pay at Chevron?

At ARCO gas sells for $ 1.13 per gallon.Gas at Chevron is 5 cents less per gallon than gas at ARCO.If you want to buy 5 gallons of gas,how much will you pay at Chevron?

At ARCO gas sells for $ 1.13 per gallon.This is 5 cents more per gallon than gas at Chevron.If you want to buy 5 gallons of gas,how much will you pay at Chevron?

At ARCO gas sells for $ 1.13 per gallon.This is 5 cents less per gallon than gas at Chevron.If you want to buy 5 gallons of gas,how much will you pay at Chevron?

The most common type of error observed in students'solutions of inconsistent problems is substituting the wrongoperation into the solution equation (i.e., using the operationthat is primed by the relational term in the problem). Toaccount for these errors, which we call reversal errors, Lewisand Mayer (1987) proposed a model in which (a) the problemsolver expects the grammatical subject of the relational sen-tence to be the unknown, (b) extra processing in building amental representation is required when this expectation isviolated, and (c) this extra processing increases the probabilityof error because of working memory demands. A potentiallymore detailed account of this extra processing is the following:Successful students build a problem model, which representsthe situation described in the problem (Greeno, 1989; Kintsch& Greeno, 1985; Paige & Simon, 1966). Consistent with thisaccount, Lewis (1989) found that training less successfulstudents to first construct a qualitative representation of therelations in the problem greatly reduced their errors on incon-sistent problems.

In this study, we examine the hypothesis that constructionof an accurate problem model during reading of the problemstatement requires more processing for inconsistent problemsthan for consistent problems. Furthermore, we examine thehypothesis that only successful problem solvers will be sensi-tive to the inconsistency in problem statements and thusperform this extra processing as they read.

Phases of Problem Solving

Mayer and his colleagues (Mayer, 1985; Mayer, Larkin, &Kadane, 1984) have identified four phases in mathematicalproblem solving: problem translation, problem integration,solution planning, and solution execution. During the trans-lation phase, the problem solver constructs an individualmental representation for each sentence of the problem. Dur-ing the integration phase, the problem solver integrates theinformation across sentences. During the planning phase, theproblem solver develops a plan for solving the problem.Finally, during the execution phase, the problem solver carriesout the computations called for in the plan.

During which of these problem-solving phases does a stu-dent construct a problem model? Previous analyses of stu-dents' errors suggest that the locus of difficulty in inconsistentproblems is not the execution phase. Lewis and Mayer (1987)

and Lewis (1989) found that students make errors in solvinginconsistent problems because they misunderstand the prob-lem and develop an incorrect solution plan, not because theymake computational errors in carrying out this plan. In thisstudy, we investigate whether the difficulty occurs during thetranslation phase or during the later integration and planningstages.

Extensions of the Consistency Model

We attempted to extend previous findings on the consis-tency effect in three ways. The first goal of this study was todetermine whether the consistency effect would be replicatedin a problem-solving situation in which the task was to statea solution plan for a problem, rather than to work out thesolution to the problem. This task requires all problem-solvingphases except execution. We predicted an effect of consistencyin which subjects would produce more errors in planning thesolution of inconsistent than consistent problems.

The second goal of the study was to determine whetherconsistency has an effect on the solution times of studentswho are accurate in planning solutions. The consistencymodel (Lewis & Mayer, 1987) predicts such an effect becauseunderstanding the relational statement in an inconsistentproblem requires more processing than in a consistent prob-lem, thereby requiring more time. We predicted an effect ofconsistency on solution times, paralleling that obtained forerror rates, because we believe that the additional processinginvolved in comprehending inconsistent problems occurs be-fore the final execution phase.

The effect of consistency on solution time has been studiedpreviously by De Corte, Verschaffel, and Pauwels (1989a,1989b; Verschaffel, De Corte, & Pauwels, 1992). They founda consistency effect for children but not for adults when theypresented one-step compare problems in which the dependentvariable was overall time to state the needed arithmetic op-eration. The failure to obtain a consistency effect for adultsmay have been due to a ceiling effect in the use of relativelyeasy problems (i.e., one-step compare problems). Thus, in thepresent study, we examine whether there is an effect ofconsistency on response time with the use of more cognitivelydemanding (i.e., two-step) problems.

A related issue concerns whether subjects who are inaccu-rate in solving word problems show the same response-time

78 M. HEGARTY, R. MAYER, AND C. GREEN

pattern as subjects who are accurate. If the low-accuracysubjects lack a metacognitive sensitivity to the linguistic struc-ture of word problems, they should not take more time forinconsistent problems than for consistent problems.

The third goal of this study was to determine whether theadditional processing for inconsistent problems occurs duringthe initial reading of the problem (the translation phase) orduring later phases (integration and planning). We expectedthis analysis to provide important new information about thetime course of comprehension of arithmetic word problems(i.e., whether problem solvers start to construct the solutionplan as soon as they read the problem or whether the solutionplan is constructed at some later stage in the problem-solvingprocess). If the solution plan is constructed immediately, theninconsistency between a relational statement and its operationshould affect the time taken to initially read the problem. Ifthe solution plan is constructed after the initial reading, theninitial reading time should be equivalent for consistent andinconsistent problems.

This second possibility is supported by De Corte et al.'s(1989b) and Verschaffel et al.'s (1992) findings with childrenthat the consistency effect is stronger during the second phaseof problem solving (i.e., all time spent after reading theproblem once to the end of the problem-solving process) thanduring the first phase (i.e., all time spent in reading theproblem once). In the present study, we assess whether thesame is true of adult arithmetic problem solvers. Furthermore,a detailed analysis of students' eye movements in this studyprovides a preliminary account of the nature of the additionalprocessing performed on inconsistent problems.

A related issue concerns the nature of the additional proc-essing performed on inconsistent problems. We predict thatif students attempt to construct a model of the problem, thenthey will frequently reread lines of the problem, focusing onthe specific numbers, variable names, and relational terms. Ifsubjects have more difficulty in building a (nonquantitative)model of inconsistent problems than of consistent problems,we expect that they will reread the relevant variable namesand relational terms (but not the numbers) more often forinconsistent problems than for consistent problems.

Method

Subjects and Design

The subjects were 38 undergraduates who were recruited from thePsychology Department subject pool at the University of California,Santa Barbara. Because of difficulty in calibrating the eye-trackingequipment, data were available for only 32 subjects. All subjects werepresented with each of four problem types generated by a 2 x 2within-subject design, so all comparisons involving problem types arewithin-subject comparisons. The first factor was language consistency;in half the problems, the relational term (e.g., less than) was consistentwith the operation to be performed (e.g., subtraction), and in theother half of the problems, the relational term was inconsistent. Thesecond factor was whether the relational term was lexically marked(cf. Clark, 1969); half the relational terms were unmarked (e.g., morethan), and half were marked (e.g., less than). The problems werepresented in four different cover stories, as described by Lewis andMayer (1987), and each subject received one of four versions of the

problems, which differed in the pairing of the different problem typeswith the cover stories. Thus, test form was a between-subjects variable.

Materials and Apparatus

The materials consisted of four sets of 18 arithmetic word prob-lems, containing 14 filler and 4 target problems. The target problemswere presented in Positions 2, 4, 6, and 8, in a different counterbal-anced order in each problem set. The filler problems were simpleone-step and two-step arithmetic word problems. Four of the fillerproblems were comparison problems involving multiplication anddivision.2

Each target problem consisted of three sentences. The first sentencewas an assignment statement expressing the value of some variable(e.g., "At ARCO gas sells for $ 1.13 per gallon"). The second sentencewas a relational statement, expressing the value of a second variablein relation to the first variable (e.g., "Gas at Chevron is 5 cents moreper gallon than gas at ARCO"). This sentence varied across problemsin its consistency and lexical marking, yielding the four problemtypes, examples of which are given in Table 1. The third sentenceasked a question about the value of some quantity in terms of thesecond variable (e.g., "If you want to buy 5 gallons of gas, how muchwill you pay at Chevron?"). Answering the question always involvedmultiplication or division of the value of the second variable by aquantity given in the third sentence.

The stimuli were presented on a DEC VR 260 Monochrome VideoMonitor, situated approximately 3 ft (0.91 m) from the subject. Thesubject's eye fixations were monitored with an Iscan corneal-reflec-tance and pupil-center eye tracker (Model RK.-426) that sampled theposition of the subject's gaze every 16 ms and output the x and ycoordinates of this position to a DEC Vaxstation 3200. In addition,the position of the subject's gaze was instantaneously displayed on asecond video monitor (out of sight of the subject) by a pair of crosshairs (indicating the x and y coordinates) superimposed on thestimulus display that the subject was viewing. The display on thissecond video monitor was recorded on videotape with a standardvideo camera and VHS video cassette recorder. The video equipmentwas also used to record the subject's verbal statement of the solutionplan by means of a microphone, situated approximately 1 ft (0.30 m)from the subject.

Procedure

Each subject was randomly assigned to one of the four test versionsand tested individually. First, the experimenter presented written andverbal instructions. The subject was told that a word problem wouldappear on the computer screen, and the subject's task was to tell howhe or she would solve the problem but not to carry out any actualarithmetic operations. To illustrate these instructions, the subject wasgiven the following sample problem: "Joe has 3 marbles. Tom has 5more marbles than Joe. How many marbles does Tom have?" Sub-jects were told that an acceptable response was the following: "I would

2 The test included four multiplication/division problems involvingrelational expressions such as twice as many as or lA as many as. Wedid not include these problems in the analysis for the followingreasons: (a) The relational terms for multiplication/division problemsdo not correspond to more widely accepted comparison terms suchas less and more, furthermore, the concept oF markedness did notseem to apply to the relational terms in multiplication/divisionproblems; and (b) we wished to compare our results to those of DeCorte et al. (1989a, 1989b), who used only addition/subtractionproblems.

ARITHMETIC WORD PROBLEMS 79

add 5 and 3 to get the answer." Subjects then practiced giving asolution plan to another problem before the experiment commenced.

Following these instructions, the subject was seated in a dentist'schair facing the display screen and microphone. A headrest was fittedcomfortably to his or her head, and he or she was asked to move aslittle as possible during the experiment. The subject was asked tovisually fixate an asterisk that appeared in the top left corner of thescreen and to push a button to begin and end each trial. As soon asthe button was pressed, a word problem appeared on the displayscreen. The subject read the problem, stated how he or she wouldsolve it, and then pushed a button when he or she was finishedanswering. No time limitations were imposed. This procedure wasrepeated for each of the problems. When the set was completed bythe subject, he or she was debriefed and dismissed.

Low-Accuracy Subjects

Results and Discussion

Scoring

We partitioned the subjects into two groups: 11 subjectswho committed two or more errors on the four target prob-lems in the set (i.e., low-accuracy subjects) and 21 subjectswho committed either no errors or one error on the targetproblems (i.e., high-accuracy subjects). Each subject's eyefixations were aggregated into units called gazes, consistingof uninterrupted sequences of fixations on a word or numberin the text. These corresponded to a video recording of thesequence of the subject's eye fixations on the problem, whichalso provided an audio recording of the subject's solutionplan for each problem.

For each subject, the following scores were determined foreach of the four target problem types: (a) whether the subjectmade a reversal error in solving the problem; (b) overall time(i.e., the time from the appearance of a problem on themonitor to the completion of the answer); (c) translation time(i.e., time to initially read the problem, which was defined asthe time from the appearance of the problem to the end ofthe first series of eye fixations on the question at the last lineof the problem); and (d) integration and planning time (i.e.,the time from the end of first eye fixations on the question tothe completion of the oral statement of the solution plan).3

Error Analysis

The first level of analysis involved determining whether theerror patterns replicated those obtained by Lewis and Mayer(1987) and by Lewis (1989), namely, that students make morereversal errors for inconsistent problems than for consistentproblems, especially when the terms are marked. Figure 1shows the proportion of reversal errors on each of the fourproblem types for the low-accuracy subjects. As can be seen,students were more likely to state incorrect solution plans forinconsistent problems than for consistent problems, F(l, 10)= 7.74, p < .02, MSC = 0.42; however, there was no significanteffect of lexical markedness, F < 1, and no significant inter-action between consistency and markedness, F(l, 10) = 3.20,p > .10, MSe = 0.11. These F ratios should be interpreted inlight of the fact that each analysis of variance (ANOVA) effect

100 r

# 9 0

I 80

- 70

£ 60

^ 50o

S,40

§ 30

°* 20

10

I I Consistent

II Inconsistent

more lessthan than

Relational Term

Figure 1. Percentage of reversal errors made by low-accuracy sub-jects in solving consistent and inconsistent compare problems.

is associated with a 3-point scale (i.e., a difference of 0, 1, or2 errors). The majority of high-accuracy students committedno errors, so no analysis of their errors was conducted.

Despite the task differences between the present study (inwhich subjects orally stated the solution procedure withoutdoing any computations) and previous work (in which sub-jects worked out complete solutions including arithmeticcomputations), a similar consistency effect was obtained. Thissimilarity in error patterns suggests that the effects of consist-ency can be attributed to the problem-solving stages precedingthe execution phase (i.e., translation, integration, and plan-ning).

Total Time Analysis for High-Accuracy Students

The left portion of Figure 2 shows the mean overall solutiontime for each of the four problem types for the high-accuracysubjects and is comparable to response-time data reported byDe Corte et al. (1989a, 1989b) and Verschaffel et al. (1992).As predicted, inconsistent problems required significantlymore time than did consistent problems, F(\, 20) = 5.89, p< .05, MSe - 16.73; in addition, problems with marked termstook significantly longer than did those with unmarked terms,F(l, 20) = 5.09, p < .05, MSC = 32.01, and the interactionwas not statistically significant, F{ 1, 20) = 1.51, p > .20, MSe

= 44.36.The pattern of reaction times over the different problem

types thus mirrors the pattern of reversal errors observed in

3 The integration and planning phases could not be differentiatedin our data because students started verbalizing plans before they hadfinished reading the problem.

80

28

26

24

22

20

1 8

16

14

1 2

10

8

6

4

M. HEGARTY, R. MAYER, AND C. GREEN

High-Accuracy Subjects Low-Accuracy Subjects

\~\ Consistent

I I Inconsistent

Translation

Integrationand Planning

morethan

lessthan

Relational

morethan

Term

lessthan

Figure 2. Mean response time (in seconds) of high-accuracy and low-accuracy subjects for consistentand inconsistent compare problems. (The bottom portion of each bar represents integration and planningtime, and the top portion represents translation time.)

previous studies (Lewis, 1989; Lewis & Mayer, 1987), sup-porting the view that understanding inconsistent problemsinvolves additional processing steps. This additional process-ing can affect students' solutions in two ways. First, it canincrease the time required to solve inconsistent problemscorrectly. Second, there is some probability of error with eachadditional processing step, so that students are more likely tomake errors on inconsistent problems.

These results also extend those of De Corte et al. (1989a,1989b) and suggest why those researchers failed to find effectsof consistency on the solution times of adults (althoughchildren's solution times did show a consistency effect). Amain difference between De Corte et al.'s studies and ours isthat they used one-step problems and we used two-step prob-lems. De Corte et al.'s failure to obtain consistency effects inadults may be due to the ease with which adults can solveone-step problems. Using a similar methodology (i.e., focusingon the overall time to state a solution plan), but with two-step problems, we obtained a strong consistency effect foradults. More recently, Verschaffel, De Corte, and Pauwels(1992) also have found a consistency effect in adults whentwo-step problems are used. Apparently, when the cognitiveload is sufficiently high, high-accuracy adults require addi-tional time to solve problems with inconsistent relationalstatements.

Componential Time Analysis forHigh-Accuracy Students

The results described above demonstrate that the consis-tency effect occurs in a task that excludes the execution phase.In this section, we further examine which of the remainingphases of problem solving are the loci of the consistencyeffect: problem translation, on the one hand, or problemintegration and solution planning, on the other.

Figure 2 shows the mean time required for initial readingof the problem (translation time) and for rereading of theproblem (integration and planning time) for each type ofproblem. Problem solvers devoted equivalent amounts of timeto their initial reading of consistent and inconsistent prob-lems, F< 1, MSC = 7.79; in addition, there was no significanteffect for markedness, F< I, MSe = 6.03, and no significantinteraction, F < 1, MSC = 4.18. Thus, the translation phase(in which each sentence is individually encoded) is not af-fected by inconsistencies between sentences in the problem.

In contrast, the time required for integration and planningof inconsistent problems was greater than for consistent prob-lems, 7=1(1, 20) = 10.23,/>< .01, MSC = 12.59, and was greaterfor marked than for unmarked relational terms, F(l, 20) =8.99, p< .01, MSt = 23.79; however, the interaction was notsignificant, F(\, 20) = 1.78, p > .20, MSe = 38.64. Consistent

ARITHMETIC WORD PROBLEMS 81

with the model, additional time for integrating the inconsis-tent information occurs after initial reading of the problemhas been completed.

These results help to clarify some undeveloped aspects ofLewis and Mayer's (1987) consistency model. Apparentlymost problem solvers in our sample constructed the individualrepresentation of each sentence in the problem (i.e., thetranslation phase) before they attempted to integrate theinformation across sentences or to plan a solution. The pictureemerging from these results is that consistency affects theintegration and planning phases rather than the initial readingof the problem or the final execution of the plan.

Total Time and Componential Time Analyses forLow-Accuracy Students

The righthand portion of Figure 2 shows the average re-sponse time produced by the 11 low-accuracy subjects oneach of the four question types. These response-time datamust be interpreted in light of the finding that subjects oftenmade errors in solving the problems. If the low-accuracysubjects are insensitive to the linguistic structure of the prob-lem, we would not expect them to spend more time readinginconsistent than consistent problems. An analysis of thesedata revealed that the average response time for consistentlanguage problems did not differ significantly from the timerequired for inconsistent language problems, F < 1, MSe =45.35; in addition, these students did not devote significantlymore time to reading problems containing marked ratherthan unmarked relational terms, F < 1, MSe = 20.91, andthere was no significant interaction between language consis-tency and markedness, F < 1, MSC = 15.02. These resultscontrast with those of the high-accuracy subjects, shown inthe left portion of Figure 2, who devoted significantly moretime to problems involving inconsistent language and markedrelational terms.

In the righthand portion of Figure 2, average overall readingtime is divided into the average time that low-accuracy stu-dents spent translating and integrating/planning for each typeof problem. As can be seen, low-accuracy students appear todevote approximately the same amount of time to the trans-lation process regardless of problem type. In support of thisobservation, an analysis revealed no significant differences intranslation times due to language consistency, F < 1, MSe =9.90, markedness, F < 1, MSC = 5.16, or their interaction, F< 1, MSC = 9.06. This pattern is consistent with the resultsfor the high-accuracy students.

Our results for high-accuracy students indicated that themain impact of inconsistency (and markedness) occurs in theintegration and planning phases of problem solving: High-accuracy students incur longer integration/planning times forinconsistent problems than for consistent problems (and forproblems containing marked rather than unmarked relationalterms). In contrast to these results, low-accuracy studentsappear to devote approximately the same amount of time tothe integration and planning processes of each problem type.An analysis revealed no significant differences in integration/planning time for low-accuracy students due to languageconsistency, F < 1, MSt = 56.16, markedness, F< 1, MSC =

15.56, or the interaction of these two variables, F < 1, MSt =17.46.

We interpret these results as indicating that low-accuracysubjects either lack or fail to use appropriate metacognitivestrategies while reading arithmetic word problems. In partic-ular, they fail to recognize the need to devote additionalprocessing to problems that conflict with typical problemstructures, such as problems containing a language inconsis-tency. Low-accuracy students, on the average, do not showevidence of adjusting their cognitive processing for differenttypes of problems. Instead, they devote approximately thesame amount of time to processing of potentially difficultproblems as they do to processing potentially easier problems.In contrast, high-accuracy subjects demonstrate a metacog-nitive awareness of the potential difficulty of problems bydevoting considerably more time to the integration/planningprocessing of difficult problems than to the processing of easyproblems.

A Closer Look at How High-Accuracy StudentsProcess an Arithmetic Word Problem

The eye movement recordings provide a trace of the loca-tion, duration, and sequence of subjects' gazes as they readeach problem. To mine this rich data base, we developed aneye-fixation protocol for each of 15 high-accuracy subjects'performance on the ARCO problem (see Table 1). (Theprotocol data from other high-accuracy subjects were notavailable.) The protocol listed each line at which the subjectgazed, along with each word fixated on that line; when asubject's eyes moved to a different line, we added that line tothe protocol, along with the words fixated on that line, andso forth. We refer to each movement to a line that has alreadybeen fixated as a rereading.

Although a detailed analysis of each subject's comprehen-sion process is beyond the scope of this study, we analyzedsubjects' eye fixations on the ARCO problem to providepreliminary evidence that must be accounted for by a modelof how students process arithmetic word problems. On theARCO problem, our subjects averaged 13.20 rereadings (SD= 4.26); in this section we focus on three preliminary patternsconcerning these rereadings, which we call the funnel effect,the selection effect, and the consistency effect.

The funnel effect refers to the observation that studentsfocus on progressively smaller proportions of the words on aline with successive rereadings. For example, whereas studentsfixated almost all the words when they first read a line (M =8.77, SD = 1.11), they fixated significantly fewer words whenthey reread a line (M = 3.91, SD = 0.87), ?(14) = 11.44, p <.001. Approximately 39% of the rereadings involved lookingat one word, and 26% of the rereadings involved looking attwo words. When we partitioned each subject's rereadingsinto the first and second halves of the protocol, we found thatstudents looked at approximately one fewer word per reread-ing (M = 0.99, SD = 1.15) during the second half than duringthe first half, r(14) = 2.14, p = .05.

The selection effect refers to the observation that whenstudents reread a line, they focused more on numbers thanon other information in the problem. Figure 3 shows the

82 M. HEGARTY, R. MAYER, AND C. GREEN

.5

At

1.4

.6

ARCO1.7

.6

gas

!.l

1.2

sells1.3

1.6

for

2.4

4.0

$1.132.9

.9

] P^.9

.9

gallon..7

2.4Gas at ChevronThis1.0 1.6 2.9 2.6

1.6 3.5 3.4

is 5

1.5

(more/less)

2.6

.9

per

.9

.9

gallon

.7

.3

than

1.7

.6

gas

1.1

.5 .5ARCO.Chevron.

1.4 1.3

c

I

1.0

how

1.1

1.0

much1.1

1.0

will1.1

.6

you

.9

1.1

pay

1.6

.5

at

1.4

.5

Chevron?1.3

Figure 3. Mean number of rereadings of each word or quantity in consistent (C) and inconsistent (I)versions of the ARCO problem (see Table 1). (Numbers in boldface type indicate that words were rereadsignificantly more times in inconsistent than consistent problems, p < .05; items enclosed in rectangleswere reread significantly more than once, p < .05.)

average number of times a word was reread for consistent andinconsistent versions of the ARCO problem. The three num-bers (along with their companion words) in the problem werereread more than three times each on average, whereas mostof the other information was reread an average of one time,/(14) = 5.60, p<, 001.

Figure 3 also shows that problem consistency affected stu-dents' eye fixations, exemplifying the consistency effect. Stu-dents reread the numbers approximately the same number oftimes in consistent and inconsistent problems, \t\ < 1, butthey reread the relevant background information (such as thevariable names and relational term) more in inconsistentproblems than in consistent problems, t(l3) = 2.78, p < .02.A possible interpretation of these results, which warrantsfurther testing, is that subjects mainly read the words of theproblem when they are still constructing the situation modelof the problem and mainly fixate numbers when they areplanning their solution. Thus, the results suggest that buildinga situation model of a problem requires more processing for

inconsistent problems than for consistent problems but fillingin the quantitative information needed for a solution plan isnot affected by the problem's linguistic structure.

To illustrate these points further, we present the protocolof a high-accuracy student solving a consistent version of theARCO problem in Table 2 and the protocol of a high-accuracystudent student solving an inconsistent version in Table 3. Inthe protocol presented in Table 2, the student began byreading all of the words in each of the four lines of theproblem in order. We propose that these first four lines of theprotocols correspond to the translation process in which thestudent converts each sentence into a separate internal rep-resentation. In solving this problem, the student reread 11times; most of these rereadings involved looking at numberssuch as $ 1.13, 5 cents, or 5 gallons; this provides examples ofthe funnel and selection effects. We propose that these reread-ings correspond to the integration and planning processes inwhich the student builds connections among the pieces ofinformation in the problem.

Table 2Protocol of Student Solving a Consistent Problem

Line no. PhraseAt ARCO gas sells for $ 1.13 per gallonGas at Chevron is 5 cents more per gallon than gas at ARCOIf you want to buy 5 gallons of gasHow much will you pay at Chevron5 gallonsGas at Chevron is 5 cents more$1.13 per gallon5 gallons$1.13Gas at Chevron is 5 cents more$1.135 cents more$1.135 cents5 gallons of gas

Note. Line numbers refer to the line of the problem in which the phrase was presented (see Table 1).

ARITHMETIC WORD PROBLEMS 83

Table 3Protocol of Student Solving an Inconsistent Problem

Line no. Phrase

123412342323121343421

At ARCO gas sells for $ 1.13 per gallonThis is 5 cents more per gallon than gas at ChevronIf you want to buy 5 gallons of gashow much will you pay at ChevronAt ARCO gas sells for $ 1.13 per gallonThis is 5 cents more per gallon than gas at ChevronIf you want to buy 5 gallons of gashow much will you paythan gas at Chevronof gasgas at Chevronwant to buy 5 gallonsgas sellsthan gasfor$1.135 gallonsyou pay at Chevron5 gallonshow much will you pay at Chevron5 cents moreAt ARCO

Note. Line numbers refer to the line of the problem in which the phrase was presented (see Table 1).

In contrast, in the protocol presented in Table 3, the studentcompletely read the first four lines of the problem and thenwent back to reread each line. Even after rereading the entireproblem, the student continued with 13 more rereadings thatfocus mostly on words rather than numbers in the problem.For example, after rereading the entire problem, the studentreread "than gas at Chevron" in Line 2 and moved on to "ofgas" in Line 3 but had to interrupt this reading of Line 3 byreturning to "gas at Chevron" in Line 2. Then the studentjumped to the beginning of Line 1 but had to interruptcompletion of this line by jumping again to Line 2. Then, thestudent moved on to the question, rereading the number "5gallons" in Line 3 and the unknown variable "Chevron" inLine 4 and then rereading each of these. Finally, the subjectagain checked the relational term in Line 2 and the pronounreferent (ARCO) in Line 1.

The protocols provide examples of the funnel effect: Afterinitially reading each line, students tend to reread only smallportions of each line. The protocols also exemplify the selec-tion effect: Students tend to focus on rereading the numbersin the problem ($1.13,5 cents, 5 gallons). Finally, a compar-ison of Tables 2 and 3 provides an example of the consistencyeffect, showing that the rereadings for the inconsistent prob-lem contain more words than those for the consistent prob-lem. We interpret rereading of words to indicate that thestudent is still constructing a situation model of the problem.Thus, the view that emerges from this evaluation of eyemovements is one of a reader who does not necessarily builda final mental representation of the problem situation fromsystematically reading each line of the problem once. Instead,the reader seems to be continually struggling to integrate theinformation across sentences, to build a progressively moreaccurate model of the problem situation.

Conclusion

In summary, the present study provides three importantextensions of earlier work on the consistency effect. First, itshows that the consistency effect occurs in a situation thatexcludes arithmetic computation, suggesting that the locus ofthe effect exists outside of the execution phase of problemsolving. Second, the study shows that the consistency effectoccurs for high-accuracy students when the dependent mea-sure is overall solution time, suggesting that additional proc-essing predicted by the consistency model actually requiresadditional time and that even for good problem solvers (i.e.,those making one or no errors), inconsistency causes diffi-culty. Third, this study shows that the consistency effect iscaused by additional processing in the integration and plan-ning phases of problem solving but not in the translation orexecution phases, suggesting that the locus of the consistencyeffect lies in additional processing for integrating across sen-tences to build a problem model and solution plan.

These results extend previous results (Lewis, 1989; Lewis &Mayer, 1987) to a new situation (namely, stating a solutionplan rather than writing a worked-out answer) and to a newdependent measure (namely, time rather than error rate).These results also explain an apparent contradiction in DeCorte et al.'s (1989a, 1989b) finding of consistency effects inchildren but not in adults when one-step problems were given.In our study we found strong evidence for consistency effectsin adults when we used more challenging problems, whichavoided a ceiling effect (i.e., two-step compare problems ratherthan one-step problems), as did Verschaffel et al. (1992).

Third, the study points to the important contribution thateye-fixation data can make to the study of the time course ofcomprehension of mathematical problems, including the fun-

84 M. HEGARTY, R. MAYER, AND C. GREEN

nel, selection, and consistency effects. The eye movementdata suggest that during the initial reading of a problem,students are not concerned with constructing a solution plan.This finding is consistent with the view that understanding amathematical problem involves progressively constructingvarious levels of representation, only the last of which is usedto generate a solution plan (Kintsch & Greeno, 1985).

Finally, this study provides preliminary information con-cerning differences between the ways in which successful andless successful problem solvers process word problems. Pre-vious research has identified two different approaches tosolving mathematical word problems, a direct translationapproach and a mental model approach (Kintsch & Greeno,1985; Lewis, 1989; Paige & Simon, 1966). Students who takethe direct translation approach attempt to derive a solutionplan directly from the verbal statement of the problem usingkey words, such as more to signal addition and less to signalsubtraction. Students who take the mental model approachfirst construct a qualitative model of the situation describedin the problem and then derive the solution plan from thatmodel. Students using the direct translation approach wouldspend the same amount of time on consistent and inconsistentproblems and would make conversion errors. In contrast,students using the mental model approach would spend moretime reading inconsistent than consistent problems, becausethe relational term primes a relation that is opposite the actualrelation between the two quantities in the described situation,and therefore would avoid making conversion errors. Thisadditional time would be spent mainly on rereading back-ground information relevant to constructing a qualitativemodel rather than on rereading the numbers in the problem.A theme in our findings that warrants further investigation isthat low-accuracy students appear to be using the directtranslation approach, whereas high-accuracy students appearto be using the mental model approach.

References

Carpenter, P. A., & Just, M. A. (1977). Reading comprehension aseyes see it. In M. A. Just & P. A. Carpenter (Eds.), Cognitiveprocesses in comprehension (pp. 109-140). Hillsdale, NJ: Erlbaum.

Clark, H. H. (1969). Linguistic processes in deductive reasoning.Psychological Review, 76, 387-404.

Corbett, A. T., & Chang, F. R. (1983). Pronoun disambiguation:Accessing potential antecedents. Memory and Cognition, 11, 671-684.

Cummins, D. D., Kintsch, W., Reusser, K., & Weimer, R. (1988).The role of understanding in solving word problems. CognitivePsychology, 20, 405-438.

De Corte, E., Verschaffel, L., & Pauwels, A. (1989a). Comprehendingcompare word problems: Empirical validation of Lewis and Mayer'sconsistency hypothesis. Unpublished manuscript, University ofLeuven, Leuven, Belgium.

De Corte, E., Verschaffel, L., & Pauwels, A. (1989b). Third-graders'comprehension of compare problems: Empirical validation of Lewisand Mayer's consistency hypothesis. Unpublished manuscript, Uni-versity of Leuven, Leuven, Belgium.

Dossey, J. A., Mullis, I. V., Linquist, M. M., & Chambers, D. L.(1988). The mathematics report card: Are we measuring up? Prince-ton, NJ: Educational Testing Service.

Ehrlich, K., & Rayner, K. (1983). Pronoun assignment and semanticintegration during reading: Eye movements and immediacy ofprocessing. Journal of Verbal Learning and Verbal Behavior, 22,75-87.

Greeno, J. G. (1989). Situation models, mental models, and genera-tive knowledge. In D. Klahr & K. Kotovsky (Eds.), Complexinformation processing: The impact of Herbert A. Simon. Hillsdale,NJ: Erlbaum.

Huttenlocher, J., & Strauss, S. (1968). Comprehension and a state-ment's relation to the situation it describes. Journal of VerbalLearning and Verbal Behavior, 7, 300-304.

Kintsch, W., & Greeno, J. G. (1985). Understanding and solvingword arithmetic problems. Psychological Review, 92, 109-129.

Lewis, A. B. (1989). Training students to represent arithmetic wordproblems. Journal of Educational Psychology, 81, 521-531.

Lewis, A. B., & Mayer, R. E. (1987). Students' miscomprehension ofrelational statements in arithmetic word problems. Journal of Ed-ucational Psychology, 79, 363-371.

Mayer, R. E. (1982). Memory for algebra story problems. Journal ofEducational Psychology, 74, 199-216.

Mayer, R. E. (1985). Mathematical ability. In R. J. Sternberg (Ed.),Human abilities: An information processing approach (pp. 127—154), San Francisco: Freeman.

Mayer, R. E., Larkin, J. H., & Kadane, J. B. (1984). A cognitiveanalysis of mathematical problem-solving ability. In R. J. Sternberg(Ed.), Advances in the psychology of human intelligence (Vol. 2,pp. 231-273). Hillsdale, NJ: Erlbaum.

Morales, R. V., Shute, V. J., & Pellegrino, J. W. (1985). Develop-mental differences in understanding and solving simple word prob-lems. Cognition and Instruction, 2, 41-57.

Paige, J. M., & Simon, H. A. (1966). Cognitive processes in solvingalgebra word problems. In B. Kleinmuntz (Ed.), Problem solving:Research, method, and theory (pp. 51-119). New York: Wiley.

Riley, M. S., & Greeno, J. G. (1988). Developmental analysis ofunderstanding language about quantities and of solving problems.Cognition and Instruction, J, 49-101.

Riley, M. S., Greeno, J. G., & Heller, J. I. (1983). Development ofchildren's problem-solving ability. In H. P. Ginsberg (Ed.), Thedevelopment of mathematical thinking (pp. 153-196). San Diego,CA: Academic Press.

Robitaille, D. F., & Garden, R. A. (1989). The IEA study of mathe-matics: Vol. 2. Contexts and outcomes of school mathematics.Elmsford, NY: Pergamon Press.

Verschaffel, L., De Corte, E., & Pauwels, A. (1992). Solving compareproblems: An eye movement test of Lewis and Mayer's consistencyhypothesis. Journal of Educational Psychology, 84, 85-94.

Received July 9, 1990Revision received August 19, 1991

Accepted August 19, 1991 •