Download pdf - Peer interaction and the learning of critical thinking skills in further education students

Instructional Science29: 1–32, 2001.© 2001Kluwer Academic Publishers. Printed in the Netherlands.

1

Peer interaction and the learning of critical thinking skills infurther education students

TONY ANDERSON1, CHRISTINE HOWE1, REBECCA SODEN2, JOHNHALLIDAY 2 & JENNIFER LOW1

1Centre for Research into Interactive Learning, Department of Psychology, University ofStrathclyde, Hills Building, 40 George St., Glasgow G1 1QE, U.K.;2Scottish School ofFurther Education, Faculty of Education, University of Strathclyde, Jordanhill Campus, U.K.

Received: 29 July 1998; in final form: 7 September 1999; accepted: 7 October 1999

Abstract. A teaching programme is reported in which critical thinking skills (in the sense ofreasoned justification of arguments; see Kuhn, 1991, 1993) were taught. The principal aimsof the study were to develop, implement and evaluate a programme for teaching evidence-based justification to vocational education students in Further Education colleges. Teachingwas via modelling and peer-based critiquing exercises in the context of the students’ projectwork. Eighty-four Further Education college students underwent a 10-session teaching inter-vention which dovetailed with their Additional Assessment integrative project work. Studentstook part in peer-based exercises in which they learned to critique imaginary examples ofproject outlines and plans, followed by similar peer-based critiquing of each others’ proposedprojects. Analysis of the students’ dialogues with each other indicated that they had learnedthe importance of justifying arguments, and content analysis of their written work indicatedthat they engaged in justification of their arguments to a significantly greater degree thancontrol groups. Several key variables in the dialogues correlated positively with justificationin the written work, suggesting that the dialogue had impacted on the written work. However,justification tended to be of a weak kind (using anecdotes or experience-based generalisa-tions), and strong (i.e. formal research-based) evidence remained relatively infrequent andsometimes inappropriately used. A psychometric test of general critical thinking skills showedno evidence of transfer of learning.

Introduction

The notion of critical thinking has received a great deal of attention in recentyears (e.g. Kuhn, 1991, 1993; Ennis, 1987; Fisher, 1990). It is not only ofinterest to academic psychologists, but has also been emphasised in the prac-tical context of further and higher education in both the United States (Ennis,1987) and the United Kingdom (The Audit Commission, 1993). Indeed, thevarious skills that are collectively termed ‘critical thinking’ are regarded asan important component of the so-called ‘transferable skills’ accrued duringhigher education.

2

The phrase ‘critical thinking’ has been used as a generic term encom-passing a wide variety of skills (Brookfield, 1987; Ennis, 1987; Kuhn, 1991;McGuinness, 1990; Paul, 1987). These skills include, for example, identify-ing assumptions, identifying and dealing with equivocation, making valuejudgements, analyzing arguments, asking and answering questions of clarifi-cation and/or challenge, judging the credibility of a source, and so on. Ennis(1987) attempted to derive a taxonomy of critical thinking skills, resultingin a lengthy list of some 118 subcategories, clearly underlining the diverserange of critical thinking skills. However, while such taxonomies are usefulfor identifying the variety of skills involved, they are too detailed for manypractical purposes. Furthermore, they do not illuminate the interrelationshipsamong the various skills (Kuhn, 1991) nor do they clarify the processes ofcritical thinking on particular occasions.

An alternative, more global and encompassing perspective on criticalthinking which conceptualises it as a type of reasoned argument and whichgives it an explicitly social dimension is exemplified by the work of Kuhn(1991; see also Billig, 1987). As Kuhn (1993) notes, thinking as argument isinvariably implicated in the beliefs people hold, the judgements they make,and the conclusions they arrive at: it is at the heart of everyday thinking.In a related vein, Billig (1987) argues that there exists an argumentativeand rhetorical component in many everyday psychological phenomena, suchas attitudes, roles and categorisations. Kuhn (1991) sees this kind of crit-ical thinking as involving the abilities to: a) differentiate opinions (or, asshe calls them, ‘theories’) from evidence, b) support opinions with non-spurious evidence, c) propose opinions alternative to one’s own and to knowwhat evidence would support these, d) provide evidence that simultaneouslysupports one’s own opinions while rebutting the alternatives, and e) take anepistemological stance which involves weighing the pros and cons of what isknown.

Kuhn (1991) provides data indicating that none of the above abilitiesis widespread in the adult population, and that the problems are especiallyacute amongst individuals whose post-school education, if present at all, isrestricted to vocational instruction. In the UK, as elsewhere, the distinctionbetween a vocational instruction and an academic education is commonlymade. It is widely believed that whereas Universities offer an academiceducation to those with the best qualifications upon leaving school, Collegesof Further Education (FE) offer only forms of vocational instruction to thosewith poorer or even no such qualifications. This has contributed to whatis known as the ‘academic-vocational divide’ which Young (1993) amongothers regrets. While there are many features of this divide that are worthinvestigation, the research that is reported here is concerned with the extent

3

to which something like Kuhn’s conception of critical thinking is taught inparts of the FE curriculum and the possibility that such a conception could betaught.

There is no doubt that although traditionally critical thinking has beenrecognised as an important aim of academic education, it has been rela-tively underplayed in vocational education, perhaps even contributing to theundervaluing of vocational education which Young (1993) laments. Yet, thislack of emphasis on critical thinking within the vocational context cannotbe seen to be necessary: practical action is as likely to benefit from theapplication of critical thinking as is academic endeavour. Similarly, there isno suggestion that students in vocational education are incapable of criticalthinking: Kuhn’s (1991) subjects tended to show some, although inconsistent,ability to display the skills of reasoned argument. They typified Fischer andFarrar’s (1988) characterisation of novice learners as showing cognitive skillswhich are partially but imperfectly developed rather than entirely absent andas demonstrating wide within-individual variability in performance acrossdifferent contexts. Such a characterisation suggests that guided practice ofthe skills involved should help improve matters, although as Kuhn (1991)notes, empirical support for this suggestion is surprisingly scarce. One aim ofour study was to provide such support.

The present research therefore focussed on the acquisition of criticalthinking skills by vocational education students in Further Education (FE)colleges, and in particular on the development of the skills of reasoned argu-ment via practice. Our specific aims were to develop, implement and evaluatea programme for teaching critical thinking skills, particularly those relatedto generating and critiquing evidence-based arguments. The programme wasguided by three broad principles. First and most generally, instruction aimedat improving the quality of the learners’ domain knowledge should be inte-grated fully with the development of more effective thinking skills. This pointlinks with Glaser’s (1984) claim that there is no sound evidence that attemptsto teach thinking skills as separate add-on courses improve students’ abilityto solve problems other than those of the type they practised during thetraining (although we acknowledge that others would disagree: e.g. Feuer-stein’s (1980) work on teaching basic cognitive skills via a programme of‘Instrumental Enrichment’). This integration principle suggests embeddinginstruction in critical thinking within the vocational content. Second, explicitdescriptions (‘models’) of what is involved in thinking effectively should bemade available to learners. This modelling of effective thinking should beaccompanied by attempts to engage students in exercises which help themdevelop insight into the nature of thinking. Third, thorough, systematic prac-tice in applying targeted skills to complex tasks which matter to students is

4

required, and there is much evidence (e.g. see the collection of papers in Foot,Howe, Anderson, Tolmie & Warden, 1994) to suggest that these exercisesmight profitably make use of peer-based social interaction.

Social interaction offers a natural corrective to the egocentrism of indi-vidual thought: we try out our ideas, and often our interlocutor is better ablethan we are to evaluate and offer a critique of them (e.g. Miyake, 1987),perhaps even offering a contrasting view and thereby overcoming our ‘mentalset’ or ‘cognitive hysteresis’. Piagetian theory emphasises the importanceof conceptual conflict as a means for provoking individual reflection andultimately superior conceptions. There is a considerable body of literaturewhich confirms that in children, peer interaction can have a positive effecton conceptual development (e.g. Damon & Killen, 1982; Doise & Mugny,1984; Weinstein & Bearison, 1985), and that the effect occurs as a functionof conceptual conflict promoting individual post-task reflection (Howe et al.,1990; Howe et al., 1992b; cf. Roazzi & Bryant, 1998, who provide evidencefor positive effects of agreement rather than social conflict).

The benefits of peer interaction need not be confined to children, however.Miyake (1987) demonstrated the beneficial effect of constructive criticismfrom a peer in adult subjects given a problem-solving task. She noted thatas few as 12% of the criticisms of hypotheses generated during the taskwere self-criticisms, and that the positive effect of an interlocutor involvedconstructive criticism of putative solutions (see also Whimbey & Lochhead,1980; Lochhead, 1985). Allen (1992) successfully employed peer critiquingin adults, giving students the task of producing compositions on topics ofinterest to themselves (for example, freedom of speech). In addition, thestudents were required to present versions of their compositions to each other,and to discuss each other’s work critically. Allen’s data suggest that thesepairwise critiques added value to what was produced. Howe et al. (1992a)also demonstrated positive effects of peer interaction with adult subjects(undergraduates) in a physics learning task. Given the weight of evidencecited above, the use of peer critiquing of students’ views was incorporatedinto our teaching programme. In summary, critiquing techniques were to bemodelled by the researchers and then practised in small groups by the studentsthemselves.

The students who participated in the programme were taking what isknown as the Level III general Scottish Vocational Qualification in SocialCare and Health Care. This qualification is usually obtained after two years’full-time study in a college. It includes elements of work experience, theacquisition of what are known as ‘core skills’ such as information tech-nology, communication, numeracy and problem solving, and the acquisitionof occupational skills related to the domains of social and health care. These

5

inclusions are brought together in a final project which requires studentsto undertake the background research involved in setting up a facility orservice (for students in Social Care) such as a childcare facility, or alterna-tively to design and implement a health promotion campaign (for HealthCare students) such as a campaign to encourage mothers to breast- ratherthan bottle-feed. The very choice of a project topic could be a vehicle forexercising dialogic argument and critical thinking, and the detailed designdecisions that require to be made during the project should again prove goodtopics for practising critical thinking and justification.

During the academic year immediately prior to that in which ourprogramme was to take place, observations and videotaped records weremade in three Further Education colleges of normal classroom teaching of theAdditional Assessment. Teaching staff were also interviewed regarding theiraims, objectives and teaching methods (Anderson et al., 1997). These initialdata revealed that the lessons were typically of two kinds: a) whole-classdiscussion, which followed Sinclair and Coulthard’s (1985) classic patternof teacher initiation – student response – teacher feedback, and b) individualinterviews between a student and a member of staff, with other students typi-cally in the room awaiting their turn. Behaviours indicative of critical thinkingwere in fact rare, a finding which is in accord with the research literature citedearlier. Our observations suggested that both teachers and students were moreconcerned to satisfy the Scottish Qualifications Authority’s written guidelinesfor what they should do than to exercise any disposition to think critically.There appeared to be little or no recognition of the opportunities for crit-ical appraisal that the process of project design allowed. These observationsinformed the details of the design of our teaching programme.

Method

Participants

The same three colleges that had participated in the observational phase werewilling to have the programme implemented. Two of the colleges (A andB) were in Glasgow; the third (C) was in a large town about thirty milesfrom Glasgow. Across the three colleges, a total of 84 students participatedat some point in the programme, 28 in College A, 20 in College B and 36 inCollege C. Inevitably, though, there was some subject attrition due to studentsdropping out of college, ultimately leaving 25 in College A, 12 in College B,and 21 in College C. While these drop out rates might appear to be high,they are regrettably all too common in many Colleges of FE and across alloccupational domains (Audit Commission, 1993; Further Education Funding

6

Council Statistics, 1998). In this respect our sample may be regarded asreasonably typical of those undertaking full time study in Colleges of FE.It should be noted that an analysis comparing the pre-test scores on the Smithand Whetton test of critical thinking of those who dropped out and thosewho persisted through to post-test revealed no significant differences: attritiondid not bias the sample in terms of ability, and was more than likely due tosocioeconomic factors or other individual differences. A control group of 35students, of similar levels of qualification and gender balance and drawn fromtwo other colleges, also participated. These students were involved in thesame type of study programme as the experimental group, and followed thenormal Additional Assessment curriculum (without experiencing any criticalthinking teaching intervention), and ultimately produced a project report that(like those produced by students in the experimental group) had to meet thestandards set by the accrediting body.

The teaching programme

The observations and interviews from the previous year indicated that peer-interaction is not characteristic of FE teaching, and hence that the peer-basedcomponent of the programme would have to be introduced gently. The obser-vations also made clear the all-pervading influence of external referencepoints, suggesting that the critical thinking exercises would have to be seento serve these ends to have a chance of success. This would have to be thecase even though the research team intended to deliver the lessons rather thanthe college staff, for there was no doubt that these reference points would behighlighted in other contexts even if they were ignored during the programme.After all, the students would be meeting with the college staff for other partsof the curriculum, and would almost certainly discuss their progress on theAdditional Assessment during these meetings. In addition, the classes forthese other parts of the curriculum would undoubtedly be delivered in thetraditional fashion.

The programme therefore took the form of a series of ten lessons whichintroduced appropriate aspects of the Additional Assessment specificationand contextualised critical thinking around decisions relevant to fulfillingthe requirements. In essence, the programme attempted to teach the studentsKuhn’s (1991) four key argumentative critical thinking skills, namely,providing evidence for one’s own theory, envisioning alternative theories,providing counter-arguments, and rebutting, in the context of devising thedesign of their projects and writing their reports. These skills provided a focusfor practising both evidence based justification and the generation of critique.Besides these four key skills, the teaching programme additionally addressedthe issue of meaning clarification and the importance of assumptions, but the

7

most substantial focus was upon evidence-based justification. The startingpoint was the fact that for the Additional Assessment students are required toproduce six ‘items of (written) evidence’, namely a project outline, a projectplan, a portfolio of evidence from project activities, a record of progress, theproject outcome, and an evaluation report. The lessons were scheduled for thefirst months of an exercise which is intended to occupy the full academic year.Thus, it was decided to focus on activities relevant to the first two items ofevidence, the outline and plan. To this end, a series of activities were sketchedout which involved: a) explication of the Additional Assessment specificationas it related to some aspect of the outline or plan, and guidance as to how therequirements might be fulfilled, b) homework where the students wrote downtheir preliminary ideas about what might be implied for their own projects, c)modelling by the research team of what peer-based critiquing would involve(by demonstration around specially prepared materials), d) deployment ofthe model by the students through the presentation and defense of their ideasin group discussion with their classmates, and e) preparation of an updatedwritten record in the light of the group discussion.

Although the three colleges involved in the programme were all workingto fulfil the GSVQ requirements, they varied in: a) the emphasis on SocialCare as opposed to Health Care, b) the time available for the teachingprogramme (from one to three hours per lesson – although note that the totaltime devoted to the Additional Assessment was constant across Colleges,so that those students with shorter lessons had time left over for reflectionand private study), c) the treatment of the outlines and plans as separate asopposed to combined entities, and d) the extra (non-Additional Assessment)goals which were set for the lessons, for example instruction in problemsolving. This variation entailed some tailoring of the details of the programmein each of the three colleges (see below).

Procedure

Once the activities had been finalised, they were implemented within thecolleges. One member of the research team led the lessons within eachcollege, although other members were present to model the peer-basedcritiquing. To preserve continuity, the same researchers were present eachweek in any given college.

The teaching across all college sites emphasised the importance of crit-ical thinking, particularly meaning clarification, justification, evidence givingand identifying and questioning assumptions. Common to all college siteswas the use of peer interaction and critiquing exercises undertaken by thestudents, initially of hypothetical materials and later of each others’ actual

8

materials. There were therefore several common threads that ran through theteaching across the sites, and also much sharing of materials and examples forcritiquing. There were, however, also slight variations of emphasis, to dovetailwith the requirements of individual colleges. For example, at College B, therewas a slightly greater emphasis on problem solving, including conceptual-isation of the Additional Assessment as a problem to be solved (the themeof problem solving being emphasised in three lessons as opposed to onein college C). Likewise, because all students at College C were students ofHealth Care and therefore committed to running a health promotion actionevent for their Additional Assessment project, persuasive communicationfeatured more explicitly in the lessons (the theme occurring in at least threelessons: see Appendix 1) than was the case in College B, for example (wherepersuasive communication emerged as an issue in two lessons). Nevertheless,such variations in emphasis were relatively subtle (students at College B didhear a little about persuasive communication, and students at College C didhear about problem-solving) and the number of common elements within theteaching programmes in the different colleges was much more striking thanthe variations of emphasis.

Appendix 1 sets out the contents of the ten lessons as they werepresented at College C. Note the use of exposition (e.g. of AdditionalAssessment requirements and of substantive topics such as problem solvingand persuasive communication), coaching and modelling of peer critiquing,critiquing of carefully constructed example items, the undertaking of peercritiquing by the students themselves of drafts of each others’ ideas, andhomework exercises for the students to undertake. This blend of teachingmethods was common across all college sites, and was intended to dove-tail with the three teaching principles articulated above. For example, keyaspects of critical thinking such as identifying and questioning assumptionswere embedded within the teaching of how to produce a good project outline,rather than being tackled as isolated topics in their own right. The modellingby the research team of how to apply critical thinking to relevant hypotheticalexamples of project outlines and plans (see Appendix 2 for examples), andthe peer-based undertaking of similar critiquing exercises by the studentsthemselves was designed to help students develop insight into the nature ofcritical thinking. And the peer-based critiquing of each others’ draft outlinesand plans attempted to give the students practice in applying the targetedskills to appropriate complex tasks.

Appendix 2 shows some example materials that the students could useto practice critiquing. These examples were putative Additional Assessmentoutline sections, for Social Care projects that proposed to take elderly peoplefrom a residential home away on holiday. Example 1 was designed to be

9

poorer in the sense of being less well thought-out and not well justified(for example: entailing questionable assumptions to the effect that elderlypeople are ‘just like any other holidaymaker’ and correspondingly likely tofall foul of practical considerations such as how health or mobility problemswould dovetail with air travel; being innocent of any research on what sortof holiday elderly people might desire or what special arrangements travelcompanies might routinely make for such clientele; and so on) and example2 was designed to be better. Each example is provided with some usefulcritiquing questions to help get the students started. Even although example2 is less prone to criticism than example 1, it nevertheless leaves room for theexercise of critical thinking skills. Similar examples were created for healthcare project outlines, and for both types of project plan. These exerciseswere in practice well received by the students, and proved a useful vehiclefor developing and articulating critical thinking skills. Ultimately, the peercritiquing process was brought to bear on the students’ own draft outlinesand plans, by which time they had had considerable modelling and practiceof the relevant skills.

Evaluation

The evaluation of the effectiveness of the programme involved three analyses:a) analysis of the participants’ dialogue with each other, b) analysis of theirwritten work and its relation to their dialogue, and c) analysis of their perfor-mance pre- and post-intervention on a standardised test of critical thinking.Each of these three analyses will be described in turn.

Videorecordings were made of a sample of the peer-based critiquingsessions in all three participating colleges, and the frequency of a seriesof dialogue variables was subsequently counted. Videorecording beganin the sixth week, by which time the students were accustomed to theresearchers and the critiquing. This permitted us to ascertain whether anypatterns of dialogue relevant to critical thinking were discernible. In addition,draft outlines and plans were gathered from all three colleges during theprogramme, and the final versions of the full Additional Assessment weregathered from College C when they were presented about six months afterthe programme’s completion. Outlines and plans from the drafts, and outlines,plans and evaluation reports from the final write-ups were content analysed interms of their use of justification. The final reports were compared to a controlsample from the same college in the previous year, permitting an examinationof whether the programme had affected the students’ writing.

It had been hoped to videorecord every student on two occasions, but thisproved impossible. Participation in the videorecording was voluntary, and

10

some students were initially hesitant. Given that the recordings were spreadover several weeks, however, initial reticence was frequently later overcome,so that the vast majority of the students were recorded at least once. Therecording equipment was in full view of the students, and no attempts weremade to hide it or to record surreptitiously. Despite the obvious presence ofthe camera, once involved and absorbed in their tasks, the students appearedto behave reasonably naturally and in an unselfconscious way. Fifty-eightstudents were recorded once, but of these only 25 were recorded twice. 22variables were used to code the dialogue that was recorded (see Table 1).

A sample of dialogues was coded by a second coder, and the inter-raterreliability coefficient for this coding was 0.90. Analyses were conductedon the absolute frequency per recording of the variables, the proportionalfrequency as a function of total utterances, the total number of utterances,and the duration of the session. Absolute and proportional frequencies werehowever highly intercorrelated, and the results below are based exclusivelyon the former.

The written work was coded (by a coder who was blind to which conditionthe written materials originated from) for each of three kinds of justification:no justification (i.e. the student simply asserts a statement without justifica-tion – e.g. ‘I have decided to promote the benefits of breastfeeding’), weakjustification (i.e. the student justifies an assertion or course of action usinganecdotal or informal evidence such as a broad generalisation, e.g. ‘I havechosen to promote awareness of AIDS and how the virus spreads, becausethere are still too many people who are ignorant of the virus’), or strongjustification (the student justifies his or her choices or claims by referenceto formal research and statistical evidence, even if obtained ‘second-hand’via health promotion leaflets – e.g. ‘I know from reading the Breast FeedingLeaflet, Natural For You, Best For Your Baby, Health Education Board forScotland, 1994, that breast milk contains antibodies from the mother whichcan help the baby fight infections’). Appendix 3 gives some further examplesof the different types of justification used. The coding of such justifica-tions was conservative in that only very clear instances of justifications werecounted: justification of a claim had to occur either within the same sentenceas the claim or in an immediately adjacent sentence. This excluded sentencesthat cited evidence but did not explicitly link it to a choice or claim made inthe immediately adjacent text. It is possible that in some such cases the writermay have intended to make more global justifications, but such instanceswould not count under our coding scheme. A randomly selected sampleamounting to 25% of the written materials were coded by two raters, andinter-rater reliabilities averaged 0.88 for the draft outlines and plans and 0.92for the final project sections.

11

Table 1. List of categories used in the analysis of the subjects’ peer critiquing interactions,with definitions or examples

Task chairing (all remarks where one group member tries to move the task to the nextstep)

Checking (requests advice or affirmation)

Minimal comments (yes/no response).

Agrees or disagreeswith what is stated.

Asserts point of view(takes a stance on one side of an issue)

Offers suggestion

Basic statements(unjustified assertion, e.g. stating his/her view if asked)

Justifies with anecdote(e.g. appeal to personal experience or broad generalisation basedon experience)

Justifies with evidence(cites formal evidence, e.g. research statistics)

Justifies with values(extrinsic or intrinsic)

Responds to anecdote(with agreement or disagreement)

Responds to evidence(with agreement or disagreement)

Responds to values(with agreement or disagreement)

Requests expansion

Requests source of knowledge

Discusses practical possibility(e.g. feasibility of intended campaign)

Weighs evidence(e.g. synthesises ideas)

Positive procedural (positive remarks about the functioning of the group or the exerciseas a whole)

Negative procedural(negative remarks about the functioning of the group or the exerciseas a whole)

References to instruction sheets

Asides(all remarks directed at extraneous activities)

Uncodable

Third, a standardised multiple choice test of critical thinking, the Verbalcomponent of the Critical Reasoning Test (Smith & Whetton, 1992), wasadministered in the second and tenth weeks to participating students in allcolleges and to a control sample studying Health and Social Care in twoadditional colleges, as a test of any effect of generalisation of learning. Thetest involves identifying implications, evaluating evidence and recognisingassumptions in a text relating to commercial practices. The reasons for usingthis test were a) the test was designed for employees with similar educa-tional backgrounds to our cohort of further education students, rather thanbeing designed to test graduates; b) recent normative data are available foremployees engaged in occupations similar to those for which the FE students

12

were being prepared, and c) the test content is based both on research and thetasks that employees actually perform. In many respects therefore, it appearedto be a suitable choice.

Results

i) Analysis of the peer dialogues

The intercorrelation between absolute and proportional frequencies of thedialogue variables (noted above) suggests that there was a common patternto the dialogues as they occurred across recordings and across colleges. Table2 shows the absolute frequencies of the 22 dialogue variables in both the firstand the second recordings. The most frequently occurring variables acrossboth sessions are minimal comments, asserts point of view, basic statements,justifies with anecdote, requests expansion and asides. The frequency of justi-fication with anecdote is much higher than for justification with evidenceor justification with values in both recordings. Two striking findings arethe significant decline in basic (unjustified) statements and the significantincrease in justification with anecdote between session 1 and session 2. Thesefindings suggest that the participants are increasingly recognising the impor-tance of supporting their claims with evidence of whatever kind they canmuster, which is usually anecdotal rather than stronger forms of evidence.It should also be noted that the categories ‘justifies with evidence’ and‘weighs evidence’ are relatively infrequent in occurrence: the participantsare improving a little in how they make their own case (Kuhn’s providingevidence for one’s own theory) by providing anecdotal evidence, but arenot providing strong evidence for their case, or weighing evidence. Thus,in terms of Kuhn’s four skills of argument, there is considerable room forimprovement, even at recording 2.

A number of variables are asterisked in Table 2 to indicate that they formedan intercorrelated cluster in both the first and second recordings. The iden-tity of the more-frequently occurring and the asterisked variables in Table2 suggests patterns of dialogue which: a) began non-critically, via minimalcomments, basic statements and the like, b) proceeded with challenge, viarequests for expansion, and c) concluded with justifications, albeit only atthe level of anecdote. Athough thepattern was constant across colleges,its frequencywas not: in both recordings, College B produced more of theasterisked variables than College A or College C. However, the dialoguesfrom College B were longer than at the other two colleges, and subsequentcomparision of written materials demonstrated no significat differences inthe students use of justification, for example, when comparing draft project

13

Table 2. Mean frequencies across students of the coded dialogue variables. Theasterisked variables form a positively intercorrelated cluster

Dialogue variable First rec. Second rec. First vs. second

(N = 58) (n = 25)

∗Task chairing 1.24 1.05 t = 0.41, ns∗Checking 1.72 1.28 t = 0.89, ns∗Minimal comments 3.00 3.72 t = 0.63, ns

Agrees or disagrees 1.28 0.76 t = 1.40, ns

Asserts point of view 2.56 3.00 t = 0.66, ns

Offers suggestion 1.24 0.56 t = 1.78, ns

Basic statement 12.04 8.28 t = 2.05,p = 0.05∗Justifies with anecdote 2.21 40.79 t = 5.10,p< 0.001

Justifies with evidence 0.16 0.36 t = 1.10, ns

Justifies with values 0.20 0.04 t = 1.44, ns

Responds to anecdote 0.20 0.12 t = 0.46, ns

Responds to evidence 0.0004 0.04 t = 0.99, ns

Responds to values 1.28 0.76 t = 1.40, ns

Requests expansion 4.88 4.40 t = 0.41, ns

Requests source of knowledge 0.04 0.48 t = 1.23, ns

Discusses practical possibility 0.16 0.08 t = 0.62, ns

Weighs evidence 0.00 0.08 t = 1.44, ns

Positive procedural 0.00 0.00 t not computed∗Negative procedural 0.12 0.00 t = 1.81, ns∗Reference to sheets 1.20 1.64 t = 0.44, ns

Asides 3.72 13.40 t = 3.07, p = 0.005

Uncodable 1.28 1.48 t = 0.41, ns

Outline sections from College B as opposed to College C. Clearly the qualita-tive pattern of the dialogues was important for the ultimate outcome, ratherthan the sheer quantity of talk involved.

Table 3 shows the intercorrelations among the dialogue variables aster-isked in Table 2.

The intercorrelations are never negative, and the majority of them are stat-istically significant. The variable that intercorrelates most weakly with theothers is task chairing, since it fails to correlate significantly with three of theother variables. All other intercorrelations are either statistically significant orvery close to being so. It is perhaps noteworthy that the strongest correlationin the table is between justification by anecdote and the variable Negative

14

Table 3. Intercorrelations among the cluster of dialogue variables identifiedin Table 2 above. Note that most of the intercorrelations are statisticallysignifiicant or very close to significance, and none are negative

Chair Check MinCom JustAn NegProc RefSheet

Chair 0.309 0.301 0.206 0.14 0.032

<0.048 <0.05 NS NS NS

Check 0.366 0.295 0.305 0.412

<0.019 <0.058 <0.05 <0.008

MinCom 0.495 0.34 0.365

<0.0015 <0.029 <0.019

JustAn 0.514 0.413

<0.001 <0.008

NegProc 0.448

<0.004

Key: Chair = Task chairing, Check = Checking, MinCom = makingMinimal Comments, JustAn = Justifies with Anecdote, NegProc = makingnegative comments about the task procedures, and RefSheet = makingreferences to the task instruction sheets. All values are Spearman’s Rhocorrected for ties.

Procedural (i.e. making negative or critical comments about the task); bothare forms of critical thinking.

The character of the students’ dialogues can best be appreciated by consid-ering some excerpts of transcript. In the following extract, one of the studentshas decided that his Social Care service will be to befriend a youngster, andthe two students are discussing the choice of a possible target group (A andB are the students, T is the teacher):

A: Do you have a target group that you’ve chosen?B: At the moment I have a target group that I’ve chosen to befriend, thesame as a few other students in the class.A: How did you find out about befriending?B: Actually before I was at the college I put my name down to become abefriender and the college started a course. So I went and started it and afew of the other ones came to it.A: Could you just explain what a befriender is?B: A befriender is somebody that takes a kid who maybe has some prob-lems, can be a wide range of problems like in the house with their parentsand things and it’s just to take them away from that situation for a coupleof hours or a day to let them see what everyday life is all about.

15

A: Do you think it’s a good thing?B: Oh I think really it’s a good thing. I think it’s a really good thing forkids.A: Why do you think it’s a good thing?B: Well, lets them see what everyday life’s about with ordinary everydaypeople you know.A: I can’t think of anything else to ask. My mind’s a blank!T: What kinds of acts do you think you’ll get involved in?B: Quite a range of acts we could participate in. First of all, you’ve gotto give kids the choice. You can’t just steer them into something. Got tocome to an agreement with them. And then maybe a wide range of thingslike taking them to McDonalds, taking them to the cinema, or away for aday. There’s no end to it, you know. A day trip to the coast.A: I can’t think of anything else to ask.

The dialogue above comes from relatively early in the sequence of lessons,and the initial choice of a project topic is being discussed. Initially, studentA seeks a number of meaning clarifications (e.g., what a befriender is, whyis befriending considered a good thing). Student B, whose proposed projectthis is, is a little vague about the justification for befriending (“let themsee what everyday life’s about”), but at this early stage student A is notyet skilled in teasing out the underlying issues, and indeed grinds to a halt(“can’t think of anything else to ask”), at which point the teacher inter-venes to seek further detail on the proposed activities, which should generatefurther issues for discussion (e.g. the practicability of some of the proposedactivities, the funding of them, and so on). Even though student A missessome opportunities for critical questioning (for example, failing to home inon B’s vague justification for his proposal), student B nevertheless is givenfood for thought. The dialogue excerpt is fairly typical of the dialogues thattook place early in the sequence; later dialogues contained much more inthe way of justifications, although as noted above these tend to be anecdotesor experience-based generalisations (e.g. ‘Homeless people have a lot ofproblems with the legislation of the housing department and causes greatproblems with them. They don’t know how to get round it. No advice isgiven to them. I’m hoping to advertise myself as someone who can givethis advice and educate them of their rights’; here a set of generalisationsis used to justify the proposed service). Overall, students improved both intheir capacity to ask the right kinds of critical questions (e.g. increasinglyspecific and relevant questions, such as “Why are you targeting 14–16 yearolds? Why not younger kids, of 10–16?”; “Have you any experience in thisfield?”; “Have you considered any other client groups? And if so, why didyou choose not to work with them?”; and so on) and to provide answers and

16

Figure 1. Frequencies of justification types in draft outline and plan sections.

justifications for their answers as the programme progressed. In summary,there is evidence from both quantitative analyses and qualitative examinationof the raw dialogue data that the programme did promote critical analysiswithin the peer-based dialogue, but did this feed through to the students’written work?

ii) Analysis of the students’ written work

The students’ first drafts of their outline and plan sections were available forstudents from Colleges A and C (n = 11 and 24, respectively; students inCollege B had progressed only to the point of having completed their draftoutline sections by the time at which these data were collected and wereaccordingly omitted from this analysis). The College C students’ final plansand outlines were also available for analysis (N = 29), as were those from acontrol sample attending the same college the year before (N = 25). In addi-tion, the final projects from both College C and the control sample containedevaluation sections which assessed the projects’ strengths and weaknesses,and which could also be coded in terms of critical thinking variables andcompared. The resulting data are shown in Table 4 and Figures 1–3.

In the case of the draft outlines and plans, the experimental groupproduced significantly more instances of the categories ‘no justification’ and‘weak justification’. Although the difference between the two groups wasin the same direction in the case of strong justification (with experimentalsubjects producing greater numbers of strong justifications), the relatively low

17

Table 4. Mean numbers of unjustified, weakly justified and strongly justified assertions inthe draft outline/plan and final outline/plan and evaluation sections of experimental (draftn= 35, finaln = 29) and control (draftn = 24, finaln = 25) groups’ project reports

Group No justification Weak justification Strong justification

Draft outlines/plans

Experimental 21.7 10.7 1.37

Control 10.25 4.46 0.25

t-value (df = 57) 4.81 3.28 1.6

Significance of difference 0.0001 0.0018 NS

Final outlines/plans


Control 50.2 15.4 0.08

t-value (df = 52) 2.57 4.07 4.05

Significance of difference 0.01 0.0002 0.0002

Final evaluations


Control 16.16 5.36 0.0

t-value (df = 52) 6.65 6.5 4.47

Significance of difference 0.0001 0.0001 0.0001

Figure 2. Frequencies of justification types in final outline and plan sections.

18

Figure 3. Frequencies of justification types in final evaluation sections.

numbers of such justifications actually produced, with many students produc-ing none at all, resulted in high variance and a non-significant comparison.The relatively low mean number of strong justifications compared to weakjustifications produced by the experimental group suggests that the studentshave learned the importance of justification but at that point in their studieslack the evidential base to produce strong justification.

In the case of the written work from the final versions of the students’projects, for all three categories of justification (none, weak and strong), theexperimental group give significantly higher numbers than do the controlgroup, for both the outlines and plans on the one hand and the evaluations onthe other (see Table 4 above). In relative terms the differences are striking:experimental group participants do not merely producemore justificationthan do control participants, but produce multiples of the controls’ levels ofjustification: for example, twice as much weak justification in the outlines andplans and more than three times as much weak justification in the evaluations.In the case of strong justification, the differences are even more dramatic:the control participants’ low mean of 0.08 in the outlines and plans acrossall participants corresponds to a mere two instances found in the writtenwork of the 25 participants, whereas experimental group participants areproducing an average of almost two such justifications each. In the evalu-ation sections, none of the control participants produce strong justifications,whereas experimental group participants produce an average of one each.Overall then, in all sections of the final written materials analysed, students

19

Table 5. Dialogue variables correlating significantly with the draft projectplans and outlines. All values quoted are for Spearman’s Rho corrected forties

Variable r p

Correlations with category ‘no evidence’

Minimal comments +0.598 <0.0019

Justifies with anecdote +0.437 <0.02

Correlations with category ‘weak evidence’

Assert point of view +0.395 <0.04

Correlations with category ‘strong evidence’

Agrees or disagrees +0.388 <0.04

Reference to Sheets −0.39 <0.04

from the participating colleges outshone their control group counterparts inall categories of evidence-giving. This is despite the fact that the final versionswere produced about six months after the teaching programme’s conclu-sion. Again, however, it should be noted that the justifications and evidenceproduced in both draft and final written work were overwhelmingly justi-fications of one’s own position; envisaging evidence for other positions andweighing the evidence overall were areas in which considerable improvementremained to be effected.

Correlational analyses were undertaken, between the various dialoguevariables and the categories of justification in on the one hand the draftoutlines and plans and on the other the final outlines, plans and evaluationsections. The significant correlations are shown in Tables 5 and 6.

There is some degree of consistency in the pattern of significant corre-lations emerging. For example, ‘Justification with Anecdote’ and ‘TaskChairing’ during the dialogues are positively associated with the categoriesof justification in both draft and final versions of the written work. Likewise,‘Minimal Comments’ and ‘Agrees or Disagrees’ are positively associatedwith justification in both draft and final versions of the written work. Overall,the pattern of correlations suggests that students who are proactive in thedialogues (by asserting their point of view, justifying themselves with anec-dotes, chairing, and agreeing and disagreeing with their peers) are more likelyto produce justifications in their written work. The slightly more passivebehaviours of responding to evidence provided by others and referring totask sheets are negatively associated with subsequent written justifications.Negative comments on the procedures during the dialogue are associatedwith justification, but in the final evaluation section only. Perhaps this reflects

20

Table 6. Dialogue variables correlating significantly with the final project plansand outlines, and evaluation sections. All values quoted are for Spearman’s Rhocorrected for ties

Variable r p

Plans and outlines



Asides +0.498 <0.02


Minimal comments +0.468 <0.03


Responds to evidence −0.436 <0.05


Task chairing +0.419 <0.05

Evaluation sections


Negative procedural +0.746 <0.0006



Agrees or disagrees +0.422 <0.05

Negative procedural +0.467 <0.03



Responds to evidence −0.471 <0.03

a critical propensity of a slightly different kind (not based on criticism ofarguments but of tasks and procedures), and one that is more appropriatelyexercised in the evaluation sections. Overall, these analyses suggest that thedialogues in which the students were engaged did indeed impact upon theirwritten work.

The third aspect of the evaluation, performance on the Smith-WhettonCritical Reasoning Test, demonstrated a) no significant differences amongthe participating colleges in scores overall (F[2,48] = 1.39,p < 0.26) or ingains (F[2,48] = 0.747,p < 0.48); b) test scores of both the participatingand control colleges improved from first presentation to second (27.6 pre-vs. 29.4 post-intervention,F[1,84] = 9.324,p < 0.003); c) no significantdifference between participating and control colleges overall (F[1,84] = 1.7,p < 0.19) and d) no significant differences in pre-post test gain between

21

participating and control colleges (F[1,84] = 0.67,p < 0.41). Thus, basedon the Critical Reasoning Test, there was no evidence that the skills masteredwithin the programme transferred elsewhere. Transfer is an important issuetheoretically, and the failure to find evidence for it in the present study isdiscussed further below.

Discussion

Essentially, we found that i) modelling and peer interaction resulted inimprovements to certain key targeted aspects of critical thinking in thestudents’ dialogues, ii) these improvements increased as our interventionproceeded, iii) the enhanced critical thinking evident in the dialogues fedthrough to the participants’ subsequent written work, resulting in enhancedjustification in participants’ projects, iv) the participants’ written work provedsignificantly better than those of controls in terms of adequacy of justifica-tion, and v) there was no evidence that these enhanced critical thinking skillsgeneralised beyond the context of learning. In discussing these findings, wewill a) draw together the main results, b) look at some of the features ofthe data which qualify the principal results to some extent, c) consider theimplications of the study for both theory and teaching practice, and d) identifysome issues for further research.

Kuhn (1991) argues that whilst many have asserted that thinking willimprove as a function of guided practice (or ‘exercise’), ‘few research data areavailable regarding how thinking improves through exercise’ (Kuhn, 1991,p. 292). We would claim that our results provide precisely the kind of datathat would throw light upon such a view, and propose further that they lendsome qualified support to the view that thinking improves with practice.Several features of our data are pertinent here. Firstly, the dialogues thattook place during the peer interaction sessions showed increases in justi-fication (albeit by anecdote) and reduction of unjustified assertions betweensuccessive sessions. A qualitative examination of the transcripts of students’dialogue suggests an improvement in the ability to frame appropriate crit-ical questions on the one hand and to anticipate and ‘defuse’ criticism bymarshalling evidence in support of one’s claims on the other. All in all,students exhibited the thinking skills that we were interested in promotingto an increasing degree as the intervention progressed: thinking does indeedimprove with guided practice.

Secondly, the positive correlations between some of the dialogue vari-ables and the justifications used in the participants’ subsequent written worksuggests that the dialogues were indeed having an impact upon the writtenwork. For example, justification with anecdote in the dialogues correlates

22

positively with evidence-giving in the final project plans and outlines, and‘negative procedural’ (another form of critical thinking) correlates signifi-cantly with evidence-giving in the final project evaluation sections. Enhancedlevels of justification in the project evaluation sections, written some monthsafter the intervention had ceased, implies (at the very minimum) that thegain observed as the intervention progressed was a sustained one. The latterfinding could even be argued to be an instance of ‘near transfer’ (positivegeneralisation; see below).

Thirdly, participants who underwent the teaching programme producedenhanced justification of their ideas relative to controls. In particular, the useof strong evidence is strikingly different in the two groups’ final reports. Inaddition to the use of strong evidence within the text of the project reports, theappending of parts of or entire research publications occurs in a number of theprojects from the experimental group. In one case, an article from theJournalof Intellectual Disability Researchwas included to indicate the prevalenceand extent of the problem of obesity in sufferers of Down’s syndrome. Suchthoroughness was completely unknown in the control group projects.

It should be noted that variation in the length of the teaching sessionsdid not alter the broad pattern of the results: as noted earlier, comparison ofdraft Outlines from college B (which had the longest lesson lengths) yieldedno significant differences from those produced at College C (which had theshortest lesson lengths). Furthermore, the data in Table 4 on final Outline/Planand Evaluation sections are from students in College C, with the shortestlesson lengths, and therefore, even if lesson length were to impact uponoutcome, our claims are mainly based on the data coming from the Collegewith shortest lessons.

Despite all the positive aspects of the above-reported results, there weresome findings and features in the data that suggest that the above gainswere to some extent limited in scope. Two centrally important areas thatfeatured such data will be considered: the students’ use of evidence and theissue of generalisation. Consider first the use of justification in the students’written work. Whilst participants outperformed controls in the use of justifi-cation, it was clear from a more qualitative examination of the experimentalstudents’ writing that their use of evidence as justification was not completelydeveloped. For example, students frequently cited strong evidence from e.g.medical research, but failed to explicitly link it to the claims that they weremaking, instead listing facts and figures in a fashion that did not directlybear on their arguments. Our coding scheme only counted instances wherea claim was justified in the same or an immediately adjacent sentence, andlong lists of facts and figures that were not explicitly related to the students’arguments counted as statements rather than justifications. Thus, the students

23

participating in our programme improved in their demonstration of criticalthinking skills vis-a-vis the controls in that they clearly recognised the needto cite strong evidence as part of their work, but they sometimes did notfully exploit the strong evidence they cited in relation to the arguments theywere making. Kuhn (1991, pp. 83–86) notes a similar failure of some of herparticipants to link evidence to argument, including instances in which theindividual concerned is clearly in command of a good deal of appropriatedomain knowledge that can be articulated but nevertheless fails to use itto bolster specific claims. Kuhn regards such instances as exemplifying atype of nonevidence. Clearly, awareness of domain knowledge is insufficientto guarantee cogent critical reasoning, even among some college-educatedsubjects (the examples of this particular type of non-evidence that Kuhn citesare from college-educated subjects, but unfortunately she does not give adetailed breakdown of the incidence of this type of non-evidence as a functionof participants’ educational level). It is worth noting that there are two keydifferences between Kuhn’s subjects and ours. One is that our participantshad undergone an intervention specifically designed to promote better criticalthinking whilst Kuhn’s subjects had not; the second is that Kuhn’s subjectsgenerated spoken responses, whilst ours generated written projects. Arguably,our subjects would have had more opportunity to organise and marshal theirthoughts, and their occasional failure to coordinate claim and evidence onpaper despite our intervention and a considerably greater amount of time toorganise their thoughts is an issue for concern and further study.

A second issue with respect to which our positive findings require to bequalified is that of generalisation or transfer. Generalisation was tested inthe present study using the Smith-Whetton Critical Thinking Appraisal pre-and post-the intervention. Such a general psychometric test would effectivelyfunction as a test of ‘far’ or remote transfer: its contents relate to criticalthinking to be sure, but in a completely different domain to that in whichthe participants learned about critical thinking. To have shown significantlygreater gains in test scores on this test relative to controls, our participantswould have had to have learned the aspects of critical thinking that we soughtto promote in a very domain-general way. The problematic status of general-isation in the literature, in which views on it range from schema-based modelsto explain the phenomenon (e.g. Reed, 1993) to claims that generalisationis an epiphenomenon (e.g. Detterman, 1993), suggests that generalisationshould not necessarily be expected. In the face of such divergent evidence,it would have been optimistic to have predicted domain-general transfer tooccur following learning within one particular domain. Our analysis had thestatus of a check to see if there was any evidence for transfer, but the negativeresults obtained are no surprise.

24

On the other hand, it is possible that our participants would have displayednear transfer (i.e. transfer within the learning domain of Care) had it beentested for, but tapping such abilities was beyond the scope of our research.In that respect, it was interesting to observe that enhanced justificationwas true of our participants’ evaluation sections, which were written somemonths after the intervention had ceased, suggesting positive near transfer.Undoubtedly, however, in some respects our intervention must have directlyinfluenced the content of the students’ evaluations (e.g. by our suggestingpre- and post-questionnaires as a measure of how successful the student’shealth promotion event had been), resulting in the citation of ‘hard’ data.Such instances could not count as examples of ‘near’ transfer. Other cases,in which students included in their Evaluation sections hard evidence fromresearch literature entirely on their own initiative, might be taken as evidenceof such transfer. Unfortunately, we have noted very few such instances in thedata. The issue of near transfer following interventions such as ours remainsone for future research; our failure to find evidence for it is in itself inter-esting, but it should be borne in mind that our intervention had a limitedand narrow focus (solely upon the Additional Assessment; students simultan-eously underwent several other learning modules in which little or no mentionof critical thinking would have been made). Whether near transfer would bemore likely to occur as a function of more broadly-based training in criticalthinking remains to be seen.

Implications

The above findings bear on a number of significant theoretical issues. Thatpeer interaction helped promote critical thinking is in itself a noteworthyfinding, but the data went further in two key respects. Firstly, the frequen-cies of occurrence of the various coding categories suggested that withinparticular dialogues there existed patterns in which points of view wereasserted, challenged and justified. Secondly, the significant changes thatoccurred between the two sessions of videotaped dialogue suggested thatpost-task reflection had occurred between the two sessions. Both findings areconsistent with Piagetian rather than Vygotskyan views on the processes ofpeer interaction, suggesting that the mechanisms underlying change involveboth conceptual conflict and subsequent reflection. These findings also tie inwith our own previous work (Howe et al., 1990, 1992b).

Another major theoretical issue within developmental psychology iswhether thinking skills are fundamentally generic or specific. Kuhn (1991)argues that her own data provide evidence that forms of thinking that tran-scend the specific contexts of their use can be identified, insofar as herparticipants sometimes exhibited similar forms of argument and justification

25

across more than one of the three topics that she used. This finding is incontrast to some of the claims that have been made in recent years about thealleged domain-specificity of thinking (e.g. Carey, 1985, 1986; Keil, 1984).Participants in the present study did demonstrate improvement in the targetedskills during the dialogues and these were carried over into the written work.However, unlike Kuhn, we were unable to test the subjects’ thinking inmore than one domain, and as noted above failed to find any evidence fortransfer. We would therefore be cautious in making any claims about domain-generality. Clearly, more intensive and systematic study of the wider effectsof exposure to a teaching intervention such as ours would be a key issue forfurther research.

Our study also bears on the issue of the efficacy of the various availableteaching methods for effecting improvement in such metacognitive skillsas critical thinking skills. Whilst some would claim success in improvingthinking skills via teachingabout those skills (e.g. see Baron & Sternberg,1987 for a review of several such programs), and others would claim successat teaching thinking skills via a Vygotskyan ‘scaffolding’ process involvingone-to-one teaching (e.g. Perkins et al., 1986) our approach involved anelement of modelling and a considerable amount of practice via peer-baseddialogue. Kuhn (1991, p. 295) raises the issue of whether peer-based dialogueis sufficient as a means of inculcating critical thinking skills, or whether someelement of modelling or coaching is necessary in the teaching programme.Our approach involved both of these elements (although weighted heavilytoward the latter) and we are not therefore in a position to make strong claimsabout the relative efficacy of these methods. However, our own previousresearch (e.g., Anderson, et al., 1996; Howe et al., 1990; Howe et al., 1992a)provides ample evidence that peer interaction is helpful for both conceptualand procedural learning. Accordingly, it would not be unduly surprising wereit to transpire that the element of our intervention that involved peer dialogueswas the key element for facilitating the learning of metacognitive skills (thepositive correlations that we found between dialogue variables and indicesof critical thinking would appear to support this view), although again thatremains an issue for future study.

A critic of our study might argue that it is hardly surprising that thestudents did learn certain oral and written uses of language that we take tobe indicative of our preferred (for the purposes of the project) conceptionof critical thinking. What is much more surprising and interesting howeveris that they continued to use some of these forms several months after theintervention had ceased (in their evaluation sections), and long after theresearchers had stopped teaching them. We did not simply tell each individualto insert a series of specific justifications in his or her project; we emphasised

26

the importance of justification, reinforcing this via the peer-based critiquingexercises, and the students themselves did much to determine which of theirideas needed justification and gathered the appropriate evidence. In somecases, as noted above, this was done in an impressively thorough way.

Whilst our study is encouraging regarding the issue of whether thinkingskills can be taught via a combination of coaching, modelling and struc-tured peer interaction, there remain a number of issues for future research.Could the present teaching method and findings could be replicated by theFE teaching staff themselves? What type of staff development would benecessary to achieve replication of the results? What would be the effect ofgreater degrees of embeddedness of critical thinking across the vocationalcurriculum on transferability? More generally, what would be the effect of apeer-based intervention such as ours on everyday reasoning (as opposed toreasoning within vocational and academic contexts) such as that studied byKuhn (1991) and Perkins et al. (1983)? Could such an approach help developa ‘critical epistemology’ (Perkins et al., 1983), and if so, what would be theconstraints on the efficacy of the relevant intervention?

If critical thinking is an educational ideal (Siegel, 1988), then its realisa-tion can only come about if there are ways of encouraging it in educationalinstitutions. The present study suggests one method by which this could beachieved in further education colleges.

Conclusions

The study reported in the present paper demonstrated that critical thinkingskills (in particular, evidence-based justification) can indeed be improvedwith guided practice, as Kuhn (1991) suggested. A beneficial effect of peer-based dialogue in effecting this improvement is also suggested by the data.However, the improvements achieved were limited in scope (with participantsimproving in their use of evidence in the justification of their own argu-ments, but showing little evidence of advance in more sophisticated skillsof evidence use such as weighing evidence), and no evidence of transfer. Wedo not claim to have constructed a definitive thinking skills curriculum, butinstead claim to have demonstrated that some of the skills of argumentationidentified by Kuhn (1991) can be successfully enhanced within an existingcurriculum, resulting in students improving in their ability to use evidence inrelation to their assertions.

27

Acknowledgements

This research was supported by the Economic and Social Research Councilof Great Britain under grant number R000221801. We would also like togratefully acknowledge the help and cooperation given to us by the staff andstudents in the three participating colleges. Finally, we would like to thankJoanne Gurney and Simon Hunter for help with the statistical analyses, andthe three anonymous reviewers for their comments.

Appendix 1

Brief outline of the lesson contents in the sessions conducted in College C. In this college,all students studied the GSVQ in health care, and teaching focussed on health care issuesaccordingly.

Lesson 1. Introduction to problem solving, factors and goals (exposition and classdiscussion); example problems – consideration of relevant factors (completion ofconstructed example items) first thoughts by students about what they wanted to focustheir project on (homework).Lesson 2. Health promotion campaigns as a type of persuasive communication (exposi-tion and class discussion); importance of having good reasons for what is promoted bya campaign and introduction to the idea of reasoned argument (modelling); CRT pre-test. Choosing a health promotion action event and giving reasons for the event chosen(homework).Lesson 3. Examples of good and poor reasons for an argument; counter-argumentand rebuttal; being persuasive (exposition, modelling and class discussion). Choosinga health promotion action event and anticipating both counter-arguments against thechoice from the GSVQ assessors and counter-arguments against the recommendedactions from members of the target group (homework).Lesson 4. Evidence-giving and persuasion (exposition); details given to students of theproject outline (exposition); group discussion of example project outlines (group discus-sion of supplied examples); students told to begin writing their own project outlines; useof evidence (group discussion of relevant psychometric test items).Lesson 5. Use of evidence (group discussion of relevant sample psychometric testitemscontinued); completeness of evidence; evidence and persuasive argument (groupdiscussion of supplied examples).Lesson 6. Start of work on project plans; presentation of improved data from relevantsubsection of psychometric test (exposition); health promotion campaign events – groupdiscussion of own proposed campaign and of previous campaigns (structured peerinteraction).Lesson 7. Return of project outlines to students; justification in terms of practicability;consideration of the presentation aspect of the health promotion campaign; talks versusother forms of campaign; advantages and disadvantages of talks vis-a-vis other meansof persuasion; the structure of effective talks (exposition); group discussionof campaignactivities; setting timescales and recording progress (structured peer interaction).Lesson 8. The distinction between aims and objectives (exposition); students work onaims and objectives for their own projects (structured peer interaction).

28

Lesson 9. Project plan specification; identifying the project stages; specifying theactivities and information and resources needed; setting timescales; thinking about feed-back on effectiveness of campaign via questionnaires (exposition); group discussionofexample project plans; students commence work on own project plans (structured peerinteraction, with group discussion).

Lesson 10. Overview of previous teaching (exposition); readministration of CRT test.

Appendix 2

Sample materials for critiquing by students.GSVQ in Care: Social Care at Level IIIAdditional AssessmentProject Outline

Example 1

Specify the type of facility or service to be provided and its target client groupTake a group of elderly people away on a holiday.

Explain why you have made this selectionAn annual holiday is an enjoyable thing to have; it gives the clients of a residential carefacility for the elderly something to look forward to before it happens and something toreminisce about afterwards, and so helps reduce boredom.

Identify factors which will be important in providing this facility or service and how youwill deal with those factorsI will need to think where I would identify a particular target group.

I will need to find information about possible holidays.I will need to think about where I will take them to (I have been thinking about this, and

thought that somewhere like Benidorm might be suitable, given that the weather is good, andlots of people like it as a place to go on holiday.

I will need to think about how they will entertain themselves when they get there (I havethought about them going on coach trips to see the sights, like any other holidaymaker).

Discuss together the following questions:Can you think of anything relevant to this proposal that has been overlooked?Is the suggested project one which you are convinced is well justified and practicable?What assumptions were being made?Were these reasonable ones?

Additional AssessmentProject Outline

Example 2

Specify the type of facility or service to be provided and its target client groupTake a group of elderly people from a residential care facility away on a touring holiday bycoach to Yorkshire.

29

Explain why you have made this selectionA one-week annual holiday is something that I believe that everyone is entitled to. Theclients of a residential care facility for the elderly tend to spend much time in one physicallocation (their residential care home), and however comfortable it is and however enjoyablethe activities that go on there, they still have less stimulation and variety in their lives than theyounger working population. A holiday would therefore be a particularly welcome change intheir routine and a particularly worthwhile thing to organise.

Identify factors which will be important in providing this facility or service and how youwill deal with those factorsI obtained information from some local residential homes for the elderly about the kind ofclients they take in from their publicity materials, and this allowed me to identify a suitablegroup of clients. Since any holiday will involve travelling and staying in unfamiliar places,the group taken will necessarily be restricted to people who don’t have dementia and who arereasonably mobile. I talked to some elderly relatives and their friends about holidays that theyhad recently had and enjoyed, and from talking to them it seems that elderly people would bequite likely to enjoy a touring holiday in a coach, to somewhere within reasonable travellingdistance like the Highlands or Yorkshire. I finally settled on Yorkshire as the location, becauseon investigation it turns out that there are many more places of interest that can be visitedwithin reasonable travelling distances. For example, during the day, the clients can have coachtrips to visit places where they film popular TV series like Last of the Summer Wine, Heartbeatand Coronation Street. These are all of interest to the elderly and all within a couple of hours’travelling distance from anywhere in Yorkshire. In the evening, I would need to organise thekinds of entertainment that elderly people enjoy, like old-time dancing and sing-songs.

From the people I spoke to, I got the names of companies that specialise in holidays forthe elderly, and my next step will be to get their brochures – companies like SAGA, Shearingsand David Urquhart Travel. I also need to find out about the cost of such a holiday, and totalk to staff and clients at the residential homes, to find out how many of them are likely towant to come, given the approximate costings that I come up with. I might then be able tonegotiate a favourable rate with one of the companies given that a whole group of holidayswill be bought at once.

Discuss together the following questions:Can you think of anything relevant to this proposal that has been overlooked?Is the suggested project one which you are convinced is well justified and practicable?What assumptions were being made?Were these reasonable ones?

Appendix 3

Examples of assertions that are not justified, weakly justified, or strongly justified in thestudents’ written work (slightly parphrased in the interests of brevity).1) Unjustified assertions:

a) I promoted a Healthy Eating and Exercise campaign for Down’s Syndrome adultsand their carers.

b) Teenagers need to be made aware of the effects of drugs on their health.c) The aim of my campaign is to encourage children to be more safety conscious.

2) Weakly justified assertions (i.e. justified by experience-based generalisations):

30

a) I chose an anti-smoking campaign because, like myself, friends and neighbours areconcerned at the numbers of children who are starting to smoke.

b) My campaign is to try and reduce drug abuse amongst teenagers. I have witnessedfrom going out at weekends and from going to parties that drug abuse is morecommon, especially amongst teenagers.

c) I aim to raise awareness that breastfeeding is beneficial to the baby. I feel that manymothers are only given the basic facts on which method they choose to feed theirbaby.

3) Strongly justified assertions (i.e. justified by formal research evidence; students cite thesources of this evidence in their reports)a) Anorexia Nervosa is steadily on the increase. Research at one London hospital shows

that the number of people seeking treatment has increased by 360% over the years.b) There are far too many children killed or seriously injured in accidents which could

have been avoided. Every year 600 children die or are seriously injured and a further10 000 disabled.

c) The fact that 1 in 5 hospital beds are occupied by a patient with an alcohol relateddisease shows how serious alcohol abuse is.

References

Allen, C.L. (1992). Multimedia learning environments designed with organising principlesfrom non-school settings. In E. DeCorte et al., eds,Computer-Based Learning Environ-ments and Problem Solving(pp. 464–484). London: Springer-Verlag.

Anderson, A., Halliday, J., Howe, C. & Soden, R. (1997).Bridging the Academic/VocationalDivide by Integrating Critical Thinking. End of Award Report to ESRC (reserch grantnumber R000221801).

Anderson, A., Howe, C. & Tolmie, A. (1996). Interaction and mental models of physicsphenomena: Evidence from dialogues between learners. In J. Oakhill & A. Garnham,eds, Mental Models in Cognitive Science: Essays in Honour of Phil Johnson-Laird(pp. 247–273). Hove, Sussex: Psychology Press.

Audit Commission/OFSTED (1993).Unfinished Business: Full-time Educational Courses for16–19 Year Olds. London: HMSO.

Baron, J.B. & Sternberg, R.J., eds (1987).Teaching Thinking Skills: Theory and Practice. NewYork: W. H. Freeman and Co.

Billig, M. (1987). Arguing and Thinking: A Rhetorical Approach to Social Psychology.Cambridge: Cambridge University Press.

Brookfield, S.D. (1987).Developing Critical Thinkers: Challenging Adults to Explore Alter-native Ways of Thinking and Acting. Milton Keynes: Open University Press.

Carey, S. (1985). Are children fundamentally different kinds of thinkers and learners thanadults? In S. Chipman, J. Segal & R. Glaser, eds,Thinking and Learning Skills(Vol. 2).Hillsdale, NJ: Erlbaum.

Carey, S. (1986). Cognitive science and science education.American Psychologist4: 1123–1130.

Damon, W. & Killen, M. (1982). Peer interaction and the process of change in children’s moralreasoning.Merrill-Palmer Quarterly28: 347–367.

Detterman, D.K. (1993). The case for the prosecution: Transfer as an epiphenomenon. InD.K. Detterman & R.J. Sternberg, eds,Transfer on Trial: Intelligence, Cognition andInstruction(pp. 39–67). Norwood, NJ: Ablex.

31

Doise, W. & Mugny, G. (1984).The Social Development of the Intellect. Oxford: Pergamon.Ennis, R.H. (1987). A Taxonomy of Critical Thinking Dispositions and Abilities. In J.B. Baron

& R.J. Sternberg, eds,Teaching Thinking Skills: Theory and Practice. New York: W. H.Freeman and Co.

Feuerstein, R. (1980).Instrumental Enrichment: An Intervention Program for CognitiveModifiability. Baltimore: University Park Press.

Fischer, K. & Farrar, M. (1988). Generalizations about generalization: How a theory of skilldevelopment explains both generality and specificity. In A. Demetriou, ed,The Neo-Piagetian Theories of Cognitive Development: Toward an Integration. New York: ElsevierNorth-Holland.

Fisher, R. (1990).Teaching Children to Think. Simon and Schuster.Foot, H.C., Howe, C.J., Anderson, A., Tolmie, A.K. & Warden, D.A. (1994).Group and

Interactive Learning. Southampton: Computational Mechanics Publications.Further Education Funding Council (1998).Quality and Standards in Further Education in

England, 1997–1998. Coventry, England: Further Education Funding Council.Glaser, R. (1984). Education and thinking: The role of knowledge.American Psychologist39:

93–104.Howe, C.J., Rogers, C. & Tolmie, A. (1990). Physics in the primary school: Peer interac-

tion and the undestanding of floating and sinking.European Journal of Psychology ofEducation5: 459–475.

Howe, C.J., Tolmie, A. Anderson, A. & Mackenzie, M. (1992a). Conceptual knowledge inphysics: The role of group interaction in computer-supported teaching.Learning andInstruction2: 161–183.

Howe, C.J., Tolmie, A. & Rogers, C. (1992b). The acquisition of conceptual knowledge inscience by primary school children: Group interaction and the understanding of motiondown an incline.British Journal of Developmental Psychology10: 113–130.

Keil, F. (1984). Mechanisms in cognitive development and the structure of knowledge. In R.Sternberg, ed,Mechanisms of Cognitive Development. New York: Freeman.

Kuhn, D. (1991).The Skills of Argument. Cambridge: Cambridge University Press.Kuhn, D. (1993). Connecting scientific and informal reasoning.Merrill-Palmer Quarterly39:

74–103.Lochhead, J. (1985). Teaching analytic reasoning skills through pair problem solving. In J.W.

Segal, S.F. Chipman & R. Glaser, eds,Thinking and Learning Skills. Volume 1: RelatingInstruction to Research(pp. 109–131). Hillsdale, NJ: Erlbaum.

McGuinness, C. (1990). Talking about thinking: The role of metacognition in teachingthinking. In K.J. Gilhooly, M.T.G. Keane, R.H. Logie & G. Erdos, eds,Lines of Thinking,Volume 2. Chichester: John Wiley and Sons Ltd.

Miyake, N. (1987). Constructive interaction and the iterative process of understanding.Cognitive Science10: 151–177.

Paul, R.W. (1987). Dialogical thinking: Critical thought essential to the acquisition of rationalknowledge and passions. In J.B. Baron & R. Sternberg, eds,Teaching Thinking Skills:Theory and Practice(pp. 127–148). New York: Freeman.

Perkins, D., Allen, R. & Hafner, J. (1983). Difficulties in everyday reasoning. In W. Maxwell,ed,Thinking: The Expanding Frontier(pp. 177–189). Philadelphia, PA: Franklin Institute.

Perkins, D., Bushey, B. & Farady, M. (1986).Learning to Reason. Unpublished manuscript,Harvard University.

Reed, S.K. (1993). A schema-based theory of transfer. In D.K. Detterman & R.J. Sternberg,eds,Transfer on Trial: Intelligence, Cognition and Instruction(pp. 39–67). Norwood, NJ:Ablex.

32

Roazzi, A. & Bryant, P. (1998). The effects of symmetrical and asymmetrical social interactionon children’s logical inferences.British Journal of Developmental Psychology16: 175–181.

Siegel, H. (1988).Educating Reason: Rationality, Critical Thinking and Education. NewYork: Routledge.

Sinclair, J.McH. & Coulthard, R.M. (1975).Towards an Analysis of Discourse. London:Oxford University Press.

Smith, P. & Whetton, C. (1992).Critical Reasoning Tests. Windsor: NFER-Nelson.Weinstein, B.D. & Bearison, D.J. (1985). Social Interaction, social observation and cognitive

development in young children.European Journal of Social Psychology15: 333–343.Whimbey, A. & Lochhead, J. (1980).Problem Solving and Comprehension: A Short Course

in Analytical Reasoning. Philadelphia: Franklin Institute Press.Young, M. (1993). A curriculum for the 21st century? Towards a new basis for overcoming

academic/vocational divisions.British Journal of Educational Studies41(3): 203–222.