ASSIGNMENT REPORT FORM

Student ID

58117156

Tutor ID

Name:

Date Student sent

assignment:06/12/

2010

Date Received by

tutor:

Date Sent to

Oscail:

Overall Mark Obtained:

Module:PSY 4 Group: 1 TMA No:

Summary of Performance*

Bands

1

Performa

nce

Compone

nts

E

x

c

e

l

l

e

n

t

(

H

1

)

M

a

r

k

s

r

a

n

g

e

:

70-100%

V

e

r

y

G

o

o

d

(

H

2

.

1

)

M

a

r

k

s

r

a

n

g

e

:

6

0

-

6

9

%

G

o

o

d

(

H

2

.

2

)

M

a

r

k

s

r

a

n

g

e

:

5

0

-

5

9

%

F

a

i

r

(

H

3

)

M

a

r

k

s

r

a

n

g

e

:

4

0

-

4

9

%

W

e

a

k

M

a

r

k

s

r

a

n

g

e

:

35-39%

P

o

o

r

M

a

r

k

s

r

a

n

g

e

:

b

e

l

o

w

3

5

%

N

o

t

a

p

p

l

i

c

a

b

l

e

2

Attention to assignment taskAnalysisStructureUse of sourcesReferencesIntroductionConclusionSpelling/GrammarPresentation (Style)

* This table facilitates the assessment of your performance in selected components of your assignment, and is

designed to alert you to the general areas of strength and/ or in need of improvement in your work. Please note that

the components are not equal in terms of contribution to your overall mark. Attention to Assignment Task,

Analysis and Structure are the three most important criteria for assessment. Please note that the total mark

indicated is based on an evaluation of your overall performance in the set assignment.

SUMMARY COMMENTS

ADVICE FOR FUTURE ASSIGNMENTS

ANNOTATED FEEDBACK

(Refer to Assignment for the sections relating to the following comments)

1.

2.

3

An Experimental Approach to Analyse Success Rates in Concrete and

Abstract Selection Tasks

Charles Brand

Dublin City University

4

Abstract

The aim of this study was to investigate whether people are more successful

in a reasoning task when rules involve thematic content and materials than

when the task involves abstract terms and relationships between objects. As

demonstrated by Johnson-Laird et al (1972) a higher success rate in

selection tasks was observed when realistic objects, terms and relations

were utilised in the selection test. The abstract condition test as developed

by Wason (1966) was shown to have a lower success rate than the concrete

condition test developed by Johnson-Laird et al (1972). This study required

both the concrete and abstract tests to be recreated and administered.

Results for 99 participants (n=99) were analysed. Participants ranged in age

from 18-65. Results showed a significantly higher success rate for the

concrete condition test (60%).

5

Introduction

Early psychological theorist Ferrero (1895) studied the analytical process of

human reasoning and logic, by way of “...reflection and comparison”

(Gardiner, 1896 p. 237) and referred to this process as “arrested mentation”

(Ferrero cited in Gardiner, 1896 p. 237). Santrock (2001) defines logic as

“...the science of thinking” (p. 422). The relationship between logic and

thinking is not inter-dependent however as every person is capable of

thinking without always necessarily being logical. The place and importance

of logic within human thought processes is not generally debated, however

identifying a predominant form of reasoning and how best to model this in

relation to our cognitive architecture has interested many researchers in the

field of cognitive psychology.

Reasoning and logic are psychological by-products of thought in humans

and both processes utilise our memory stores, or knowledge. Inferences are

the means by which we use our knowledge to make decisions and

judgements about situations based on that knowledge (Evans, Newstead &

Byrne, 1993). The use of inferences is central to our reasoning capacity and

ability. Deductive inferences, a type whereby their “...conclusions

necessarily follow from their premises or assumption” (Evans, Newstead &

Byrne, 1993) are the basis for deductive reasoning. When investigating

these concepts psychological research makes a distinction between explicit

and implicit inferences. Implicit inferences relate to the internal and

unconsciousness processes that facilitate people's comprehension and

perception, whereas explicit inferences are the slower, voluntary and more

obvious processes that underlie people's problem solving methodology

(Johnson-Laird & Wason, 1977). Within experimental cognitive psychology it

is the explicit kind of inference that is usually under examination.

Early cognitive psychological experiments in reasoning and logic (see

Wordsworth & Sells, 1935) utilised a syllogistic reasoning form in order to

examine the explicit inferential processes that people employed. The

syllogisms they utilised for the experimental task were classic in form:

Some A are B

6

No C are B

Therefore

Some C are not A.

Participants in these experiments were usually asked to demonstrate

whether or not they understood the logical validity of the syllogistic

argument presented to them. This experimental methodology is still

prevalent in cognitive research today as the primary means of investigating

reasoning and logic processes in humans (Evans, Newstead & Byrne, 1993).

One reason for using a conditional format (if...then) to pose questions to

participants in experimental settings, as Wason (1966) did, is that it

minimises contextualisation of the proposition or the introduction of beliefs

on the part of the participants (Verschueren et al, 2006).

The Wason selection test (1966) examines a participants' reasoning skills by

asking them to verify that a statement was true or false from evidence

presented to them on 4 cards with numbers on one side and letters on the

other. A minimum of two cards must be turned over in order to correctly

verify the truth or falsity of the statement. Wason (1960) had conducted

earlier experiments in relation to inductive reasoning. During these

experiments he developed his concept of confirmation bias. This describes a

cognitive deficit whereby people tended to search for evidence to confirm

their hypotheses or beliefs and avoided that which did not. In the selection

test Wason attempted to illustrate the confirmation bias cognitive deficit.

The type of reasoning utilised for the experiment was proposition logic (if p

then q) and its main aim was to observe if participants would apply a modus

tollens reasoning pattern, a method by which the falsity of a statement can

be deduced (Poletiek, 2001). Wasons' (1966) selection test demonstrated

that his earlier theory on confirmation bias was valid and less than 10% of

participants chose the correct two cards.

Johnson-Laird, Legrenzi & Legrenzi (1972) argued that the abstract nature of

the instructions, context and materials used Wason's experiment made

success in the task unduly difficult. They proposed that using more

recognisable materials (envelopes and stamps) and introducing a more

concrete context (a postal worker scenario) instead of the abstract

7

number/letter combination used by Wason would yield a higher rate of

success. Their experiment was shown to support their hypothesis with a

62.5% success rate and Griggs & Cox (1982) labelled this as the thematic

materials effect. Adaptations of both the abstract and concrete selection

experiments have been developed to probe a variety of hypotheses

involving logic and reasoning. These include psychological investigations

into the reasoning structures of pathological participants (see Dudly et al,

1998), the development of the field known as conditional reasoning

research (see Verschueren, Schaeken, & Schroyens, 2006; Beller &

Kuhnmünch, 2007) as well as Wason's own modification of the original

basic form of the experiment using thematic materials (See Wason &

Shapiro, 1971).

The aim of this study was to verify that people will more often successfully

reason in a logical manner when the rules involve thematic content and

materials, as demonstrated by the Johnson-Laird et al (1972) experiment

than in the abstract condition test as developed by Wason (1966). This

required both the concrete and abstract tests to be recreated and

administered to a population sample in order to support or reject the

following hypothesis:

H1

There should be a significant difference between the rate of success on the

concrete selection task and the rate of success on the abstract selection

task.

Method

Design

An independent groups design was used and participants were randomly

assigned to either the abstract and concrete groups. Participants were

administered the concrete or abstract selection tests and the success rate of

8

each participant was then recorded. The time taken by each participant to

complete their task was also recorded.

Participants

A subset of 10 participants comprised of a self-selected sample of 10 friends

and relatives of the researcher. Of the subset, 5 participants completed the

abstract test and 5 participants completed the concrete test. Participants

were made aware before taking the tests that it was part of a psychological

experiment.

Male = 6 (4 abstract condition & 2 concrete condition).

Female = 4 (1 abstract condition & 3 concrete condition).

Age Range = 21-63

Participants in the subset received no incentive in order to take part in the

experiment. Occupational status of participants in the subset were as

follows:

Unemployed = 1

Student = 1

Professional musician = 1

Engineer = 1

Publican = 1

Home Makers = 2

Property Developer = 1

Sales Executive = 1

University Lecturer/Researcher = 1

Participants in the remainder of the sample comprised a self-selected

sample of 89 friends and relatives of the students from the Oscail Cognitive

Psychology class, Group 1.

Males = 37

Females = 55

Age Range = 18-65

9

Results of subset tests were incorporated into the over-all sample and

yielded the follow results:

Abstract condition = 49 participants.

Concrete condition = 50 participants.

Materials

See details see Appendix B

Procedure

Participants were tested concurrently and in the same room. As each

participant entered the room they were seated and the following statement

was made by the researcher in order to address any pre-test anxiety with

regard to under performance on the test:

“Thank you for agreeing to take part in this psychological experiment. This

test is a cognitive test concerned with finding out about how people use

logic and reasoning to solve problems. The test you are about to take has

been used to evaluate this issue for over 40 years and literally millions of

people have taken it. While I'm sure that you will be trying to get the correct

solution to the test, I would like to stress that failure or success is not the

primary question that the experiment is concerned with. This research is

mainly concerned with why people reason in the way that they do.”

Participants were then requested to sign a consent form (see appendix B).

Participants were allocated to either the abstract or concrete test by means

of a lottery using the relevant aforementioned materials.

The materials for both the abstract and concrete tests were pre-prepared on

the table, facing the participant and under a large enough cloth sheet to

cover both sets of test materials. Upon allocation to each test as per the

lottery procedure, the instruction sheet for each test was then placed in

front of the participant face down.

Abstract test

The following instruction was given to each participant:

10

“Please turn over the [statement] sheet, read the instructions and when

you're satisfied that you understand the instruction let me know.”

Once the participant indicated they understood the instruction on the

instruction sheet the cloth sheet was folded back to reveal the set of

abstract cards. The stopwatch was then started. The stopwatch was stopped

once they indicated that they had turned over as many cards that they felt

had verified the statement on the statement sheet to be either true or false.

The configuration of the cards was as follows:

Corresponding numbers or letters on the opposite side:

8 5 E Q

Concrete test

The following instruction was given to each participant:

“We're going to pretend that you work in a post office sorting depot. You are

seated in front of a conveyor belt and your job is to inspect letters as they

pass to ensure they adhere to certain rules about the type of stamps that

are put on envelopes depending on whether they're open or closed. On the

instruction sheet is a rule that has to be verified”. Once the participant

indicated they understood the instruction on the instruction sheet the cloth

sheet was folded back to reveal the set of envelopes. The stopwatch was

then started. The stopwatch was stopped once they indicated that they had

chosen as many envelopes that they felt had verified the statement on the

statement sheet to be either true or false. The configuration of open/closed

envelopes, stamp showing/stamp not showing and stamp type was as

follows:

Visibly Sealed

Stamp face down

Visibly Unsealed

Stamp face down

Not visibly Unsealed

Stamp face up

Not visibly Sealed

Stamp face up

11

55c 10c 55c 5c

Many participants in both test types looked to the researcher for

confirmation of the correctness of their choices after selecting one, two

three or all four cards or envelopes by asking “Is that right?” or “do I need

to keep going?”. If this happened the researcher said “If you're satisfied that

your current selection verifies or falsifies the statement on the statement

sheet then the test is over”.

Once the stopwatch was stopped participant details such as time taken to

complete the test, age, success rate and occupational status were recorded.

Participants were then invited to ask any questions they may have about

the particular test they completed or the experiment in general. Once this

discussion was complete a debriefing sheet (see appendix C) was given to

each participant.

Results

In order to successfully verify the truth or falsity of the statement on the

statement sheet for both tests 2 cards or envelopes must to be turned over

and checked by participants. Data from 99 participants was included in the

analysis. Success and failure rates for each of the tests are

included in Table 1 and Figure 1:

Table 1 - Success and Failure Rates for Abstract and Concrete Conditions

Test Condition Number of Participants Number Successful Number Failed

Abstract 49 4 45

Concrete 50 32 18

12

Figure 1 - Success and Failure Rates for Abstract and Concrete Conditions

The failure rate for the abstract condition was 60% higher than that of the

concrete failure rate. The success rate for the concrete condition was 87.5%

higher than that of the abstract condition.

The frequency of the number of cards/envelopes chosen by each participant

for the each selection test is detailed in Table 2, Table 3 and Figure 2. For

both the concrete and abstract conditions combined, participants selected,

on average, the correct number of cards/envelopes on 36.5% of tests.

Table 2 - Number of Cards Turned by Frequency

Abstract Condition - Number of Cards Turned by Frequency

1 card 11

2 cards 31

3 cards 1

4 cards 6

total 49

Table 3 - Number of Envelopes Turned by Frequency

13

4

32

45

18 Abstract Condition Success

Concrete Condidtion Success

Abstract Condition Failure

Concrete Condidtion Failure

Concrete Condition - Number of Envelopes Turned by Frequency

1 card 6

2 cards 42

3 cards 2

4 cards 0

total 50

Figure 2 - Number of Cards and Envelopes Turned by Frequency

The time ranges participants for completion of the tests in the overall

sample are shown in Table 4

Table 4 – Time Ranges for Completion of Each Condition

Abstract time range 15-660 seconds

Concrete time range 2-700 seconds

Mean times for completion of the tests are shown in Table 5

Table 5 – Mean Times for the Completion of Each Condition

Abstract 155.02

Concrete 140.06

14

1 Card/envelope 2 cards/envelope 3 cards envelopes 4 cards/envelopes

0

5

10

15

20

25

30

35

40

45

Concrete Condition

Abstract condition

Nu

mb

er

of

Pa

rtic

ipa

nts

A Mann-Whitney U Test was conducted on the overall success rate for the

participants in both the abstract and concrete conditions. The participants

in the concrete condition were significantly more successful at choosing the

correct cards than were those in the abstract condition U(97) = 541.00, z=

-5.75, p ,.001 two-tailed.

Ranks

Condition N Mean Rank Sum of Ranks

Overall Success

dimension1

Abstract 49 63.96 3134.00

Concrete 50 36.32 1816.00

Total 99

Test Statistics

Overall Success

Mann-Whitney U 541.000

Wilcoxon W 1816.000

Z -5.745

Asymp. Sig. (2-tailed) .000

a. Grouping Variable: Condition

Discussion

Figure 1 shows a significantly higher rate of success for participants in the

concrete condition test than for participants in the abstract condition test.

While a significant majority of participants selected the correct number of

cards/envelopes in both the abstract and concrete conditions, a higher

proportion of participants in the abstract condition were more likely to

choose either too few or too many cards as shown in Figure 2. Also, the total

completion time ranges as shown in Table 4 indicate that some participants

in the abstract condition required less overall time to complete the test.

However, once the mean time for completion, as shown in Table 5 and the

minimum time required for some participants in the concrete test, as shown

in Table 4 is considered, it can be observed that participants in the concrete

15

condition took less time on average to complete the test and some took up

to 60% less time to complete the test.

The Wason (1966) card selection test and envelope selection test developed

by Johnson-Laird et al (1972) were recreated in order to verify that the use

of thematic materials would increase the success rate of the participants in

this condition. A significant difference was observed between success rates

for the concrete and abstract selection tests with a statistical chance of the

difference being due to chance being less than 1% (P>00.1). The resultant z

value (z> 1.96) suggest that the null hypothesis can be rejected.

The higher success rate for the concrete test is in-line with similar

experimental results (see johnson-Laird et al, 1972). The possible causes of

a higher success rate in this study may have been due to a number of

possible factors and is not conclusive. The question considered relevant in

order to better understand possible reasons for the difference in success

rates was: whether it was the terms or relationships between the terms

used in the experiments i.e. envelopes and stamps as opposed to letters,

vowels and consonants that facilitated a higher success rate in the concrete

condition test (Johnson-Laird & Wason, 1977). Bracewell & Hidi (1974) found

that only the relation between terms that provided insight to the

participants and could be responsible for the higher success rate. Gilhooly &

Falconer (1974) found opposing evidence in their experiment which

suggested that the realistic nature of the terms was a more important factor

in participants successfully completing the concrete condition test. Van

Duyne (1976) somewhat reconciled these dichotomous experimental results

by suggesting that a more complex process was involved that introduced

degrees of realism which affected participants performance.

The impact of these factors on the results of this study are, however

mitigated by a lack of test reliability and procedural consistency.

Participants in this study varied in the number of cards and envelopes they

turned over before they were satisfied that they had verified that the

statement on the statement sheet. Due to a lack of complete procedural

data pertaining to the order in which the cards and envelopes were placed

16

in front of all participants, no statistical information can be calculated in

relation to the frequency of particular selection combinations or sequence of

selections. A number of different researchers were responsible for

conducting all of the tests but without a single definition of test

requirements or test materials e.g. type of timing device, when to begin and

end the timing sequence. Test location was also highly varied and

conceptual problems regarding the methodology of administering the test to

participants were also observed. Ethical concerns regarding the nature of

the tests aired by some researchers may have distorted the delivery of

instructions to participants and due to the fact that family members of

researchers were used as participants may have also introduced anomalous

behaviour on the part of the researchers, All these confounds render the

test results unreliable.

References

Bracewell, R. J. and Hidi, S. E.(1974) The solution of an inferential problem

as a function of stimulus materials. The Quarterly Journal of Experimental

Psychology, 26 (3), 480 -488.

Beller, S., Kunmünch, G. (2007) What Causal Conditional Reasoning Tells

Us About People’s Understanding of Causality. Thinking and Reasoning, 13

(4), 426 – 460.

Evans, J. St., B. P., Newstead, E., Byrne, R. M. J.(1993) Human reasoning:

The psychology of deduction. Hove, England:Erlbaum.

Gardiner, H. N. (1896) Psychological literature: Arrested mentation.

[Review of the article Ferrero, G. (1895). Arrested Mentation. The

Monist, Vol. 6, 60-75.] October, 1895 Psychological Review, 3(2), 237.

Gilhooly, K.J., Falconer, K.A. (1974) Concrete and Abstract Terms and

Relations in Testing a Rule. The Quarterly Journal of Experimental

Psychology, 26, (3), 355-359.

17

Griggs, R. A., Cox, J. R. (1982) The elusive thematic-materials effect in

Wason's selection task. British Journal of Psychology, 73(3), 407-420.

Johnson-Laird, P.N., Legrenzi, P., Legrenzi, M.S.(1972) Reasoning and a

sense of reality. British Journal of Psychology,63(3), 395-400.

Johnson-Laird, P.N., Wason, P.C. (1977). Reasoning with Quantifiers In

P.N. Johnson-Laird & P.C. Wason (Eds) Thinking: Readings in Cognitive

Science. Bath, U.K.:Cambridge University Press. pp. 307-326.

Poletiek, F. (2001) Hypothesis-testing behaviour-Essays in cognitive

psychology. East Sussex, U.K.:Psychology Press.

Santrock, R.L. (2001). Cognitive Psychology. Boston, MA:Allyn & Beacon.

Van Duyne, P.C. (1974) Realism and Linguistic Complexity in Reasoning.

British Journal of Psychology, 65(1), 59-67.

Verschueren, N., Schaeken, W., Schroyens, W. (2006). Necessity and

Sufficiency in Abstract Conditional Reasoning. European Journal of

Cognitive Psychology, 18 (2), 255-276.

Wason, P.C. (1960). On the failure to eliminate hypotheses in a conceptual

task. Quarterly Journal of Experimental Psychology, 12, 129-140.

Wason, P.C. (1966). Reasoning. In Foss, B. M. (Eds), New horizons in

psychology. Harmondsworth, U.K.:Penguin.

Wason, P.C., Shapiro, D. (1971). Natural and Contrived Experience in a

Reasoning Problem. Quarterly Journal Of Experimental Psychology, 23, 63-

71.

18

Appendix A

LETTER OF CONSENT

Re: Wason Selection Test – Humanities Degree Program (HONS) Oscail Psychology 4 Research

This research is being undertaken by Charles Brand, who is a student at Dublin City

University, Dublin 9.

The study will examine the logical thinking and reasoning by way of a card selection

test. The test will take no longer than 10 minutes and involves nothing more than

participants selecting either cards or envelopes from a selection.

If you agree to participate in this study your identity will be coded for anonimity

purposes and the information that you provide will be destroyed immediately after

inputting into a dataset. You will be at liberty to withdraw from the study at any time

without prejudice. Your rights under the Freedom of Information Act 1977 (amended

2003) will be respected at all times.

Charles Brand,

c/o Oscail, Dublin City University, Dublin 9.

e-mail: Charlesbrand@gmail.com

Please place an X in the box to indicate that you have read and understood the above and that you

agree to participate in this study.

19

Appendix B

Materials required for the concrete test consisted of:

4 plain white envelopes

4 postage stamps – 2 stamps 55c, 1 stamp 10c, 1 stamp 5c.

1 A4 plain white paper sheet containing the following statement: “Which

envelope(s) would you choose to verify if the following rule is true? 'If the

envelope is sealed, then it must have a 55c stamp on it'”.

Materials required for the abstract test consisted of:

4 plain white A3 cards with numbers on one side and and letters on the

opposite side in the following configuration:

A:8 B:5 4:E 7:Q

1 A4 plain white paper sheet containing the following statement: “Which

card(s) must be turned over to verify that the following statement is true?

'If a card has a vowel on one side, then it has an even number on the other

side'”.

1 consent form per participants.

1 debriefing form per participant.

1 stopwatch to record completion times for all test attempts by participants.

A software model was used at http://http://www.online-stopwatch.com/

1 laptop (model Dell Latitude E6400I RAM-2Gb. Processor Core Duo

P8600@2.4 Ghz Centrino. Operating system - Windows Vista Business).

1 table big enough to hold both sets of test cards and envelopes.

1 dark coloured cloth sheet, big enough to cover the table.

5 post-its© containing the letter C and 5 post-its© containing the letter A

for test type allocation lottery.

All participants were tested in the same room which was brightly lit. All

participants sat opposite the researcher who was seated with their back to a

dark coloured wall with no windows.

20

Appendix C

Debriefing Sheet

Thank you for participating in this research.

The study examined how people use their logic and reasoning skills in relation to

concrete and abstract problems.

_

The test that you completed was developed in order for psychologists to better

understand how we utilise different aspects of our knowledge to reason and perform

logical tasks. Success or failure by the participants in the test is not the only or

primary concern. The process by which participants come to their conclusions are of

most relevance to helping improve our understanding of our mental processes.

Further information and results can be obtained by contacting

Charles Brand,

c/o Oscail, Dublin City University, Dublin 9.

e-mail: Charlesbrand@gmail.com

Should you have been affected by any of the issues touched upon in this study, the following services are

available to you:

Samaritans – jo@samaritans.org Tel: Lo-Call - 1850 609090

21