Curriculum evaluation noted_25

Critical Evaluation at the Curriculum Level

James Atherton1 December 2010

http://bedspce2.blogspot.com/

Much evaluation takes place at the level of the taught session or a teaching sequence; this lecture will concentrate on critical evaluation of curricula as a whole.

“Critical” in this context means making use of some kind of external framework to throw certain features into relief.

Gibbs’ cycle

DescriptionWhat happened?

ConclusionWhat else couldyou have done?

Action PlanIf it arose again,

what would you do?

FeelingsWhat were you

thinking and feeling?

EvaluationWhat was good and

bad about the experience?

AnalysisWhat sense

can you makeof the situation?

Just a reminder of the components of an evaluation and

development cycle, after

Graham Gibbs

• Incidentally, do read Gibbs G (2010) Dimensions of Quality York; Higher Education Academy (available on-line for free—link is on blog)

• It’s one of the best thingson evaluation (albeit inHE), to appear foryears.

Brookfield’s four “lenses”

There’s nothing exclusive about choosing these particular lenses or perspectives. It is their multiplicity which is important.

Brookfield S D (1995) Becoming a Critically Reflective Teacher San Francisco; Jossey-Bass

Four lenses

•Autobiographical•Student

•Peer

•Theoretical

Four lenses

•Autobiographical•Student

•Peer

•TheoreticalThis lens is the default for most of us—seeing from the perspective

of our own experience as the teacher.

Four lenses

•Autobiographical

•Student•Peer

•Theoretical

There has recently been an emphasis

on the “student experience”

particularly in HE

This is the formalcurriculum

This is the formalcurriculum

This is broadly where it works; you set out to teach something ,

and the students learn it

This is the formalcurriculum

This is broadly where it works; you set out to teach something ,

and the students learn it

This is what you teach but the

students do not learn; it’s a waste of

time

This is the formalcurriculum

This is broadly where it works; you set out to teach something ,

and the students learn it

This is what you teach but the

students do not learn; it’s a waste of

time

And this is the hidden

curriculum; stuff you did not set out to

teach, but the students learned

anyway.

•All social practices have a sub-text, or “send a message”.

•… usually about values and relationships

•There is no way to avoid these messages The only question is whether they are

“good” messages or “bad” ones

Conveyed by practically everything that goes on apart from the formal taught material:• from the quality of the furnishings• to the assessment regulations• and accessibility ...They all say something about what the institution thinks of its students... And that interacts with the formal curriculum...

Formal/intentional message

Covert/unintended message

“Good”: covert message

reinforces formal message

“Bad”: covert messagecontradicts/underminesformal message

Irrelevant/neutral

• Bowles and Gintis: suggest that the systems of US education have developed to socialise children to join a capitalist workforce

• Hunter: Suggests that many values associated with schooling were not deliberately developed, but are the by-products of the “social technology” of the institution

• Illich: argues that the values of the school are inimical to education

• Becker, Snyder: see the most effective training undertaken in colleges is how to survive the system

• Apple: deconstructs practices (and materials) to expose embedded values

Commentators—go back a long way in uncovering the hidden

curriculum

See blog for more detail/links

Four lenses

•Autobiographical

•Student

•Peer•Theoretical

We teach as part of a

community of practice, living with all the tensions that entails. Assumptions about teaching and learning… Concern for inclusivity, and for standards

Based on Wenger E (1998) Communities of Practice Cambridge; CUP p. 63

Participation

Reification

meaning

worldexperience

negotiation

living in the world

membershipacting

interacting

mutuality

forms

points of focus

documentsmonuments

instrumentsprojection

This is Etienne Wenger’s take on a community of

practice. Much of it is not formally designed as an

organisation; it develops organically...

Based on Wenger E (1998) Communities of Practice Cambridge; CUP p. 63

Participation

Reification

meaning

worldexperience

negotiation

living in the world

membershipacting

interacting

mutuality

forms

points of focus

documentsmonuments

instrumentsprojection

...out of the interaction of participants, and the way in which procedures take on a life of their own and may even take physical form (reification)

Hunting Assumptions

Assumption 1• It’s common sense to cut lecturing down to a minimum, since

lecturing induces passivity in students and kills critical thinkingAssumption 2• It’s common sense that students like group discussion because

they feel involved and respected in such a setting. Discussion methods build on principles of participatory, active learning.

Assumption 3 • It’s common sense that respectful, empathic teachers will

downplay their position of presumed superiority and acknowledge their students as co-teachers.

Etc….(Brookfield, 1995)

In the same vein, Brookfield encourages us to explore the

taken-for-granted assumptions implicit in the curriculum and pedagogic approach: where’s the evidence for all this stuff? What’s it say about us that we

believe it?

Espoused theoriesand theories-in-use

Argyris and Schön differentiate between

•espoused theories: what people say they are doing, and

•theories-in-use: what they are “in fact” doing, as it might appear to an informed outsider

So what are the theories-in-use in the curriculum?Note that theories-in-use are often regarded as “inferior” to espoused theories, both technically and morally. But sometimes they are actually better—we just don’t know how to explain and communicate them.

Working myths

•When theories-in-use are articulated

•usually as stories•they become taken-for-granted

“working myths”

Ideas are shared in communities of practice not as formal theories,

but as stories which embody values

Four lenses

•Autobiographical

•Student

•Peer

•TheoreticalWe’ve touched on this

lens, implicitly, but as the preceding point about

stories makes clear, it’s perhaps the least important one...

Kirkpatrick

1. Reaction

2. Learning

3. Behaviour

4. Impact (results)

Kirkpatrick D and Kirkpatrick J (2006) Evaluating Training

Programs: The Four Levels (3rd edn.) NY; Berrett-Koehler

To end with a reminder of what you already know, just as we started with one.

Kirkpatrick

1. Reaction

2. Learning

3. Behaviour

4. Impact

Kirkpatrick D and Kirkpatrick J (2006) Evaluating Training

Programs: The Four Levels (3rd edn.) NY; Berrett-Koehler

Did the students/participa

nts enjoy it?

Kirkpatrick

1. Reaction

2. Learning

3. Behaviour

4. Impact

Kirkpatrick D and Kirkpatrick J (2006) Evaluating Training

Programs: The Four Levels (3rd edn.) NY; Berrett-Koehler

Did the students/participa

nts enjoy it?

From evaluation

instruments

Kirkpatrick

1. Reaction

2. Learning

3. Behaviour

4. Impact

Kirkpatrick D and Kirkpatrick J (2006) Evaluating Training

Programs: The Four Levels (3rd edn.) NY; Berrett-Koehler

Did they actually learn

anything?

Kirkpatrick

1. Reaction

2. Learning

3. Behaviour

4. Impact

Kirkpatrick D and Kirkpatrick J (2006) Evaluating Training

Programs: The Four Levels (3rd edn.) NY; Berrett-Koehler

Did they actually learn

anything?

From assessment

results

Kirkpatrick

1. Reaction

2. Learning

3. Behaviour

4. Impact

Kirkpatrick D and Kirkpatrick J (2006) Evaluating Training

Programs: The Four Levels (3rd edn.) NY; Berrett-Koehler

Do they do anything

differently?

Kirkpatrick

1. Reaction

2. Learning

3. Behaviour

4. Impact

Kirkpatrick D and Kirkpatrick J (2006) Evaluating Training

Programs: The Four Levels (3rd edn.) NY; Berrett-Koehler

Do they do anything

differently?

From observation

of subsequent

practice

Kirkpatrick

1. Reaction

2. Learning

3. Behaviour

4. Impact

Kirkpatrick D and Kirkpatrick J (2006) Evaluating Training

Programs: The Four Levels (3rd edn.) NY; Berrett-Koehler

Does that make any difference to the performance

of the organisation?

Kirkpatrick

1. Reaction

2. Learning

3. Behaviour

4. Impact

Kirkpatrick D and Kirkpatrick J (2006) Evaluating Training

Programs: The Four Levels (3rd edn.) NY; Berrett-Koehler

Does that make any difference to the performance

of the organisation?

From ?

http://bedspce2.blogspot.com/