www.kent.ac.uk

Six Guiding Principles

BRACE (Building Research in Australasian Computing Education)

First Workshop, Dunedin, 23-26 January 2004

2

A good way to spend $20 …

Scientific Research in EducationRichard J. Shavelson & Lisa Towne, Editors

2002, National Academy Press, Washington DC

Committee on Scientific Principles for Education Research, National Research Council

http://www.nap.edu/books/0309082919/html/

3

The six principles …

• Pose significant questions that can be answered empirically

• Link research to relevant theory• Use methods that permit direct investigation of the

question• Provide a coherent and explicit chain of reasoning• Replicate and generalize across studies• Disclose research to encourage professional scrutiny

and critique

4

For this workshop

• For this workshop, we’ll be most concerned with the first three – next year, the last three will come into their own

5

Especially for now: Link research to relevant theory

All these words/phrases are used in this area• Theory• Explanatory Theories• Empirical Laws• Models (sometimes “explanatory models”)• Conceptual Frameworks (sometimes “conceptual

lenses” – which I actually prefer)• Intermediate concepts

6

Unpicking vocabulary (i)

Theory• Understanding that will generalise across situations or

events. Predictive. Characteristic of Natural Sciences

“Modern science is a gloriously imaginative way of describing things, brilliantly successful for the purpose for which it was developed - namely, predicting and controlling phenomena.”

7

Unpicking vocabulary (ii)

• Explanatory Theories (also called “mid-range” theories)

The purpose of an explanatory theory is simply to explain observed human behaviour (i.e. it is not predictive) – although generalized theoretical understanding is still a goal. “Deep” and specific. Characteristic of Social Sciences.

8

Unpicking vocabulary (iii)

Empirical Laws• Offer simple quantitative predications of human performance.

Largely derived from Cognitive Science. (Medical statistical evidence fits here, too)

Models• Generalized, hypothetical descriptions, often based on an

analogy, used to analyse or explain something not directly observable. eg molecules, culture

Conceptual Frameworks/Conceptual Lenses• Conceptual frameworks define a particular point of view within a

discipline from which the researcher focuses his or her study. This "theoretical perspective" identifies underlying assumptions from which particular kinds of questions are generated.

9

Unpicking vocabulary (iv)

Intermediate ConceptsIntermediate concepts are intermediate in that they are between

concepts (theories, theoretical principles, conceptual lenses) and empirical observations, materials, and data. An intermediate construct is not given at the beginning of research but rather is built from what researchers come to know in the field - how concepts “live”, how they are situated in multiple contexts. Intermediate concepts are means for reciprocally making sense of field research and making sense of concepts in relation to empirical research and theory-building … intermediate concept construction contributes to the basis for generalization from particularized qualitative case examples.

Judith Gregory Activity Theory in a “Trading Zone” for Design Research and Practice (Doctoral Education in Design Conference, La Clusaz 9-12 July 2000)

10

So? So why are you saying all this?

• The kinds of question you can ask are (partly) dictated by your “philosophical” stance

• This underpins (and constrains)• what kinds of question you can ask• what is a legitimate question• what is an appropriate question

• And• what questions are allowable

A social psychologist and an epidemiologist on an airplane …

11

How does this fit with CSEd Research?

• Two interesting thoughts:• relationship between phenomena & evidence• the “trading zone”

12

Phenomena & evidence (i)

• When a picture of a sub-atomic particle trail is recorded in a cloud chamber are we making a phenomenon visible, or producing evidence of its existence?

13

Phenomena & evidence (ii)

• When Edward Jenner made case notes on patients afflicted with cowpox & smallpox was he recording phenomena concerning Sarah the dairymaid and George the cattleman? or producing evidence of the existence of a link between these diseases?

14

Evidence & CSEd

• Much of CSEd (called “research” or not) has been concerned with noticing phenomena• “This is what happens in my classroom”• “This is what happens when you teach x in this way”• “If I teach x differently, something else happens”

• What moves recognition of phenomena to evidence is purposeful investigation & relationship of that to “theory”. Can we do that?

15

The Trading Zone

I intend the term “trading zone” to be taken seriously, as a social, material, and intellectual mortar binding together the disunified traditions of experimenting, theorizing, and instrument building [in subcultures of physics]. Anthropologists are familiar with different cultures encountering one another through trade, even when the significance of the objects traded-and of the trade itself-may be utterly different for the two sides

(Peter Galison Image and Logic: A Material Culture of Microphysics, University of Chicago Press1997: 803).

16

The Peter Schaghen letter

High and Mighty Lords,

Yesterday the ship the Arms of Amsterdam arrived here. It sailed from New Netherland out of the River Mauritius on the 23d of September. They report that our people are in good spirit and live in peace. The women also have borne some children there. They have purchased the Island Manhattes from the Indians for the value of 60 guilders. It is 11,000 morgens in size [about 22,000 acres]. They had all their grain sowed by the middle of May, and reaped by the middle of August They sent samples of these summer grains: wheat, rye, barley, oats, buckwheat, canary seed, beans and flax. The cargo of the aforesaid ship is:7246 Beaver skins178½ Otter skins675 Otter skins48 Mink skins36 Lynx skins33 Minks34 Weasel skins

Many oak timbers and nut wood. Herewith, High and Mighty Lords, be commended to the mercy of the Almighty,

Your High and Mightinesses' obedient, P.Schaghen

17

The Peter Schaghen letter

High and Mighty Lords,

Yesterday the ship the Arms of Amsterdam arrived here. It sailed from New Netherland out of the River Mauritius on the 23d of September. They report that our people are in good spirit and live in peace. The women also have borne some children there. They have purchased the Island Manhattes from the Indians for the value of 60 guilders. It is 11,000 morgens in size [about 22,000 acres]. They had all their grain sowed by the middle of May, and reaped by the middle of August They sent samples of these summer grains: wheat, rye, barley, oats, buckwheat, canary seed, beans and flax. The cargo of the aforesaid ship is:7246 Beaver skins178½ Otter skins675 Otter skins48 Mink skins36 Lynx skins33 Minks34 Weasel skins

Many oak timbers and nut wood. Herewith, High and Mighty Lords, be commended to the mercy of the Almighty,

Your High and Mightinesses' obedient, P.Schaghen

18

An equal bargain?

Alfred Frederick's The Purchase of Manhattan Island from the Indians

The idea of land ownership did not exist among Native Americans.The Lenape were “trading” the right to use the land – which everyone had as a right – for money and goods. These were valuable; they used uncommon raw materials and were troublesome and time-consuming to produce.

19

Not just physics …

As a design scientist in my field, I attempt to engineer innovative educational environments and simultaneously conduct experimental studies of those innovations. This involves orchestrating all aspects of a period of daily life in classrooms, a research activity for which I was not trained. My training was that of a classic learning theorist prepared to work with “subjects” (rats, children, sophomores), in strictly controlled laboratory settings. The methods I have employed in my previous life are not readily transported to the research activities I oversee currently.

20

“Six principles” in action …

Even though the research setting has changed dramatically, my goal remains the same: to work toward a theoretical model of learning and instruction rooted in a firm empirical base. I regard classroom work as just as basic as my laboratory endeavours, although the situated nature of the research lends itself most readily to practical application … I attempt to engineer interventions that not only work by recognizable standards but are also based on theoretical descriptions that delineate why they work, and thus render them reliable and repeatable.

21

Real problems …

Indeed, the first grant proposal I ever had rejected was about 10 years ago, when anonymous reviewers accused me of abandoning my experimental training and conducting “Pseudo-experimental research in quasi-naturalistic settings” This was not a flattering description of what I took to be microgenetic/observational studies of learning in the classroom

(Ann L. Brown Design Experiments: Theoretical and Methodological Challenges in Creating Complex Interventions in Classroom Settings The Journal of the Learning Sciences 1992 2(2) 141-178)

22

The Trading Zone & CSEd

• It may be that every inter-disciplinary field is a “trading zone” (or grows from one)

• Necessity of being able to speak with “trading partners”• If we use quantitative methods, would a statistician recognize

the “truth” of our conclusions?• If we use ethnographic investigation, would an anthropologist

recognize the validity of our methodology?

• Can we develop distinctive areas of working – moving from pidgin to creole? Can we “grow up”?

23

Place of theory

• The world didn’t need Isaac Newton to know that apples fall off trees. It did need Newton to give a general theory that explains why apples fall off trees.