Methodology and ExplanationXX50125

Lecture 2: Experiments

Dr. Danaë Stanton Fraser

Methodology and Explanation 2005

Lectures

1. Introduction to Methods

2. Experiments

3. Interviews/questionnaires/surveys

4. Usability testing

5. Ethnography

Methodology and Explanation 2005

Aim

• To answer a question or test an hypothesis that predicts a relationship between two or more events, known as variables.

• E.g. Will people be quicker and more accurate at completing a task

collaborating at a distance using just verbal instructions or with verbal instructions with gesture input?

Will spatial knowledge be superior following exploration of a VR simulation of a building or from exploring a model of a building?

Methodology and Explanation 2005

Variables

• Such hypotheses are tested by manipulating one or more of the variables.

• The variable that is manipulated is called the independent variable (the conditions to test this variable are setup independently before the experiment starts). In the study mentioned media type (VR vs model) is the independent measure

• The dependent variable would be accuracy of spatial information (ie time to reach point A from point B) because the time to carry out the task depends on the media explored.

• What are the IV and DV for the gesture study?

Methodology and Explanation 2005

Variables and Conditions

• In order to test an hypothesis the experimenter sets up the experimental conditions e.g

• Condition 1 Pretest Explore model Post test

• Condition 2 Pretest Explore VR simulation Post test

• Control condition – against which to compare the results

• What are the conditions in the gesture study?

Methodology and Explanation 2005

Participants

• Within participants designCounterbalancing required

• Between participants designTwo drawbacks – no. of participants needed, individual differences

Advantage – no order effects

• Matched pairs design (on characteristics such as gender and expertise)

Difficult to match across all variables

Methodology and Explanation 2005

Practicalities

• Where will the experiment be carried out?

• How will the equipment be set-up?

• How does one introduce participants to study?

• What scripts are need to standardise the procedure?

• Always include pilot study

Methodology and Explanation 2005

Data Collection and Analysis

• Data – performance measures are taken e.g response times, no. of errors, angle estimation…

• Use graphs• The data should be averaged across conditions to

examine any differences• Statistical tests such as t-tests and ANOVAs can

reveal if the differences are significant• Software packages such as SPSS are often used• If there is no significant difference then the

hypothesis is refuted.

Methodology and Explanation 2005

Data captured

• Record quantative results

• However often also able to gain qualitative data

e.g dialogue turns, gesture and non verbal behaviour

• Good practice to collect both qualitative and quantative data

Methodology and Explanation 2005

Summary of Experimental Design

• State a causal hypothesis• Manipulate independent variable• Assign participants randomly to groups• Use systematic procedures to test hypothesised causal

relationships• Use specific controls to ensure validity

Methodology and Explanation 2005

Quasi-Experimental Design

• State a causal hypothesis

• Separate participants based on some characteristic e.g level of mobility as child, gender, occupation

• Cannot assign participants randomly to groups

Methodology and Explanation 2005

A Variety of Experimental Designs

• (Intervention) - Post test

• Pre test - (Intervention) - Post test

• Pretest - (Intervention) - Post test - Delayed Post Test

• Interrupted time-series

• Cross-over

Methodology and Explanation 2005

Preparation for Practical 2/3Topic: Examining the use of gestures in co-located collaboration.

We will design and run a small experimental study to examine gesture in co-located collaboration

In preparation for the practical please:Read up on experimental design e.g Robson, C. (1994). Experimental Design and Statistics in Psychology.

Aylesbury, UK. Penguin Psychology.

Coursework 1: Write up the experiment carried out in Practicals 2 and 3. Deadline for report Friday 18th November.