1

Keeping Track: Coordinating Multiple Representations in

Programming

Pablo Romero, Benedict du Boulay & Rudi LutzIDEAs Lab,

Human Centred Technology Research Group,

School of Science and Technology,

University of Sussex.

2

Background

• Computer programs can be looked at from different perspectives

• Experienced programmers’ mental representations include multiple perspectives and mappings between them

• SDEs often provide multiple, multimodal, linked and concurrent displays (code, visualisation(s), output)

• Programming requires co-ordination of these

BlueJ Java tutoring

Environment

Kolling (2000)

Monash University

3

Some Research Questions

• Are there patterns of interaction which characterise better debugging performance?– Representation use (attention and switches)– Tool use

• Are graphical representations more helpful than textual ones?

• Is debugging performance associated with any “individual difference” variable values?

4

Contents

• Programming is a multi-representation and multi-faceted activity

• Debugging• Experiments

– Static System– “Dynamic” system

• Conclusions

5

Representations - The Problem

• The original problem in real world terms;• The abstraction of the problem, in real world

terms, but omitting irrelevant detail;• The realisation of the real world abstraction in

terms of programming forms e.g. general computational methods of approach;

• The detailed realisation of the program in terms of the kinds data-structures and algorithms available to be used in the given language;

6

Representations - The Code

• The code itself as data-structures and algorithms;

• The input/output behaviour of the code when run;

• Dynamic representations of different aspects of the code, e.g. control-flow and data-structure representations;

7

Representations - The Context

• The workings of the underlying virtual machine• The workings of the programming environment

itself;• Descriptions of the code produced by

static/dynamic analysis tools;

8

Programming Skills : Debugging

• Subdivision of the task– Strategies for divide and conquer and for keeping track

within and between representations– Reasoning about overall strategy

• Reasoning in the problem and programming domains– Forwards from code to behaviour, and backwards from

behaviour to code– Between representations

• Instrumenting the situation– Tools for observing hidden events e.g. print statements

9

Dynamic SDE experiment• 42 participants (1st & 2nd year ug)• Verbal and graphical abilities pre-tests• 5 programs

– 1 control,– Graphical-textual visualisations,

– Control-flow, data structure errors

• 1 error each• Static SDE with 4 windows

– Code– Objects– Call sequence– Output

• Breakpoints

10

The experimental tool• Modified version of

Restricted Focus Viewer

• Presents blurred image stimuli

• User moves unblurred ‘foveal’ area around screen

• Logs user actions and records audio

11

Experimental data• Debugging performance

– Accuracy– Spotting time

• Representation use– Fixation periods– Switching

• Individual differences• (Verbal protocols)

12

Debugging accuracy results

13

Window fixation results

14

Window switching results

15

Window switching results

16

Breakpoint fixation results

17

Critical window fixation results

18

Results summary

• Visualisations helpful

• More interaction for intermediates

• Graphical condition associated with more efficient use for best performers

• Positive correlation between graphical literacy and debugging accuracy

19

Questions and Conclusions

• How far does window blurring alter programming strategy? e.g. Peripheral vision

• How far can we assume that the unblurred window is the focus of attention?

• How do we interpret time spent focusing on a representation? Utility vs. difficulty

• How do we interpret different patterns of switches? E.g. rapidly between a pair or reps., leisurely between a pair of reps.

• Should we expect stronger interactions between graphical/spatial/textual abilities and representation preferences

20

AcknowledgementsRichard Cox

EPSRC

www.cogs.susx.ac.uk/projects/crusade/

21

The End

22

BlueJ Java tutoringEnvironment

Kolling (2000)Monash University

23

Background• Computer programs can be looked at from different

perspectives

• Experienced programmers’ mental representations include multiple perspectives and mappings between them

• SDEs often provide multiple, multimodal, linked and concurrent displays (code, visualisation(s), output)

• Programming requires co-ordination of these

24

Some of the Tasks….

• Understanding the problem

• Constructing general approach

• Specifying sequences, entities and relationships

• Coding

• Debugging (at any of the above stages)

• Maintenance

25

Static SDE experiment

• 48 participants (1st & 2nd year UG)• Verbal and spatial abilities pre-tests• 4 programs to debug• 4 errors each• Static SDE with 3 windows

– Code

– Visualisation

– Output

26

Debugging strategy results

• Qualitative analysis for 2 contrasting participants– Experience

– Verbal ability

• Initial code browsing episode• Two ways of detecting bugs

– by spotting something odd in code browsing episode

– by coordinating representations

27

Window fixation results

• Poor performers focus on the code a lot

• Good performers also focus on objects window

28

Visualisations

30

Representation Issues….

• Cognitive– Relationships between internal and external representations– Structure of knowledge and nature of expertise

• Educational– Evolution of knowledge and its organization from novice to expert– Activities to support learning– Scaffolding needed at various stages

• Aesthetic and Motivational– Nature of the experience– Quality of the artifacts

31

Crusade overall aims• Investigate the co-ordination of multiple external

representations in OO program comprehension

• Investigate role of perspective, modality and individual differences in representation co-ordination in programming.

• Develop set of design principles for program comprehension tools

• Develop computerised experimental tool to study representation co-ordination in programming