Different approaches and techniques of behaviour coding Yfke Ongena Workshop on Behaviour Coding...

Different approaches and techniques of behaviour

codingYfke Ongena

Workshop on Behaviour Coding

Wivenhoe House, University of Essex

16 February 2007

What is behaviour coding?

•Data reduction for Systematic analysis

•Behaviour of interviewer and respondent

•Understanding the question-answer process

•Deviations from the ‘paradigmatic’ Q-A sequence

Paradigmatic Q-A sequence

I: How many days a week do you watch television?

R: Seven days

I: Okay, Thank you

Non-Paradigmatic Q-A sequence (1)

I: How many days a week do you watch television?

R: Excuse me?

I: How many days a week do you watch television?

R: Seven days

I: Okay, Thank you

Non-Paradigmatic Q-A sequence (2)

I: How many days a week do you watch television?

R: Well, that would be most days

I: Seven days a week?

R: Yes

I: Okay, Thank you

Diagnostic instrument

Interaction

Validity

•Evaluating questions/ data-collection procedures

•Monitoring interviewers

Problem-solving instrument

Cognitive or Social/Communicative problems

Interaction

•CASM movement

•(Further) development of cognitive and conversational theories

History of behaviour coding

•Cannell, Fowler & Marquis, 1968: fairly simple scheme

•1969 tape recorder: Increasing number of codes(12 47)

•Use of computer: semi-automatic coding & increased possibilities of analysis

•Current data sets: ±500 interviews, 20.000 QA sequences, 100.000+ utterances

Typical codes included in schemes

Interviewer behaviour N coding schemes

Range % of occurrence

Reads question exactly as scripted

26 28-97%

Reads question with minor change

21 1-32%

Reads question with major change

35 0-25%

Question skipped/ not verified 16 0-22%

Non-directive probe in interviewer’s words

23 5-80%

Suggestive/directive probe 15 0-33%

Typical codes included in schemes

Respondent behaviour N coding schemes

Range % of occurrence

Adequate answer 25 75-95%

Inadequate answer 21 2-27%

Don’t know answer 17 1-6%

Refusal to answer 21 0-1%

Request for clarification 18 0-23%

Interruption 18 0-36%

Qualified answer 14 2-20%

Goals of Behaviour Coding•Prior to actual data collection:

•Pre-testing questions*

•Pre-testing data-collection procedures

•During data-collection

•Monitoring interviewers*

•After data collection

•Evaluating data quality

•Explore causes and effects of behaviours*

Pre-testing questions

•Most frequent use of behaviour coding

•Respondent codes: request for clarification, qualified answer, inadequate answer, don’t know/refusal

•Problems with questions may be visible in very subtle ways

•Interviewer codes: minor/major changes in question wording

•Might be more important to know what is changed in question wording

•Benefits of behaviour coding: reliable and quantitative

Interviewer monitoring

•During data-collection: Supervision

•But also prior/after (pre-testing/evaluation)

•Codes based on interviewer training

•Unconditional versus conditional behaviour

•Major benefit of Behaviour coding: objectivity

Interaction analysis

•Sequential information in QA sequence is analyzed

•‘Non-problematic’ behaviours should also be included

•Full versus selective coding

Unit of coding

•Roughest unit: whole interview

•Most frequently used: QA sequence

•Intermediate level: Exchange

•Most detailed level: Utterance

•Selective coding: only utterances that are within the set of pre-specified codes are coded

•Full coding: all utterances are coded

Conventional coding (selective coding at the level of the QA

sequence)Interviewer Respondent codes

E Exact 1 Interruption with answer

S Slight change 2 Clarification

M Major change 3 Adequate answer

4 Qualified answer

5 Inadequate answer

6 Don’t know

7 Refusal to answer

Conventional codingExercise

• Question 1

• Question 2

• Question 3

• Question 7

• Question 8

Exchange level coding

•Exchange of initial question reading and initial response

•Exchange of prompt by interviewer and a possible second answer by the respondent

•Ignore ‘insignificant behaviours’ in between exchanges (acknowledgement token, silence, laughter)

Exchange level codingExercise

• Question 10

Interaction coding (full coding at the utterance level)

•Sequence Viewer program

•Transcripts available in text window

•Semi-automatic coding

•Multivariate coding

•Multiple variables from general to specific

•Each variable only a few values

Example of interaction coding in Sequence Viewer

Example of full coding at the utterance levelI: First, How many persons live in your

household, counting all adults and children and including yourself?

R: Four

I: Okay

------

------------

Transcription

Audio/videofiles

Example of full coding at the utterance level

I: First, How many persons live in your household, counting all adults and children and including yourself?

R: Four

I: Okay

- - - - - -

I: First, How many persons live in your household, counting all adults and children and including yourself?

R: Four

I: Okay

Actor

Exchange

Distance

Specification

Adequacy

Direction

I-----

------

------

I: First, How many persons live in your household, counting all adults and children and including yourself?

R: Four

I: Okay

Actor

Exchange

Distance

Specification

Adequacy

Direction

IQ----

------

------

I: First, How many persons live in your household, counting all adults and children and including yourself?

R: Four

I: Okay

Actor

Exchange

Distance

Specification

Adequacy

Direction

IQ0---

------

------

I: First, How many persons live in your household, counting all adults and children and including yourself?

R: Four

I: Okay

Actor

Exchange

Distance

Specification

Adequacy

Direction

IQ0C--

------

------

I: First, How many persons live in your household, counting all adults and children and including yourself…

I: Would it one, two, between three and five or more than five?

Actor

Exchange

Distance

Specification

Adequacy

Direction

IQ0C--

IQ0A--

I: First, How many persons live in your household, counting all adults and children and including yourself?

R: Four

I: Okay

Actor

Exchange

Distance

Specification

Adequacy

Direction

IQ0CA-

------

------

I: First, How many persons live in your household, counting all adults and children and including yourself?

R: Four

I: Okay

Actor

Exchange

Distance

Specification

Adequacy

Direction

IQ0CAz

------

------

I: First, How many persons live in your household, counting all adults and children and including yourself?

R: Four

I: Okay

Actor

Exchange

Distance

Specification

Adequacy

Direction

IQ0CAz

RA0AA4

IP0nxx

Comparison full and selective coding

• Question 1

I: First, How many persons live in your household, counting all adults and children and including yourself?

R: Four

I: Okay

Selective codes: ‘Exact’, ‘3’ (Adequate answer)

‘Full’ codes: IQ0CAz, RA0AA4, IP0nxx

• Question 2 (part 1)I: …..how confident are you that the Census Bureau… with other government agencies? Very confident, somewhat confident, not too confident, or not at all confident?

R:Share it with what other governments?

Selective code: ‘Exact’, ‘2’ (Requests Clarification)

‘Full’ codes: IQ0CAz, IQ0AAa, RR0rxx

Comparison full and selective coding

• Question 2 (part 2)I: Well the question doesn’t specify but it just says other government agencies.

R: oh probably very confident

I: Okay

Selective codes: 4 (Qualified answer)

‘Full codes’: IQ0MAz, RA0AT1, IP0nxx

Comparison full and selective coding

• Question 7

Codes: Exact, Interrupts, Don’t know

Other codes of the exercise

• Question 3

Codes: Major change, Inadequate answer

• Question 8Codes: Exact, Interrupts, Inadequate answer

Practical application of coding

•Live coding

•Live coding with tape

•Tape coding

•Tape coding with filled out questionnaire

•Coding from transcripts

•Coding from transcripts+ direct access to audio

Incre

asin

g

assista

nce

& h

en

ce

possib

ilities

New Technologies

•CARI:Computer as tape recorder

•Access to additional information from CATI

•Electronic documentation of coder’s notes

•Semi-automatic coding

•Fully automatic coding from log-files•Do interviewers open screens with question texts, help info, enter data correctly etc.•Use of respondent laptop: are show cards shown?