Qualitative Inquiry in Social and Cultural Contexts

The world’s libraries. Connected.

Qualitative Inquiry in Social and Cultural Contexts

The Critical Incident Technique

CoLIS,

Copenhagen, Denmark

August 22, 2013

Lynn Silipigni Connaway, Ph.D.

Senior Research ScientistOCLC

@LynnConnaway

[email protected]


• Flanagan (1954)

• Qualitative technique

• Focuses on most memorable event/experience

• Allows categories or themes to emerge rather than be imposed

Critical Incident Technique (CIT)

(Flanagan, 1954)


• Sir Francis Galton

• Aviation Psychology Program of US Army Air Forces in WWII

• Analysis of failure in learning to fly

• Used in proceedings of elimination boards

• Research for selecting pilots

• Reasons for failures of bombing missions

Origins of CIT

(Flanagan, 1954, p. 2)


• Combat leadership studies

• Gather specific incidents of behavior

• Helpful or inadequate in accomplishing mission

• “Describe the officer’s action. What did he do?”

• Resulted in “critical requirements” of combat leadership

Origins of CIT

(Flanagan, 1954, 2)


• Radford (2006) Tice (2001)

• Evaluate CLASP (Connecting Libraries and Schools Project)

• YA attitudes toward public libraries

• Fisher & Oulton (1999)

• Staff development needs

• Library decision-making

• Tool for librarian entering management positions

• Hamer (2003)

• Information seeking of LGBT youth coming out

• Wilkins & Leckie (1997)

• Investigate information needs & information-seeking behavior of university staff

• Used in addition to questionnaire

CIT in LIS


Critical Incident Technique Examples: Our Research


• Studied habits & needs of virtual reference services (VRS) users and non-users to identify characteristics for informing library system and service development

• Four phases:

• Focus group interviews

• Analysis of 850 QuestionPoint live chat transcripts

• Online surveys of VRS

• 137 VRS Users

• 173 VRS Librarians

• 134 VRS Non-users

• Telephone interviews

• 76 VRS Users

• 100 VRS Librarians

• 107 VRS Non-users

• Generalizable through large sample sizes, multiple methods of data collection, and triangulation of results.

Seeking Synchronicity: Evaluating Virtual ReferenceServices from User, Non-User & Librarian Perspectives

http://www.oclc.org/research/activities/synchronicity/default.htm

http://www.oclc.org/research/activities/synchronicity/default.htm


Think about one experience using VRS in which you felt achieved (or did not achieve) a positive result

• Please describe the circumstances and nature of your question.

• Describe why you felt the encounter was successful (or unsuccessful).

• Did the chat format help your experience to be successful (or unsuccessful)? If yes, how?

Example: Seeking Synchronicity CIT

VRS User Online Survey Questions

(Connaway & Radford, 2011)


Think about one experience in which you felt you achieved (or did not achieve) a positive

result after seeking library reference services in any format.

• Think about one experience in which you felt you did (or did not) achieve a positive result after seeking library reference services in any format.

• Describe each interaction.

• Identify the factors that made these interactions positive or negative.

Example: Seeking Synchronicity CIT VRS Potential User Online Survey Questions

(Connaway & Radford, 2011)


“The librarian threw in a cordial sign-off and encouraged me to pursue the reading. It was like talking to a friendly librarian in person.”

–VRS User (online survey)

Example: Seeking Synchronicity CITVRS User Result


• Investigate theory of digital residents & visitors with students in the various stages of education

• Increase understanding of learners’ online engagement

• How can educational services & systems attract & sustain new group of lifelong learners?

• Trans-Atlantic partnership supports comparison of students' digital learning strategies in different cultural contexts.

Digital Visitors and Residents: What Motivates Engagement with the Digital Information Environment?

http://www.oclc.org/research/activities/vandr/default.htm

(Connaway and White for OCLC Research, 2012.)

http://www.oclc.org/research/activities/vandr/default.htm


• Several methods:

• Semi-structured interviews (qualitative)

• Diaries (qualitative)

• Online survey (quantitative)

• Enables triangulation of data

Example: Digital Visitors and ResidentsTriangulation of Data

(Connaway et al., 2012)


• Ethnographic data collection technique

• Get people to describe what has happened

• Center on defined events or moments

Diaries

(Connaway & Powell, 2010)


Example: Digital Visitors and ResidentsDiaries


Think of a time when you had a situation where you needed answers or solutions and you did a quick search and made do

with it. You knew there were other sources but you decided not to use them. Please include sources such as friends,

family, teachers, coaches, etc.

Example: Digital Visitors and Residents CITDiarist Follow Up Interviews


• Explain a time in the past month when you were SUCCESSFUL in completing an ACADEMIC assignment. What steps did you take?

• Think of a time fairly recently when you struggled to find appropriate resources to help you complete an ACADEMIC assignment. What happened?

• Explain a time in the past month when you were successful in getting what you needed in a PERSONAL situation. What steps did you take?

• Explain a time in the past month when you were NOT successful in getting what you needed in a PERSONAL situation. What steps did you take?

Example: Digital Visitors and Residents CITDiarist Follow Up Interviews


• New model for VRS

• Collaboration with Social Q&A (SQA) community

• Provide evidence for modeling new library collaborative services

• Three phases

• Analysis of transcripts

• 500 QuestionPoint

• 1000 Yahoo Answers Q & A pairs

• 200 QuestionPoint live chat

• 200 QuestionPoint Qwidget session transcripts

• Telephone interviews & analysis

• 50 librarian interviews, 50 user interviews

• Construct design specifications

Cyber Synergy: Seeking Sustainability through Collaboration between Virtual Reference & Social Q&A Sites

http://www.oclc.org/research/activities/synergy/default.htm



Example: Cyber Synergy CIT User Phone Interviews

Please recall one specific VR

interaction...that you would

consider successful and

describe.

Please recall one specific VR

interaction...that you would

consider unsuccessful and

describe.

(Radford, Connaway, & Shah, 2011-2013)


“I was looking for books on bilingual communication in the setting of sports teams and I needed at least three book references and I wasn't able to get them on time, so I think it was unsuccessful because I ended up not getting the information I needed for research I was working on.” Cyber Synergy (VS43, Male, Age 19-25)


• Can be applied to a range of studies

• Allows categories or themes to emerge rather than be imposed

• Provides clear examples that prove the value of your services

• Users speak for you

Using CIT in YOUR Research


Connaway, L. S. & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from http://www.oclc.org/reports/synchronicity/full.pdf

Connaway, L. S., Lanclos, D., White, D. S., Le Cornu, A., & Hood, E. M. (2012). User-centered decision making: A new model for developing academic library services and systems. IFLA 2012 Conference Proceedings, August 11-17, Helsinki, Finland.

Connaway, L. S., & Powell, R. R. (2010). Basic research methods for librarians. Santa Barbara, CA: Libraries Unlimited.Fisher, S., & Oulton, T. (1999). The critical incident technique in library and information management research. Education for Information, 17(2),

113–126.Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358.Geertz, Clifford. (1973). The interpretation of cultures: selected essays. New York: Basic Books, 6. Hamer, J. S. (2003). Coming-out: Gay males’ information seeking. School Libraries Worldwide, 9(2), 73–79.Radford, M. L. (summer, 2006). The critical incident technique and the qualitative evaluation of the Connecting Libraries and Schools Project.

Library Trends 54(1), 46-64. Radford, M. L. (1999). The Reference Encounter: Interpersonal Communication in the Academic Library. Chicago: Association of College &

Research Libraries, A Division of the American Library Association. (Publications in Librarianship #52).Radford, M. L., Connaway, L. S., & Shah, C. (2011-2013). Cyber Synergy: Seeking Sustainability through Collaboration between Virtual

Reference and Social Q&A Sites. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC. Retrieved from http://www.oclc.org/research/activities/synergy/default.htm

Tice, M. (2001). Queens Borough Public Library and the Connecting Libraries and Schools Project. Journal of Youth Services in Libraries, 14(3), 11–13

White, D., & Connaway, L. S. (2011). Visitors and residents: What motivates engagement with the digital information environment. Funded by JISC, OCLC, and Oxford University. Retrieved from http://www.oclc.org/research/activities/vandr/

Wilkins, J. L. H., & Leckie, G. J. (1997). University Professional and Managerial Staff: Information Needs and Seeking. College & Research Libraries, 58(6), 561-74.

References

http://www.oclc.org/reports/synchronicity/full.pdf


http://www.oclc.org/research/activities/vandr/


Questions?

Lynn Silipigni Connaway, [email protected]