Methodological Challenges for the 21 Century/file/1120crow.pdf · Methodological Challenges for the 21st Century Graham Crow . Deputy Director, NCRM . ... Netnography Robert Kozinets

NCRM is funded by the Economic and Social Research Council


1

Methodological Challenges for the 21st Century

Graham Crow Deputy Director, NCRM

University of Southampton, UK



Outline of Presentation • Historical Context • Five Methodological Challenges • Case Studies of Developing Methodological

Innovations • NCRM’s Role in Meeting Methodological

Challenges

2



3

Historical Context • A century is a long time in the history of the social

sciences • ‘In science, each of us knows that what he has

accomplished will be antiquated in ten, twenty, fifty years’ (Max Weber, 1958: 138)

• Weber’s direct experience of innovation in a 1908 survey of occupational careers, ‘obtaining information from the workers themselves’ (1971: 146) rather than from employers



Historical Context • Establishing trust central to the research

relationship, then, now and into the future (Crow and Pope 2008)

• Methodological innovations always have a history • In the current context, pressure to innovate can be

linked, among other things, to the rapid pace of social change (Crow et al. 2009; O’Connor 2011)

4



Five Methodological Challenges • 1. Keeping up with and exploiting technological

change • Argument of Mike Savage and Roger Burrows that

‘the sample survey and the in-depth interview are increasingly dated research methods’ (2007: 885)

• Rapid development of on-line methods to keep up with technological revolution in communications (Hooley et al. 2012)

• Developments in administrative data (Boyle 2011)

5



Five Methodological Challenges • 2. Enhancing research capacity in an integrated

way • Many developments require increasingly specialist

knowledge and skills, but social science is a collaborative endeavour requiring mutual understanding across interdisciplinary teams

• Mixed methods, methodological pluralism • ‘What is?’ presentations e.g. K Charmaz ‘What is

grounded theory?’ http://eprints.ncrm.ac.uk/208/ 6



Five Methodological Challenges • 3. Respecting research ethics • Questions about the whole research process, from

research design and access to data analysis and ownership, continually re-surface in new forms

• Ethics of use of data from social media an on-going challenge for NCRM TALISMAN node

• Research involves risks; researchers need to make the argument for their ethical choices (Wiles 2012)

7



Five Methodological Challenges • 4. The democratisation of social research • Working with research partners beyond academia • Pursuing collaborative research and the ‘impact’

agenda does not always lead straightforwardly to a meeting of minds/practices (Crow 2010)

• NCRM Network of Methodological Innovation ‘Dancing with new partners’, next event at the University of Leeds on 10 May 8



Five Methodological Challenges • 5. Keeping a sense of purpose, and of history • Methodological innovations are more than technical

exercises – importance of keeping in mind why we pursue them (to answer new research questions and to generate better quality data and analyses).

• J Mason (2006) on rationales for mixing methods • A Krishnan (2009) on interdisciplinarity • Treat claims to innovation with caution

9



10



Developing Methodological Innovation • Varying definitions but key elements:

– Should be rooted in genuine attempt to improve some aspect of the research process (not just gimmickry or innovation for innovation’s sake)

– Can comprise developments to established methods as well as new methods

– Should be some level of dissemination (inclusion in text books), acceptance and take-up in research community

– (Coffey and Taylor 2008;Travers 2009; Xenitidou and Gilbert 2010)

11



Developing Methodological Innovation • NCRM Hub research led by Rose Wiles • Project 1 – analysis of innovation ‘claims’ in

qualitative research in social science 2000-2009 • Project 2 – exploration of the social process of

development and the academic ‘impact’ of three qualitative ‘innovations’

12



Study 1: Exploring Innovation ‘Claims’

• Systematic review of innovation claims in social science journals 2000-2009; narrative review of claims conducted.

• Journal contents searched using innovat*, new, novel and emerg* in the title or abstract.

• Focus: authors’ self-definition of innovation • 57 papers identified • Categorised by area/topic • Categorised by type of claim [Inception (n=32); Adaptation

(n=6); Adoption (n=19)]

13



Key Findings

• Study highlights e-research, participatory and creative methods as sites of innovation

• Innovations at interface with arts-based approaches dominate

• Supports the finding that researchers are increasingly claiming innovation

• Limited evidence of wholly new methodologies or designs • Majority of ‘innovations’ appear to be developments to

existing methods or transfer and adaptation of methods from other disciplines

14



Study 2: Innovation Case Studies • 3 case studies of established innovations to explore the social process of

development and the response of researchers. • Criteria:

– Been in existence for some time to allow for take up – In broad areas identified as sites for innovation in study 1 – Been identified as innovations – Have specific individual associated with their development – Qualitative methods

Methods: • interviews with developer of the method/approach • Interviews with users, reviewers and established academics • Review of citations

15



The Cases Online/Virtual ethnography … Netnography

Robert Kozinets Creative methods … Lego Serious Play

David Gauntlett Child-led research … Children as Researchers

Mary Kellett



Response of the Academic Community: Citations

• Systematic search of UK social science bibliographic databases 1999-2010 for published journal articles/conference papers, in which:

(a) Method/approach is applied/adapted/discussed/ referred to/related book reviewed

(b) Method/approach is specifically linked to innovators we are studying.

(c) Innovators are not authors/co-authors, or otherwise linked to instance of uptake.

17



Citations of Cases by Year

18

0

5

10

15

20

25

30

35

40

1999 2003 2004 2005 2006 2007 2008 2009 2010 Year

Num

ber

child-led research Creative Research Methods netnography

Gauntlett & Holzwarth (2006)

Kozinets (1998)

Kozinets (2002)

Kellett (2004) Kellett (2010)

Gauntlett (2007)

Kellett (2005a,b)

Kozinets (2010)



Academic response to innovations

• All acknowledged by the social science community and some take-up

Netnography – 138 citations; 42% apply the method; 76% MBS; evidence of global spread but mostly in specific disciplines of MBS.

Child-led research – 76 citations; 80% referred/referenced the method; citations across SS disciplines; 71% UK based but some international recognition (North America and Australasia).

Creative Research Methods – 37 citations; 76% referred/referenced or discussed the method; media and communication studies but also other disciplines; 51% UK based but some international recognition (EU, Australasia).



Uptake: Innovators’ Views “It’s taken off, I would say, big time now, because it’s all the way round the World. I mean, my email box is just full from people all over the place wanting either advice or suggestions because they want to do something … I mean, you know, Mexico, Indonesia, all over the place” (Kellett)

“It’s now perceived, and I would say fairly accurately, as being fairly deep, fairly widespread in terms of a whole global network of qualitative consumer researchers who know this method and a number of them are using it and adopting it and adapting it” (Kozinets)

“It’s become noticed. You know, it’s the kind of thing that, when people are compiling their handbooks of qualitative methods, they probably nowadays tend to think, well we should probably have a chapter on that” (Gauntlett)

20



Factors impacting on uptake • Time • Timeliness • Support and championing • Accessibility and feasibility of uptake • Maturity of innovation • Dissemination and marketing

21



Academic Responses: Interviewees • Innovation important • People willing to push boundaries, take risks

and experiment • Cases seen as important methodological

developments • But reservations about the drive for innovation

in general

22



Reservations about Innovation • The nature of innovation in the social sciences: is

anything really new? • Encouragement to disseminate developments at an

early stage: fuelled by the impact agenda, publishers and social media

• Codification of innovations – to make them accessible and transferable

• The rush to uptake: researchers feel they should be using ‘new and exciting’ methods

23



Reservations about Innovation “I worry about innovation being, well I worry about the politics of it. I worry about it being this big machine, you know, which the whole aim is to just churn out something just because it’s new and I don’t think, just because it’s new means it’s going to be any good … what I don’t like, what I suppose I would worry about, is people sort of thinking, here’s a new trendy, tricksy method, I’m just going to try it for fun, you know without thinking, now why would I do that, and what would it do and how would you understand the kind of data that might emerge from that”

24



Conclusions to Research on Qualitative Innovation

• Exploring contemporary innovation is challenging • Social context encourages innovation • Methodological developments and their dissemination

speeding up • Academic community wants innovation but distrusts the

drive to innovation • Acceptance and uptake of the least risky innovations? • Comparison with quantitative innovation will be interesting

(Williams and Vogt 2011)

25



NCRM’s Role in Meeting Methodological Challenges

• Promoting methodological innovation involves risk • Not all developments get taken up, and those that

do take time, especially to get into the mainstream (e.g. research methods textbooks, the curriculum)

• Availability on NCRM website of resources from nodes (e.g. LEMMA multilevel modelling, SIMIAN agent-based modelling simulations, MRS systematic reviewing) and beyond

26



NCRM’s Role in Meeting Methodological Challenges

• Training and events database with details of forthcoming opportunities, and resources captured from previous events

• Includes DTC training opportunities • Flagship event the Research Methods Festival: 5th

RMF will take place 2-5 July 2012, Oxford • www.ncrm.ac.uk

27

http://www.ncrm.ac.uk/



A reminder regarding historical context • Q.23 If you are employed in a chemical works, in a factory,

in the metal working industry, or in any other industry which is particularly dangerous, enumerate the safety measures introduced by your employer.

• Q.56 If you are paid piece rates, how are the rates fixed? Does your employer or his representative resort to trickery in order to defraud you of a part of your earnings?

• Q.71 Have you noticed, in your personal experience, a greater rise in the price of the necessities of life, such as food and shelter, than in wages?

28



References • Boyle, P. (2011) ‘Making better use of administrative data in

research’ NCRM Annual Lecture http://www.ncrm.ac.uk/TandE/video/PaulBoyle.php

• Crow, G. (2010) ‘Democratising social research’, MethodsNews http://eprints.ncrm.ac.uk/1594/

• Crow, G., Bardsley, N. and Wiles, R. (eds) (2009) ‘Methodological innovation and developing understandings of 21st century society’, Introduction to special issue of 21st Century Society, 4 (2), 115-18.

29

http://www.ncrm.ac.uk/TandE/video/PaulBoyle.php

http://eprints.ncrm.ac.uk/1594/



References • Crow, G. and Pope, C. (2008) ‘The future of the research

relationship’, Introduction to special issue of Sociology 42 (5), 813-19.

• Hooley, T., Wellens, J. and Marriott, J. (2012, forthcoming) What is online research? London: Bloomsbury Academic.

• Krishnan, A. (2009) ‘Five Strategies for Practising Interdisciplinarity’ http://eprints.ncrm.ac.uk/782/

• Marx, K. (1963) ‘Marx’s Enquête Ouvrière’ in T. Bottomore and M. Rubel (eds) Karl Marx: Selected Writings in Sociology and Social Philosophy. Harmondsworth: Penguin. 30




References • Mason, J. (2006) ‘Six strategies for mixing methods and

linking data in social science research’ http://eprints.ncrm.ac.uk/482

• O’Connor, W. (2011) What's New: Reflections on Current Developments in Qualitative Social Research, SRA 2011 conference http://www.the-sra.org.uk/presentations.htm

• Rees, A. (1950) Life in a Welsh Countryside. Cardiff: University of Wales Press.

• Savage, M. and Burrows, R. (2007) ‘The coming crisis of empirical sociology’, Sociology 41 (5), 885-99.

31

http://eprints.ncrm.ac.uk/482



References • Taylor, C., and Coffey, A. (2008) ‘Innovation in qualitative

research methods: possibilities and challenges’, Cardiff: Cardiff University.

• Travers, M. (2009) ‘New methods, old problems: a sceptical view of innovation in qualitative research’, Qualitative Research 9 (2), 161-79

• Weber, M. (1958) ‘Science as a vocation’, in H. Gerth and C. Mills (eds) From Max Weber: Essays in Sociology. New York: Oxford University Press, 129-56.

32



References • Weber, M. (1971) ‘A research strategy for the study of

occupational careers and mobility patterns’, in J. Eldridge (ed.) Max Weber. London: Michael Joseph, 103-55.

• Wiles, R. (2012, forthcoming) What are qualitative research ethics? London: Bloomsbury Academic.

• Williams, M. and Vogt, W. (eds) (2011) The Sage Handbook of Innovation in Social Research Methods. London: Sage.

• Xenitidou, M. and Gilbert, N. (2010) The processes of methodological innovation: successful development and diffusion, http://eprints.ncrm.ac.uk/2047/

33


Documents

Methodological Challenges for the 21 Century/file/1120crow.pdf · Methodological Challenges for the 21st Century Graham Crow . Deputy Director, NCRM . ... Netnography Robert Kozinets