6
T here is often an assumption that qualitative research equates to primary research involving original fieldwork. However, when policymak- ers require qualitative evidence, they rarely consider the potential for examining existing data and there is a drive among policymak- ers and funders to conduct original field- work afresh with each new piece of research. As part of a programme of work exploring policy around ageing in the UK, the Institute for Public Policy Research (ippr) undertook a unique collaborative study with Joanna Bornat and Bill Bytheway at the Open University, who are conducting an ongoing qualitative longitudinal project called ‘The Oldest Generation’ (TOG). ippr’s small pilot study investigated the value of reusing the TOG data to inform our policy work, the first time that ippr has conducted secondary analysis of qualitative data. ‘The Oldest Generation’ is one of seven empirical projects that makes up the Economic and Social Research Council (ESRC)’s qualitative longitudinal initiative Timescapes. The seven projects have been designed to span the life course, and have generated detailed evidence which is both retrospective and prospective, in exploring change as it happens, in ‘real time’. It also explores the perspectives of family members of different generations over time. The methodological approach used by TOG involves repeat life history interviews with a family member who was over the age of 75 at the time of being recruited. Another family member is keeping a diary and is taking pho- tographs over 18 months. Twelve families are involved. Thepotentialforreusing qualitativedata My interest in conducting secondary analy- sis of qualitative data arose from a sense that important types of evidence, and the perspectives of some groups of people, are being overlooked by the policy community. Reuse of existing data has the potential to redress these gaps in a number of ways. Policy-focused research often has to be delivered within relatively short time- frames, which can preclude the use of methodologies that bring longitudinal and historical perspectives to policy issues. The pressure on researchers to be ‘original’ and up-to-date, and the under-resourcing of dis- semination, can also make it difficult for new policy researchers to build on the con- tinuing history and ongoing debates sur- rounding particular issues. Yet in a wide range of policy areas, ageing policy being an important example, there is a particular need to understand social change in the long term. Ageing policy needs to be informed by evidence that is holistic and continuous, and that explores how different areas of policy impact (or fail to impact) in the long term in people’s lives. It has also been claimed that many government departments do not have a good institution- al memory of what policy initiatives did and did not work in the past (O’Neill 2009, © 2009 The Author. Journal compilation © 2009 ippr publicpolicyresearch–June-August2009 97 Breakinga ‘strangesilence’ RuthSheldon exploresthepotentialforreusing existingqualitativedatatoinformpolicy,withafocus onageingpolicy

Breaking a ‘strange silence’

Embed Size (px)

Citation preview

Page 1: Breaking a ‘strange silence’

There is often an assumptionthat qualitative researchequates to primary researchinvolving original fieldwork.However, when policymak-

ers require qualitative evidence, they rarelyconsider the potential for examining existingdata and there is a drive among policymak-ers and funders to conduct original field-work afresh with each new piece of research.

As part of a programme of work exploringpolicy around ageing in the UK, the Institutefor Public Policy Research (ippr) undertook aunique collaborative study with JoannaBornat and Bill Bytheway at the OpenUniversity, who are conducting an ongoingqualitative longitudinal project called ‘TheOldest Generation’ (TOG). ippr’s small pilotstudy investigated the value of reusing theTOG data to inform our policy work, the firsttime that ippr has conducted secondaryanalysis of qualitative data.

‘The Oldest Generation’ is one of sevenempirical projects that makes up theEconomic and Social Research Council(ESRC)’s qualitative longitudinal initiativeTimescapes. The seven projects have beendesigned to span the life course, and havegenerated detailed evidence which is bothretrospective and prospective, in exploringchange as it happens, in ‘real time’. It alsoexplores the perspectives of family membersof different generations over time. Themethodological approach used by TOGinvolves repeat life history interviews with afamily member who was over the age of 75 atthe time of being recruited. Another family

member is keeping a diary and is taking pho-tographs over 18 months. Twelve families areinvolved.

The�potential�for�reusingqualitative�dataMy interest in conducting secondary analy-sis of qualitative data arose from a sensethat important types of evidence, and theperspectives of some groups of people, arebeing overlooked by the policy community.Reuse of existing data has the potential toredress these gaps in a number of ways.

Policy-focused research often has to bedelivered within relatively short time-frames, which can preclude the use ofmethodologies that bring longitudinal andhistorical perspectives to policy issues. Thepressure on researchers to be ‘original’ andup-to-date, and the under-resourcing of dis-semination, can also make it difficult fornew policy researchers to build on the con-tinuing history and ongoing debates sur-rounding particular issues. Yet in a widerange of policy areas, ageing policy beingan important example, there is a particularneed to understand social change in thelong term. Ageing policy needs to beinformed by evidence that is holistic andcontinuous, and that explores how differentareas of policy impact (or fail to impact) inthe long term in people’s lives. It has alsobeen claimed that many governmentdepartments do not have a good institution-al memory of what policy initiatives didand did not work in the past (O’Neill 2009, ©

200

9 T

he A

utho

r. J

ourn

al c

ompi

latio

n ©

200

9 ip

pr

public�policy�research�–�June-August�2009 97

Breaking�a‘strange�silence’�Ruth�Sheldon�explores�the�potential�for�reusingexisting�qualitative�data�to�inform�policy,�with�a�focuson�ageing�policy

Page 2: Breaking a ‘strange silence’

Porter 2007, Lodge and Rogers 2006). Researchers can respond to this deficit

by ensuring that historical and longitudinalevidence informs current policy decisions.For example, longitudinal data may be builtin to the design of the dataset itself, as is thecase with the evidence collected throughTimescapes. Other data sets, which may notbe longitudinal in design, can be revisitedfrom the perspective of particular policyinterests to provide insights into that natureof policy at different points in time.

The short time frame of policy-drivenresearch can also result in the exclusion ofsome people from research. For example,people in the ‘older old’ age group (oftendefined as people over the age of 85) havepreviously been excluded, sometimes becauseof their frailty and limited mobility (Carr2004). Drawing on existing research withvery elderly people via secondary analysis isone practical way of including their perspec-tives in evidence used to make policy.

Why�has�there�not�beenwider�reuse�of�qualitativedata?As Paul Thompson has argued, there wasuntil recently a ‘strange silence’ among thesocial science academic research communi-ty about reusing qualitative data that hasbeen collected by other researchers(Thompson 2000). There appears to be alack of communication between social sci-entists (who may adopt historical approach-es, such as longitudinal or life historymethodologies) and historians, and the twohave been portrayed as competing for poli-cy influence. This apparent tension seems tocreate unnecessary barriers betweenresearchers from different disciplines whoare interested in ensuring that historical evi-dence is used in policy. Social researchtends to play down the historical context,prioritising the here and now and theinevitabilities of social structures and politi-cal processes. But reusing qualitative data

gathered over different time periods neces-sarily brings a historical perspective to poli-cy problems.

There also appears to have been resistanceto using historical evidence in ‘evidence-based’ policymaking. As academics involvedin the History and Policy network1 haveargued, history can be inconvenient andembarrassing for policymakers who may notwant the public to be reminded of the genesisof their ‘new’ policy ideas. Research hasfound that it can be difficult for politicians toadmit to something being tried before(Berridge 2007). Where historical evidence isused by politicians, it often plays a rhetoricalrole which is dependent on particular policysituations and political cycles (ibid).

The complexity of qualitative evidencemay also explain why secondary analysis ofquantitative evidence is more common.Quantitative data tends to simplify thecomplicated and is more amenable to beingused to support policy decision-making; asa result, policymakers have tended to preferit (Ritchie and Lewis 2003). In contrast,administrators, policymakers and politicianshave often dismissed qualitative research as‘anecdotal’ because it is based on small sam-ple sizes. Because qualitative research is pri-marily concerned with understanding socialphenomena in context, the reuse of qualita-tive data raises additional epistemological,methodological and ethical questions abouthow data that is collected for one purposecan be used for another.

Finally, until relatively recently, qualita-tive data sets have not been easily availableto researchers because they have not beenarchived. This is now changing with thedevelopment of Qualidata2 and the fundingof Timescapes by the ESRC.

Re-using�qualitative�datato�inform�policy:�TheOldest�Generationippr’s short pilot study of The OldestGeneration data found that accessing exist-

public�policy�research�–�June-August�200998

© 2

009

The

Aut

hor.

Jou

rnal

com

pila

tion

© 2

009

ippr

1 www.historyandpolicy.org 2 ESDS Qualidata (www.esds.ac.uk/qualidata) is part of the UK Data Archive which provides access and support to over

5000 social science qualitative data sets including many classic historical and sociological studies.

Page 3: Breaking a ‘strange silence’

ing longitudinal data enabled explorationof the role that policies play in people’s livesbeyond a particular policy cycle. For exam-ple, one TOG participant, Mrs Watson3,had provided a caring role for her disabledbrother for the past 70 years, in an unstableand largely unsupportive policy context. Inher interview, and in the diary evidencerecorded by her son, Mrs Watson describedhow her life had been intertwined with herbrother’s. She talked about her sense of stig-ma on being associated from a young agewith a brother with cerebral palsy and howher caring responsibilities had impacted onher life choices. Her account highlighted theimplications of changing cultural attitudestowards disability at a personal level, inrelation to service provision and policy:

Today I have seen on TV that a footballer ... whohas a son with cerebral palsy has raised £100,000from an appeal the family made to send the boy toDüsseldorf for some specialist treatment. Howtimes change. That child reminds me of my brotherJack at that age. My mother had to make an eightmile round trip on a bus for Jack’s physio. No helpof any kind. I myself after her death decided to givehim two walking sticks and I suppose I bulliedhim to walk. I ponder which is the most humaneprocedure. In those far-gone days families were leftto get on with difficulties, both in health and socialcare. Apart from no financial help there were notany practical services of any kind, often only strangeexpressions. (Diary entry, 1 February 2008)

Accessing this data also provided evidenceof the interplay of different policy areas overtime; for example the ways in which pastemployment policies, family or health poli-cies impact on people later in their lives.

Reusing data in this way can provide newperspectives on policy ‘problems’: we wereable to search the TOG data for topics thatwere not planned into the original researchdesign or generated with particular policyframeworks in mind. This approach, referredto as ‘using browsing and serendipity’ bysome researchers (Timescapes website), hasparticular advantages. Where participants

speak about topics unprompted, this providesan opportunity to understand the kind ofintensity that the subject has for them, thelanguage that they use to speak about issuesand the contexts within which particularissues arise (Bishop 2007, Bornat 2003).

The pilot study enabled ippr to accessdata from research participants who werenot sampled and recruited through moretypical research gatekeepers such as serviceproviders and agencies. The TOG partici-pants were not sampled to explore a partic-ular policy agenda, but instead were foundby the Open University’s networks aroundthe country. This methodological approachprovided new perspectives on policy prob-lems in a number of ways.

First, it highlighted the potential for sec-ondary analysis to provide more detailedunderstanding of hidden or stigmatisedissues. One example is the policy agendaaround mental health in later life. There isevidence that suggests that there are risinglevels of depression and emotional distressamong older people but an ippr scopingstudy found that evidence about well-beingin later life often contradicts this, perhaps inreflection of the stigma associated with dis-cussing poor mental health in public contexts(Allen 2008, Age Concern and the MentalHealth Foundation 2006, Age Concern2007). Reusing TOG data enabled percep-tions of well-being to be explored indirectly,rather than through primary research explic-itly aiming to generate evidence on this sub-ject. The longitudinal nature of the projectmay have meant that participants felt moreable to open up about these issues to TOGresearchers as compared with a researcherworking on a shorter-term project.

Second, the pilot provided an understand-ing of the ways that tensions between differ-ent policy approaches are reflected in peo-ple’s lived experiences. For example, therecan be a tension between policy approachesdesigned to support independence in later lifeand preventative strategies based on earlyintervention. This tension between needingsupport and wanting independence was

© 2

009

The

Aut

hor.

Jou

rnal

com

pila

tion

© 2

009

ippr

public�policy�research�–�June-August�2009 99

3 Participants’ names have been changed throughout

Page 4: Breaking a ‘strange silence’

reflected in Mr Roberts’s account of thechoices he had made in caring for his wife:

I’ve been offered help by the doctors and the vicarand everybody, but as long as you can manageon your own you want to manage on your own.They say, well you can get home help and thatand now they’ve even offered a visitor to comeand take her out for days. But you don’t wantthat if you can possibly help it. (Interviewwith Mr Roberts, 19 September 2007)

As this example highlights, this data set hadthe advantage of preceding as well as fol-lowing possible referrals to health and socialcare agencies. It therefore provided evi-dence about the decisions people makeabout professional intervention, set againstthe backdrop of their everyday lives.

Third, analysing data that was not collect-ed for a particular policy purpose raisedquestions about the paradigms underpinningpolicy thinking, in this case how paradigmsof old age relate to people’s experience ofageing. For example, Bowling has highlight-ed how previous policy and practice frame-works have adopted ‘positive’ or ‘negative’paradigms of old age. ‘Positive’ paradigmstake a view of old age as a positive and natu-ral period of life while negative or ‘patholo-gy’ models focus on issues of dependencyand need (Bowling 2005). Yet these para-digms or models which have underpinnedpolicymaking may well be disconnectedfrom older people’s experiences (AgeConcern and the Mental Health Foundation2006). Reality is more complicated: peoplemove between these different paradigms of

ageing and have different strategies for deal-ing with these two polarities.

For example one couple in their lateeighties, living in a very rural area to whichthey had moved following retirement, werealso far from their children and were highlydependent on being able to drive and thesupport of friends and neighbours. In herinterview fieldnotes, Joanna reflected onwhat might happen in the future:

I wondered if I should ask about planning fortheir future, but they didn’t offer any kind of leadin so I left it. Yet again in one of these inter-views, happiness and contentment are beingradiated and I don’t like to disturb this account.But, I do wonder what things will be like whenI come back in eighteen months’ time.(Fieldnotes from interview with MrsArthur, 3 September 2007)

The longitudinal nature of the data and theincorporation of other family perspectiveswas able to reveal times when things weremore difficult.

Fourth, the pilot highlighted ways inwhich reusing qualitative data can enablepolicymakers to draw on evidence aboutpeople’s existing ways of life. For example,analysing TOG evidence provided anopportunity to trace people’s social contactsand social networks over time. This level ofdetailed observational data revealed sourcesof support and networks, and gaps in sup-port, which might be overlooked byresearchers, policymakers and by partici-pants themselves when asked about thisdirectly. This is extremely important forpolicymakers who are concerned withensuring that communities meet the needsof an ageing population. Existing researchhas suggested that social relationships andcommunity engagement are among themost important drivers of well-being in laterlife (Allen 2008). Policy that responds tothis needs to start from an understanding ofthe forms of social and community net-works that are important to people.

Finally, the process of conducting the

public�policy�research�–�June-August�2009100

© 2

009

The

Aut

hor.

Jou

rnal

com

pila

tion

© 2

009

ippr

Existing�research�hassuggested�that�socialrelationships�and�communityengagement�are�among�themost�important�drivers�ofwell-being�in�later�life

Page 5: Breaking a ‘strange silence’

analysis raised ippr’s awareness of issuesthat was absent from our planned work onageing: notably, the subject of death, theprocess of dying and the longer termimpact on families of absence throughdeath.

Analysing The Oldest Generation dataalso provided an opportunity to explorehow reported large-scale trends play out atan individual and family level over time.For example, the greater geographical dis-persal of families, changes in residentialcircumstances and living arrangementshave been identified in quantitative studiesas key factors impacting on later life (Hydeand Higgs 2004, Young and Grundy2006). The TOG data provides an oppor-tunity to explore the different experiencesof older people whose families were livingclose by compared with those who weremuch more distant. The evidence also pro-vides detailed insights into the processesby which older people make decisionsabout where they live.

For example, several of the TOG familieshad transnational links and these hadimportant consequences for the oldest gen-eration. Mr Shaw, 83 years old at the time ofbeing interviewed, described how his deci-sions about where to live were closely relat-ed to the movements of his daughter Josie,who was previously living in Australia. Hewas part way through applying to movethere to join her when she decided toreturn to England.

We were glad that she moved back before we gotthere [to Australia] otherwise we’d have been land-ed out there without Josie and we’d have had totravel back to England to see her. It’s good, you

know, from the point that when Josie was here [inEngland] if I was ill she was down there straightaway to see, look after me and do anything . She’sbeen a good daughter in other words. (Interviewwith Mr Shaw, 1 October 2007)

The reuse of qualitative data for policy pur-poses could also result in more innovativecollaborations between academics andapplied policy researchers. It appears thatTimescapes’ concern with archiving anduser engagement is helping to break downmore traditional academic resistance to col-laboration. This feeds into broader debatesabout the extent to which policy agendas(and therefore the decision-making of poli-cymakers) should shape the focus of aca-demic research (Hammersley 2006).

Conducting a secondary analysis along-side the primary research team raisesmethodological, ethical and practical issueswhich need to be considered when policyresearchers reuse qualitative data. For exam-ple, reusing data requires a reflectiveapproach to the potential and limitations ofthe sampling strategy used to generate theoriginal data set. It also requires engagementwith complex questions about ethics, in rela-tion to the consent and anonymity ofresearch participants (Bornat 2003).Throughout this article, we have used pseu-donyms for participants in order to protecttheir identity. This raises questions both forother researchers reusing this data and for theTimescapes team involved in archiving. Is itimportant to ensure that future researchersrefer to participants in a consistent way so thatparticipants featured can be located in thearchive? How can we ensure that partici-pants’ anonymity is maintained in a contextwhere data may be used in a cumulative wayby different researchers?

Existing literature about reusing qualita-tive data has focused particularly on theimplications of doing so while having limitedknowledge of the original research context(for example see Corti et al 2005). In appliedqualitative research, knowledge of the origi-nal research context is needed in order tounderstand the extent to which qualitativeevidence can be generalised to other times ©

200

9 T

he A

utho

r. J

ourn

al c

ompi

latio

n ©

200

9 ip

pr

public�policy�research�–�June-August�2009 101

The�reuse�of�qualitative�datafor�policy�purposes�could�alsoresult�in�more�innovativecollaborations�betweenacademics�and�applied�policyresearchers

Page 6: Breaking a ‘strange silence’

and places (Ritchie and Lewis 2003). As suchthere are complex questions for academicsinvolved in archiving qualitative data aboutwhat ‘meta-data’ (additional data about thecontext of the research) should be included.Policymakers can input in to this process byadvising on the usefulness of different contex-tual data. Our pilot study found that wheresecondary analysis is conducted alongside theprimary research team, the unrecorded reflec-tions and interpretations of the primaryresearchers themselves can become an impor-tant source of contextual data for secondaryanalysis. In our pilot analysis, discussionsabout which data the secondary analysisshould start with were based on Joanna andBill’s existing familiarity with and interpreta-tion of the data.

Conclusionippr’s pilot study demonstrates that there ismuch potential for secondary analysis ofqualitative data to be used more widely toinform policy. Doing so will open up accessto new types of evidence that are currentlyoverlooked – historical and longitudinalevidence, data about everyday life and theperspectives of people who may be exclud-ed from more traditional policy researchdesign. This will provide an opportunity touse qualitative evidence that is not pre-scribed by the time-scales of particular poli-cy cycles but is concerned to assess thelong-term impacts of policy in the lives ofpeople, families and communities.

Ruth Sheldon is a senior researcher in the QualitativeResearch Unit at the National Centre for SocialResearch. She was formerly a researcher at ippr.

This article is based on a collaborative study betweenippr and the Open University. Many thanks are owedto Joanna Bornat and Bill Bytheway for their exten-sive feedback and insightful comments.

Age Concern (2007) Improving Services and Support for OlderPeople with Mental Health Problems London: Age Concern(Second report of the UK Inquiry into Mental Healthand Well-Being in Later Life)

Age Concern and the Mental Health Foundation (2006)Promoting mental health and well-being in later life London:Age Concern and the Mental Health Foundation (Firstreport of the UK Inquiry into Mental Health and Well-Being in Later Life)

Allen J (2008) Older People and Wellbeing London: ippr, avail-able at www.ippr.org.uk/publicationsandreports/publication.asp?id=620

Berridge V (2007) History Matters? History’s role in health policy-making History and Policy Research Report, available atwww.historyandpolicy.org/docs/health_policymak-ing.pdf

Bishop L (2007) ‘A Reflexive Account of ReusingQualitative Data: Beyond Primary/Secondary Dualism’Sociological Research Online, Volume 12, Issue 3, available atwww.socresonline.org.uk/12/3/2.html

Bornat J (2003) ‘A second take: revisiting interviews with adifferent purpose’, Oral History, Spring, pp. 47-53

Bowling A (2005) Ageing Well: Quality of Life in Old AgeMaidenhead: Open University Press

Carr S (2004) Has service user participation made a difference tosocial care services? Bristol: The Policy Press for the SocialCare Institute for Excellence

Corti L, Witzel A and Bishop L (eds.) (2005) ‘Secondaryanalysis of qualitative data’, Forum QualitativeSozialforschung/Forum: Qualitative Social Research, 6(1), availableat qualitative-research.net/fqs/fqs-e/inhalt1-05-e.htm

Hammersley M (2006) ‘The contested issue of quality inqualitative research’, presentation to Natcen QualitativeResearch Unit conference, available at www.natcen.ac.uk/natcen/pages/news_and_media_docs/courses/Martyn_Hammersley.pps

Hyde M and Higgs P (2004) ‘The shifting sands of time: Resultsfrom the English longitudinal study of ageing on multipletransitions in later life’ Ageing International 29 (4): 317-332

Johnson J, Rolph S and Smith R (2007) ‘Revisiting “TheLast Refuge”: present day methodological challenges’ inBernard M and Scharf T (eds.) Critical perspectives on age-ing societies. Ageing and the Lifecourse series. Bristol: ThePolicy Press, pp. 89–103

Lodge, G and Rogers, B (2006) Whitehall’s Black Box:Accountability and performance in the senior civil serviceLondon: ippr, available at www.ippr.org.uk/publicationsandreports/publication.asp?id=486

O’Neill, Baroness (2009) IUSS select committee meeting,February 2009, available atwww.publications.parliament.uk/pa/cm200809/cmselect/cmdius/uc168-ii/uc16802.htm

Porter M (2007) Presentation ‘History and the Public’, avail-able at www.historyandpolicy.org/resources/research.html

Ritchie J and Lewis J (2003) Qualitative Research Practice: Aguide for social science students and researchers London: Sage

Thompson P (2000) ‘Reusing Qualitative Research Data: aPersonal Account [50 paragraphs]’ Forum QualitativeSozialforschung / Forum: Qualitative Social Research, 1(3), Art.27, available at http://nbn-resolving.de/urn:nbn:de:0114-fqs0003277

Timescapes website, www.timescapes.leeds.ac.uk/secondary-analysis/

Young H and Grundy E (2006) ‘Old, sick and alone?Living arrangements, health and well being amongstolder people’ Presentation at UPTAP session at theRGS-IBG Annual Conference, London, 30 August-1September, available at www.uptap.net/project08.html

public�policy�research�–�June-August�2009102

© 2

009

The

Aut

hor.

Jou

rnal

com

pila

tion

© 2

009

ippr