NPT: Potential uses in Quantitative Research

Tracy Finch

Institute of Health and Society, Newcastle University

Objectives

To describe the development of a structured research instrument (questionnaire) to measure constructs of the NPM/T in relation to a particular intervention (E-health)

To consider wider issues for use of NPT in Quantitative research

Understanding Normalisation in relation to E-health

‘Understanding the implementation and integration of e-health services’. Mair, May, Finch, Murray, O’Donnell, Wallace et al. NHS SDO Funded study. (April 2006 – Jan 09).

Focused on barriers and facilitators to use of e-health systems by health professionals within the NHS

One work-package (out of 4) involved developing a structured research instrument to assess ‘readiness’ for e-health

The NPM and NPT framed data collection and analysis throughout project (Began with NPM – finished with NPT)

WP3 ‘Technology Adoption Readiness Scale’ (TARS)

Objective: To develop a structured, predictive instrument to test the contextual readiness of a health care setting for uptake and routine use of a specific ehealth system by health professionals.

Contextual readiness: combination of individual and organisational factors

Perspective: Professionals

TARS Study Structure

Phase 1: Instrument development included online survey of experts

Phase 2: Generic instrument ‘factors’ rated in importance

Phase 3: Specific instrument ‘questions’ about specific e-health system

being used

TARS Phase 1: Item development

Aim was to develop an initial item set, using expert review

Generate potential items through translation of construct statements into plain language and single dimensions

Item development: Example….

Step 1 Factor/ Issue:Concerns about security and confidentiality and standards

Step 2 Question:Does X affect your confidence in knowledge available to you? (Relational Integration)

Step 3 Question for ehealth study:How important are the following factors in affecting the use of e-Health systems in the everyday work of health professionals: ….how much the e-Health system affects the users’ confidence in their ability to conduct their work safely and efficiently (scale: Not at all important – Extremely important)

Phase 1: Expert Survey

Survey: 27 Items for inclusion (IW, RI, SW, CI) 5pt scale of ‘importance’ (+ ‘don’t know) Free-text space for ‘any factors missed’ Conducted electronically

Sample: Authors of reviews relevant to field

Recruitment: Email invitation

Phase 1: Results

Response: 63 completed surveys (25% of 252 invitations

presumed received)

Data analysis: Descriptive (mean ratings, correlations)

Findings: Importance of items generally highly endorsed Most correlations low-moderate (little redundancy)

and higher within than across NPM constructs Free-text responses useful in identifying further

factors to include

Phase 2: Development of TARS Generic

Aimed to: Refine factor set using expert survey data Test ‘relative importance’ from perspective of

professionals

Revisions to item set (31 Items): Some items re-worded, dropped or combined New items added from free-text responses

(phase 1) New items for NPT: Coherence, Cognitive Participation,

Reflexive Monitoring

Phase 2: Testing TARS Generic

Survey of health professionals’ perspectives of importance of factors affecting uptake and use of e-health

Sample: Regional NHS Hospitals Trust (potentially 10,000+ respondents). Extensive use of e-Health.

Recruitment: Via site contact (Technical Director)

Response: Extremely low (51 responses)

Analysis: Not particularly useful, but suggestive of different patterns of response between experts and professionals

Phase 3: TARS Specific

Aimed to: Develop a version containing questions framed for

an individual’s assessment of a particular e-health system

Test through data collection at (2) different sites:

Site 1: Community nurses using PDAs (relatively new)

Site 2: Established use of several e-health systems as basis of work (algorithms, information resources, etc)

TARS Specific Instrument

Demographics (eg. Professional role, system/s used, length of time using)

Comfort with using computer-based technology

TARS Items: 30 Items rated from ‘agree strongly’ to ‘disagree strongly’ (7 pt) (Items on hand-out)

‘Normalisation’ questions:

Whether system was ‘not at all’, ‘partly’, or ‘completely’ in routine use

Perceived likelihood of it becoming routine (5 pt scale)

Phase 3: Sample

SITE 1 % (n) SITE 2 % (n)

Response: 46/243 (19%) Response: 231/1351 (17%)

Age: 72% aged 45+; 29% aged <35 Age: 40% aged 45+; 29% aged <35

Sex: All Female Sex: 86% Female

Working role:Community Enrolled Nurse Community Staff Nurse District Nursing Sister/Charge Nurse Practice Development Nurse Senior Nurse

0 (0)28 (13)61 (28)9 (4)2 (1)

Working role:Call handlersNurse advisorsTeam leadersHealth Information advisorsOther

47 (109)24 (56) 9 (21)3 (7) 16 (38)

Perceived level of routinisation of e-Health Not at all Partly Completely

0 (0) 68 (30) 32 (14)

Perceived level of routinisation of e-Health Not at all Partly Completely

1 (2) 17 (35) 83 (174)

Non-parametric (cross-tab) analysis Groups perceiving e-health as ‘completely’ rather than

‘partly’ routine differed in expected direction:

on 12/30 Items at Site 1

(CI=4; RI=4; IW=1; Co; CP; RM

on 9/30 Items at Site 2

(CI=3; RI=3; SW=1; Co; RM) At Site 2, comparison of call handlers with nursing &

related staff indicated differences on 4 items

Key results: Normalisation Perceptions

Summary of Results of TARS

Development of NPM/T based questionnaire for assessment of individual’s perceptions of factors relating to normalisation of e-health Operationalising of NPM/T constructs into plain

language questions

Support for NPM/T in terms of constructs – patterns of relations between items

Potential of items representing NPM/T constructs for discriminating between levels of perceived normalisation of e-health

TARS - Limitations

Low response rates – insufficient for scale development work (statistical properties)

Constraints of ‘real’ environments:

‘Readiness’ assessment is dependent on timing and site characteristics

Lack of access to participants/control over survey reminders etc

TARS should be used/tested in further studies, in sites where predictive utility of TARS can be assessed prospectively

Using NPM/T in Quantitative Research: Wider issues

Potential Benefits

Challenges:

1. Translating theory into plain language

2. Addressing multiple perspectives

3. Standardisation vs specification

4. Operationalising ‘normalisation’

Summary

Using NPM/T Quantitatively:Potential Benefits

The ‘How much?’ question:

Structured surveys have the potential to collect data efficiently, and on a large scale

The ‘what is likely to happen?’ question:

Surveys, used prospectively, may have some predictive utility with respect to outcomes

Potentially useful in comparative research

Surveys are appealing to practitioners and researchers - facilitate take-up of the Theory!

Challenge 1: Translating constructs into plain language

Example:How important are the following factors in affecting the

normalisation of e-health…..“…… the extent to which organizational effort is allocated to an

ehealth system in proportion to the work that the system is intended to do” (CI)

Problem: Multi-dimensional constructs difficult to capture in single questions/statements

Possible solutions? Clear definitions of terms (eg effort) or Establish understanding of terms of reference (eg agreement on

what the system is intended to do?) and use several questions to build understanding

Challenge 2: Addressing multiple perspectives

Questions not to be framed around ‘intention’ – instead reflect judgements about others/the organisation

Which stakeholder groups should be included? How do we combine/weight their ratings?

Need for customising questions (or question sets) for different stakeholder groups

Challenge 3: Standardisation vs specification

Quantitative validation of NPT would be facilitated by: Focused effort on scale development in

appropriate settings and with adequate resources, and

Comparative analysis of quantitative research using NPT survey across different settings

However..... Can we develop a useful ‘generic’ NPT based

structured survey instrument that is useable across settings?

(and if we do, are we denying the complexity that the NPT embraces?)

Challenge 4: How to operationalise ‘normalisation’?

Does the NPM/T yet define ‘normalisation’ adequately for quantitative measurement of it as an ‘outcome’?

Are ‘perceptions’ of how much an intervention/technology/ practice has become ‘part of everyday work/life’ sufficient to test the constructs of the model?

i.e. Are ‘objective’ measures needed also?

Summary & Final Thoughts

Quantitative use of NPT brings challenges, but potential benefits are huge

TARS represents a useful starting point in developing quantitative use of the NPM/T

Need for more focused effort on scale development and validation (MRC Methodology Programme grant planned)

Other quantitative studies are underway (eg. May, Rapley et al. ‘BSPAR Survey’; Newton et al. Midwifery, Melbourne).