DISCUSSION

RATIONALE

ABSTRACT

INTRODUCTION

RESULTS

DISCUSSION

METHODS

Developing a Psychometrically Sound Instrument of Best Practices Specific to the PBRN Context

Victoria Neale, Kendra Schwartz, Monina Bartoces, Jason Booza, Joseph Monsur, Marilyn Wayland: Wayne State University; and Brenda Hudson: Indiana University

Objective: To develop a self-assessment tool for PBRNs to evaluate and improve their research culture.

Design: Multi-step formative process to develop consensus on best practices. Setting: Internet-based survey data collection. Participants: 3 respondents per 42 PBRNs estimated the extent to which the

PBRN followed 59 research best practices. Outcome: A psychometrically-sound self-assessment tool of research best

practices specific to the PBRN context.

Progress to Date:

1) An extensive list of research integrity best practices was developed from the published literature and suggestions from PBRN experts. The wording for these PBRN-specific best practices was reviewed by 20 survey research experts recruited from the NAPCRG Consultants Directory.

2) 59 research best practices were organized into 7 domains: (study launch; study orientation; study supervision; data management & analysis; dissemination; professional training/development; and PBRN management).

3) “Essential Phase”: 76 PBRN experts rated the 59 items as essential or not.

4) “360 Phase”: 3 respondents from 42 PBRNs evaluated the use of 59 research best practices in their network. Respondents were in the role of : a) PBRN director/research director; b) network coordinator or experienced research assistant; and c) community-based staff person.

The practice-based research network (PBRN) movement has resulted in decentralized community-based research, which raises particular challenges to research integrity.

While practice-based primary care research offers the advantage of greater external validity, it also presents challenges that include: implementing standardized research protocols in decentralized settings; training and monitoring of ethical research practices among the community-based clinic staff; and data management and quality control issues.

To date, there is no PBRN-specific infrastructure to promote the integrity of studies conducted in decentralized settings.

Analytic Strategy:

1) Essential Phase:

Content Validity Coefficients (CVCs) identified the items that a majority of 76 PBRN experts rated as “essential” research best practices.

12/59 items had negative CVCs and were eliminated; 47 items were retained.

2) 360 Phase: a) Cronbach’s coefficient alpha computed by role of respondent:

Directors/Research Directors (n = 33): α = 0.89 Staff/Research Assistants ( n = 34): α = 0.93 Community-based Staff ( n = 37): α = 0.94 All respondents combined (n = 104): α = 0.93

b) Factor Analysis 31 items loading on 5 factors retained based on a priori criteria: eigenvalue > 1;

factor loading > 0.45; accounting for > 5% of variance.

Factor 1: Data Management (10 items): overall α = 0.88

Director/Research Director: α = 0.65

Staff/Research Assistants: α = 0.83 Community-based staff: α = 0.89

Factor 2: Study Supervision (4 items): overall α = 0.85

Director/Research Director: α = 0.91

Staff/Research Assistants: α = 0.71 Community-based staff: α = 0.80

Factor 3: PBRN Policies (9 items): overall α = 0.83 Director/Research Director: α = 0.73

Staff/Research Assistants: α = 0.87 Community-based staff: α = 0.85

Factor 4: Study Management (5 items): overall α = 0.81 Director/Research Director: α = 0.90

Staff/Research Assistants: α = 0.54 Community-based staff: α = 0.82

Factor 5: Ethical Considerations (3 items): overall α = 0.63 Director/Research Director: α = 0.63

Staff/Research Assistants: α = 0.69 Community-based staff: α = 0.59

Innovation and Significance To date, the national research enterprise has functioned in a decentralized fashion, resulting in duplicative or undocumented processes, without acknowledged best practices.

This study used collective wisdom of experienced survey researchers and practice- based researchers to identify essential PBRN-specific research best practices.

The final product will be a self-assessment checklist of best practices that PBRNs can use to assess their current research culture.

The checklist has the potential to improve PBRN research culture by encouraging PBRNs to prioritize activities that strengthen the research professionalism of their organization.

By promoting research professionalism, this project has the potential to contribute to greater public willingness to participate in research that can build the evidence- base of primary care.

Next Steps:

1) Develop Standard Operating Procedures (SOPs) for Each Best Practice 2007-2008 funding from CTSA Community Engagement Supplemental Award: Laura Mae Baldwin is Co-Investigator; Victoria Neale and Rowena Dolor are consultants. We will conduct a PBRN survey in early 2009 to: a) determine routine procedures and SOPs currently used by PBRNs; and b) ask PBRNs for insights into how SOPs could best be implemented in their setting.

2) Disseminate and Evaluate the Utility of the Best Practices and their SOPs Recruit PBRNs as test sites to implement and evaluate the SOPs. Evaluate utility of the best practices and SOP model for improving PBRN research professionalism. Determine motivators for PBRNs to engage in the self-assessment process and to use the Best Practices and their SOPs for continuous quality improvement.