Upload
mollie-thacher
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Evaluation of Written Prescription Information Provided in Community
Pharmacies: A National Study
Bonnie L. Svarstad, Ph.D.
Jeanine K. Mount, Ph.D. and R.Ph.
University of Wisconsin - Madison
Study done in cooperation with...
• FDA• National Association of Boards of Pharmacy • National expert panel
National Expert Panel
• 16 experts• Nominated by seven pharmacy organizations• Pharmacy practitioners and experts in
pharmacotherapy and communications • Faculty at 9 pharmacy colleges and universities
Panelists• Mary Amato
• Heidi Anderson-Harper
• Robert Beardsley
• C. A. Bond
• Marie Gardner
• Betty Dong
• Carole Kimberlin
• Duane Kirking
• Sharlea Leatherwood
• Matt Osterhaus
• Anthony Provenzano
• Mary Pubentz
• Betsy Sleath
• Jenene Spencer
• Judith Sommers Hanson
• Gayle Dichter
• Bonnie Svarstad, Chair
Past Studies
• Distribution of patient information increased from 16% in 1982 to 74% in 1998
• 1999 study in 8 states found 87% given information but quality was variable
[Svarstad & Bultman, 1999]
How does this study differ?
• Pharmacies sampled from national list
• Professional shoppers visited pharmacies
• Experts and consumers rated information
• Additional analyses performed
Primary Aims of Study • What percent of patients are given written
prescription information?
• How do experts rate this information?
• How do consumers rate this information?
• How well does information adhere to criteria for useful medication information?
Secondary Aims of Study
• How do expert/consumer ratings compare?
• Do ratings vary by consumer background?
• Do ratings vary by pharmacy type?
• Do ratings vary by leaflet characteristics?
Objectives for Today
• Review study design and procedures
• Review evaluation criteria and forms
• Present results:- Leaflet distribution and ratings
- Factors that might influence ratings
Study Design
• Shoppers acting as patients presented four prescriptions at each pharmacy
• Patient information leaflets mailed to UW
• Expert panelists rated leaflets
• Consumer panelists rated leaflets
• Rating forms mailed to UW for analysis
Sampling of Pharmacies
• Excluded: hospital, clinic, LTC, mail order, IV infusion, dispensing physician, government settings, Alaska, Hawaii, Puerto Rico, states prohibiting filling of prescriptions for research
• Selected from 57,157 community pharmacies
• Simple random sample of 384 pharmacies
• 35% independent, 65% chain
• Data collected in 44 states
Observer Protocol
• Shoppers hired by professional shopper firm
• 72% of visits by females
• 66% of visits by persons 45 years or older
• Mean age = 50
Standard Scenario
• Presented new prescriptions for atenolol, glyburide, atorvastatin, nitroglycerin SL
• Did not ask questions or initiate talk
• If asked, patient-observer reported:- new diagnoses: heart disease,
diabetes - no prior use of study drugs
Observer Protocol (cont.)
• Shopper mailed materials to shopper firm
• Firm removed pharmacy/pharmacist names
• All leaflets, brochures, other information items mailed to researchers
• All items referred to as “leaflets”
Expert Evaluation Forms
• Each form included 8 general criteria and 62-63 sub-criteria
• Eight criteria from 1996 Action Plan for “useful” information
• Sub-criteria based on approved labeling
• Forms revised until all panelists approved
Eight Criteria from 1996 Action Plan
for Useful Information 1. Drug names and indications
2. Contraindications and what to do before using
3. Specific directions about how to use, monitor, get most benefit
4. Specific precautions and how to avoid harm
Eight Criteria from 1996 Action Plan
for Useful Information5. Serious and frequent adverse reactions, what to do
6. General information, encouraged to ask questions
7. Scientifically accurate, unbiased, up-to-date
8. Readily comprehensible and legible
Scoring Method for Expert Forms
• Each criterion rated by 4-10 sub-criteria • Each sub-criterion rated by adherence level:
full (2), partial (1), no adherence (0)
• Computer calculated % points obtained
• Scale ranged from 0 to 100% (best)
Inter-rater Reliability
• Experts assigned to one of four drug groups
• Independently rated same sub-set of leaflets
• Good inter-rater reliability obtained
• Pearson r for overall adherence = 0.90-0.97
Expert Rating Process
• Each expert assigned to one of four drugs
• Each leaflet rated by one expert
• Experts rated 1,367 pharmacy-generated leaflets and 31 manufacturer-generated leaflets (mean = 87 per expert)
Consumer Evaluation Form
• 1-page form with 12 items
• Based on 1996 Action Plan and Krass et al
• Items scored 1 (poor) to 5 (good)
• Overall scale ranged from 0 - 100% (best)
Scoring Categories for Consumer Form Level of Adherence Point Range %
5 80-100
4 60-79
3 40-59
2 20-39
1 0-19
Three Legibility Items
• Poor-good print size
• Poor-good print quality
• Poor-good spacing between lines
Six Comprehensibility Items
• Poorly-well organized
• Poor-good length
• Unclear-clear
• Unhelpful-helpful
• Incomplete-complete
• Hard-easy to find important information
Test-Retest Reliability
• Nine consumers independently rated 18 leaflets at two sessions
• Good test-retest reliability obtained
• Pearson r for overall score = 0.82
Consumer Rating Process
• Recruited 154 consumer raters in 11 states
• Recruited at senior centers, clinics, work and social organizations
• Facilitator met with 8-15 raters per session
• Each rater independently rated ~ 10 leaflets
Rater Characteristics
• Mean age = 61 yrs (20-89 yrs)
• 68% female, 89% white
• 77% used medication daily
• 8% not completed high school, 38% completed high school, 54% beyond h.s.
% Patient-Observers Given Any Written Information
atenolol 89.6
glyburide 88.8
atorvastatin 89.3
nitroglycerin 88.3
Expert Ratings, All Criteria (n=1,367 leaflets)
3 4 3 410 11 11 10
33
2024
17
42
59
49
56
0
25
50
75
atenolol glyburide atorvastatin nitroglycerin
Level 1 Level 2 Level 3 Level 4 Level 5
Expert Ratings by Criterion• Highest Ratings: #7:
accuracy, unbiased
• Moderate Ratings: #1: names, indication#3: directions
• Low Ratings:#5: ADRs, what to do#6: general
information
• Lowest Ratings:#2: contraindications#4: precautions#8:
legibility/comprehens.
Highest Expert Ratings - Criterion 7 Scientifically Accurate, Unbiased, Up-to-date
1 3
95
0
25
50
75
100
Level 1 Level 2 Level 3 Level 4 Level 5
<1<1
Moderate Expert Ratings - Criteria 1 & 3
12 1216
3
29
19
11
47
32
19
0
25
50
75
1 Name, indications 3 Directions
Level 1 Level 2 Level 3 Level 4 Level 5
Low Expert Ratings - Criteria 5 & 6
15
32332525 26
14
4
13 13
0
25
50
75
5 ADRs, what to do 6 General information
Level 1 Level 2 Level 3 Level 4 Level 5
Lowest Expert Ratings - Criteria 2 & 4
1522
3628
16
2927
14
5 7
0
25
50
75
2 Contraindications 4 Precautions
Level 1 Level 2 Level 3 Level 4 Level 5
Lowest Expert Ratings - Criterion 8Legibility/Comprehensibility
1
24
57
18
0
25
50
75
Level 1 Level 2 Level 3 Level 4 Level 5
Consumer Ratings, All Items (%)
9 106 6
16 15 1712
20
30 2832 34
2420
28 29
17
27
20
0
25
50
atenolol glyburide atorvastatin nitroglycerin
Level 1 Level 2 Level 3 Level 4 Level 5
Consumer Scores by Item • Scores varied by item • Lowest scores: print size, print quality, spacing,
overall readability • Moderate-high scores: easy to understand, useful• 36% of leaflets given low ratings on readability
Consumer Scores on Readability &
Understandability Items
19
8
17
11
21 2223
33
20
25
0
20
40
Ease of reading Ease of Understanding
1 (poor) 2 3 4 5 (good)
Expert Versus Consumer Ratings• Low correlation between total scores
• Expert rating of usefulness was related to consumer usefulness item
• Expert rating of usefulness was unrelated to consumer readability item
Expert Rating of Usefulness by Consumer Rating of Usefulness Item (n=339 atenolol leaflets)
Consumer Score Mean expert (%)
1 (poor) 40.3
2 46.3
3 50.0
4 55.0
5 (good) 55.0 (p<.001)
Conclusions -- Part I
• 89% given some information
• Ratings varied by criterion
• Experts most critical of information on contraindications and precautions and legibility/comprehensibility
• Consumers most critical of readability
Results -- Part IIWhat factors might influence leaflet ratings? • Consumer characteristics?
• Pharmacy type?
• Leaflet characteristics?
• Changes made by pharmacy organization or pharmacy integrator?
Leaflet Ratings by Consumer Characteristics
• Ratings unrelated to gender, age, education, current drug use
• White raters gave higher scores than non-white raters for nitroglycerin leaflets
• Race unrelated to other ratings
Leaflet Distribution and Ratings by Pharmacy Type
79
4349
98
5565
0
25
50
75
100
% given leaflet Expert rating Consumer rating
Independent Chain
P<.001 all tests
Expert Ratings by Leaflet Length
44 44 45 46
56 55 5560 63
57 57 57
0
25
50
75
atenolol glyburide atorvastatin nitroglycerin
<5.6 inches5.6-11 inches11-16 inches
P<.001 all tests
Consumer Ratings by Leaflet Length
52 5058 56
6358
6468
74697066
0
25
50
75
atenolol glyburide atorvastatin nitroglycerin
<5.6 inches
5.6-11 inches
11-16 inches
P<.001 all tests
Consumer Ratings by Font Size
5550
5459
64 6368 70
0
25
50
75
atenolol glyburide atorvastatin nitroglycerin
< 10 point
>= 10 point
P<.001 all tests
Consumer Ratings of Readability by Font Size
2.72.4
2.7 2.8
3.4 3.43.6 3.6
1
2
3
4
5
atenolol glyburide atorvastatin nitroglycerin
< 10 point>= 10 point
P<.001 all tests
Data Vendors Examined
• Vendor - not ascertainable 13.5% (46)
• Vendor 1 - partial message 5.0% (17)
• Vendor 1 - full message 81.5% (277)
• Vendor 2* (comparison leaflets,institutional)
Leaflet Length by Vendor
Vendor % leaflets <5.6 inches
Vendor - not ascertainable 82.6%
Vendor 1- partial message 94.1%
Vendor 1- full message 27.1%
Vendor 2* 0.0%
P<.001
Results By Data Vendor
• Substantial variability by data vendor
• Highest: Vendor 1 and Vendor 2
• Lowest: Partial messages and unidentified vendors
Expert Ratings by Vendor
32 31 33 302824
3026
56 56 5661
75
59
77
63
0
25
50
75
atenolol glyburide atorvastatin nitroglycerin
Vendor-NA Vendor1-partial Vendor1-full Vendor2*
P<.001 all tests
Ratings of Atenolol Leaflets by Vendor
32 3228
1828
188
28
56
74
6151
75
100
62
51
0
25
50
75
100
1 Name,indication
2 Contraind. 3 Directions 4 Precautions
Vendor-NA Vendor1-partial Vendor1-full Vendor2*
.
P<.001 all tests
Ratings of Atenolol Leaflets by Vendor
28
14
91
47
20
6
85
4444 44
91
44
99
43
94
83
0
25
50
75
100
5 ADRs 6 General 7 Accuracy 8 Leg/comp
Vendor-NA Vendor1-partial Vendor1-full Vendor2*
P<.001 all tests except Criterion 7 (p>.05)
Consumer Ratings by Vendor
4842
53 53
40 43 4035
62 6065 68
89 8590 87
0
25
50
75
100
atenolol glyburide atorvastatin nitroglycerin
Vendor-NA Vendor1-partial Vendor1-full Vendor2*
P<.001 all tests
Consumer Ratings of Atenolol Leaflets by Vendor
3 3.22.8
2.3
3.22.8
3.13.6 3.8
4.7 4.5 4.4
1
2
3
4
5
Easy to read Easy toUnderstand
Useful
Vendor-NA Vendor1-partial Vendor1-full Vendor2*
P<.001 all tests
Data Vendor by Pharmacy Type
24
9132
64
89
0
25
50
75
100
Independent Chain
Vendor-NA Vendor1-partial Vendor1-full
P<.001
Analysis of Vendor 1 Leaflets
• Analyzed full leaflets used by 16 organizations
• 5 organizations used Version 1 (33 leaflets)
• 5 organizations used Version 2 (50 leaflets)
• 6 organizations used Version 3 (72 leaflets)
• Compared leaflets with prototypes
Results • Ratings varied by leaflet version
• No prototype fully met criteria
• Some organizations added and deleted information from prototype
Expert Ratings by Leaflet Version
83
34
5951
85
40
6759
54
83
67
48
0
25
50
75
100
1 Name, Ind. 2 Contraind. 3 Directions 4 Precautions
Leaflet V1 Leaflet V2 Leaflet V3
P<.001 all tests
Expert Ratings by Leaflet Version
36 33
96
4744
67
92
4453
29
85
42
0
25
50
75
100
5 ADRs 6 General 7 Accuracy 8 Leg/compreh
Leaflet V1 Leaflet V2 Leaflet V3
P<.001 for all criteria
Additions/Deletions from Prototypes
• V1: One organization deleted publisher and disclaimer information; two used leaflet+label
• V2: One deleted additional information and added label
• V3: Five deleted warning box and overdose sections; four deleted drug names and notes; five added disclaimer; 1 added publisher section
Ratings of Distributed Versus Prototype Leaflets -- Atenolol Version 1
83
34
5851
36 33
9686
29
64
4433
57
100
0
25
50
75
100
1 2 3 4 5 6 7
Criterion
Distributed V1 Prototype V1
Ratings of Distributed Versus Prototype Leaflets -- Atenolol Version 2
85
40
59 59
44
67
9286
50
71
56
33
86
100
0
25
50
75
100
1 2 3 4 5 6 7
Criterion
Distributed V2 Prototype V2
Ratings of Distributed Versus Prototype Leaflets -- Atenolol Version 3
54
83
67
48 53
29
85
57
8679
61
78
36
100
0
25
50
75
100
1 2 3 4 5 6 7
Criterion
Distributed V3 Prototype V3
Conclusions (1)
• Highest ratings for scientific accuracy and being nonpromotional
• Lowest ratings for contraindications and precautions
Conclusions (2)
• Lowest ratings for leaflets:- < 5.5 inches long or
- font size < 10 point
• Lowest ratings for leaflets from: - independent pharmacies
- unidentified vendors
Conclusions (3)
• No prototype fully met criteria– Experts, consumers both critical of legibility
• Pharmacy organizations can influence ratings by – selecting vendor and leaflet version – modifying leaflets themselves