Full Terms & Conditions of access and use can be found athttps://www.tandfonline.com/action/journalInformation?journalCode=khvi20

Human Vaccines & Immunotherapeutics

ISSN: 2164-5515 (Print) 2164-554X (Online) Journal homepage: https://www.tandfonline.com/loi/khvi20

Implementing pharmacy-located HPV vaccination:findings from pilot projects in five U.S. states

William A. Calo, Parth D. Shah, Melissa B. Gilkey, Robin C. Vanderpool, SarahBarden, William R. Doucette & Noel T. Brewer

To cite this article: William A. Calo, Parth D. Shah, Melissa B. Gilkey, Robin C. Vanderpool,Sarah Barden, William R. Doucette & Noel T. Brewer (2019) Implementing pharmacy-located HPVvaccination: findings from pilot projects in five U.S. states, Human Vaccines & Immunotherapeutics,15:7-8, 1831-1838, DOI: 10.1080/21645515.2019.1602433

To link to this article: https://doi.org/10.1080/21645515.2019.1602433

View supplementary material Accepted author version posted online: 04Apr 2019.Published online: 07 May 2019.

Submit your article to this journal Article views: 163

View related articles View Crossmark data

Citing articles: 1 View citing articles

RESEARCH PAPER

Implementing pharmacy-located HPV vaccination: findings from pilot projects infive U.S. statesWilliam A. Calo a,b, Parth D. Shah *c, Melissa B. Gilkeyd,e, Robin C. Vanderpoolf, Sarah Bardeng, William R. Doucetteh,and Noel T. Brewer d,e

aDepartment of Public Health Sciences, Penn State College of Medicine, Hershey, PA, USA; bPenn State Cancer Institute, Hershey, PA, USA; cTheHutchinson Institute for Cancer Outcomes Research, Fred Hutchinson Cancer Research Center, Seattle, WA, USA; dDepartment of Health Behavior,Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC, USA; eLineberger Comprehensive Cancer Center, University ofNorth Carolina, Chapel Hill, NC, USA; fDepartment of Health, Behavior and Society, College of Public Health, University of Kentucky, Lexington, KY,USA; gMichigan Pharmacists Association, Lansing, MI, USA; hHealth Services Research Division, University of Iowa College of Pharmacy, Iowa City, IA,USA

ABSTRACTPharmacies are promising alternative settings for human papillomavirus (HPV) vaccination because oftheir population reach, convenience, and existing infrastructure for vaccine delivery. However, pharma-cies in the US are rarely used for adolescent HPV vaccination. We sought to document challenges andopportunities of implementing pharmacy-located HPV vaccination services in five US states by mappingprocess evaluation results onto key implementation science constructs: service penetration, acceptabil-ity, appropriateness, feasibility, fidelity, adoption, and sustainability. Pilot projects were planned in NorthCarolina (k = 2 pharmacies), Michigan (k = 10), Iowa (k = 2), Kentucky (k = 1), and Oregon (no pharmacyrecruited) with varying procedures and recruitment strategies. Sites had open enrollment fora combined 12 months. Despite substantial efforts in these states, only 13 HPV vaccine doses wereadministered to adolescents and three doses to age-eligible young adults. We identified two majorreasons for these underperforming results. First, poor outcomes on service penetration and appropri-ateness pointed to engagement barriers: low parent demand and engagement among pharmacystaff. Second, poor outcomes on feasibility, adoption, and sustainability appeared to result from admin-istrative hurdles: lacking third party reimbursement (i.e., billing commercial payers, participation inVaccines for Children program) and limited integration into primary care systems. In summary, pilotprojects in five states all struggled to administer HPV vaccines. Opportunities for making pharmaciesa successful setting for adolescent HPV vaccination include expanding third party reimbursement tocover all vaccines administered by pharmacists, increasing public awareness of pharmacists’ immuniza-tion training, and improving care coordination with primary care providers.

ARTICLE HISTORYReceived 16 January 2019Revised 11 March 2019Accepted 25 March 2019

KEYWORDSHPV vaccine; pharmacies;pharmacists; alternativevaccination settings; scopeof practice; implementationscience

Introduction

Up-to-date human papillomavirus (HPV) vaccination in theUS has increased since the vaccine’s introduction over a decadeago to 49% of adolescents ages 13–17 in 2017.1 However,vaccination coverage remains far below the Healthy People2020 goal of 80% for adolescents ages 13–15.1,2 As a strategyto improve uptake, the President’s Cancer Panel3,4 and theNational Vaccine Advisory Committee5 have recommendedexpanding HPV vaccine provision in pharmacies.

Pharmacies can play a meaningful role in increasing HPVvaccination for several reasons. First, pharmacies are geogra-phically accessible to most families, including many ruralcommunities.6 Most U.S. residents (89%) live within fivemiles of a pharmacy.7 Second, pharmacies have extendedhours of operation and normally give vaccinations withoutan appointment.8,9 Third, 48 US states and the District ofColumbia allow pharmacists to administer HPV vaccination

(the exceptions are New York and New Hampshire).10 Fourth,many pharmacists are trained immunizers and alreadyadminister vaccines.

Pharmacists’ ability to increase HPV vaccine uptake is limitedby their scope of practice to vaccinate age-eligible adolescents,which varies greatly by state.10 For instance, state laws may limitvaccination practices to certain ages or the arrangement underwhich pharmacists can administer HPV vaccine (e.g., indepen-dent authority, collaborative practice agreement, or by prescrip-tion only).11 Recent national surveys of primary care physiciansand parents show that most supported HPV vaccination ofadolescents by trained pharmacists.12–14

Pharmacists administer millions of vaccine dosesevery year,8,15 but HPV vaccines represent a vanishinglysmall fraction of those doses delivered.8 Although severalauthors have noted challenges and barriers related to estab-lishing pharmacy-located HPV vaccination programs,13,16–19

CONTACT Noel T. Brewer ntb@unc.edu Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina, 325A RosenauHall CB 7440, Chapel Hill, NC 27599, USA

*shared first authorship; authors were affiliated with the University of North Carolina at Chapel Hill while conducting the studySupplemental data for this article can be accessed on the publisher’s website.

HUMAN VACCINES & IMMUNOTHERAPEUTICS2019, VOL. 15, NOS. 7–8, 1831–1838https://doi.org/10.1080/21645515.2019.1602433

© 2019 Taylor & Francis Group, LLC

no study we are aware of has documented the experiences ofimplementing HPV vaccination programs in real-world phar-macy settings. Our study evaluated the implementation ofHPV vaccination services in community pharmacies throughpilot projects in several states, mapping this process ontoconstructs from implementation science (Table 1).

Results

Service penetration

All sites experienced low or no service penetration of pharmacy-located HPV vaccination for adolescents. In North Carolina, fiveHPV vaccine doses were given to adolescents; four doses wereadministered by nursing staff using the standing order protocoland one by a pharmacist who was a clinical pharmacist practi-tioner (CPP). In Michigan, pharmacists administered no HPVvaccine doses to adolescents and three doses to adults (first dosefor a female and third doses for two males). In Kentucky, threevaccine doses were delivered to adolescents. Five HPV vaccinedoses were given by a pharmacist in Iowa, while none wereadministered in Oregon whose program did not successfullyrecruit a pharmacy site.

Acceptability

Parents who got HPV vaccine for their children in participat-ing pharmacies found the service highly acceptable. In NorthCarolina, for example, parents completed a satisfaction survey(while being monitored for potential post-vaccination sideeffects) and a follow-up survey (mailed 3 months after vacci-nation). The parents of the five adolescents who received HPVvaccine at a pharmacy in North Carolina reported that it wasconvenient, the waiting time was acceptable, the privacy andconfidentiality were satisfactory, the pharmacy’s appearancewas what they expected from a place that provides qualityhealth care, and that they would recommend other parents getHPV vaccine for their children at a pharmacy. Parents inNorth Carolina also reported that the pharmacists and nurseswho educated about and administered HPV vaccine commu-nicated clearly, spent enough time with them, made theirchildren feel comfortable during vaccination, and gave thevaccination safely.

Appropriateness

Appropriateness of pharmacy-located HPV vaccination variedby pharmacy site and stakeholder. Participating pharmacistswere knowledgeable about vaccines in general not just HPVvaccine, had the training to immunize adolescents, and wereable to report vaccines administered to state immunizationregistries. In addition, in Michigan, the study team conductedfour stakeholder surveys with providers, parents, patients, andpharmacists about HPV vaccination in pharmacies. Theappropriateness of delivering HPV vaccines in pharmacieswas supported by a majority of those interviewed. However,some pharmacy staff exhibited resistance to offering HPVvaccination, which challenged the implementation of thepilot projects. Three pharmacy locations in Michigan, forexample, had pharmacy staff who did not fully engage in theproject because they did not approve of the vaccine. The Iowaproject also encountered some reluctance from a physicianwho had concerns about the potential revenue loss caused bysending existing patients to complete dose series ata pharmacy. This concern was alleviated during recruitmentof clinics into the study when the clinic manager expressedinterest in participating in the project. Limited time was alsoa significant challenge for researchers and pharmacy teams. InIowa, Kentucky, and Oregon, investigators faced delays infunding disbursement that impeded their ability to commenceplanned recruitment of adolescents and young adults to getHPV vaccination.

In addition, pilot project offerings were not completelyaligned with pharmacies patient demographics. In Iowa, forexample, one of the two pharmacies was in a rural town of6,000 people, yielding a reduced pool of eligible adolescentsavailable for participation. Additionally, some participatingpharmacies in Michigan and Iowa served populations thatspeak little English (e.g., French, Swahili). Promotional andeducational materials were only available in English andSpanish, and pharmacists were not readily able to counselthese patients in their native language.

Feasibility

The pilot projects faced substantial challenges related to fea-sibility. Third party reimbursement for HPV vaccination

Table 1. Definitions and example measures of implementation outcomes assessed in pilot projects.

Construct Definitiona Example measure

ServicePenetration

Extent to which an evidence-based practice is integrated within a servicesetting and its subsystems. Also known as “reach.”

Number of HPV vaccine doses given to vaccine eligiblepatients contacted during the pilot project

Acceptability Extent to which implementation stakeholders perceive a service to beagreeable, palatable, or satisfactory

Parents’ satisfaction with the waiting time, privacy, pharmacy’sappearance, and interactions with staff

Appropriateness Perceived fit, relevance, or compatibility of the evidence-based practice fora given practice setting, provider, or consumer

Pharmacy staff’s support for HPV vaccination

Feasibility Extent to which a new practice can be successfully used or carried out withina given setting

Number of health insurance plans covering HPV vaccine dosesgiven at a pharmacy

Fidelity Degree to which an implementation strategy was delivered as prescribed in theoriginal protocol

Percentage of protocol items implemented as planned

Adoption Intention, initial decision, or action to try or employ an evidence-base practicein a service setting. Also called “uptake.”

Number of physicians referring patients to the pharmacy

Sustainability Extent to which a recently implemented practice is maintained andinstitutionalized within a service setting‘s ongoing operations

Number of pharmacies applying and obtaining for VFCprogram designation

a Definitions adapted from Gerke D, Lewis E, Prusaczyk B, Hanley C, Baumann A, Proctor E. Eight toolkits related to Dissemination and Implementation.Implementation Outcomes. 2017. [accessed 2018 Nov 28]. https://sites.wustl.edu/wudandi/.

1832 W. A. CALO ET AL.

within the pharmacy was a major challenge for all sites.During the pilot period, most commercial health insuranceplans did not cover the cost of HPV vaccine administered inthe pharmacy. Pharmacies can bill the medical benefit insome instances, but there is no guarantee of reimbursement,which would result in the patient receiving a bill if the plandid not cover the cost of the vaccine. As many health insurerscover HPV vaccination for adolescents through the medicalbenefit when administered in a physician’s office, convincingpatients to pay out-of-pocket for the same service ina pharmacy was a significant barrier. In Iowa and Kentucky,for example, some parents turned away vaccination in thepharmacy due to insurance not covering the benefit in thatalternative setting.

Some pharmacies’ inability to participate in the federalVaccines for Children program, which is administered differ-ently by each state, was another major feasibility challenge.Medicaid patients up through age 18 can receive vaccinesthrough the program, but most states do not yet includepharmacies as qualified providers. In Kentucky and NorthCarolina, the study pharmacies were not enrolled as providersin the Vaccines for Children program and were deterred fromdoing so due to time constraints and programmatic require-ments. In contrast, one participating pharmacy in Iowa andanother in Michigan were able to be recognized as Vaccinesfor Children program providers.

Limited pharmacy staffing was another challenge. Someparticipating pharmacies were understaffed to perform theirtypical daily workload, so offering a new service like adoles-cent HPV vaccination added too great of a burden on existingstaff. In Michigan, this issue was aggravated when severalpharmacies experienced large staff turnover during the projecttime period. When they hired new staff, adding project train-ing on top of their basic job training was not possible.Another challenge that hampered the feasibility of the projectswas low consumer awareness of HPV vaccine availability inpharmacies. Despite displaying an impressive array of com-munity-oriented advertisements promoting in-pharmacy vac-cination offering, many parents were not aware ofpharmacists’ immunization training and the availability ofadolescent vaccination in pharmacies. In Iowa, for example,multiple parents were surprised to learn their children can getHPV vaccination in pharmacies.

Fidelity

Excluding the challenges noted above, across sites, pharma-cies implemented the study protocols with high fidelity. InNorth Carolina, for example, the study team developed an18-item fidelity checklist to help pharmacists and nursesimplementing all aspects of the protocol when enrollinga patient in the study (online supplement). The investigatorsused the fidelity checklist to calculate the percentage ofprotocol items implemented as planned. With five patientsgetting HPV vaccine, the fidelity score was 100%. Other sitesalso developed a range of workflow materials to supportpharmacies implementing study protocols, which are avail-able online.20

Adoption

Pharmacy study sites had low adoption of HPV vaccinationservices. Protocols and procedures were not well integratedinto pharmacy workflow. Pharmacists were not successful inintegrating into the broader primary care ecosystem.Pharmacists across sites required a physician agreementauthorizing them to give HPV vaccine to adolescents, greatlylimiting their integration into the “medical neighborhood.”Some sites struggled finding physician partners for their pro-jects. In Michigan, for example, SpartanNash pharmacies hadan existing standing order protocol for adult patients but thesigning physician was not willing to provide a similar protocolfor adolescents. For their pilot project, the pharmacists foundanother physician who provided the needed standing orderprotocol for HPV vaccination for adolescents. Michiganfound that some physicians were very interested in referringpatients to pharmacies because they could not afford to stockHPV vaccines. However, the process of referring patients toa limited number of pilot pharmacies and not knowingwhether the pharmacy could bill for vaccination services pre-cluded many Michigan physician offices to agree ona standing order protocol.

Similarly, the research team in Kentucky was not successfulin engaging primary care providers to sign for a standingorder protocol, but the outreach efforts opened the opportu-nity to partnering with the local health department whichreadily agreed. This type of pharmacist-physician agreementalso restricts the ability of pharmacists to give HPV vaccine tochildren not covered by protocols or prescriptions. In NorthCarolina, the researchers employed a combination of CPPs forhandling physician referrals and registered nurses for givingvaccine to patient walk-ins. This is a model most pharmaciescould not integrate within their existing service infrastructuredue to budgetary and space requirements, though.

Sustainability

The pilot projects did not establish sustainable HPV vaccina-tion practices. Only one of the pharmacies continued provid-ing HPV vaccine to patients after the project ended. Manychallenges presented above, like limited third party reimbur-sement, inability to participate in the Vaccines for Childrenprogram, and low parent demand, were also reported as majorchallenges for maintaining stable HPV vaccination services inpharmacies.

Discussion

The 2018 President’s Cancer Panel report on HPV vaccina-tion reaffirmed their 2014 recommendation to use pharmaciesas one way to improve access to vaccination.4 Despite phar-macies’ noted conveniences and accessibility to patients,4

pharmacies continue to be underused for HPV vaccinationof adolescents. The multistate pilot projects struggled in dis-tributing vaccinations to adolescents, and our evaluation ofmapping these challenges onto implementation science con-structs identified two broad reasons for the poor performanceof pharmacy-located HPV vaccination. First, poor outcomes

HUMAN VACCINES & IMMUNOTHERAPEUTICS 1833

on service penetration and appropriateness point to engage-ment barriers: low parent demand and commitment frompharmacy staff. Second, poor outcomes related to feasibility,adoption, and sustainability appeared to be the result ofadministrative hurdles: lacking payer reimbursement and lim-ited integration into clinic systems. These findings also high-light the importance of including concepts fromimplementation science to improve program adoption atpharmacies.

The first set of challenges that resulted in underperform-ing vaccination programs were related to engagement bar-riers with vaccine provision. Low parent demand for HPVvaccines in pharmacies was a major challenge. The observedfinding may be explained by parents’ limited knowledge thatpharmacists can vaccinate adolescents.12,13 Unfamiliaritywith obtaining health care for children outside the tradi-tional medical office may leave parents unaware, or evenhesitant, to get vaccination at alternative settings. Studieswith parents of adolescents show that those who have hadprevious experience with vaccinating their children in phar-macies are more willing to get HPV vaccine frompharmacists.12 Pharmacies recruited in our pilot projectshad no prior experience administering Tdap or meningo-coccal vaccines to adolescents, which also left parents unfa-miliar with adolescent vaccinations at these settings.Pharmacies interested in implementing new or expandingupon existing immunization programs to include adolescentHPV vaccination may benefit by educating their clients andthe general public about pharmacists’ scope of practice andimmunization qualifications. Parent demand may alsoincrease by knowing how administered vaccines will becommunicated back to children’s primary care providers.

Low engagement of pharmacy staff was another significantbarrier for some sites. We noticed that for pharmacies withexisting immunization programs, it was less disruptive toincorporate adolescent HPV vaccination into their workflow.However, asking for additional study documentation andreporting may have created disengagement among somestaff. Researchers need to minimize new burdens on pharma-cies’ daily operations when implementing study protocols.When allowable by state law, implementation of adolescentvaccination programs should involve pharmacy technicians orstudent pharmacists. Low engagement was also observedwhen some pharmacies were incapable of fully involvingstaff due to their personal beliefs about HPV vaccination.Successful implementation of an innovation (e.g., practice,service) may be a function of intervention-values fit.21 Itmay be important to distinguish between unenthusiasticimplementation, which occurs when a novel intervention fitspoorly with staff values, and committed implementation,which occurs when an intervention fits well with staffvalues.21 Pharmacies interested in HPV vaccination shouldwork to stimulate an organizational climate that makes itmore appealing to the staff to support and implement thisnew offering. In such a way, the climate for implementationwould reinforce the intervention-values fit for pharmacist-delivered HPV vaccination.

The second set of challenges that resulted in underper-forming vaccination programs related to administrative

barriers. Lack of medical insurance reimbursement alongwith limited participation in public programs such asVaccines for Children, severely hindered pharmacies frommaximizing their participation in these pilot projects. Forinstance, although participating pharmacy sites in Michiganwere able to become Vaccines for Children program, provi-ders, time and resource constraints on the Michigan StateDepartment of Health to conduct mandatory inspectionsonly allowed one pharmacy to participate in this public pro-gram. Variable vaccination coverage through the pharmacybenefits also deterred parents from seeking HPV vaccinationfor their adolescent children in pharmacies. Several empiricalstudies and reviews have noted this as a chief reason forpharmacies not adopting vaccination platforms.9,15,17,18,22

Fragmented coverage between pharmacy benefits and medicalbenefits from insurance providers need to be reconciled andbetter aligned to the current changes in pharmacy practice,notably around the expanded role pharmacists play in vacci-nation. Future projects may benefit from partnering withpharmacies in states (e.g., Washington) where pharmacistsare recognized providers by third party insurers and can billfor vaccinations and participate in Vaccines for Childrenprogram.

Limited integration into primary care systems was alsoa significant administrative challenge for pharmacy-locatedHPV vaccinations. Due to each state’s differing pharmacypractice laws, most of the pilot projects had to use someform of a collaborative practice agreement to allow pharma-cists to vaccinate adolescents, ranging from more restrictiveCPP agreements (North Carolina) to more open standingorder protocols (Michigan). However, establishing collabora-tive practice agreements can be labor intensive for pharma-cists, requiring buy-in from physicians in many circumstancesand regulatory oversight by state medical and pharmacyboards. It was encouraging to find that pilot pharmaciescould use diverse methods for vaccination dose reportingand verification. By reporting doses to their state immuniza-tion information systems, pharmacists facilitate vaccinationmanagement and accountability, and help providers identify-ing missed opportunities for vaccinations.23

These pilot studies highlighted the importance of usingconstructs from implementation science to evaluate howwell the projects were implemented in diverse pharmacysites; specifically, we assessed well-established implementationoutcomes. Implementation outcomes serves as indicators ofthe implementation success, proximal indicators of imple-mentation processes, and key intermediate outcomes forintervention effectiveness.24 Only by mapping our processevaluation findings onto this set of implementation outcomeswe identified the main factors that drove the limited outcomesof the pilot projects and, at the same time, the potentialsolutions to overcome similar challenges in future projects.As such, when implementing new immunization services inpharmacies, practitioners and researchers must includea catalog of implementation science constructs to evaluatethe mechanisms and results of their implementation strate-gies. Although no precise guidelines exist to guide the choiceof implementation constructs to assess in studies, someauthors24,25 suggest taking into account the potential barriers

1834 W. A. CALO ET AL.

and opportunities to implementation, the novelty of the evi-dence-based practice to be implemented, the setting in whichimplementation will occur, the resources for evaluation, theunit of analysis, and the stage of implementation (e.g., earlyvs. later implementation).

In terms of strengths, our pilot projects were in five stateswith a range of pharmacy sites, yielding data for a range ofcapacities to implement HPV vaccination programs. Thesepilots also collected implementation outcome data usingmixed methods, offering a comprehensive look at the researchquestion and study findings from the perspective of manystakeholders (pharmacists, pharmacy staff, parents, primarycare providers, and researchers). The study limitations includethe brief study period for many of the projects, which wasinadequate to fully implement the proposed activities. Giventhe limited timeline, it was also not possible to formulateconclusions about the sustainability or cost of these pilots inthe long run. It is also important to acknowledge that theseprojects were in states where pharmacists do not have inde-pendent authority to administer HPV vaccine to adolescents.As such, experiences of pharmacists in states where no priorapproval from an accredited prescriber is required may differfrom those we reported. Finally, not all pilot projects collectedthe same implementation data nor used the same instruments,making some head-to-head comparisons among projects notpossible.

Conclusion

Pharmacies have great potential for vaccinating adolescents inthe US as demonstrated by their participation in adult vacci-nation. However, these diverse pilot projects suggest thatmuch of the unrealized potential is due to significant chal-lenges in implementation barriers related to administrationand engagement. Our findings suggest that pharmacists inter-ested in implementing effective HPV vaccination programsshould focus on increasing parent demand, gaining supportfrom their staff, expanding payer reimbursement, andimproving integration into clinic systems. Future studiesthat look to deliver HPV vaccines in pharmacies would ben-efit greatly by assessing implementation outcomes that wouldfacilitate the timely evaluation of implementation processes.Because any new interventions will not be effective if they arenot implemented well, the promise of pharmacy-located HPVvaccination cannot be advanced without attention to imple-mentation science. Future studies should also identify thosepharmacies already delivering HPV vaccination to many oftheir clients to document the elements that predict successfulimplementation.

Methods

Pilot project sites were in North Carolina, Michigan, Iowa,Kentucky, and Oregon, US with different study and vaccina-tion protocols at each site. Activities completed in Iowa,Kentucky, and Oregon are presented together as their projectwas sponsored as a single study with multiple sites. We out-lined the study protocols used for the different pilot projects,describing provider recruitment and training and patient

recruitment and vaccination protocols. We then evaluatedthe success of the pilot projects by mapping reported resultsto key constructs from implementation science: service pene-tration, acceptability, appropriateness, feasibility, fidelity,adoption, and sustainability.25 These implementation con-structs are commonly used in the literature to appraise howwell an evidence-based practice, like HPV vaccination, wasimplemented into a service setting.24 We provided a definitionfor each individual construct alongside an example on howthe constructs were evaluated from pilot projects in Table 1.For pilot results from North Carolina, we relied on studyrecords not published elsewhere. For the results from thepilot projects conducted in Michigan, Iowa, Kentucky, andOregon, we relied on reports publicly available on the websiteof the National HPV Vaccination Roundtable, which providedfunding for these projects.20

North Carolina

Provider recruitment and trainingThe University of North Carolina partnered with two inde-pendent pharmacies in Durham, NC to conduct theVaccination in Pharmacies study. North Carolina pharmacypractice laws allow pharmacists to administer HPV vaccinesto adolescents if pharmacists are designated as CPP. As a CPP,a pharmacist enters into a collaborative practice agreementwith a state-licensed physician to provide specific health careservices for a referred patient. The collaborative practiceagreements can be conducted as individual practice agree-ments (one pharmacist and physician) or collectively asa group (multiple pharmacists and physicians). No CPP foradolescent HPV vaccination existed in North Carolina priorto this project. The investigators recruited a family medicinedoctor in a local primary care clinic, who had an existingrelationship with one of the pharmacy sites, to serve as theprescriber on the CPP. They obtained CPP approval for twopharmacists, one at each pharmacy site, from the NorthCarolina Board of Pharmacy and the North CarolinaMedical Board, after extensive consultation with the boardsand advocacy by the University’s legal counsel. The CPPagreement included protocols for administering adolescentvaccines, documenting vaccines given, communicatingpatients’ immunization status to their primary care providers,and managing short-term side effects and other adverseevents. Since the CPPs were limited to seeing only referredpatients to the pharmacy, the investigators also employedregistered nurses at each pharmacy, for up to 10 hours perweek, to give HPV vaccines to walk-ins through a standingorder protocol supervised by a pediatrician.

Prior to launching the project, pharmacists and nursingstaff received in-person educational training on study proce-dures, including obtaining parental consent, administering thevaccines, managing potential side effects, billing insurers,using the North Carolina Immunization Registry to documentvaccines delivered, documenting vaccine administration usingstudy-specific process evaluation instruments, and adminis-tering study surveys to parents. The study team worked withparticipating pharmacists to integrate these protocols into thepharmacy’s clinical and administrative workflow.

HUMAN VACCINES & IMMUNOTHERAPEUTICS 1835

Patient recruitment and vaccination protocolPatient recruitment occurred from July 13, 2015 throughDecember 16, 2015. Recruitment efforts included: direct mail-ing to all families with one or more age-eligible children usingthe pharmacies’ patient databases, physician referrals to CPPs,print newspaper advertisements, posting fliers in locallibraries, grocery stores, and laundromats, promotions atcommunity events (e.g., Boys and Girls Clubs), and in-pharmacy promotion using posters, bag stuffers, and roadsidesigns. Additionally, online recruitment occurred using paidsocial media advertisements (e.g., Facebook) and parent list-serv at local elementary schools. Per protocol, when a parentexpressed interest in immunizing their adolescent child,a pharmacist or nurse screened the patient for adolescentvaccine eligibility and reviewed the adolescent’s immunizationregistry record to determine vaccination needs. After provid-ing parental consent, the adolescent received the vaccine anda pharmacist or nurse monitored the patient for 15 minutesfor potential post-vaccination side effects while parents com-pleted a survey questionnaire. The pharmacists then updatedthe adolescent’s immunization registry record and faxeda copy of the patient’s record to his/her primary care provider.A simplified flow diagram of this study protocol is depicted inFigure 1.

Michigan

Provider recruitment and trainingThe Michigan Pharmacists Association partnered with a regionalchain, SpartanNash Pharmacies, to conduct their pilot project in10 pharmacy locations. Five pharmacies were in the Grand Rapidsmetropolitan area (Kent County) and each of the other fivepharmacies was in one of five rural counties (Ottawa, Barry,Clare, Cass and Tuscola). In Michigan, state law allows pharma-cists to administer HPV vaccines through standing orders fromcollaborating physicians. SpartanNash pharmacies had an existingadult standing order protocol in place for HPV vaccination forpatients aged 18 to 26 years. For this project, a second physician inGrand Rapids provided a new adolescent standing order protocol

for HPV vaccination for patients aged 11 to 17 years. Prior tolaunching the project, pharmacists and pharmacy technicians ateach of the 10 participating pharmacies completed an educationaltraining program on HPV and HPV vaccination, either in a liveclassroom session or a hybrid session combining videos andfollow-up conference calls.

Patient recruitment and vaccination protocolThe recruitment period was from May 1, 2016 through July 31,2016. Based on an initial assessment of the workflow at one ofthe pharmacies, a screening tool was developed to identifypotential patients. Two versions were created, one for adoles-cents and another for adults. The screening tool was placed inthe patients’ prescription bag alongside an age-appropriate vac-cination education handout. When the patient came into thepharmacy to pick up their medication, they were asked tocomplete the screening tool during their prescription transac-tion. Of the 2,342 screening tools distributed, 429 (18%) werereturned. The pharmacist then reviewed the screening tool anddiscussed HPV vaccination options with the parent of the ado-lescent patient or the adult patient using a resource guide to aidvaccination discussions. For patients interested in getting thevaccine whose insurance covered administration in the phar-macy or who were willing to pay cash, the pharmacist adminis-tered the vaccine. As a part of the screening for vaccinationeligibility and dose documentation process, pharmacists usedthe Michigan Care Improvement Registry, an electronic state-wide system for tracking childhood and adolescent vaccination.

Iowa, Kentucky, and Oregon

Provider recruitment and trainingA group of three academic institutions that were members ofthe Centers for Disease Control and Prevention (CDC)-funded Cancer Prevention and Control Research Network(CPCRN) in Iowa, Kentucky, and Oregon conducted thepilot project in their respective states. The institutions werethe University of Iowa, the University of Kentucky, andOregon Health & Science University in collaboration withthe Northwest Portland Area Indian Health Board. The inves-tigators developed a common protocol for local pharmacies topartner with healthcare clinics to implement coordinateddelivery of HPV vaccines to patients ages 11–18 years. Theinvestigators then approached pharmacies for study participa-tion. Interested pharmacies provided contact information fora clinic they wished to collaborate with; the investigators thencontacted clinics for recruitment. In Iowa, two local indepen-dent pharmacies, Towncrest Pharmacy (Iowa City) andOsterhaus Pharmacy (Maquoketa), partnered with SoutheastIowa City Clinic and the Medical Association of Maquoketa,respectively. In Kentucky, Total Care Pharmacy, a pharmacybelonging to a local chain with six locations, partnered withthe district health department.26 In Oregon, six tribes withpharmacies were approached to discuss project participationbut none were recruited. Prior to launching the project,investigators offered in-person training for pharmacy-clinicpartners that included review of study protocol, distributionof project and HPV educational materials, discussion of rolesand responsibilities, enrollment in the state’s Vaccines for

Figure 1. Study protocol from North Carolina pilot project. Abbreviations:NCIR = North Carolina Immunization Registry; EHR = Electronic health records;VAERS = Vaccine Adverse Event Reporting System; PCP = Primary care provider;MD = Physician.

1836 W. A. CALO ET AL.

Children program, and an assessment of the compatibility ofelectronic medical record systems.

Patient recruitment and vaccination protocolThe recruitment period in Kentucky was from July 2016through September 2016; Iowa started recruiting inSeptember 2016; and Oregon was not able to start their project.Patient recruitment efforts varied by site but overall, activitiesincluded: displaying CDC posters in pharmacies, direct mailingto existing customers, running 237 30-second ads through localradio stations, biweekly newspaper advertising, and promotionsat back-to-school events. Per study protocol, clinics adminis-tered the first dose of the HPV vaccine series, and transmittedprescription orders for second and third doses to partneringpharmacies. Pharmacies then scheduled doses with patients;after three failed attempts to contact patients, pharmacies noti-fied the health clinic of non-response. Pharmacies transmittedvaccination information to corresponding state immunizationregistries and sent immunization information to the patient’shealthcare provider via fax or electronically. The protocol wastailored as necessary to match the local needs at respectivepharmacy-clinic partners and investigators’ institutions.

Disclosure of potential conflicts of interest

Dr. Brewer has received commercial research grants from Merck andPfizer and served as a paid advisory board member for Merck. He ischair of the National HPV Vaccination Roundtable which is funded byCDC and hosted by the American Cancer Society. The other authorshave no financial disclosures or potential conflicts of interest to report.

Funding

This study was funded by Merck Sharp & Dohme Investigator StudiesProgram (grant #50928 to North Carolina) and the American CancerSociety (grant #32608 to Iowa, Kentucky and Oregon). Authors’ time weresupported in part by training or career development awards from theNational Cancer Institute (K22 CA186979 to Gilkey and R25 CA116369to Calo) and the Agency for Healthcare Research and Quality (T32HS000032 to Shah). Funders played no role in: 1) study design; 2) thecollection, analysis, and interpretation of data; 3) the writing of the report;or 4) the decision to submit the manuscript for publication; AmericanCancer Society [#32608]; Merck Sharp and Dohme [#50928].

ORCID

William A. Calo http://orcid.org/0000-0001-8452-5063Parth D. Shah http://orcid.org/0000-0002-3536-6219Noel T. Brewer http://orcid.org/0000-0003-2241-7002

References

1. Walker TY, Elam-Evans LD, Yankey D, Markowitz LE,Williams CL, Mbaeyi SA, Fredua B, Stokley S. National, regio-nal, state, and selected local area vaccination coverage amongadolescents aged 13–17 years—United States, 2017. MMWRMorb Mortal Wkly Rep. 2018;67(33):909–17. doi:10.15585/mmwr.mm6733a1.

2. Reagan-Steiner S, Yankey D, Jeyarajah J, Elam-Evans LD,Singleton JA, Curtis CR, MacNeil J, Markowitz LE, Stokley S.National, regional, state, and selected local area vaccination cover-age among adolescents aged 13-17 years–United States, 2014.MMWR Morb Mortal Wkly Rep. 2015;64(29):784–792.

3. President’s Cancer Panel. Accelerating HPV vaccine uptake:urgency for action to prevent cancer. A report to the presidentof the United States from the president‘s cancer panel. Bethesda(MD): President’s Cancer Panel; 2014.

4. President’s Cancer Panel. HPV vaccination for cancer preven-tion: progress, opportunities, and a renewed call to action.A report to the president of the United States from the chairof the president’s cancer panel. Bethesda (MD): President’sCancer Panel; 2018.

5. National Vaccine Advisory Committee. Overcoming barriers tolow HPV vaccine uptake in the United States: recommendationsfrom the national vaccine advisory committee: approved by thenational vaccine advisory committee on June 9, 2015. PublicHealth Rep. 2016;131(1):17–25. doi:10.1177/003335491613100106.

6. Vanderpool RC, Stradtman LR, Brandt HM. Policy opportunitiesto increase HPV vaccination in rural communities. Hum VaccinImmunother. 2019 Jan 4. [accessed 2019 Jan 8]. doi:10.1080/21645515.2018.1553475.

7. National Association of Chain Drug Stores. Home page. [accessed2018 Dec 1]. https://www.nacds.org/.

8. Goad JA, Taitel MS, Fensterheim LE, Cannon AE. Vaccinationsadministered during off-clinic hours at a national communitypharmacy: implications for increasing patient access andconvenience. Ann Fam Med. 2013;11(5):429–36. doi:10.1370/afm.1542.

9. Goad J, Bach A. The role of community pharmacy-based vaccina-tion in the USA: current practice and future directions. IntegrPharm Res Prac. 2015;4:67–77.

10. American Pharmacists Association. Immunization Center -Immunization Authority [accessed 2018 Dec 11].https://www.pharmacist.com/immunization-center?is_sso_called=1.

11. Brewer NT, Chung JK, Baker HM, Rothholz MC, Smith JS.Pharmacist authority to provide HPV vaccine: novel partners incervical cancer prevention. Gynecol Oncol. 2014;132:S3–S8.doi:10.1016/j.ygyno.2013.12.020.

12. Calo WA, Gilkey MB, Shah P, Marciniak MW, Brewer NT.Parents‘ willingness to get human papillomavirus vaccination fortheir adolescent children at a pharmacy. Prev Med.2017;99:251–56. doi:10.1016/j.ypmed.2017.02.003.

13. Shah PD, Calo W, Marciniak MW, Gilkey MB, Brewer NT.Support for pharmacist-provided HPV vaccination: national sur-veys of US physicians and parents. Cancer Epidemiol PrevBiomarkers. 2018;27(8):970–78. doi:10.1158/1055-9965.EPI-18-0380.

14. Shah PD, Calo WA, Marciniak MW, Golin CE, Sleath BL,Brewer NT. Service quality and parents‘ willingness to get adoles-cents HPV vaccine from pharmacists. Prev Med. 2018;109:106–12.doi:10.1016/j.ypmed.2018.01.002.

15. Burson RC, Buttenheim AM, Armstrong A, Feemster KA.Community pharmacies as sites of adult vaccination:A systematic review. Hum Vaccin Immunother. 2016;12(12):3146–59. doi:10.1080/21645515.2016.1215393.

16. Westrick SC, Hohmann LA, McFarland SJ, Teeter BS, White KK,Hastings TJ. Parental acceptance of human papillomavirus vacci-nations and community pharmacies as vaccination settings:A qualitative study in Alabama. Papillomavirus Res.2017;3:24–29. doi:10.1016/j.pvr.2016.12.003.

17. Hastings TJ, Hohmann LA, McFarland SJ, Teeter BS,Westrick SCJP. Pharmacists’ attitudes and perceived barriers tohuman papillomavirus (HPV) vaccination services. Pharmacy(Basel). 53). 2017;5(3):45. doi:10.3390/pharmacy5030045.

18. Shah PD, Gilkey MB, Pepper JK, Gottlieb SL, Brewer NT.Promising alternative settings for HPV vaccination of USadolescents. Expert Rev Vaccines. 2014;13(2):235–46.doi:10.1586/14760584.2013.871204.

19. McRee A-L, Reiter PL, Pepper JK, Brewer NT. Correlates ofcomfort with alternative settings for HPV vaccine delivery. HumVaccin Immunother. 2013;9:306–13.

20. National HPV Vaccination Roundtable. Homepage. [accessed2018 Nov 8]. http://hpvroundtable.org/.

HUMAN VACCINES & IMMUNOTHERAPEUTICS 1837

21. Klein KJ, Knight AP. Innovation implementation: overcoming thechallenge. Curr Dir Psychol Sci. 2005;14(5):243–46. doi:10.1111/j.0963-7214.2005.00373.x.

22. Islam JY, Gruber JF, Lockhart A, Kunwar M, Wilson S, SmithSB, Brewer NT, Smith JS. Opportunities and challenges of ado-lescent and adult vaccination administration within pharmaciesin the United States. Biomed Infor Insights.2017;9:1178222617692538.

23. Community Preventive Services Task Force. Recommendation foruse of immunization information systems to increase vaccinationrates. J Public Health Manag Pract. 2015;21(3):249–52.doi:10.1097/PHH.0000000000000092.

24. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G,Bunger A, Griffey R, Hensley M. Outcomes for implementationresearch: conceptual distinctions, measurement challenges, andresearch agenda. Adm Policy Ment Health. 2011;38(2):65–76.doi:10.1007/s10488-010-0319-7.

25. Gerke D, Lewis E, Prusaczyk B, Hanley C, Baumann A, Proctor EEight toolkits related to dissemination and implementation.Implementation Outcomes. [accessed 2018 Nov 28]. https://sites.wustl.edu/wudandi/.

26. Vanderpool R, Pilar M, Barker J, Freeman P. Increasing HPVvaccination through community pharmacy partnerships: lessonslearned from a pilot project. Ky Pharm. 2017;12:33–36.

1838 W. A. CALO ET AL.