TBM ORIGINAL RESEARCH

Understanding adaptations to patient-centered medicalhome activities: The PCMH adaptations model

Tristen L. Hall, MPH,1 Jodi Summers Holtrop, PhD, MCHES,2 L. Miriam Dickinson, PhD,3

Russell E. Glasgow, PhD2

AbstractPrimary care practices have increasingly adopted thepatient-centered medical home (PCMH) model and oftenadapted quality improvement efforts to fit local context.This paper implemented a modified framework for un-derstanding adaptations in the context of primary carePCMH transformation efforts. We combined an adapta-tionsmodel by Stirman et al. that categorized adaptationsto evidence-based interventions in research studies withdimensions from the RE-AIM framework, as well as itemsspecific to PCMH. The resulting constructs were translatedinto a Bplain English^ adaptations interview. We con-ducted interviews with 27 practices and used resultingdescriptive categories to inform exploratory analyses ofthe relationships between adaptation characteristics andimprovement outcomes in PCMH domains of team-basedcare and data capacity. Practices most commonly focusedon development and use of disease registries and en-hancements to team-based care (not disease-specificoutcomes). Adaptations were common, with practicesmost frequently making changes to format or personnel.Adaptations were most often intended to increase effec-tiveness and based on pragmatic considerations. Gener-ally similar adaptation themes emerged across differentcontent topics (registry and quality improvement team).Adaptations initiated or carried out by the entire team ormade in early to middle stages of the project were mostrelated to outcome measures of team-based care anddata capacity. This paper extends adaptationmodels fromspecific interventions in research studies to PCMH qualityimprovement efforts. Despite limitations, the PCMH Ad-aptations Model provided a useful framework to under-stand adaptations in this context.

Keywords

Adaptations, PCMH, Quality improvement,Assessment, Fidelity, Registry

IntroductionMany papers examining implementation of evidence-based programs have reported on fidelity to interven-tion protocols and consistency of delivery of key in-tervention program components [1, 2]. This makesgood sense, given that one of the primary reasons forprogram failure is the lack of ability to replicate results

of successful programs when applied in communityand clinical settings [3]. That is, failure to implementthe intervention as designed. This, however, is onlypart of the picture, as implementation science andbehavior change literatures have also documentedthe importance of context and of tailoring interven-tions to local settings. Indeed, there is evidence thatsuccessful implementers are those who know how tosuccessfully adapt an intervention to specific settingsand circumstances [4, 5]. Therefore, there is a need tostudy how programs, policies, and quality improve-ment efforts are adapted from an original protocol orplan, and for frameworks to understand and measuresto assess these adaptations [6]. The purposes of thispaper are to investigate (a) the characteristics andperceived impact of different types of adaptations inthe context of quality improvement efforts based uponthe patient-centered medical home (PCMH) model[7–9]; (b) associations between those adaptation char-acteristics and outcomes on the Practice Monitor, anassessment tool measuring level of implementation ofkey elements of PCMH transformation; and (c) the fitof a proposed expanded conceptual model of adapta-tions, the PCMH Adaptations Model, for this newcontext.Adoption of the PCMHmodel [7] has been among

the most influential recent developments in primary

1Department of Family Medicine,University of Colorado AnschutzMedical Campus, 12631 E. 17th Ave.,Aurora, CO 80045, USA2Department of Family Medicine &Adult and Child Consortium forOutcomes Research and DeliveryScience (ACCORDS) Disseminationand Implementation Program,University of Colorado AnschutzMedical Campus, Aurora, CO, USA3Department of Family Medicine &Adult and Child Consortium forOutcomes Research and DeliveryScience (ACCORDS),University of Colorado AnschutzMedical Campus, Aurora, CO, USACorrespondence to: T HallTristen.Hall@ucdenver.edu

Cite this as: TBM 2017;7:861–872doi: 10.1007/s13142-017-0511-3

# Society of Behavioral Medicine 2017

ImplicationsPractice: Adaptations to quality improvement andPCMH transformation components are commonand can result in positive impacts on effectiveness.

Policy: Effective primary care quality improve-ment and PCMH transformation initiatives mustanticipate the potential for and evaluate the impactof adaptations during program implementation.

Research: Future research is needed to further in-vestigate, improve, and test the generalizability ofmodels to describe and understand adaptationsthat primary care practices make in the course ofPCMH transformation and similar quality im-provement initiatives.

TBM page 861 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

health care. PCMH is essentially a philosophy aboutcare bound by a set of principles and achievementsthat, taken together, focus on the goal of improving thequality, efficiency, and patient- and system-specificoutcomes of primary care. Features of the medicalhome include care that is patient-centered, compre-hensive, coordinated, accessible, and committed toquality and safety [8]. Meeting, documenting, and sub-mitting evidence of the achievement of these princi-ples can result in recognition as a PCMH-designatedpractice, which can result in improved reimbursementor other positive outcomes [9]. Evaluations of PCMHefforts have found that generally, PCMH-designatedpractices produce improved outcomes compared tonon-designated practices [10–12].A regional quality improvement and PCMH trans-

formation program in Colorado offered an opportuni-ty to apply and expand concepts of adaptations fromstrictly evidence-based interventions to a new contentarea, practice transformation. Most studies of adapta-tion have focused on tightly defined, evidence-based,and typically Bprotocolized or manualized^ interven-tions [13, 14]. This paper extends the study of adapta-tions to quality improvement efforts that are less tight-ly structured, but still designed to employ standardmethods and processes, such as team-based care andthe development of patient registries to guide efforts toimprove patient health outcomes in primary care.To study adaptations in this PCMH context, we

began with the framework of Stirman et al. [13], whichis based upon a review of the literature on adaptations.Stirman et al. developed a model to describe adapta-tions or modifications that organizations make to fit anevidence-based intervention to their setting. Theirmodel was developed based upon a systematic reviewof adaptations to evidence-based interventions. Thisand other studies have illustrated that there are severalchallenges to diffusion and implementation ofevidence-based interventions and that both plannedand unplanned adaptations are frequent [15, 16]. Fur-ther, various factors may function as either facilitatorsor barriers to implementation of evidence-based inter-ventions, depending on context. These data suggestthat adaptations are widespread, underscoring the im-portance of understanding the effectiveness of variousadaptations. Despite this, relatively little is knownabout adaptations, their context, positive and negativeoutcomes associated with various levels of fidelity oradaptation, and the robustness of adaptation conceptsacross different contexts and types of interventions[13]. To address these issues, the Robert Wood John-son Foundation funded grants to study local adapta-tions under a program envisioned by Leviton andcolleagues [17]. This report is the result of one of theseprojects.

MethodsThis study was funded by the Robert Wood JohnsonFoundation (Grant #71732) and approved by the Col-orado Multiple Institutional Review Board at the

University of Colorado Denver. Data were collectedin 2015 and early 2016, and analyses occurred in 2015and 2016.

Recruitment and settingParticipants were recruited from Colorado primarycare practices that participated in a year-long practicetransformation initiative consisting of trained practicefacilitators providing regular technical assistance onquality improvement (QI) and PCMH concepts.Prominent goals and frequent activities in the initiativefocused on the development of team-based care andpatient registries to guide chronic disease manage-ment. Patient registries are an important componentof the PCMHmodel because they allow the care teamto identify, maintain, and track patients with a particu-lar risk factor or condition, such as uncontrolled hy-pertension or diabetes, and perform specific activitiesto improve patient health outcomes. Most participat-ing practices specialized in family medicine, and justover half were part of a group (e.g., network, hospitalsystem, independent practice association). Key infor-mants who took part in interviews represented bothclinical and non-clinical roles and held positions suchas practice administrator, office manager, coordinator,director, or physician.For this study, a research assistant emailed and

phone called practices after an initial introduction bythe initiative’s practice facilitators, offering interviewsto theQI team leader or coordinator at the 56 practicesthat completed the initiative.

PCMH adaptations modelInitial pilot work and discussions with PCMH col-leagues revealed that although useful, the Stirmanet al. model might not include all the factors involvedin understanding adaptations made during quality im-provement efforts for PCMH transformation andwould need to be modified to enhance applicabilityto this topic and context. To address this, we addedcomponents from the RE-AIM planning and evalua-tion framework [18, 19] and others based upon prag-matic issues in primary care settings to help us betterunderstand the characteristics and purpose of adapta-tions, creating the BPCMH Adaptations Model.^Questions were framed in a Bwho, what, why, when,how^ format intended to be user-friendly and under-standable by practice personnel. Interview questionsfrom the Stirman et al. model included the domains ofWho: the individual(s) making the decision to modify,and What, consisting of both the type of content mod-ification and context of the adaptation. To this model,we added domains based on the RE-AIM framework(Reach, Effectiveness, Adoption, Implementation, andMaintenance) [18–20] and our study team’s experi-ence related to PCMH improvement efforts. This in-cluded the domains of Why: the reasoning behind theadaptation (e.g., to increase reach, effectiveness, adop-tion, implementation and/or maintenance; adding

ORIGINAL RESEARCH

TBMpage 862 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

response options to allow for the possibility that exter-nal factors influenced the adaptation’s development),and Impact: the perceived positive and/or negativeoutcomes of the adaptation. Interviewees wereinstructed to provide their own assessment of the im-pact of the adaptation on their original program goals,acknowledging that this was their personal perceptionof impact. To complete our PCMHAdaptations Mod-el, we added questions to represent the How domain:the adaptation’s basis, and When: the point during theprogram at which the adaptation was made. Table 1presents all questions and response options by domainfrom our structured interview for adaptations based onthe model.We tested this model and structured interviewmeth-

od in a pilot study of eight primary care organizationsthat participated in the Robert Wood Johnson Pre-scription for Health program [21]. The interview ques-tions and procedures worked well for these respon-dents and confirmed that the additional items helpedto address the PCMH topic and context. Minor word-ing modifications to clarify questions and responseoptions were made based on these interviews.

Data collectionWe used a structured interview guide based on ourPCMH Adaptations Model domains (Who, What,When, How, Why, Impact) designed to documenthow practices in the PCMH initiative adaptedtheir original plans. A study investigator (JH) andresearch assistant (TH) trained in qualitative datacollection and analysis conducted telephone inter-views with practice key informants. The first sev-eral study interviews were conducted includingthe study PI (RG) to ensure fidelity to the inter-view protocol and provide any necessary adjust-ments to interview methods. The remaining inter-views were conducted individually by the investi-gator or research assistant. Interviewers firstdiscussed the concept of adaptation with partici-pants, defined in the interview as Ba change toyour original plans or goals of the initiative,^ andasked participants to identify up to two separateadaptations. This definition of adaptation in thecontext of PCMH transformation was added basedon pilot interviews and has implications for par-ticipants’ characterization of the perceived positiveor negative impacts of the adaptation. Interviewersthen walked through each domain of the PCMHAdaptations Model and asked questions abouteach of these domains for up to two adaptationsthat the interviewee identified. Each interviewquestion had set response options, and partici-pants were asked to identify a primary and sec-ondary selection from these options (Table 1),along with providing a narrative explanation fortheir selections.Interviews took place over a ten-week period from

January to April 2015 and lasted between 25 and60 min. Interviewers took extensive notes

supplemented by audio recordings. Participants re-ceived a $50 gift card for their participation.

MeasuresThere are two types of data included in this paper: (1)descriptive information on the characteristics of adap-tations provided in interviews and (2) exploratoryanalysis of associations between adaptation character-istics and outcomes data for the practice changes madethrough the PCMH transformation. These data onperceived outcomes were collected via a separate as-sessment tool, the PCMH Practice Monitor describedbelow, as part of an independent evaluation of thePCMH initiative conducted by different research staffand separately from the PCMH adaptations interview.The study investigator and research assistant con-

ducted basic descriptive analysis to determine thetypes of adaptations identified by interviewees andthe frequency of corresponding response options, bydomain of the PCMH Adaptations Model. Responseswere organized into a Microsoft Excel spreadsheetaccording to categories based on the following do-mains of the PCMHAdaptations Model: Who, What,Why, When, How, and Impact.

Improvement outcomesAssessing outcomes in practice improvement in thePCMH initiative was complex and challenging be-cause practices focused on different disease conditionsand employed different change strategies. We devel-oped a composite outcome measure that was indepen-dent of adaptations and related to individual practicegoals and improvement progress based on previouslycollected assessment data. This measure was comput-ed using relevant scores from the separately adminis-tered Practice Monitor [22], a practice-based assess-ment for participating primary care clinicians and staffat baseline and 12-month follow-up during the initia-tive (and conducted by different staff independent ofthe structured adaptation interviews). The PracticeMonitor measures progress on 11 domains—Staff En-gagement, QI Processes, Data Capacity, PopulationManagement, Patient & Family Engagement, Team-BasedCare, Coordination of Care, Cost Containment,Access & Continuity, Behavioral Health Integration,and Leadership—that align with key elements ofPCMH transformation [23]. Each domain consists ofbetween five and nine items on which practices arescored on a five-point scale ranging from 0 (not at all)to 4 (completely). The study team carefully reviewedand discussed a priori the items in each domain anddetermined that the team-based care and data capacitydomain items best aligned with practice goals andactivities in this study and therefore were the mostrelevant domains to analyze for association with adap-tations. The team-based care domain includes fiveitemsmeasuring, for example, themaintenance of careteams and daily team huddles. The seven items con-stituting the data capacity domain include the frequen-cy of reporting and reviewing quality measures, the

ORIGINAL RESEARCH

TBM page 863 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

Table1|PCM

Had

aptatio

nsmod

el,correspon

ding

interviewqu

estio

ns,a

ndrespon

seop

tions

Dom

ain

Descriptio

nSo

urce

Interviewqu

estio

nsan

drespon

seop

tions

Who

Person

(s)who

initiated

thead

aptatio

nStirm

anWho

was

prim

arily

respon

sibleforinitiatingthismod

ificatio

n?–En

tireor

mostof

team

–Practitione

r–Adm

inistrator

–Re

searcher

–Develop

er–Stakeh

olde

r–Co

alition

Wha

tCo

nten

tof

theinterven

tion

Stirm

anWhich

ofthefollowingelem

ents

was

prim

arily

chan

gedas

partof

thisad

aptatio

n?–Thesetting

–Theform

at–Pe

rson

nelinvolved

–Thetarget

popu

latio

n–How

theinterven

tionispresen

ted

–Other

Which

ofthefollowingwas

theprim

arytype

ofchan

geinvolved

?–Tailo

ring

toindividu

als

–Add

ingacompo

nent

–Re

movingacompo

nent

–Co

nden

sing

acompo

nent

–Extend

ingacompo

nent

–Su

bstitutingforacompo

nent

–Ch

anging

theorde

rof

compo

nents

–Integratingwith

othe

rprog

ramswearedo

ing

–Re

peatingacompo

nent

–Loosen

ingthestructureor

protocol

–Otherwisechan

ging

theinterven

tion

Whe

nWhe

ndu

ringtheprojectthead

aptatio

nwas

mad

eStud

yteam

Atwhich

ofthefollowingpo

ints

intheprojectwas

thischan

geFIRS

Tmad

e?–Duringplan

ning

stages

before

beganinterven

tion

–Early

durin

gfirst

fewweeks

ofinterven

tion

–Duringthemiddlestages

–In

thelaterstages

–Ator

closeto

theen

dof

project

How

design

edHow

thead

aptatio

nwas

design

ed;on

wha

tba

sis

Stud

yteam

Wha

twas

theprim

aryba

sison

which

thischan

gewas

mad

e?–Based

onou

rvision

orvalues

–Based

onafram

ework(for

exam

plePC

MH)

–Based

onou

rkn

owledg

eor

expe

rienceof

working

with

patie

nts

ORIGINAL RESEARCH

TBMpage 864 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

–Based

onQId

ata,

summaryinform

ation,

orresults

–Based

onpragmatic/practical

considerations

(for

exam

ple

Bthisistheon

lyway

itwou

ldwork^)

–Based

onfin

ancial

incentives/paymen

t–Based

onfeed

back

orsuggestio

ns(practicefacilitator/coa

chor

othe

r)–Other

Why

Reason

ingbe

hind

thead

aptatio

nRE

-AIM

Which

ofthefollowingwas

theprim

aryreason

behind

thischan

ge?

–To

increase

thenu

mbe

ror

type

ofpa

tientscontacted(reach)

–To

enha

ncetheim

pact

orsuccessof

theinterven

tionforallo

rim

portan

tsubg

roup

s(effectiven

ess)

–To

makeitpo

ssible

toinvolvemoreteam

s,team

mem

bers

orstaff(ad

optio

n)–To

maketheinterven

tionde

livered

moreconsistently;to

better

fitou

rpractice,

patie

ntflo

wor

EHR;

forpractical

reason

s(im

plem

entatio

n)–To

institu

tiona

lizeor

sustaintheinterven

tion(m

ainten

ance)

–To

respon

dto

external

pressuresor

policy

–To

save

mon

eyor

othe

rresources(im

plem

entatio

n)–Other

Impa

ctRe

sults

ofthead

aptatio

n,po

sitiveor

negative

RE-AIM

Which

ofthefollowingresults

orim

pact

was

theprim

aryresultof

thisad

aptatio

n?–Nomajor

chan

ges

Increasedor

decreased:

–Num

beror

type

ofpa

tientsen

gaged(reach)

–Qua

lityof

care

orothe

rou

tcom

es(effe

ctiven

ess)

–Pa

rticipationby

team

sor

staff(ad

optio

n)–Co

nsistent

deliveryof

quality

care

orcosts(im

plem

entatio

n)–Mainten

ance

orsustaina

bilityof

theinterven

tionin

thepractice(m

ainten

ance)

–Mainten

ance

orsustaina

bilityof

thepa

tient

with

intheinterven

tion(m

ainten

ance)

–Re

imbu

rsem

ento

rfin

ancial

implications

forthepractice

–Efficiency(gettin

gmoredo

nefaster

orwith

less

resources)

ORIGINAL RESEARCH

TBM page 865 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

ability to extract data from the medical record systemfor patient registries, and data accuracy. Team-basedcare and data capacity domain summary scores werecomputed by transforming responses to a scale of oneto five, calculating the sum of responses to each item inthe domain, and scaling to a summary score out of 100possible points to facilitate comparison across scales.Practice Monitor summary scores reflect pre–post as-sessments in key criteria for PCMH recognition andare to be distinguished from the ongoing adaptationsinvolving the process of implementation assessed inthis study.BSuccess^ in this initiative was challenging to define.

Practices addressed different chronic illnesses andtypes of improvement, so it was not possible to selecta single common outcome as in most randomizedcontrolled trials. Some practices were new to the facil-itated practice improvement initiatives, while othershad been previously involved in PCMH initiatives andwere focusing on new areas of change after previouslymaking substantial gains. Thus, a simple change scoreon the Practice Monitor subscales was not a goodindicator of Bsuccess^ that could be applied to allpractices, since practices’ baseline levels of experience,ski l ls , PCMH status, and activit ies varied.Operationalization of the outcome measures fromthe Practice Monitor subscales is described below.

AnalysesThe study team conducted basic descriptive analysis(frequencies of response options selected by adapta-tion topic) to identify the most common characteristicsof adaptations in the areas of team-based care andchronic disease registries. These descriptive analyseswere used to inform selection of a subset of adaptationcharacteristics for analyses of relationship to out-comes. Topical categories of adaptations that emergedfrom initial descriptive analysis indicated the mostrelevant domains of Practice Monitor data to includein this analysis. To maintain a reasonable number ofpotential predictor variables, it was necessary to in-clude in quantitative analyses all adaptations describedrather than considering QI team and registry adapta-tions separately, and to reduce the adaptations datasetby collapsing similar response options. The entirestudy team utilized a group discussion process to de-cide which interview response options to combine,based on existing research frameworks, interview re-spondents’ conceptualization of response options, andthe frequency and relationship of selected responseoptions (e.g., deleting those options almost never se-lected or always selected). These decisions were madeindependently of and before considering outcomes.For instance, in response to the Why question ofBWhich of the following was the primary reason be-hind this change?^ response options related to ele-ments of the RE-AIM framework (reach, adoption,implementation, maintenance) were combined be-cause they clustered together, while the Beffectiveness^response option was left independent due to its high

selection frequency. Each adaptation characteristic se-lected for exploratory analysis was coded yes/no (en-dorsed versus not endorsed) for each practice (N = 27)for either adaptation described in the structuredinterviews—that is, a practice would be endorsed asByes^’ for the What—Format response option if thiswas their primary or secondary response option select-ed for either of the two adaptations they described.Outcome variables were created based on Practice

Monitor scores for the data capacity and team-basedcare subscales as described above at baseline and 12-month follow-up. Exploratory analyses on the largersample of practices (N= 58; this number is inconsistentwith the interview sample due to two interviews takingplace with representatives of multi-specialty clinicswhich completed two separate Practice Monitors) inthis initiative using hierarchical agglomerative clusteranalytic approaches suggested three general patternsof change: (1) lower baseline scores with less improve-ment (n = 3) or decline (n = 1); (2) lower baselinescores with substantial improvement (n = 23); and (3)high baseline scores that were maintained over time(n = 31). However, except for the single practice thatactually declined, these distinctions are somewhat ar-bitrary, so a continuous outcome variable was opera-tionalized by calculating total area under the curve(AUC) for baseline to follow-up scores for each sub-scale and across subscales, thus capturing overall ca-pacity plus improvement spanning the 12-monthtimespan of the PCMH initiative. AUC has been com-monly used in other scenarios to combine multiplemeasures over time to reflect some total quantity overa designated time span, and the combined measurecan be an outcome or predictor [24–26]. Kendall’s tauwas used to examine the bivariate associations be-tween the dichotomized adaptation variables (pres-ent/absent) and the continuous AUC outcome vari-ables. Due to the small sample size and limited num-ber of variables that could be included in a multivari-able model, multivariable linear regression using aforward stepwise approach with 0.1 to enter the model(in order of lowest p value) was used to explore therelationship between adaptations and the AUCoutcomes.

ResultsTwenty-seven practices participated out of 56 totalpossible initiative-participating practices. Most partic-ipating practices specialized in family medicine, withsmall representation from internal medicine. Abouthalf were identified as part of a group such as a net-work or hospital system. Practices which declined torespond or participate were relatively similar to thosethat took part in interviews: most specialized in familymedicine (p = 0.19), followed by internal medicine;about two-thirds were part of a system or group(p = 0.60), compared to just over half of participatingpractices. Measured by the number of clinicians at thepractice, comparison indicates that non-participatingpractices may have been slightly smaller, with a mean

ORIGINAL RESEARCH

TBMpage 866 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

of just under three providers compared to just underfour among participating practices (p = 0.09). SeeTable 2 for comparison of select characteristics ofparticipating and non-participating practices.

Adaptation topicsA total of 49 adaptations were described by the 27practices. Responses to probes at the beginning of theinterview to elicit the topical focus of the adaptationindicated that changes to registry and changes to theQI team were by far the most common categories ofadaptations, though these were also the focus of muchof the program guidance received, so this may be aresult of the initiative in which the practices participat-ed. The most common adaptation topic involved theregistry (37%, n = 18 adaptations), characterized by achange in the process, goals, or outcomes of develop-ing a registry. For example, multiple practices experi-enced a change in the vendor or software utilized forthe electronic health record or for patient registriesduring the initiative, as a result of factors outside ofthe QI team’s control. Others experienced challengesobtaining access to accurate data and registry reportsto guide improvement efforts. These issues led to de-lays and forced practices to adapt and utilize whateverinformation they could access to guide PCMH trans-formation efforts.Modifications to the QI team and movement

toward team-based care aspects of the PCMH,described as a change in team structure, member-ship, or meeting times, made up one-third ofadaptations (33%, n = 16). Examples of theseadaptations include adding meetings aligning withorganizational structure at practice and leadershiplevels to coordinate QI efforts, or reducing thenumber of QI team members involved in allmeetings due to levels of staffing, resources, orphysician engagement.

Characteristics of adaptationsReview of all 49 adaptations, across topic categories,indicated that within the Stirman et al. frameworkdomain of What, personnel changes (e.g., adding orremoving roles from meeting attendance, revising thepersonnel responsible for tasks) occurred most often(n = 27, 55% of adaptations), followed by changes informat (e.g., revising the frequency and schedule ofmeetings, utilizing Microsoft Excel for patient regis-tries instead of the EHR system) (n = 23, 47% ofadaptations). Within the Who domain, the entire teamwas most often responsible for initiating or carryingout the adaptation (n = 25, 51%), and adding a com-ponent (What) was the primary type of adaptationinvolved (n = 18, 37% of adaptations). Coding oncontextual and RE-AIM factors of How and Whyshowed that most adaptations were based primarilyon pragmatic or practical issues (n = 24, 49% of adap-tations) and performed with the intention or hope ofincreasing either or both program effectiveness

(n = 22, 45%) or implementation (n = 21, 43%). Thevast majority took place (When) during the early (21,n = 43%) or middle (21, n = 43%) stages of the pro-gram. See Table 3 for frequencies of the most com-monly selected response options in each domainacross all adaptations.Characteristics of adaptations reported for

modifications to the registry in comparison toQI team were similar for many domains includ-ing Who (most often made by the team), When(most often in the early or middle stages of im-provement activities), and Impact (the vast major-ity of adaptations being perceived as having pos-itive impact in comparison to having not madethe adaptation at all). There were some differ-ences in response patterns between registry andQI team adaptations on the How domain, withpragmatic and data-based responses chosen moreoften for registry adaptations, and feedback, prac-tical, and vision reasons being similarly commonwith one another for QI team adaptations. For-mat changes were more frequent for registry thanQI team adaptations, though as mentionedabove, this was still a fairly common characteris-tic described in both categories of adaptations. Interms of specific type of adaptation, tailoring wasmuch more common for QI team modificationsthan registry adaptations. Finally, for the Whyquestion, in order to increase effectiveness wasselected much more often for QI team adapta-tions, while implementation was a more commonreason within the category of adaptations to theregistry. Frequency counts of the most commonresponse options in each domain of the PCMHAdaptations Model are presented for all adapta-tions, and separately for categories of registry andQI team adaptations in Table 3.

Relationship of adaptation characteristics to outcomesIn order to maintain sufficient ratio of subjects tovariables for analyses of association, characteris-tics of adaptations were considered without re-gard for the type of adaptation (e.g., QI team orregistry); that is, analyses included all 49 adapta-tions described. Further, these analyses were per-formed at the practice level (N = 27), consistentwith practice-level outcomes on the identifiedPractice Monitor summary outcome measures ofdata capacity and team-based care. For analyses,adaptations characteristics were coded by wheth-er each practice selected a given response optionfor either of the adaptations they described (i.e.,endorsed vs. not endorsed). Table 4 summarizesresults of analyses relating adaptation character-istics to Practice Monitor outcomes of (a) datacapacity, (b) team-based care, and (c) both ofthese outcome categories combined.There were no significant negative associations be-

tween identified adaptations and outcomes. Because oflimited power, bivariate associations are noted for all

ORIGINAL RESEARCH

TBM page 867 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

Table2|C

haracteristicsof

participatingan

dno

n-pa

rticipatingpractices

Participatingpractices

(n=27

)Non

-participatingpractices

(n=29

)

Practicespecialty

#Practices

%Practices

#Practices

%Practices

Family

med

icine

2281

1862

Internal

med

icine

27

310

Wom

en’she

alth;OB-GYN

14

27

Pediatrics

00

27

Other

–Co

llege

health

–Co

mbina

tion(ped

iatrics,family

med

icine,

internal

med

icine,

and/or

urgent

care)

27

414

Practiceow

nershipa

#Practices

%Practices

#Practices

%Practices

Partof

grou

p(network,

hospita

lsystem,IPA

)15

5616

67Not

partof

grou

p12

448

33Practicesize

b

Rang

e(m

in–max)

Mean,

med

ian

Rang

e(m

in–max)

Mean,

med

ian

Num

berof

clinicians

(MDor

DO)

0–12

3.8,

3.0

0–9

2.8,

2.0

aDue

toincompleteprog

ram

applicationda

ta,informationregardingpracticemem

bershipin

grou

pispresen

tedfor24

of29

non-pa

rticipatingpractices

bPracticesize

characteristicsarepresen

tedforthe23

practices

forwhich

data

wereavailable,

ofthe29

practices

that

declined

topa

rticipatein

interviews

ORIGINAL RESEARCH

TBMpage 868 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

p values < 0.15 (instead of the customary 0.05). Thestrongest bivariate associations were between adapta-tions to promote the impact or success (Why) of theintervention and better team-based care (p = 0.0018).Adaptations initiated and/or carried out by the team(Who) were associated with both better team-basedcare (p = 0.0136) and data capacity (p = 0.1022). Ad-aptations that occurred in the early or middle stages ofthe intervention (When) were associated with betterteam-based care (p = 0.0586) and data capacity(p = 0.0370). Adaptations to modify content of theintervention to fit the target population (What) andadaptations of tailoring tomake the intervention betterfit the practice and its resources (Why) were associatedwith better team-based care (p = 0.1176; p = 0.1437).Responses from the category of Impact were

not included in analyses relating adaptations tooutcomes because of potential confounding of

characteristics of adaptations (perceived Impact)with their (separately measured) outcomes. Mul-tivariate analyses, although considered prelimi-nary due to the small number of practices, re-vealed that variables from the categories of Whenand Why contributed the most to the summaryoutcome measures (see Table 4).

DiscussionThis paper extends application of adaptationmodels from specific interventions to general pro-gram plans, quality improvement, and the PCMHmovement. We found it helpful to add componentsto the original Stirman et al. framework to fit thePCMH, and possibly other primary care contexts.Consistent with literature in other content areas,

Table 3 | Counts of most frequent response selections for registry and QI team adaptations

Type of adaptation

PCMH adaptations model domain All adaptations(n = 49)

Registry(n = 18)

QI team(n = 16)

Number of adaptationsa

What (element)Personnel 27 11 11Format 23 12 6How presented 14 3 7

What (type)Adding 18 6 6Tailoring 11 1 6Integrating 10 3 6

WhoTeam 25 9 8Practitioner 18 7 7Administrator 13 4 6Researcher (practice facilitator) 7 4 2

WhyImprove effectiveness 22 2 12Implementation 21 9 6Maintenance 8 3 3External pressure 6 5 0

WhenEarly 21 7 8Middle 21 8 8

How designed (basis)Pragmatic or practical 24 10 6Data 11 8 2Feedback 14 1 7Vision or values 13 4 5

Perceived impactPositive 38 13 13Effectivenessb 23 10 7Adoptionb 17 4 8Efficiencyb 12 3 7Implementationb 12 6 2

a Number of registry and QI team adaptations reported with each characteristic as a primary or secondary selection, of 49 total adaptations. Exclusion ofinfrequently selected response options means that the sum within each domain for each type of adaptation is variable, and exclusion of personnel and focus-related adaptations means that counts within QI team and registry adaptations will not always sum to BAll adaptations^ columnb In any direction (positive or negative impact)

ORIGINAL RESEARCH

TBM page 869 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

adaptations to PCMH transformation were com-mon [13, 14, 27] and may have positive results.Across all 27 practices’ adaptation activities,

regardless of topic, practices most commonlymade adaptations derived from the Stirmanet al. framework: modifications to personnel(Who) from those originally responsible (endorsedin at least one adaptation by 21 practices) as wellas change in program format (What) (endorsed inat least one adaptation by 20 practices). Resultsfrom the RE-AIM elements indicated that adap-tations were most often made (Why) for pragmat-ic reasons (endorsed in at least one adaptation by22 practices) in an effort to increase effectiveness(Why-endorsed in at least one adaptation by 19practices). Other interview items relevant toPCMH indicated that the majority of adaptationsoccurred around early to middle stages (bothendorsed in at least one adaptation by 21 prac-tices) of PCMH enhancement efforts. Despite theoverall similarities, there were some differences

between adaptations to the registry and qualityimprovement team in the domains of How, What,and Why; contributing to our understanding ofthe specific characteristics of adaptations of vari-ous topics.Preliminary analysis of the association between

adaptation characteristics and Practice Monitoroutcomes indicated that the adaptations moststrongly related to relevant outcomes on PracticeMonitor domains were the following:

& in response to the Who question, based on theoriginal Stirman model, adaptations initiated orcarried out by some or most of the team;

& in response to Why questions, based on theRE-AIM framework, adaptations performedto enhance effectiveness or implementation;

& in response toWhen questions, added to the originalStirman model to fit primary care interventions,adaptations performed in early or middle stages ofthe program.

Table 4 | Associations between practice-level adaptations and practice monitor outcomes

Bivariate associations

Domain: responseoption

# Practices endorsing(of 27)

Area under the curve(AUC)

Team AUC Data AUC

Kendall’s tau Kendall’stau

Kendall’stau

What: format 20 −0.0361 0.1475 −0.1094What: personnel 21 −0.0571 −0.0778 0.0048What: target population 12 0.1034 0.2602* 0.0362What: tailoring toindividuals

10 0.1392 −0.0502 0.0827

Who: team 17 0.2456* 0.4100* 0.2688*Why: effectiveness 19 0.0346 −0.2035 0.2405*Why: implementation 17 0.0737 0.5188* −0.0992When: early 19 0.0173 0.3142* 0.0569When: middle 18 0.3523* 0.0729 0.3431*How: vision or values 17 0.0573 0.1506 −0.0620How: pragmatic orpractical

22 0.1120 −0.1144 0.1748

How: data 11 −0.1127 0.1563 −0.0853

Multivariable linear regression

Domain: response option AUC Team AUC Data AUCCoef (p value) Coef (p value) Coef (p value)

What: formatWhat: personnelWhat: target population 10.17 (0.0271)What: tailoring to individualsWho: team 18.59 (0.0236)Why: effectiveness 15.78 (0.0642)Why: implementation 39.1 (0.0685) 26.83 (<0.0001)When: earlyWhen: middle 58.2 (0.0106) 13.70 (0.0102) 19.83 (0.0183)How: vision or valuesHow: pragmatic or practicalHow: data*Indicates p < 0.15 for Kendall’s tau

ORIGINAL RESEARCH

TBMpage 870 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

These significant associations share similarities withinterviewees’ most frequently selected response op-tions in these domains:

& most or all of the Team was the most commonparty responsible for initiating or carrying out theintervention (Who);

& improving effectiveness and implementation wereby far the most common reasons to perform adap-tations (Why);

& adaptations taking place in the early or middlestages of the program made up the vast majorityof adaptations (When).

These commonalities between descriptive analysesand analysis of association with improvement out-comes as measured by a separate standardized toolreinforce the relevance of these domains of the PCMHAdaptations Model. The association of relevant im-provement outcomes with response options originat-ing from domains of all sources (the original Stirmanmodel, the RE-AIM framework, and the study team’sadditions for the PCMH context) suggests that main-taining as well as extending the original Stirman et al.model categories to include these additional factorswas warranted. The PCMH Adaptations Model canhelp practices to better anticipate, categorize, and un-derstand the types of adaptations commonly made inPCMH transformation efforts and aid decision-making by providing insight as to how these adapta-tions might relate to potential outcomes.In the context of practice improvement and PCMH

transformation, adaptations identified by practiceswere defined as changes in their original plans to enactimprovement activities, often mid-course correctionsto uncontrollable outside factors rather than changesto specificQI tools or elements of the PCMHmodel. Itmay be beneficial for future research to explore adap-tations tomore specific and strictly defined elements ofPCMH transformation, such as enhanced patient ac-cess or coordinated care, or other primary care ororganizational change efforts, to enhance knowledgeof the outcomes associated with these types of adapta-tions or facilitate use of the PCMHAdaptationsModelby focusing the discussion of adaptations to a morespecific topic area.Our PCMHAdaptations Model was able to charac-

terize important adaptations, but challenges and limi-tations were also noted. Providing an example of whatwe meant by an adaptation at the start of the interviewhelped interviewees understand the concept, butcarries the possibility of biasing respondents’ recollec-tion of their practices’ adaptations. Still, some respon-dents had difficulty differentiating adaptations toplanned goals of the change program from more gen-eral changes in their clinic. There were also relativelyhigh numbers of response options to choose from inmany domains, ranging from 5 (When) to 11(What—type), making it difficult for some respondentsto differentiate or choose among responses. The

terminology of some questions and response optionspresented challenges in interpretation. For example,what is a Bcomponent^ when the term is applied to aquality improvement effort (vs. a clearly defined re-search protocol)? Based on these findings and lessonslearned from these interviews, we have recently mod-ified the interview framework to include more open-ended responses, which are then coded into the vari-ous response categories. This revised interview is be-ing used in recently initiated investigations and is pub-lically available from the senior author.One limitation is the relatively small sample that

involved only a limited, although diverse, selection ofpractices in Colorado that participated in one PCMHtransformation initiative. Participants self-selected, andwhile participating and non-participating practices hadsimilar specialties and only minor differences in size,we cannot determine the extent to which the resultsare generalizable to other practices. Given the slightlygreater mean number of providers in participatingpractices, it is possible that practice sizemay be a factorin practices’ level of implementation and types ofadaptations made throughout PCMH transformation.The focus in the PCMH initiative onQI team activitiesand chronic disease registries, among other topics,likely influenced participants’ identification of adapta-tions. Interviews occurred in some cases one year ormore after the conclusion of program participation, soparticipants’ recollection of specific details and infor-mation may have been incomplete. As previouslydescribed, assessment of the impact of adaptations onprogram progress and goals was based on inter-viewees’ perceptions rather than objective outcomesdata. Finally, in most cases we interviewed only onerepresentative per practice, this being the leader orcoordinator of practice QI activities, whose experi-ences and perspective of adaptations and perceptionof their impact may have been different from otherteam members.Due to these limitations, the perceived impacts of

adaptations as described using the structured PCMHAdaptations Model interview guide should beinterpreted with caution. This interpretation shouldinclude the understanding that these are the subjectiveinterpretation of interview participants. Associationsbetween various adaptation characteristics and im-provement outcomes asmeasured by the data capacityand team-based care domains of the Practice Monitordemonstrate more concrete assessment of progresstoward program goals for improvement. Althoughadmittedly exploratory, use of cluster analytic andregression approaches helped to understand the com-plex relationships among adaptations and outcomes.Future investigations should determine if these resultscan be replicated with larger samples and other prima-ry care, quality improvement, and health care teaminterventions.In summary, despite challenges associated with the

nature of PCMH transformation as a topic and theuser-friendliness of some interview questions, thePCMH Adaptations Model provided a useful

ORIGINAL RESEARCH

TBM page 871 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020

framework with which to identify and characterize pat-terns of adaptations made in the course of PCMH trans-formation.We identified common types of adaptationsand found differential relationships of different typesof adaptations to improvement. Future research isneeded to further investigate and improve the PCMHAdaptations Model, to replicate these results, and totest their generalizability.

Acknowledgements:We would like to acknowledge the HealthTeamWorksorganization for their valuable knowledge and assistance in recruiting andproviding historical data on practices; recognize the contributions of the primarycare practices that participated in this study; and thank the Robert WoodJohnson Foundation for financial support making this study possible.

Compliance with ethical standardsThe findings reported have not beenpreviously published, and this manuscript is not being simultaneously submit-ted elsewhere.The authors have full control of all primary data and agree to allow the journalto review the data if requested.

Funding:This study is funded by the Robert Wood Johnson Foundation grant#71732.

Ethical approval: All procedures performed in studies involving humanparticipants were in accordance with the ethical standards of the institutionaland/or national research committee and with the 1964 Helsinki Declaration andits later amendments or comparable ethical standards.

Statement on the welfare of animals: This study did not involve thestudy of animals.

Disclosure of potential conflicts of interest: The authors declare thatthey have no conflicting or competing interests related to this study.

Informed consent: Informed consent was obtained from all individualparticipants included in this study.

1. Rohrbach LA, Dent CW, Skara S, Sun P, Sussman S. Fidelity ofimplementation in project Towards No Drug Abuse (TND): a com-parison of classroom teachers and program specialists. Prev Sci .2007; 8: 125–132.

2. Allen J, Linnan L, Emmons K. Fidelity and its relationship toimplementation, effectiveness, adaptation, and dissemination.In: Brownson R, Colditz G, Proctor E, eds. Dissemination and Imple-mentation Research in Health. New York: Oxford University Press;2012:281–304.

3. Bellg A, Borrelli B, Resnick B, et al. Enhancing treatment fidelity inhealth behavior change studies: best practices and recommen-dations from the NIH Behavior Change Consortium.Health Psychol.2004; 23(5): 443–451.

4. Cohen D, Crabtree B, Etz R, et al. Fidelity versus flexibility: trans-lating evidence-based research into practice.Am J PrevMed. 2008;35(5 Suppl): S381–S389.

5. Heifetz R, Grashow A, Linsky M. The Practice of Adaptive Leadership:Tools and Tactics for Changing Your Organization and the World. Bos-ton, MA:Harvard Business Press; 2009.

6. Rabin BA, Purcell P, Naveed S, et al. Advancing the application,quality and harmonization of implementation science measures.Implement Sci. 2012; 7: 119.

7. Kellerman R, Kirk L. Principles of the patient-centered medicalhome. Am Fam Physician. 2007; 76(6): 774–775.

8. Patient-Centered Primary Care Collaborative. Defining the medicalhome: a patient-centered philosophy that drives primary careexcellence 2015 [Available from: http://www.pcpcc.org/about/medical-home.]

9. National Committee for Quality Assurance. Patient centered med-ical home recognition 2015 [Available from: http://www.ncqa.o r g / P r o g r a m s / R e c o g n i t i o n / P r a c t i c e s /PatientCenteredMedicalHomePCMH.aspx.]

10. Shi L, Lee D, Chung M, Liang H, Lock D, Sripipatana A. Patient-centered medical home recognition and clinical performance inU.S. community health centers. Health Serv Res. 2017; 52(3): 984–1004

11. Jackson G, Powers B, Chatterjee R, et al. Improving patient care.The patient centered medical home: a systematic review. AnnIntern Med. 2013; 158(3): 169–178.

12. Alexander J, Markovitz A, Paustian M, et al. Implementation ofpatient-centered medical homes in adult primary care practices.Med Care Res Rev. 2015; 72(4): 438–467.

13. Stirman SW, Miller CJ, Toder K, Calloway A. Development of aframework and coding system for modifications and adaptationsof evidence-based interventions. Implement Sci. 2013; 8: 65.

14. Castro FG, Barrera Jr. M, Martinez Jr. CR. The cultural adaptation ofprevention interventions: resolving tensions between fidelity andfit. Prev Sci 2004; 5: 41.

15. Lundgren L, Amodeo M, Cohen A, Chassler D, Horowitz A. Modifi-cations of evidence-based practices in community-based addic-tion treatment organizations: a qualitative research study. AddictBehav. 2011; 36(6): 630–635.

16. Embry D, Biglan A. Evidence-based kernels: fundamental units ofbehavioral influence. Clin Child Fam Psychol Rev. 2008; 11(3): 75–113.

17. Leviton L, Henry B, eds. Better Information for GeneralizableKnowledge: Systematic Study of Local Adaptation. American Pub-lic Health Association 139th Annual Meeting and Exposition;Washington, DC 2011.

18. Gaglio B, Shoup JA, Glasgow RE. The RE-AIM framework: a system-atic review of use over time. Am J Public Health. 2013; 103(6):e38–e46.

19. Kessler R, Purcell E, Glasgow R, Klesges L, Benkeser R, Peek C.What does it mean to “employ” the RE-AIM model? Eval HealthProf. 2013; 36(1): 44–66.

20. Reach Effectiveness Adoption Implementation Maintenance(RE-AIM): Virginia Tech; 2016 [Available from: http://re-aim.org.]

21. Green L, Cifuentes M, Glasgow R, Stange K. Redesigning primarycare practice to incorporate health behavior change: prescriptionfor health round-2 results. Am J Prev Med. 2008; 35(5): S347–S3S9.

22. Fernald DH, Jortberg BT, Dickinson LM, et al. Tools to assesspatient centered medical “homeness” in primary care. AHRQNational PBRN Research Conference; Bethesda, MD 2011.

23. DickinsonWP, Jortberg BT, O'Neill C, editor. The PCMHmonitor: aneffective tool to assess PCMH progress in family medicine prac-tices. Society of Teachers of Family Medicine Annual Spring Con-ference; Seattle, WA 2012.

24. Whitelaw DC, Clark PM, Smith JM, Nattrass M. Effects of the neworal hypoglycaemic agent nateglinide on insulin secretion in type2 diabetes mellitus. Diabet Med. 2000; 17: 225–229.

25. Yeh S. Using trapezoidal rule for the area under a curve calcula-tion. Proceedings of the Twenty-Seventh Annual SAS®User GroupInternational (SUGI) Conference. 2002; 27.

26. Joerger M, Huitema AD, Krahenbuhl S, et al. Methotrexatearea under the curve is an important predictor in patientswith primary CNS lymphoma: a pharmacokinetic-pharmaco-dynamic analysis from the IELSG no. 20 trial. Br J Cancer.2010; 102(4): 673–677.

27. Aarons G, Green A, Palinkas L, et al. Dynamic adaptation processto implement an evidence-based child maltreatment intervention.Implement Sci. 2012; 7: 32.

ORIGINAL RESEARCH

TBMpage 872 of 872

Dow

nloaded from https://academ

ic.oup.com/tbm

/article-abstract/7/4/861/4810353 by University of C

olorado systems user on 13 January 2020