Practical Challenges Of Systems Thinking an Practical Challenges Of Systems Thinking And Modeling in Public Health

8/6/2019 Practical Challenges Of Systems Thinking an Practical Challenges Of Systems Thinking And Modeling in Public Health

1/10

P r a c t i c a l C h a l l e n g e s o f S y s t e m s T h i n k i n g a n dM o d e l i n g i n P u b l i c H e a l t h

Wiii iam M, Troc him, PhD, Derek A, Cabrera , iVlA, Bobby M ilste in, iVIPH, Richard S, G al lagher, BS, and Sc ott J, Leischow , PhD

Modem public health practice encompasses acomplex, loosely coupled system' of actors in-cluding govemmental entities at the intem a-tional, national, regional, and local levels; adiverse conglomeration of nongovemmentalorganizations (such as foundations, advocacyand specicil interest groups, coalitions andpartnerships, for-profit and nonprofit medicalsystems, and businesses); and citizens in thepublic at large. The broad array of threats towell-being, ranging from obesity and tobaccouse to violence and infectious diseases, canbe most aptly portrayed from a complex andadaptive system perspective.

Systems thinking and modeling are broadclasses of intellectual endeavors that are beingincorporated increasingly into contemporarypublic health. Research has proven both thegeneral potential of systems thinking^"" andapplications in specific areas,'^" * Empiricalstudies related to complex systems have ap-peared of late in n otable medical journals, in-cluding ihe Joumal ofthe American MedicalAssociation, Lancet, and the N e w EnglandJoumal of Medidne?^'^^ The authors of anInstitute of Medicine re port. Crossing th eQuality Chasm: A N e w Health System for th e21st Century,"^''''^'^^ used a systems perspec-tive to delineate 10 "simple mles to guide theredesign of the health care system" and de-scribed the entire health care system as acomplex adaptive system:

A health care system can be defined as a set ofconnected or interdependent parts or agents-including caregivers and patientsbound by acommon purpose and acting on tiieir knowi-edge. Heaith care is complex because of thegreat number of interconnections within andamong smail care systems.'"' ' ' ' '"

Systems thinking encompasses and is conso-nant with ecological models^^""** familiar topublic health practitioners, including the ideasof human ecology, population health, and thesocial determ inants of public health. But it goesbeyond these models, incorporating advances

Objectives. Awareness of a n d support for systems thinking and mode ling in thepublic health field are growing, yet there are many practical challenges to im-pleme ntation. We sought to identify and describe these challenges from the per-spectives of p racticing public health professionals.Methods. A systems-based methodology, concept mapping, was used in astudy of 133 participants fro m 2 systems-based pu blic health initiatives (the Ini-tiative for the Study and Implementation of Systems and the Syndemics Pre-vention Network), This method identified 100 key challenges to implem entation

of systems thinking and modeling in public health work.Results. The project resulted In a map ide ntifying 8 categories of challenges andthe dynamic interactions among them.Conclusions. Implementation by public health professionals of the 8 simplerules we de rived from the clusters in the map iden tified here will help to addresschallenges and improve the organization of systems that protect the public'shealth, (Am J Public Health. 2006;96:538-546, doi:10,2105/AJPH,2005,066001)

over the pa st decades, particularly infieldssuchas system dynamics and complexity theory.

It is relatively easy to identify examples ofpublic health issues that can be understoodaccurately only by examining the complexand dynamic part-and-whole interactionsthat make up systems. For instance, considerthe area of tobacco control. At the policylevel, it is reasonab le to argue that the 196 4surgeon general's report on smoking^^ hadprofound effects on the policy debate withconsequences for smoking prevalence andconsumption to this day. The report itselfwas the product of a complex series ofevents that led to its production. In tum, itset off a cascade of events and changes. It isvirtually impossible to determine the effectsof that important event in isolation, as a partthat is separable from the whole.

For instance, the report was most likely animportant catalyst in creating a public policyclimate that enabled the litigation that led tothe Tobacco Settlement Agreement severaldecades later, to increased taxation of ciga-rettes by states, to legal restrictions on smokingin public places, and to tobacco counteradver-tising. By the same token, the report may haveled to unanticipated "negative" consequencesby spurring the tobacco industry to adapt its

product marketing, lobbying, and public reltions and perhaps indirectly contributed to creation offi-ontgroups and covert efforts tounderm ine tobacco control research,""*

There are countless other examples. Smers may react to fears about the harmful efects of smoking by sw itching to so-called"light" cigarettes (i,e,, low tar and nicotine mulations as determined by machine smoktests). Yet, despite their manufacturers' clai"light" cigarettes may actually increase theprevalence of a more lethal type of lung cacer, probably because of the way the ir venlated filters alter the physical act of smok-ing,^'"*^ Or consider the way the industryresponds to youth advertising restrictions.Billboard advertising was outlawed by theMaster Settlement Agreement, In response,the tobacco companies increased advertisinand promotion in retail stores, effectively saurating the retail environment with producimages,'*'' Thus, banning billboard advertisihas led to even more children being exposeto pro-tobacco messages as they go abouttheir daily Uves,

Even the cigarette itself is a complex system, a highly engineered pro duct designed extreme elasticity of delivery.''^ And tobacccontrol as a public endeavor can be viewed


2/10

mimm hm mmmi

fighting o-control, as in all other

odeling is ubiquitous.Despite the growing cognizance of and

most recent developments. Sec-overview of public health

modeling, particularly the

odeling in public health.

Systems thinking is a general conceptualween parts and their relationships to

even greater whole. It is an-

dynamics of the economy. But sys-of the

porary science. Systemsmethodological tradition that

Computational modeling and simulation, as acomplement to experimentation and theory,are hallmarks of recent systems thinking andthe systems sciences.''

The depth eind breadth of systems sciencecan be bewildering, particularly as one first isintroduced to its underlying principles andformulations. Consider just a few of the topicsassociated with contemporary systems think-ing: causal feedback''*; stock-flow structuresand open and closed systems'*; centralized,decentralized, heterarchical, hierarchical, andself-organizing systems^'^'; autopoiesis^^"^'';nonlinear systems and chaos^^; complexadaptive systems' "^; boundciry conditions,seeding, power laws, phase transitions, univer-sality, and renormalizadon^^'^"; silo effects^';^^^''; celluleir automata''; fractalself-similarity'"^; general systemscybernetics'''''^'; control theory^*; informationtheory"^; computational simulation''^; deci-sion and game theory"; system dynam-ics'^"^'; evolution, biology, and ecology'*"*';small world phenomena^"*''; and set, graph,and network theoiy.=^'^'*'''*2'3'*^'''

The vastness of the literature alone can beoverwhelming, and it is not easily summa-rized. We offer 2 organizing ideas (dynamicsand complexity) and 2 infiuential metaphors{mechanical and biological) that can help usunderstand this daunting array. In addition,we consider 2 common misconceptions aboutsystems thinking that are important to an un-derstanding of these ideas and metaphors.Dynamics and Complexity

Dynamics. Whether a system settles into astate of equilibrium, changes in repeating cy-cles, or changes in even more complicatedways, a common theme is change. A fieldcalled dynamics, with its own rich history dat-ing back to Isaac Nevrton, provides a vocabu-lary and a methodology for understandingthese changes.^^ Terms such as the "butterflyeffect"*'** and the "tipping point,"*'' whichare now part of the public vernacular, havetheir source in the study of chaotic systems.In chaos, "a deterministic system exhibits ape-riodic behavior that depends sensitively onthe initial conditions, thereby rendering long-term prediction impossible."^^''''" This conceptis just one of the many useful ideas in thefield of dynamics.

Complexity. Most systems in the publichealth arena eire complex in that they consistof many interacting stakeholders with oftendifferent and competing interests. Agents inthese networks must constantly adapt to theactions of others and to a changing environ-ment that is in turn affected by the actionsof the agents themselves. Such systems are notcontrolled centrally; they are self-organizing.Complexity theory, or the study of complexadaptive systems,^''^* focuses on understand-ing systems of this typ e. Most defmitions ofsuch systems include some notion of the rela-tionship between the emergent or unpre-dictable behav ior of a system and au-tonomous agents self-organizing by simplerules. For example, one description of a com-plex adaptive system is independent variablesfollowing simple local rules leading to emer-gent comp lexity,"'"'''" and another descriptionsuggests that a working definition of a com-plex system is "one whose properties are notfully explained by an understanding of itscomponent parts."""''"' Simple rules, networksof adaptive agents, feedback, self-organization,and emergence are hallmarks of complexadaptive systems.

A good example of a complex adaptivesystem that is familiar to virtually everyone isthe emergency medical services (EMS) systemthat comes into play in some way in virtuallyevery medical or health emergency that anindividual experiences. It illustrates well howa complex adaptive system is made up ofvarious independent agents following simplerules and interacting locally with other inde-pendent agents in the system. In the EMS,each agent has a role to play and a simpleset of rules to follow. These roles and rulesare graduated in a linked chain from first re-sponder to emergency room doctor to reha-bilitation specialist

For a citizen in first responder role, therules typically instruct the agent to activatethe EMS system by "calling 91 1" and thenadminister basic care until professional helparrives. Next in the chain, the professionalrescuer, firefighter, or emergency medicaltechnician (EMT) plays a different role andfollows a different rule set with primaryresponsibility for advanced field care andtransport. In tum, the EMT initiates theinvolvement of the next set of agents by


3/10

communicating with the emergency room. Ifneeded, the emergency room connects withthe agents of hospital inpatient care, andthese agents in tum join subsequently withposthospital rehabilitation.Today, except in rural and more isolatedcommunities in the United States, we takethe EMS system for granted, and it is difficultto remember a time when it did not exist tohelp individuals in an emergency. But therecent horrific hurricane in New Orleans andsurrounding areasremindsus that a comparableregional-, state-, or federal-level emergencydisaster response system has not yet evolvedsufficient capacity to deal with large-scalecatastrophes.

Although the vast array of participatinggovemme ntal, medical, public health, andnonprofit organizations involved in the EMSsystem have coordinated their efforts in vari-ous ways for several decades,^^ it is not a cen-trally and hierarchically controlled system.Tens of thousands of people are trained invarious roles by a wide variety of entitiesranging from medical and public healthschools to nonprofit organizations such asthe Red Cross,^^ Although individual agentsmay be aware of the existence of the broadersystem, their training concentrates on theirspecific part in it and how it connects withadjacent partsthe role they play and therules they followleading to a system thatcan adapt to a gre at variety of individualmedical em ergencies.Mechanical and Biological Systems

In addition to the 2 broad organizingideas just discussed, it is useful to distinguish2 metaphors for systems that are both preva-lent and infiuentiar'''"'^''-'=: systems asmechanical and systems as biological. In themechanical metaphor, systems are construedas machines made up of parts or subsystemsthat interact in complex ways to producecertain characteristic behaviors. In the bio-logical metaphor, systems are living andevolving entities, in turn often composed ofsubsystems that are themselves evolving andadapting to the environment. Studies of sys-tems, infiuenced by both types of metaphors,have led to many significant scientific discov-eries. The biological metaphor appears to beincreasingly prevalent, but some systems.

even complex and nonlinear systems, be-have more like machines than like biologicalorganisms. There are also mixes of themetaphors, such as in bioengineering, inwhich cells are thought of as tiny biologicalmachines,^^ Although systems thinking in-herently is not either mechanistic or biologi-cal, particular phenom ena may be aptlycharacterized by one or the other metaphoror by some combination of the 2 metaphors.Misconceptions

Finally, 2 misconceptions about the systemsapproach need to be addressed. First, systemsthinking is not a rejection of traditional scien-tific views that are linear, reductionist, mecha-nistic, or atomistic and framed by mechanical,spatial, or temporal metaphors,''"'"'''*"''^"'^''^''In a study focusing on the enablers of, barriersto, and precursors to systems thinking, Davidzet al, noted: "It is important to rem ember theembedded nature of systems. What is consid-ered a holistic, systems view is considered areductionist view w hen the boundaries of thesystem are redrawn,"^^'"''"^' Contrary to pop-ular claims, systems thinking encompasses andincludes reductionism; it does not replace orreject it,

A second misconception is that systemsthinking lacks scientific rigor,' This fallacyprobably stems from popular literature por-traying systems thinking as "soft" or in opposi-tion to scientific or analytic thinking. Accord-ing to Von Bertalanffy, systems epistemology"shares the same scientific attitude" with sci-entific or analytic thinking,^^'"'''^''' Systemsthinkers achieve a holistic view of complexphenomena0'^'^'''^=^'''^'o precisely becausethey approach the study of relationships as adistinct and legitimate form of inquiry.""Consequently, most of the techniques usedfor systems thinking and modeling arerooted in mathematics as well as the physi-cal, biological, and social sciences, and theyhave been used to conduct some of themost rigorous and sophisticated experimentsever devised,CURRENT SYSTEMS THINKINGEFFORTS IN PUBLIC HEALTH

The field of public health is adapting tothe evolution of systems thinking and its

accompanying modeling approaches. Morescholars are studying and writing on thetopic, more research is emphasizing a sys-tems view, and ambitious attempts are unway to focus practitioners on improving ovall system performance,'"^ A good exam plISIS, a project of the Tobacco Control Re-search Branch of the US National Cancer Istitute, The purpose of ISIS is to explorewhether systems thinking can serve as afoundation for more effective public healthefforts to com bat tobacco u se, particularlythe face of countervailing forces such as thefforts of the tobacco industry,

ISIS brought together a transdisciplinarygroup of leaders in fields such as system dynamics, network analysis, knowledge man-agement and informatics, tobacco control,management sciences, and health policy todevelop afi-ameworkfor systems action. Tnetwork of thinkers considered some corequestions: How can the flow in botk direc-tions between research and practice be optmized? How can systems structure and funtion be best characterized to be useful to thpublic health community? Which approachcan be used for better understanding and otimization of networks? Through which strgies do information and knowledge bec omthe currency for change?

The ISIS team concluded that systemsthinking in public health cannot be encompassed by a single discipline or even a singapproach to "systems thinking" (e,g,, systemdynamics); instead, it consists of a trans-disciplinary integration of public healthapproaches that strive to understand andreconcile linear and nonlinear, qualitativeand quantitative, and reductionist and holithinking and methods into a federationof approaches to systems thinking andmodeling,'^

The ISIS team also recognized that thecomplexity and b readth of systems thinkingmay be dismissed as being too complicatedthe public health community, from clinicianto policymakers, is to value systems thinkinas a guiding approach, it must be practical,manageable, and accessible. Toward that enISIS supported efforts that resulted in practcal examples of systems ideas in public heacontexts: development of a system dynamicmodel for characterizing the complex state


4/10

ions, a concept mapping project to

actual networks for global tobacco

ure approaches to systems thinking in pub-c health.ISIS is hardly the only effort to assess the

Syndemics Preven-^ supported by the Centers

(substance a buse, violence,

h the methodological techniques

W. K.

W.K. Kelloggr H eal th,"' the efforts of the Insti-

es movement,"^ the Partnership

If systems thinking cind modeling are to be

his end, we conducted an initial study withthe Syndemics Prevention Network) in

METHODSParticipants

Invitations to participate in the projectwere sent to 359 individuals who were onthe e-mail distribution lists of the SyndemicsPrevention Network and the ISIS project asof December 15, 20 04 . The Syndemics Pre-vention Network included more than 300members from 11 countries, and the ISIS listincluded approximately 60 participants fromCanada, Australia, and the United States.Most of these individuals are practicing pub-lic health professionals (e.g., researchers, pro-gram managers, policymakers); however, asignificant percentage also identified them-selves as having special expertise and train-ing in systems thinking and modeling. (Moredetails on the methods and results of thisproject are available from the first author onrequest.)Concept iVIapping

Concept mapping is a systems method thatenables a group to describe its ideas on anytopic'^' and represent these ideas visually ina map. The general procedure for conceptmapping has been described in detciil byTrochim.'^' The method has been used in awide range offields,'^^ ncluding health ser-vices research'^^'^'' and public health.'^^

To accomplish this project, participantsbrainstormed or free listed a large set ofstatements addressing an agreed-upon focusstatement for the project. All participationwas via the Web. Each generated statementcompleted the following focus prompt: "Onespecific practical challenge that needs to beaddressed to e ncourage and support effectivesystems thinking and modeling in publichealth work i s . . . . " The group generated 318statements that were synthesized and editedto a final set of 100 statements used duringthe remainder of the project.

Each participant was invited to sort thesestatements into groups of similar ones andrate each statement in terms of its relativeimportance as a challenge that must be ad-dressed to encourage and support systemsthinking and modeling in public health.These data were assessed in a sequence ofmultivariate statistical cUiEilyses that includedmultidimensional scaling*^*"^' cind hierardiicsil

cluster analysis'^*'^^ methods. The resultingmaps showed each of the statements, withmore similar ones located nearer each other,and illustrated how the statements weregrouped into clusters. Initial interpretation ofthe maps followed the general process de-scribed by Trochim.'^'RESULTS

In the brainstorming phase, 133 partici-pants visited the Web page. In the sorting andrating phase, 56 psirticipants completed thesorting task, and 54 completed the ratings.The stress value is the usual statistic reportedin multidimensional scaling analyses to indi-cate goodness of fit, with a lower stress valueindicating a b etter fit. In a study of the relia-bility of concept mapping, Trochim''"' re-ported an average stress value across 33projects of 0.285, with a range from 0.155 to0.352. The stress value in the present analy-sis was 0.300. An 8-cluster solution was se-lected as the one that preserved the most de-tail and yielded substantively interpretableclusters of statements. The key materials usedin the interpretation of the results includedthe statements produced through brainstorm-ing, listed by cluster; the point map showingeach statement; and the cluster map showingthe 8-cluster solution.

Figure 1 shows the final map with thecluster labels arrived at through a consensusprocess that involved a subgroup of the par-ticipants. Table 1 lists, for each cluster, the 3challenges to implementing systems thinkingthat were assigned the highest average impor-tance ratings.

Here we describe each cluster briefiy, mov-ing from highest to lowest in cluster averageimportance rating as listed in Table 1. Thecluster labeled "Expeind Cross-CategoryFunding" consisted of 10 statements prima-rily related to financial issues. These state-ments challenged traditional funding cate-gories and explicitly encouraged a moreintegrative, systems-based view of financing.The cluster labeled "Support Dynamic andDiverse Networks" contained 8 statementsabout encouraging networks, collaborations,teams, and partnerships that span traditionaldisciplines and boundaries and value diverseperspectives.


5/10

Support Dynamicand Diverse Networks

Use SystemsMeasuresand Models

InspireIntegrativeLearning

Foster SystemsPlanningand Evaluation

ExpandCross-CategoryFunding

Show Potential ofSystems Appro aches

UtilizeSystemIncentives

Explore SystemsParadigms andPerspectives

F IGU R E 1 - E lg h t - c lu s t er co n cep t m ap o f p rac t ica l ch a l len g es t h a t n eed t o b e ad d ressed t oen co u rag e an d su p p o r t e f f ec t ive sys t em s t h in k in g an d m o d e l in g in p u b i ic h ea i t h wo rk .

The "Use Systems Measures and Models"cluster (10 statements) related to the creationof methods and tools for systems-based mea-surement and data collection, use and evalua-tion of systems methods and models, and de-velopment of systems tools and approaches."Inspire Integrative Learning" included themost statemen ts (23), reflecting the fact thatthe topics contained in this cluster were mostfrequently brainstormed by participants. Exam-ples are training and education, disseminationand diffusion of systems thinking and ap-proaches, use of interactive and Web-based re-sources, and a broad emphasis on understand-ing the area of systems thinking and modeling.

"Explore Systems Paradigms and Perspec-tives" included 15 statements addressing valuesand implications of a potential paradigmchange involved in systems thinking and theinfluence of such thinking on the perspectivespeople bring to public health work. The 11statements in the "Show Potential of SystemsApproaches" cluster described the value andimpact of systems thinking and modeling. "Fos-ter Systems Planning and Evaluation" (9 state-ments) suggested the integration of systemsthinking and modeling into traditional planningand evaluation. Finally, "Utilize Systems Incen-tives" (14 statements) emphasized the need to

address political and sodal factors that influ-ence use of systems thinking, including issuesof bureaucracy, people's fears (e.g., apprehen-sions about job loss, job difficulty, or change),jurisdictional conflicts among organizations,and factors in academic environments that limitadoption or use of systems approaches.D I S C U S S I O N

The map developed in this project depicts8 distinct clusters of practical challenges thatneed to be addressed to encourage and sup-port effective systems thinking and modelingin public health. The label for each cluster ofchallenges was carefully reverse checked withthe cluster statements to ensure that theywere adequately represented. In many cases,single words in the cluster name represent en-tire fields of inquiry such as integrative learn-ing, networks, planning, and evaluation, andeach term in the cluster name modifies theothers. T hus, the modifying effect of "sys-tems" on "planning" in the Foster SystemsPlanning and Evaluation cluster is not merelysemantic; it reflects a perspective that differsdramatically from traditional planning inwhich planning precedes action and evalua-tion follows in a rational and linear fashion.

Instead, the statements suggest 2 challengeto systems thinking in public hea lth: thatplanning and evaluation are not yet suffi-ciently systemic and that planning should bcontinuous and adaptive, with constant feeback am ong planning, action, and evaluatio

The cluster in the center of the map(Figure 1), Show Potential of Systems Ap-proaches, can be considered central bothgraphically and conceptually. In systems jargit might be thought of as a central attractor ithe d)mamic cycle of the overall map. As theclusters in the exterior ring interact in variouways, their activity converges on the centralcluster, where assessment and dissemination(e.g.,research nto what works) are representhe disseminated content in tum excites newactivity in the exterior "ring" of dus ters.

The Figure 1 map can be viewed througthe lens of systems thinking itself Earlier, woffered 2 broad organ izing ideas that help make sense of the often bewildering and diverse landscape of systems thinking: Systemare dynamic and systems are complex. Frothe perspective of dynamic systems, the macan be interpreted as a collection of interacing "cluster agents" each affecting the otherFrom the perspective of complex systems,clusters can be viewed as simple rules thatencourage emergence and adaptation. Eacis considered briefiy.

In terms of dynamics, the clusters can bethought of as interacting conceptual or the-matic agents that can influence other dusteagents when coupled. Each cluster resembla semi-autonomous agent functioning in ahighly integrated system. When one clusteinteracts with another, they affect each othFor example, when interacting with E xplorSystems Paradigms and Perspectives, the Inspire Integrative Learning cluster takes on aslightly different meaning thcin when it isconsidered in connection with Use SystemsMeasurement and Modeling. In the first caslearning is centered on systems paradigmsand perspectives at a conceptual or epistemlogical level. In the second case, learning ismore formal and adaptive; systems-basedmeasurement and modeling are used to in-form decisionmaking and action.

In another exeimple, the Support Dynamand Diverse N etworks cluster interacts withExpand Cross-Category Funding to create


6/10

TABLE 1 - P a r t ic ipa nt s ' Ra t ings o f P ra c t ic a l Cha i ie nge s t o E f f e c t iv e Systems Thinit ing a n diViodel ing in Public Heaith

Cluster and C hal lenge Rat ingExpand Cross-Categoiy Funding (average: 3.86 )

Ident i f y and develop funding sources that wi l l encourage systems approaches to publ i c heal th 4 .1 3Develop funding for demonst rat ion projects that val idate systems approaches to publ i c heal th 4 .0 9Increase funding for t ransdiscip l inary and interagency col laborat i ve projects wi th a systems focus 4 .0 6

Suppor t Dynamic and Diverse Networks (average: 3.50)Encourage col laborat ions between researchers and prac t i ti oners by c lar i f y ing the l ink of systems th inking 3 . 7 4

and model ing to eveiyday pract i ce in publ i c heal thUse par t i c ipatory act ion approaches to par tner wi th communi t ies to co-def ine publ i c heal th problems, 3 . 7 4

chal lenges, needs, assets, and resourcesSustain mul t id isc ip l inary teams f rom a broad range of heal th and science backgrounds and th inking 3 . 6 5

( e . g . , deduc t i ve / induc t i ve , r esea r ch / p r ac t ice )Use Systems Measures and Models (average: 3.39)

Development of methods and tools that encourage systems approaches in research and evaluat ion 3 . 9 3Develop new evaluat ion approaches that wi l l help demonst rate the value of systems approaches, 3 . 7 8

such as syndemics, in publ i c heal thIdent i fy p r ior it y pu bl i c hea l th i ssues ( e . g . , tobacc o, HIV, obesi ty) as p ossib le t ipping points for ear ly 3 . 4 6

examples of systems th inking and model ingInspi re Integrat ive Learning (average: 3.38)

Ident i f y and disseminate examples of "best pract i ces" or "what works" in systems th inking inside and 3 . 9 3outs ide of publ i c heal th

A cr i t i cal mass of pract i t i oners who are able to approach publ i c heal th f rom a nonl inear perspect i ve 3 . 7 2Train ing and educat ion in systems research techniques for publ i c heal th professionals 3 . 7 0

Explore Systems Paradigms a nd P erspect ives (average: 3 .26)Recognize the impor tance of a systems paradigm to publ i c heal th ( e . g . , ecolog ical , systemic, hol i s t ic , 4 .0 0par t i c ipatory, mul t id imensional , const ruct i v i st , adapt ive, complex, and nonl inear f rameworks)Recognize the l imi tat ions of the dominant paradigm in publ i c heal th ( e . g . , l i near causal i t y , reduct ionism, 3 . 5 7

posi t i v ism, ob ject i v ism, the medical mod el , logic m odels, program- focused, d isease- focused f rameworks)Get t ing government and publ i c heal th of f i c ia ls at state and federal levels to appreciate the value of 3 . 5 4

communi ty-based approaches and highl ight c i t i zenship and local governance in publ i c heal thShow Po tent ia l of Systems Approaches (average: 3 .25)

Rigorous research that demonst rates the value of systems th inking, methods, approaches, and research 3 . 7 6Set pr ior i t ies by analyzing system-wide issues rather than simply ranking by disease burden or attr ibutable r isk 3 . 7 4Connect systems th inking and model ing to the ser ies of recent Inst i tute of Medic ine repor ts ( e . g . , br idging 3 . 6 3

the qual i t y chasm, reducing heal th care er rors, e l iminat ing heal th and heal th care dispar i t ies)Foster Systems P lanning and Evaluat ion (average: 3 .20)

Integrate organizat ional p lanning and evaluat ion funct ions around a systems approach 3 . 7 2Apply systems th inking to physical and mental heal th problems af fect ing indiv iduals, fami l ies, and 3 . 6 5

communi t ies throughout the human l i fe cycleDevelop a uni f ied mission/v is ion across sectors ( e . g . , publ i c heal th, educat ion, publ i c safety, behavioral 3 . 5 0

heal th) and between layers ( e . g . , nat ional , s tate, communi ty) regarding the systems approachUt i li ze System Incen t i ves (average: 3.05)

Provide incent i ves that encourage systems th inking 3 . 5 6Reduce the overemphasis on imm ediate p osi t i ve program impacts by taking a longer term view 3 . 5 4Address i ssues of pol i t i cs and bureaucracy that h inder systems th inking ( e . g . , pol i t i c ians' ignorance of how 3 . 3 5

thei r systems work, publ i c employee unions that avoid employee accountabi l i t y , c i v i l serv ice systemsthat encourage stagnat ion)

Note . Shown are the 3 chal lenges in each c luster wi th the highest average impor tance rat ings.

diverse and dynamic networks of monetaryflows and feedback loops. In contrast, whenShow Potential of Systems Approaches, whichcontains statements relating to research, re-searchers, and best practices, is combinedwith Support Dynam ic and Diverse Networks,innovative notions about supporting diversenetworks of researchers and practitionersemerge. The cluster-as-agent interpretationgenerates 56 possible pairings. Increasing incomplexity, the clusters can be thought of ascombinations interacting in triples or largergroupings. This interpretation is dynamic be-cause, with just 8 simple clusters, a diverseand complex number of possible innovativeideas can result. For example, the map mightlead us to ask what a dynam ic network ofbest practices in systems thinking and model-ing would look like, with differing combina-tions and juxtapositions of clusters suggestingspecific novel practices.

With respect to complexity, the map can beviewed from the perspective of the theory ofcomplex adaptive systems, which holds th atsimple local rules lead to emergent complex-ity. Each cluster label is worded as a simpleobject-action "rule" that can be used by publichealth professionals to encourage and m an-age a complex and adaptive system. Eachlabel completes the prompt that w as the focusof this project and constitutes a practical chal-lenge to be addressed. Th e especially intrigu-ing hypothesis from a systems point of viewis that if multiple independent agents (individ-uals and organizations) follow the 8 rules rep-resented by the cluster names and are pro-vided with appropriate feedback about whatis occurring as a result, systems-oriented or-ganizations and networks will natunillyemerge in the public health system.

This study involved seyeral limitations. Thesample consisted of self-selected participantsinvited from the e-mail lists of 2 prominentinitiatives for systems thinking in publichecilth, so the generalizability of our results islimited to similar groups. However, generaliz-ability was not a major purpose of this study,which was closer in nature to a focus groupor expert panel than to a sample survey. Inaddition, this was an unfunded project accom-plished und er considerable time constraintsthat may have limited participation. Brain-storming occurred during late December and


7/10

early January and conflicted with busy holi-day seasons. The entire project was accom-plished within 2 months.

Because of time and technology con-straints, it was not feasible to engage a broadcross section of the entire participant groupin an interpretation of the map results or toengage the broader public health communitydirectly. To help address this limitation, wecreated a Web site (available at: http://www.greaterthanthesum.net) to distribute detailedresults of this project and encourage furtherinteraction and discussion of these issues andthe resulting concept map. Many of the limita-tions of this study would be best amelioratedthrough independent replications and studiesinvolving the use of alternative methods foridentiiying practical challenges to support ef-fective systems thinking and modeling in pub-lic health work.

This project is significant for several rea-sons. First, the concept map results provide abasis for subsequent action by various actorsin public health agencies (and beyond). Anygroup or organization can examine the clus-ters, or the statements contained within them,and detennine the degree to which they con-stitute or suggest actions they might take toaddress practical challenges to systems think-ing and modeling in pubHc health. Second,the results clearly point to a need for educa-tion and leaming in systems thinking andmodeling, including everything from potentialcurriculum topics (participatory methods,nonlinear dynamics, simulations) to enhanc-ing leaming capacity (development of centersand of electronic materials).

Third, the map provides a conceptual modelthat serves as a basis for developing role playsimulations that can enable public health or-ganizations to try out different actions andexplore in a controlled context how theiradoption could potentially change the out-comes of public health efforts. The map con-stitutes a structure and a set of rules that dif-ferent agents can use depending on theirroles in the public health system. Such simula-tions would make it possible for the field ofpublic health to leam more dynamicallyabout the effects of systems thinking andmodeling and to cinticipate better where fund-ing resources might be most effectively allo-cated to encourage systems evolution.

Finally, a significant aspect of having donethe concept mapping is that it helps to estab-lish an appropriate and vndely shared bound-a r y for thinking about the many issues inquestion, particularly at a relatively early stagein our efforts to build institutional support fora systems orientation. The number of clustersand the diversity of their themes now serve asa check against planning and capacity-buildinginitiatives that might otherwise be scoped toonarrowly or too abstractly. Equipped with thepractical insights from the concept mapping,we can now embark on more productivemultistakeholder dialogues and think togetherabout precisely what kinds of supports areneeded if an authentic system orientation isto thrive in public health agencies.

This study provides an initial identificationof th e challenges, a map that can be used tonavigate them, and a set of 8 simple rules forfacing these challenges and moving toward ef-fective implementation of systems approachesin public health efforts. It has practical utilityin terms of helping organizational practition-e r s , researchers, policymakers, and the generalpublic face these challenges. The results re-ported here, major reports such as Crossing th eQuality Chasm, and initiatives such as the Syn-demics Prevention Network and ISIS can beviewed from a systems perspective as dynami-cally interacting components in the growingawareness and support of systems thinkingand modeling in public health, and they offerthe promise that more effective public healthsystems will consequently emerge.

A b o u t t h e A u t h o rsWilliam M. Trochim is with Ih e Department of PolicyAnalysis and Management, Cometl University, Ithaca,NY. Derek A. Cabrera is with th e Department of Educa-tion, Cornell University. Bobby Mibtein is with the Centersfor Disease Control and Prevention, Atlanta, Ga. RichardS. Gallagher is with Gallagher and Associates, Ithaca,NY. At the time ofthe study, Scott f. Leischow was withth e National Cancer Institute, Bethesda, Md.

Requests for reprints should b e sent to William M.Trochim, P h D , Policy Analysis and Management, 249MVR Hall, Cornell University, Ithaca, NY 14853 (e-mail:wmtl @comell.edu).

This article was accepted September 19, 2005.

C o n t r i b u t o r sThis article was a collaborative effort of all of the authors,who contrihuted to and revised the article, discussed theresearch at all phases, and participated in the interpreta-tion of results. W.M. Trochim originated the study andcollaborated with D.A. Cabrera in implementing the

study. D.A. Cabrera contributed much ofthe content osystems thinking. W.M. Trochim, D.A. Cabrera, and RGallagher collaborated on the conceptualization andstructure of the article. B. Milstein and S. J. Leischow ctributed to the section on current systems thinking effin public health.

H u m a n P a r t ic i p a n t P r o t e c t i o nThis study was approved by the institutional reviewboard of Cornell University.

R e f e r e n c e s1. Committee on Assuring the Health of the Pubin the 21st Century. The Future ofthe Public's Healtin the 21st Century. Washington, DC: Institute of Micine. 2002.2 . Green LW, McGinnis M, Phillips TS, DevereauxMontes H. Strategies for Promoting Health for SpeciPopulations. Washington, DC: US Ofiice of Health Iformation and Health Promotion; 1981. DHHS pubcation PHS 81 50169.3 . Green LW. Should health education abandon atude change strategies: perspectives from recent re-search. Health Educ Monogr. 1970;t:24-48.4 . Anderson RA, Issel LM, McDaniel RR. Nursinghomes as complex adaptive systems. Nurs R e s . 200352 :12 -21 .5 . Anderson RA, McDaniel RRJ. Managing healthcare organizations: where professionalism meets complexity science. Health Care Manage R e v . 2000;25:83-92.6. Anonymous. Ways of understanding. BMJ [serionline]. 200 1:323 . Available at: http://bmj.bmjjoumcom/cgi/content/full/ 323/7315/O/a. Accessed No-vember 27, 2005.7. Suchman AL. Linearity, complexity and well-being. Med Encounter. 2002;16:17-19.8. Ashmos DP, Duchon D, McDaniel R. Physicianand decisions: a simple rule for increasing connectioin hospitals. Health Care Manage R e v . 2000;25:109-115.9. Begun JW, Zimmerman B, Dooley K. Health caorganizations as complex adaptive systems. In: Mick Wyltenbach ME, eds. Advances in Health Care Orgazation Theory. San Francisco, Calif: Jossey-Bass; 20253-288.1 0 . Glouberman S, Zimmerman B. Complicated anComplex Systems: What Would Successful Reform ofMedicare Look Like? Toronto, Ontario, Canada: Commission on the Future of Health Care in Canada;2002.1 1. Institute of Medicine. Crossing the Quality ChaA New Heatth System for the 21st Century. WashingDC: National Academy Press; 2001.1 2 . PIsek PE, Greenhalgh T. The challenge of com-plexity in health care. BMJ. 2001;323:625-628.1 3 . Plsek PE, Wilson T. Complexity, leadership, andmanagement in healthcare organisations. BMJ. 2001323:746-749.1 4 . McDaniel RR, Driebe DJ, Blair J, Fottler M,Savage G, eds. Complexity Science and Health Careagement Amsterdam, the Netherlands: Elsevier ScienLtd; 2001.


8/10

. Ashmos DP, Duchon D, McDaniel R. PhysidansHealth Care Manage Rev. 2000;25:109-n5.

. Baskin K, Goldstein J, Lindberg C. Merging, de-2000;26(3):20-25.

Dardik I. The origin of disease and health, heartFrontier Perspect. 1997;6:

Dekker J, Crow RS, Folsom AR, et al. Low heart

Circulation. 2000;102:1239-1244.Fmmkin H. Beyond toxicity: human health and

AmJ Prev Med. 2001:30:

0 . Godin P], Buchman TG. Uncoupling biological os-s: a complementary hypothesis concerning mul-

Crit Care Med. 1996;

. Goldberger AL. Fractal variability versus patho-Perspect Biol Med 1997;40:543-561.

2. Goodwin JS. Chaos and the limits of modem med-

. Lindberg C, Herzog A, Merry M, Goldstein J. LifePhysician Executive. 1998:24(l):6-20.

4. McDaniel RD, Dean J. Complexity science andJ, Fottler M, Savage G,

Advances in Health Care Management. Vol. 2.

. Miller W. Understanding change in primary carePract. 1998;

6 . Seely A, Nicolas V. Multiple organ dysfunctionCrit Care Med 2000;28:2193-2200.

. Weibel ER. Fractal geometry: a design principleAmJ Physiol. 1991 ;261:

8 . Zimmerman B, Lindberg C, Plsek P. Edgeware:ence for Health Care Leaders.

as, Tex: VHA Inc; 2001.9 . Cole C. Heart-rate recovery immediately after ex-

N EnglJ Med. 1999;

0 . Ivanov PC, Amaral LAN, Goldberger AL, et al.Nature.

. Pennisi E. In nature, animals that stop and startScience. 2000;288:83-85.

2. Schmidt G. Heart-rate turbulence after ventricularLancet 1999;353:1390-1396.

3 . Lipsitz LA, Gotdberger AL. Loss of 'comptexity'to senescence.//lAM. 1992 ;267 :180 6-1809 .

4 . Goldberger AL, Rigney DR, West BJ. Chaos andSci Am. 1990;262:

35 . Goldberger AL. Non-linear dynamics for clinicians:chaos theory, fractals, and complexity at the bedside.Lancet 1996;347:1312-1314.36 . Syme SL, Balfour JL. Social determinants of dis-ease. In: Wallace RB, ed. Public Health and PreventiveMedicine. 14th ed. Stamford, Conn: Appleton & Lange;1998:795.37 Healthy People 2010. Washington, DC: US Deptof Health and Human Services; 2000.38 . Stokols D, Pelletier KR, Fielding JE. The ecologyof work and heatth: research and policy directions forthe promotion for employee health. Health Educ Q.1996;23:137-158.3 9 . Smoking and Health: Report ofthe Advisory Com-mittee to the Surgeon General ofthe Public Heatth Service.Washington, DC: US Dept of Health. Education, andWelfare; 1964.40 . Trochim W, Stillman F, Clark P, Schmitt C. Devel-opment of a model of the tobacco industry's interfer-ence with tobacco control programs. Tob Control.2003:12:140-14741 . Hoffmann D, Hoffrnann 1. The changing cigarette,1950-1995. / Toxicol Environ Health. 1997;50:307-364.42 . Wynder EL, Hoffmann D. Smoking and lung can-cer: scientific challenges and opportunities. Cancer Res.1994;54:5284-5295.43 . Wakefield MA, Terry YM, Chaloupka FJ, et al.Changes at the point-of-sale for tobacco following the1999 tobacco billboard ban. Am J Public Health. 2002;92:937-939.44 . Clark PI, Djordjevic MJ. llie role of smokingtopography in assessing human smoking and its utilityfor informing machine-smoking protocols. Report to theWorld Health Organization, Scientific Advisory Councilon Tobacco Regulation. 2003:1-2345 . Stacey RD, Griffin D, Shaw P. Complexity andManagement: Fad or Radicat Challenge to Systems Think-ing? New York, NY: Routledge; 2000.46 . McKelvey B. Complexity theory in organizationscience: seizing the promise or becoming a fad? Emer-gence. 1999:1:5-32.47 . Gilbert GN, Troitzsch KG. Simulation for the SocialScientist. Philadelphia, Pa: Open University Press; 1999.48 . Richardson GP. Feedback Thought in Social Scienceand Systems Theory. Philadelphia, Pa: University ofPennsylvania Press; 1991.49 . Sterman JD. Business Dynamics: Systems Thinkingand Modeling for a Complex World. New York, NY:McGraw-Hill/Irwin; 2000.50. Kauffman SA. At Home in the Universe: The Searchfor Laws of Self-Organization and Complexity. New York,NY: Oxford University Press Inc; 1995.51 . Kauffman SA. The Origins of Order: Self-Chganizationand Selection in Evolution. New York, NY: Oxford Uni-versity Press tnc; 1993.52 . Maturana HR. Autopoiesis: reproduction, heredityand evolution. In: Zeleny M, ed. Autopoiesis, DissipativeStructures and Spontaneous Social Order. Boulder, Colo:Westview Press; 1981:48-80.53 . Maturana HR, Varela FJ. AutopoiesLs and Cognition:The Realization ofthe Living. Dordrecht, the Nether-lands: D. Reidel Publishing Co; 1980.

54. Maturana HR, Varela FJ. The Tree of Knowledge:The Biological Roots of Human Understanding. Boston,Mass: Shambala; 1992.55 . Strogatz SH. Nonlinear Dynamics and Chaos: WithApplications to Physics, Biology, Chemistry, and Engi-neering. Reading, Mass: Addison-Wesley; 1994.5 6. Gell-Mann M. Let's call it plectics. Complexity.1995;1:5.57 . Gell-Mann M. The Quark and the Jaguar: Adven-tures in the Simple and the Complex. London, England:Abacus; 2003 .58 . Holland J. Hidden Order: How Adaptation BuildsComplexity. Reading, Mass: Perseus Books; 1995.59. Watdrop MM. Complexity: The Emerging Science athe Edge of Order and Chaos. New York, NY: Simon &Schuster; 1992.60 . Yeomans JM. Statistical Mechanics of Phase Transi-tions. Oxford, England: Oxford University Press Inc; 1992.61 . Cote M. A matter of trust and respect. CA Maga-zine [serial online]. March 2002. Available at; http://www.camagazine.com/index.cfm/ci_id/6798/la_id/l.htm. Accessed November 27, 2005.62. Holland JH. Emergence: From Chaos to Order.Reading, Mass: Addison-Wesley; 1998 .63 . Johnson S. Emergence: The Connected Lives of AntsBrains, Cities, and Software. New York, NY: Scribner;2001 .64. Wolfram S. A New Kind of Science. Champaign, III:Wolfram Media; 2002.65 . Von Bertalanffy L. The history and status of gen-eral systems. In: Klir J, ed. Trends in General SystemsTheory. New York, NY: Wiley-Interscience; 1972:423.66. Francois C. International Encyclopedia of Systemsand Cybernetics. 2nd ed. Munich, Germany: KG Saur;2004.67 . Young JF. Cybernetics. New York, NY: ElsevierPublishing Co; 1969.68 . Sontag ED. Mathematical Control Theory: Deter-ministic Finite Dimensional Systems. 2nd ed. New York,NY: Springer; 1998 .69. Shannon C. A mathematical theory of communi-cation. Bell Syst TechJ. 1948;27:379-423.70 . Resnick M. Turtles, Termites, and Traffic Jams: Ex-plorations in Massively Parallel Microworlds. CambridgeMass: MIT Press; 1994.71 . von Neumann J. Morgenstem 0. Theory of Gamesand Economic Behavior Princeton, NJ: Princeton Uni-versity Press; 1944.72. Forrester JW. Industrial Dynamics. Cambridge,Mass: MIT Press; 1961.73 . Forrester JW. Leaming through system dynamicsas preparation for the 21st century. Paper presented at;Systems Thinking and Dynamic Modeling Conferencefor K-12 Education, June 1994, Concord, Mass.74 . Forrester JW. Roadmaps: A guide to leaming sys-tem dynamics. Available at: http://web.mit.edu/sdg/www/roadmaps.html. Accessed March 18, 2003 .75 . Richardson GP. Problems for the future of systemdynamics. Syst Dynamics Rev. 1996;12:141-15776 . Sterman J. System dynamics modeling: tools forlearning in a complex world. Calif Manage Rev. 2001;43:8-25.


9/10

77. System Dynamics Society. What is system dynamics?Available at: http://www.systemdynamics.org. AccessedDecember 19, 2002.78. Capra E TheWeb of Life: A NewSynthesis of Mindand Matter. London. England: Elamingo; 1997.79 . Capra E The Hidden Connections: Integrating theHidden Connections Among the Biological, Cognitive, andSocial Dimensions of Life. New York, NY: Doubleday;2002.80. Capra F. From the Parts to the Whole: System'sThinking in Ecology and Education. Mill Valley Calif:Mill Valley School District; 1994.81 . NewParadigm Thinking [audiotape]. San Eran-cisco, Calif; New Dimensions Eoundation; 1993.82 . Strogatz SH. Sync: The Emerging Science of Sponta-neous Order. New York, NY: Hyperion; 2 0 0 3 .83 . Watts DJ. Small Worlds: The Dynamicsof NetworksBetween Order and Randomness. Princeton. NJ: Prince-ton University Press; 1999.8 4 . Watts DJ. TheStructure and Dynamics of Small-World Systems [dissertation]. Ithaca, NY: Cornell Uni-versity; 1997.85 . Newman MEJ. The structure and function of com-plex networks. arXiv. March 25, 2003 : 1 .86 . Watts DJ. Six Degrees: The Science ofa ConnectedAge. New York: WW Norton &Co; 2 0 0 3 .87 . Lorenz E. Predictability: does the flap of a butter-fly's wings in Brazil set off a tornado in Texas? Paperpresented at: Annnal Meeting of the American Associa-tion for the Advancement of Science, December 1979,Washington, DC.88 . Gleick J. Chaos: Making a New Science. New York,NY: Viking; 198789 . Gladwell M. TheTipping Point: How Little ThingsCan Make a Big Difference. Boston, Mass: Little Brown& Co; 2000.90 . Cabrera DA. Patterns of knowledge: knowledge asa complex, evolutionary system, an educational impera-tive. In: Miller R, ed. Creating Leaming Communities[online edition]. Brandon. Vt: Solomon Press; 2002.91 . Lewin R. Complexity: Life at the Edge of Chaos. 2nded. Chicago, III; University of Chicago Press; 1999.92 . Institute of Medicine. Accidental Death and Dis-ability: The Neglected Disease of Modem Society. Wash-ington, DC; National Academy of Sciences; 1966.93 . American Red Cross. CPft/AEDfor the Profes-sional Rescuer. Yardely, Pa:Staywell; 2002.94 . Mindwalk [videotape]. Hollywood, Calif: Para-mount; 1992.95 . TheTuming Point: A Transformative Vision for anEcological Age [audiotape]. Emeryville, Calif: EnhancedAudio Systems; 1991.96 . Cabrera DA. Bacterial burritos, mixed meta-phors and a man in the middle: a scientific life at thecrossroads of biology, chemistry and engineering.Available at; http://www.genomics.comell.edu/news/news_story.cfm?id=52. Accessed November 24 ,2 0 0 5 .97 Von Bertalanfly L. The History and Status of Gen-eral Systems Theory. New York, NY: Wiley-Interscience;1972 .98 . Valera EJ. On being autonomous; the lessons

of natural history for systems theory. In: Klir G, ed.Applied Systems Research. New York, NY; PlenumPress; 1977;77-85.99 . Davidz HL, Nightingale DJ, Rhodes DH. Enablers,Barriers, and Precursors to Systems Thinking Develop-ment: The Urgent Need or More Information. Cam-bridge, Mass; MIT Press; 2004.100. Von Bertalanfly L. General System Theory: Founda-tions, Development, Applications. New York, NY:Braziller; 1969.101. Richmond B. Systems thinking; critical thinkingskills for the 1990s and beyond. Syst Dynamics Rev.1993;9:113-134.102. The World Health Report 2000: Health Systems:Improving Performance. Geneva. Switzerland: WorldHealth Organization; 2000. Report 924156198X.103. Krygiel AJ. Behind the Wizard's Curtain: An Inte-gration Environment for a System of Systems. Washing-ton, DC: Institute for National Strategic Studies; 1999.104. Spotlight on Syndemics. Atlanta. Ga: Centers forDisease Control and Prevention;2001.105 . Baer HA, Singer M, Susser 1. Medical Anthropologyand the World System. 2nd ed. Westport, Conn:Praeger; 2 0 0 3 .106. Singer M. A dose of drugs, a touch of violence, acase of AIDS; conceptualizing the SAVA syndemic. FreeInquiry Creative Sodol. 1996;24;99-110.107. Singer M, Clair S. Syndemics and public health;reconceptualizing disease in bio-social context. MedAnthropol Q. 2003;17;423-441.108 . Singer M, Snipes C. Generations of suffering: ex-periences of a treatment program for substance abuseduring pregnancy. / Health Care Poor Underserved.1992;3:222-234.109. Milstein B. Syndemics. In: Encyclopedia of Evalua-tion. Newbury Park, Calif: Sage Publications; 2004;4 0 4 - 4 0 5 .110. Bammer G. Integration and implementation sci-ences: building a new specialization. Ecology Soc [serialonline]. 2005;10(2):6. Available at: http://www.ecologyandsociety.org/vol 10/iss2/ar t6. Accessed No-vember 24, 2 0 0 5 .111. Emirbayer M. Manifesto for a relational sociology.Amf Sodol 1997;103:281-317.112. Midgley G. System Intervention: Philosophy, Meth-odology, and Practice. New York, NY: Kluwer Academic;2 0 0 0 .113. WK Kellogg Eoundation. Partnerships; a powerfultool for improving the well-being of families and neigh-borhoods. Available at; http;//www.wkkf.oi^/Pnbs/YouthED/Pub591.pdf. Accessed November 24 . 2 0 0 5 .114. US Dept of Health and Human Services. Creatingpartnerships, improving health; the role of community-based participatory research. Available at: http://www.ahrq.gov/research/cbprrole.htm. Accessed November24 , 2 0 0 5 .115. University of Washington. Community-CampusPartnerships for Health (CCPH). Available at:. AccessedNovember 24. 2 0 0 5 .116. Institute for Healthcare Improvement. About us.Available at: http://www.ihi.org/ihi/about. AccessedSeptember 15. 2004.

117. Intemational Healthy Cities Eoundation. Whatthe Healthy Cities movement? Available at: http://www.healthycities.org. Accessed October 5, 2004.118. Partnership for the Public's Health. About the pnership. Available at: http://www.partnershipph.org/coll/about/overview.html. Accessed April 20, 2 0 0 5119. Tuming Point National Program Office, from Sto Systems: Using Performance Management to ImprthePublic Health. Seattle, Wash: University of Washton; 2 0 0 3 .120 . World Health Organization, Commission on SoDeterminants of Health. Erom knowledge to action;global country partners address social determinanof health. Available at; http;//www.who.int/social_determinants/en. Accessed March 18, 2 0 0 5 .121. Trochim W An introduction to concept mappinfor planning and evaluation. Eval Program Plann.1989;12:1-16.122. Trochim W Concept mapping: soft science orhard art? Eval Program Plann. 1989:12:87-110.123. Batterham R, Southem D, Appleby N, et al. Costruction of a GP integration model. Soc Sd Med.2 0 0 2 ; 5 4 : 1 2 2 5 - 1 2 4 1 .124. DeRidder D, Depla M, Severens P, Malsch M. liefs on coping with illness: a consumer's perspectiveSoc Sd Med. 1997;44:553-559.125 . Trochim W, Milstein B, Wood B, Jackson S,Pressler V. Setting objectives for community and systems change: an application of concept mapping forplanning a statewide health improvement initiative.Health Promotion Pract. 2004:5:8-19.126. Davison ML. Multidimensional Scaling. New YNY: John Wiley & Sons Inc; 1983.127. Kniskal JB. Wish M. Multidimensional Scaling.Beverly Hills. Calif: Sage Publications; 1978.128 . Anderberg MR. Cluster Analysis for ApplicatioNew York, NY: Academic Press Inc; 1973.129.Everitt B. Cluster Analysis. 2nd ed. New York.Halsted Press; 1980.130. Trochim W. Reliability of concept mapping. Pappresented at; Annual Conference of the American Evuation Association, November 3-6, 1993, Dallas, Te


10/10

Documents

Practical Challenges Of Systems Thinking an Practical Challenges Of Systems Thinking And Modeling in Public Health