An Evaluation of Subsidized Rural Primary Care Programs: 1. A

An Evaluation of Subsidized Rural Primary CarePrograms: 1. A Typology of Practice Organizations

CECIL G. SHEPS, MD, MPH, EDWARD H. WAGNER, MD, MPH, WARREN H. SCHONFELD, PHD,GORDON H. DEFRIESE, PHD, MIRIAM BACHAR, PHD, E. F. BROOKS, MBA,

DENNIS B. GILLINGS, PHD, PRISCILLA A. GUILD, MSPH, T. ROBERT KONRAD, PHD, CURTIS P.MCLAUGHLIN, DBA, THOMAS C. RICKETTS, MPH, CONRAD SEIPP, PHD, AND JANE S. STEIN, MS

Abstract: The design of a comprehensive evalua-tion of subsidized rural primary care programs on alarge national scale is described. Its major purpose isto derive data whose analysis will answer major policyquestions about the factors influencing the outcome ofthe major types of such programs in different commu-nities. This first paper also delineates a typology which

In the 1960s, the long recognized problem of makinggood quality personal health services available to people inrural areas was given greatly increased attention in theUnited States. For many years prior to this, various mea-sures to correct this persistent deficiency had been tried byrural people themselves, by agencies such as governments atlocal, state, and national levels, and by local and nationalfoundations. Few of these efforts seemed to provide satisfac-tory solutions. During the 1970s, growing concern about thisproblem led to greatly expanded and strengthened effortsand some new approaches which included the requirementthat subsidized primary care projects adopt specific pro-grammatic goals and particular methods of organization,staffing, and operation.

Toward the end of the 1970s, national economic con-straints produced increasing competition for the public dol-lar, thus heightening an interest in comprehensive evaluationstudies that might lead to policy changes for future sup-port." 2 A review of the literature indicates that, although agood many evaluation studies have been done in this field,they have been limited by the representativeness of thepractices studied and the examination of a limited set ofrelevant characteristics. For example, several studies haveexamined factors associated with financial self-sufficiencywithout simultaneously examining the impact of the programon the community being served. A partial, but repre-sentative, list is included in the references.3-19

From the Health Services Research Center, University ofNorth Carolina, Chapel Hill. Address reprint requests to Dr. CecilG. Sheps, Health Services Research Center, Chase Hall 132-A,University of North Carolina, Chapel Hill, NC 27514. This paper,submitted to the Journal January 21, 1982, was revised and acceptedfor publication July 22, 1982.

© 1982 American Journal of Public Health

was developed of five principal organizational forms ofthese programs. This classification appears to providesuitable operational definitions of forms of rural prac-tice as a basis for evaluating the differential impact ofalternative types of primary care programs. (Am JPublic Health 1983; 73:38-49.)

In 1978, the Health Services Research Center of theUniversity of North Carolina at Chapel Hill, with supportfrom the Robert Wood Johnson Foundation and in coordina-tion with the Office of the Assistant Secretary for Planningand Evaluation (Health), US Department of Health andHuman Services (DHHS), began a comprehensive nationalevaluation of subsidized primary health care programs whichwere providing services to rural communities in the UnitedStates. The background and conceptualization of recentstrategies for providing personal health services in under-served rural areas was published in 1981.1 The purpose ofthis paper is to set forth the basic objectives and approach ofthis evaluation project, and the development and initialexamination of a typology of practice organizations.

The National Evaluation of Subsidized RuralPrimary Care

Major Evaluation Questions

We attempted to develop an overall formulation whichwould enable us to evaluate the achievements of alternativestrategies for developing and providing primary care indifferent kinds of communities possessing different con-straints and opportunities. The fundamental question weasked was "What kinds of program development strategiesproduced the greater benefit in what types of communities interms of access, stability, and consumer satisfaction?"There are three basic issues encompassed in this question:

1. What forms of health program organization generallyemerge from certain developmental strategies?

2. What levels of outcomes (community access, pro-gram stability, and consumer satisfaction) are associatedwith particular organizational forms, regardless of develop-mental strategy?

AJPH January 1983, Vol. 73, No. 138

EVALUATING RURAL PRIMARY CARE PROGRAMS

ENVIRONMENTAL FACTORSCOMMUNITY INTERNAL TO LOCAL COMMUNITY

FIGURE 1-Key Variables in Study Design

3. What levels of access, stability, and satisfaction areassociated with particular strategies of health program devel-opment?We postulated that, in each of these questions, the natureand role of the community plays an important part.

Major Study Variables

Developmental Strategies-Variables considered to beof major significance in the initiation of rural health careprograms include: the source of the initiative whether localor external, professionals or laymen, etc.; the nature anddegree of involvement of the sponsoring agency(ies) inprogram development; the requirements of the sources offunding; and the quantity and quality of technical assistancereceived.

Elements ofProgram Organization-Variables of primeimportance in defining the structure and function of the ruralprimary care projects that have been developed are thosewhich might be expected to influence the impact of theservices upon the target population. We classified these intothree groups (described in detail in an earlier paper).'

1. characteristics of professional personnel;2. the characteristics of health care content;3. program organization and administration.A typology of organizational forms of these programs

was developed. It is presented in detail later in this paper.Characteristics of the Community-Characteristics of

the community and its relationship to the project include:socioeconomic status, demography, density of population,rurality indices, other medical resources in the area, levels ofcommunity understanding and expectations, the degree ofcommunity involvement, legislation affecting medical prac-tice or reimbursement, etc.'

Access, Stability, and Consumer Satisfaction-A fulldescription of our approaches to measuring these variables isbeyond the scope of this paper. Briefly, access has beenconceptualized as a set of perceptual phenomena20 withbehavioral consequences in the form of health care utiliza-tion, particularly in relation to need.2'

Two aspects of practice stability are being examined:the ability of the program to maintain a full complement ofproviders over time; and the ability of the program togenerate adequate demand and to control costs in order tomove toward self-sufficiency and reduce the dependence onsubsidies. Consumer satisfaction is measured by the extentto which consumers express positive feelings about andpreferences for various aspects of their care.

Research Objectives

Figure 1 illustrates the key variables in a conceptualmodel to answer the major evaluation questions. In additionto reflecting the scope of the study, it illustrates the way inwhich certain variables are dependent at one stage of theanalysis, and independent at another stage.

Having developed this conceptual framework, our ob-jective was to develop a research plan which would enableus to:

1. delineate the relationship between each type of de-velopmental strategy and the organizational form of primarycare;

2. delineate the generic nature of each organizationalform in terms of specific characteristics such as mode ofoperation, scope of services, and the association of eachcategory of organizational form with various strategies forprogram development;

AJPH January 1983, Vol. 73, No. 1 39

SHEPS, ET AL.

3. determine the relationship between the differentforms of program organization and their outcomes in termsof access, stability, and consumer satisfaction;

4. and finally, generate a set of recommendations con-cerning the relative importance of key elements of ruralhealth programs in the achievement of access, communitysatisfaction, and program stability in different types ofcommunities.

Study Design

The character and scale of data needed to answer theresearch questions led to a study design in which specificsets of data were collected at four different levels or tiers.While certain types of information have been obtained onlyin one tier, other areas of inquiry were pursued by obtainingmore detailed information on a smaller number of programsin another tier. This partitioning was done randomly toachieve generalizability from one tier to the next and tocompare information on certain variables from one tier toanother.

Tier I was a descriptive inventory of as many of thesubsidized rural primary care programs in the United Statesas could be reached. Information was obtained in the fall of1979 by the US Department of Health and Human Services(known then as the Department of Health, Education, andWelfare) via a questionnaire sent to all known subsidizedrural primary care programs. The published list consists of1,4% health organizations.22

Limited information was sought regarding general char-acteristics of the program such as the size of the serviceareas, estimates of numbers of users and encounters, staffcharacteristics, the types of services provided, record sys-tems, linkages with other providers, technical assistance,program governance, and problems faced in program devel-opment. Information was also requested on financing of theprogram including total expenditures and the percentage ofreceipts coming from various sources, plus methods ofcharging patients and paying providers. Tier I provided thedata needed for identifying eligible programs for our evalua-tion and classifying them into the organizational forms,described ater,and serve as the sampling frame forsubsequent tiers of the study.

Tier II consisted of a telephone survey followed bymailed questionnaires designed to acquire more detailed datafrom each of 259 programs selected by a stratified randomsampling procedure to represent the various organizationalforms nationwide. Information was obtained about develop-mental history, current organizational characteristics andclinical program staffing, financial stability, and the nature ofthe community served. In addition, a mail survey wasconducted of all physicians and new health professionals(nurse practitioners and physician assistants) in these pro-grams, seeking information on their background, their rea-son for being employed in these programs, their level ofsatisfaction, and plans with regard to their continuing towork in these programs. A second mail questionnaire wasused to obtain detailed information about the financialoperations of each program.

Tier III consisted of site visits to 40 randomly selectedprograms to obtain a direct view of the operation of theprogram, verifying and expanding upon the type of dataobtained by telephone in Tier II.

Tier IV consisted of a random household survey ofresidents in each of 36 of the 40 communities studied in TierIII plus eight comparison communities. Four Tier III com-munities were excluded from Tier IV because of fundinglimitations; these four were selected because of languagedifficulties or other problems determined during the sitevisits. In selecting eight additional comparison communities,the objective was to find communities which were lackingadequate primary care services, but which resembled thecommunities where subsidized rural primary care programsare presently located.

A Typology of Rural Practice Organizations

Tier I variables considered to be of prime importance indefining the structure and function of subsidized rural pri-mary care programs were those which might be expected toinfluence the impact of the services upon the target popula-tion. A review of relevant literature' 2,19,23-31 and the fieldexperience of our research group led to a typology basedupon three relatively stable characteristics of such pro-grams: 1) the form of sponsorship/governance of the pro-gram; 2) the size and mix of the program's staff; and 3) thespatial dispersion of the program and its clinical facilities(i.e., whether the program operated from a single site ormultiple clinical site). This typology builds on the work ofDavis and Marshall23 and includes five forms:*Comprehensive Health Centers (CHCs)

These programs are characterized by comprehensive programdevelopment on a relatively large scale, together with substantialcommunity involvement and control. Social and health objectivesare to be achieved by a relatively broad range of non-clinicalservices to support and extend the impact of basic medical services.Examples are the neighborhood health centers and family healthcenters (mainly supported by DHHS), of which a sizable numberserve rural populations.Operational Definition

Any program, not externally controlled [i.e., not form IEP(institutional extended program)], having a governing communityboard, at least three full-time equivalent primary care providers(i.e., physicians and/or new health professionals), and providingoutreach services.Organized Group Practices (OGPs)

These programs consist of at least two full-time physicians ingroup practice operating autonomously, through a pooled incomearrangement, not providing any outreach services. Some, like thepractices fostered and supported by the Robert Wood Johnson RuralPractice Project, emphasized leadership by physicians, sophisticat-ed administration and staff development.Operational Definition

Any program, not form CHC or not externally controlled,having at least two full-time physicians who are compensated other

*Note: These definitions apply only to those primary careprograms operating from a single clinical practice site.



OrganizationalFormsConditions

External Control?( (yes) IEP{Institutional Extension}

(no)

I- Three or more

Community Salaried FTE outreacGoverning -(yes Primary Care + service - CHCBoard? Providers (Comprehensive Health Center}

(no) (no)

Two or mo MD Payment-basedFull-time -(yes) on other than (yes) - - OGPMDs? fee for service? {Organized Group Practice)

Traditionalno Group Practice

(n(exactly

) 1 full-time -(yes)MD?*

I community Traditional' (no) _ involvement (no Solo

Practice

(Y's)

Community -InvolvementI {Primary Care Center}

*This means no NHPs or part-time MDs working in practice site.

FIGURE 2-Algorithm for Classifying Programs by OrganizationalForm

than on a fee-for-service basis (although the practice may charge forpatient care on a fee-for-service basis).Primary Care Centers (PCCs)

These are smaller programs stimulated and/or subsidized byindigenous community initiative, with or without financial assist-ance from outside the community, and often involve the use of newhealth professionals with physician backup, on site or elsewhere.There is no formal institutional affiliation.

Operational DefinitionAny program not form CHC or OGP or externally controlled

having a governing community board or both local government orcitizen group involvement in program initiation and current financialsupport from local civic or community groups.

Institutional Extension Practices (IEPs)These are services developed by existing institutions such as

hospitals, health departments, group practices, etc. The leadingexample of this approach is the W. K. Kellogg Foundation-support-ed Innovative Ambulatory Primary Care Award Program. There arealso rural satellites developed by health departments, establishedgroup practices and university medical centers or Health Under-served Rural Areas (HURA) programs sponsored by DHHS.

Operational DefinitionAny program, regardless of its other characteristics, for which

an external health agency or organization (i.e., public health depart-ment, private medical practice, hospital, or medical school) controlsthe program budget and/or hiring and firing decisions.

Traditional Solo Practice (SPs)Here the objective is to place some medical manpower into a

community with little or no emphasis on particular health program

elements, administration, or institutionalization. The placement ofan isolated physician in a community has been the traditionalapproach to rural health care delivery.Operational Definition

Any program not form IEP or CHC or OGP or PCC having onefull-time physician, no new health professionals, no dental orsupport service staff and providing only basic medical services.

Other Forms of PracticeIn very few cases, the characteristics of a single-site program

may be such that it cannot be classified into one of the five forms. Insuch circumstances it is classified as "other."

Programs with more than one service delivery site (i.e.,multiple-site programs) can be classified using a three-stageprocess.

1. If an external agency controls the budget or personnel of theentire program, it is classified as an IEP (i.e., multiple-siteinstitutional extension program).

2. If the program is not externally controlled, each full-time,year-round rural site of a multiple-site program is classifiedaccording to the same criteria as for single-site programs.

3. The entire multiple-site program is classified according to itsmost complex site, where CHC is considered more complexthan OGP, and OGP is more complex than PCC. By defini-tion a multiple-site program cannot be clas fed as SP. Anymultiple-site program not classified as , CHC, OGP, orPCC is classified as "other."

The empirical procedure by which the typology oforganizational form was operationalized is illustrated in thealgorithm in Figure 2.

Methods

In the Tier I survey outlined above, questionnaires weresent to 1,496 organizations located in rural areas or servingrural populations and offering some primary care servicessuch as internal medicine, family or general practice, pediat-rics and/or obstetrics/gynecology.

Responses were received from 998 programs in thecontiguous United States (excluding Alaska and Hawaii).After eliminating responses with inadequate or inconsistentdata, further review of the data regarding program character-istics showed that not all of the respondents should beincluded in the study. In order to eliminate programs, thefollowing criteria were used:

1. Support Criterion-The program must have received somegovernment or private foundation support either during itsdevelopment or at some time during its operation, but notnecessarily at the time of the Tier I survey.

2. Content of care criterion-at least one site in the programmust deliver primary care not restricted on a statutory basisto a particular subgroup of the population or category ofhealth problems.**

3. Days open criterion-the program must have at least one siteopen to provide services at least four days a week essentiallythe year round.

4. Rurality criterion-the program must have at least one sitelocated in a rural area. A site was considered rural if it waslocated more than 15 road miles along an interstate or more

**Health programs which exclusively serve military or Indianreservations were not included. Programs which serve migrantworkers were included. Programs providing diagnosis and/or treat-ment of specific conditions only, such as tuberculosis, venerealdisease, or immunizations were not considered to provide primarycare according to this criterion.


I

41

SHEPS, ET AL.

TABLE 1-Distribution of Rural Primary Care Programs by Organizational Form

Single-Site Multiple-Site All ProgramsOrganizational Form Number (%) Number (%) Number (%)

Comprehensive Health Center 21 (6.9) 28 (17.4) 49 (10.6)Organized Group Practice 59 (20.5) 39 (24.2) 98 (21.1)Primary Care Center 134 (45.9) 45 (28.8) 179 (38.6)Institutional Extension Practices 63 (21.2) 25 (15.5) 88 (19.0)Other, Including Solo Practices* 26 (5.1) 24 (14.9) 50 (10.8)Total Classified 303 (100.0) 161 (100.0) 464 (100.1)

*Includes 15 solo practices, programs not meeting the criteria for any of the defined forms, and programs thatcould not be classified due to missing information.

than 10 road miles on non-interstate roads or across a stateboundary line from the nearest city of 25,000 or more people.

Six hundred twenty-seven programs met these fourcriteria.*** For evaluation, however, we needed a group ofprograms which had provided service for a long enoughperiod of time to have become established and serve aregular clientele and this led to a fifth criterion.

5. Program age criterion-the program must have starteddelivering services prior to 1978; multiple-site programswere considered to have met this criterion if any delivery sitestarted delivering services prior to 1978.

There were 464 programs, a little less than half of therespondents, which met these five criteria.

Information needed to classify the programs which wasomitted from the returned questionnaires was obtained bytelephone follow-up. Of the 464 programs, 92 per cent couldbe classified into the five organizational forms alreadydefined. There were only 35 programs which did not provideenough information to be classified or did not meet thecriteria for the five categories of organizational form; 22 ofthese were multiple-site programs whose principal site wasnot in a rural area, although at least one associated site wasproviding primary care to one or more rural communities.

In addition to the questionnaire data, certain informa-tion was obtained about the county in which the programwas located from the "Area Resource File," a computer-based, county-specific data bank containing demographic,economic, and health resource items made available by theHealth Resources Administration of the US Department ofHealth and Human Services.32

ResultsDistribution of Organizational Forms

The classification of programs by organizational form isshown in Table 1. The most prevalent organizational form isthe Primary Care Center (PCC). Almost one-half of thesingle-site programs and one-third of the multiple-site pro-grams fall into this category. The Organized Group Practice

***The precise number of existing programs in the US isunknown, but we developed an estimate (see Appendix), whichsuggests that this subset of programs contains about 75 to 80 percent of all programs qualifying by these four criteria.

(OGP) is the next most frequently encountered. Althoughtwo-thirds of all programs are single sites, the majority ofComprehensive Health Centers (CHC) are multi-site pro-grams, a finding consistent with the large scale of theseorganizations by definition. Institutional Extension Practices(IEPs) represent only one-fifth of all programs. Further dataon IEPs suggest that, in terms of sponsorship, the majorityof these programs are extensions of health departments.Finally, the traditional Solo Practice (SP) represents only asmall fraction of programs, by definition existing at onlysingle sites. Because of the small number of these programs(N = 15) and their declining policy relevance, this form willbe omitted from subsequent comparisons of organizationalforms.

Time of Program Development

The different forms varied in their recency of develop-ment, as measured by when they first began deliveringservices. More than three-fourths of the programs firstprovided services after 1970. Although the years 1970-1977included the period of most rapid development for allorganizational forms, the pattern of development was not thesame for all forms. In particular, IEPs had the earliestsizable development, with over 20 per cent of them startedbefore 1960 and nearly 40 per cent before 1970. In contrast,20 per cent or less of programs of the three other forms wereinitiated prior to 1970. Single-site OGPs also had earlydevelopment, with 21 per cent delivering services before1960, followed by a lull in further development until 1970.Comprehensive Health Centers are of more recent origin;only three programs (6 per cent) were initiated before 1960and nearly 60 per cent began between 1970-1975-the finalyears of OEO activity. Primary Care Centers represent anewer development in rural practice with over 85 per centhaving started since 1970 and over 40 per cent beginningservices after 1975.

Sources of Financial and Personnel Support

Limited data dealing specifically with sources of pro-gram support, (either financial or personnel) are shown inTable 2. Eighty-six per cent of all programs were supportedin some way by the federal government, notably the CHCs.The large majority of programs of other organizational formsalso were federally subsidized, although not quite to the



TABLE 2-Percentage of Programs Receiving Different Kinds of Public and Private Financial orPersonnel Support, by Organizational Form of the Program*

Single-Site Programs Multiple-Site Programs

Organizational Form Organizational Form

TotalCHC OGP PCC IEP CHC OGP PCC IEP Programs

Source of Support n = 21 n = 59 n = 134 n = 63 n = 28 n = 37 n = 45 n = 25 n = 464

Public Sources % % % % % % % % %Federal (any type) 100 81 89 79 100 92 78 81 86US DHHS 100 68 82 57 100 87 78 87 80US Office of Economic

Opportunity 14 3 6 8 46 8 13 1 1 10Appalachian RegionalCommission 14 17 16 24 14 22 16 26 18

US Dept. of Agriculture(WIC, FmHA, Supple-mental Foods) 33 7 13 29 61 27 49 52 26

US Dept. of Labor (e.g.,CETA) 57 34 37 37 75 43 67 48 45

State or Local 38 41 37 67 71 41 49 52 49Private Foundations 33 27 22 33 18 24 20 27

Robert Wood JohnsonFoundation 10 15 6 6 4 14 7 4 8

W. K. Kellogg Foundation 10 15 4 10 7 3 2 7 5Local Groups 67 34 60 35 39 49 51 30 53

*Column percentages do not add to 100% due to presence of multiple sources of funding in most programs.

extent of CHCs. State and local governmental agenciessupported 49 per cent of the programs, with IEPs their majorfocus.

Private sources of support played a significant but muchsmaller role. About 25 per cent of programs received founda-tion support, and about 47 per cent received support fromlocal private groups. There was no consistent pattern ofprivate support among the organizational forms, althoughthe Robert Wood Johnson Foundation's Rural PracticeProject may be described as the prototype for OGPs and alarger percentage of these practices received Johnson Foun-

dation support than other organizational forms. The out-standing characteristic of Table 2 is the multiplicity ofsupport sources for all types of programs.

Program Staffing

The wide variation in staff size of subsidized ruralprimary care programs is illustrated in Table 3. Single-siteprograms varied considerably in staff size and this variationwas heavily influenced by organizational form. Comprehen-sive Health Centers were staffed most heavily (median,21.4), while PCCs tended to have the smallest number of

TABLE 3-Percentage of Programs by Organizational Form and Number of Full-Time EquivalentTotal Staff

Single-Site Programs

Organizational Form

Multiple-Site Programs

Organizational Form

Number of Full-Time Equivalent

Total StaffCHC OGP PCC IEP CHC OGP PCC

n = 20 n = 54 n = 119 n = 56 n = 27 n = 36 n = 41IEP

n = 20

Less than 5 5 10 57 435-10 9 25 20 13 0 14 14 2111-20 36 34 15 25 7 35 44 2121-50 50 31 8 19 48 32 40 29More than 50 0 0 0 0 44 19 2 29Total 100 100 100 100 100 100 100 100Median 21.4 14.6 4.78 9.50 47.7 20.0 18.6 26.3


SHEPS, ET AL.

TABLE 4-Percentage of Programs by Organizational Form and Proportion of All Primary CareProviders Who are New Health Practitioners


Proportion ofOrganizational Form Organizational Form

Providers Who areNew Health CHC OGP PCC IEP CHC OGP PCC IEPPractitioners n= 20 n= 54 n = 119 n= 56 n= 27 n =36 n = 41 n = 20

0 5 57 40 39 7 19 20 10.01-.29 5 28 1 1 1 33 25 10 10.30-.50 70 13 23 10 30 44 30 30.51 or more 20 2 36 40 30 11 40 50

staff (median, 4.8). As expected, the staff size of multiple- Table 4 shows the proportion of the full-time equivalentsite programs was larger than single-site counterparts. primary care provider staff consisting of new health profes-

One-fifth of all IEPs and 15 per cent of single-site PCCs sionals in the different organizational forms. In 90 per cent ofhad no physician presence at all. Similarly, 25 per cent and 5 single-site CHCs, new health professional personnel accountper cent of multiple-site IEPs and PCCs, respectively, had for 30 per cent or more of their full-time equivalent primaryless than a single full-time physician. In contrast, approxi- care providers, but only about 15 per cent of the single-sitemately one-half of multiple-site CHCs and OGPs had large OGPs use such personnel this extensively and over one-halfphysician staffs consisting of five or more. Overall, 55 per of these groups have none at all. In multiple-site programscent of single-site and 85 per cent of multiple-site and 11 per the presence and intensity of use of new health professionalscent of single site programs employed the services of new is somewhat greater than is the case in single-site programshealth professionals (either nurse practitioners and/or physi- and varies less by organizational form.cian assistants). Ninety per cent of single site CHCs had newhealth professionals on staff as contrasted with approximate- Scope of Services Providedly 40 per cent of single-site programs of the other three We examined whether different organizational forms offorms. providing primary care offered a different mix and scope of

TABLE 5-Percentage of Programs which Provide Different Types of Services, by Organiza-tional Form of the Program



CHC OGP PCC IEP CHC OGP PCC IEPTypes of Services Provided n = 21* n = 59* n = 134* n = 62* n = 28* n = 37* n = 45* n = 25*

%% % % % % % %Prenatal Care and

Family Planning 95 89 84 91 100 97 96 96WIC 55 30 33 53 68 44 63 75Well-Child Care 95 88 86 91 93 100 89 96Nutrition Counseling 95 75 83 88 93 95 84 100Outreach 100 24 39 52 100 44 65 70Transportation 55 10 24 37 85 19 36 42Environmental 25 1 1 18 46 33 17 29 69Screening or Health

Education 95 58 71 77 96 78 86 88Dental 70 28 19 41 81 59 52 48Mental Health 30 27 25 27 46 49 31 32Organized Home Care 38 17 28 30 36 32 29 36Physical Therapy 35 15 19 37 27 38 16 32Mean Number of Services 7.9 4.7 5.3 6.7 8.6 6.7 6.8 7.9

*Because of incomplete data, the n varies from service to service.



TABLE 6-Percentage of Programs Which Have Different Types of Medical Specialists*Available, by Organizational Form of the Program


Organizational Form Organizational FormType of Medical

Specialist in CHC OGP PCC IEP CHC OGP PCC IEPProgram n =20 n =54 n = 119 n =56 n =27 n =36 n =41 n =20

Internal Medicine 33 42 22 19 52 49 18 33Pediatrics 58 41 9 27 62 32 16 55Obstetrics andGynecology 38 29 5 17 19 24 9 19

General Surgery 25 25 10 13 26 24 2 11Ophthalmology 15 12 3 3 4 8 4 0Psychiatry 20 7 4 0 4 19 4 19

*Other than family or general practitioners.

services, since only the provision of outreach services wasincorporated into the definitions of organizational form. Thepattern of health and social services provided by programs ofvarious forms is displayed in Table 5. Nearly all programs,regardless of form, were providing basic primary care serv-ices such as prenatal and well-child care. By definition,CHCs had to provide outreach services. However, they alsowere most likely to offer other special services, such astransportation, screening or health education, dental serv-ices, and organized home care. IEPs also frequently provid-ed non-medical services consistent with the frequency ofhealth departments as the sponsoring agency. Multiple-siteprograms more often had a given service available than didsingle-site programs of the same organizational form.

Table 6 shows the availability of various medical spe-cialty services other than family/general practice withinprograms. As would be predicted by their staff size relative

to other forms, CHCs and OGPs more frequently have otherthan family/general practice physicians than do PCCs orsingle-site IEPs. Consistent with the dominance of newhealth practitioners in PCCs, these medical specialists arerelatively rare in such practices, even in multiple-site pro-grams. The relatively high frequency of pediatricians, asopposed to internists, in CHCs and IEPs is noteworthy.With respect to IEPs, we suspect that this reflects theimportance of the local health department and its traditionalemphasis on immunization and well-child care.

Community -Setting

An analysis of socioeconomic and demographic datafrom the counties where rural primary care programs arelocated suggests that different organizational forms tend toserve different population subgroups. Since programs oftenserve only a portion of a given county, the county data

TABLE 7-Percentage of Organizational Forms in Study Which Are Located in Counties withLow Median Income, Large Proportion of Black and Hispanic Population

Single-Site Programs Multiple-Site Programs*


CHC OGP PCC IEP CHC OGP PCC IEPCounty Characteristics n = 21 n = 59 n = 130 n = 62 n = 28 n = 37 n = 45 n =25

Per cent located incounties with a medianfamily income of lessthan $3,000 in 1975 38 5 13 10 25 8 19 18

Per cent of programslocated in countieswhere the populationis at least 20% Black 24 7 15 10 25 5 9 11

Per cent of programslocated in countieswhere the populationis at least 20% Hispanic 29 3 12 5 1 1 3 7 15

*County of principal site is used in table.


SHEPS, ET AL.

TABLE 8-Distribution of Each Organizational Form by Location in Counties of DifferentPopulation Size

Single-Site Programs Multiple-Site Programs*


CHC OGP PCC IEP CHC OGP PCC IEPSize of County n= 21 n= 59 n = 134 n= 62 n= 28 n = 37 n= 45 n = 25

Less than 10,000 0 5 18 6 4 8 11 1510,000-24,999 33 25 21 29 32 38 29 2625,000-49,999 24 34 20 27 21 16 16 19Greater than 50,000 43 34 41 38 43 38 44 41

*County of principal site used in table.

presented in Table 7 provide only a general indication of thecommunity setting in which these programs operate. Formultiple-site programs the location of the principal sitedesignated by the program was used for these analyses. Alarger percentage of all CHCs than of other forms are locatedin "disadvantaged areas," i.e., in counties defined as havinga median annual family income of less than $3,000 in 1975 orat least 20 per cent Black or Hispanic population. Thelocation patterns of CHCs and OGPs are very different:more than one out of four CHCs, both single-site andmultiple-site programs, are in disadvantaged areas; fewerthan one in ten OGPs are in such areas.

There is little difference among organizational formswith respect to the total population of the counties in whichprograms are located, as shown in Table 8. Depending on theform of practice, between 36 per cent and 44 per cent ofprograms are in counties with populations over 50,000.However, there are higher percentages of PCCs and multi-ple-site IEPs in counties with population under 10,000 thanfor the other forms. Thus, PCCs and IEPs are the mostprevalent fornm of subsidized practice organization in coun-ties with small populations.

Service Area Population

The size of the service area population as reported bythe programs themselves is shown in Table 9. For single-siteprograms, CHCs and OGPs have a very similar distributionof service area populations, with almost half of these pro-grams having service areas containing between 15,000 and40,000 people and none serving more than 40,000 or fewerthan 3,000 people. Single-site IEPs tend to serve slightlysmaller areas; single-site PCCs have the smallest serviceareas. The service area population reported by multiple-siteprograms includes the population of the total area served byall sites in each program and is, therefore, greater than forsingle-site programs. The pattern for multiple-site programsis not quite as clear as for single-site programs, but generallyfollows the same lines.

Discussion

This evaluation constitutes a comprehensive, althoughnot fully complete, approach to the evaluation of subsidizedrural primary care programs. It makes it possible to charac-

TABLE 9-Percentage of Programs Which Have Service Area Populations of Different Size, byOrganizational Form of the Program



Population of the CHC OGP PCC IEP CHC OGP PCC IEPService Area n =21 n =59 n = 130 n =62 n =27 n =37 n =44 n =25

3,000 or fewer 0 0 17 8 0 0 0 03,001-7,000 10 12 34 18 0 3 1 1 247,001-15,000 43 39 22 34 4 30 30 415,001-40,000 48 49 27 40 33 49 39 28Over 40,000 0 0 0 0 63 19 21 44Median ServiceArea Size 14,400 14,800 6,900 12,600 40,000t 23,700 20,800 34,600



terize three major clusters of variables operating in thiscomplex field-the communities, the developmental strate-gies, and the organizational forms of the practice. It shouldbe possible to relate them to each other and, finally, to theactual outcomes which they produce in their communities.We believe this evaluation plan differs from previous effortsto evaluate rural primary care programs in several significantways:

* It is national in scope, encompassing private andpublic national and regional efforts;

* It focuses on multiple program outcomes rather than asingle one and allows some consideration of how variousdesired outcomes may reinforce or inhibit one another;

* It examines different organizational forms and pro-gram components in different types of community settingssimultaneously, thus making it possible, we hope, to assessthe relative impact of each of these major factors; and

* It assesses the relationship between the original inten-tions of diverse intervention strategies and the nature of theirrealization in concrete organizational forms, processes, andachievements.

The concept of organizational form of practice appearsto be a useful tool for describing and classifying ruralprimary care programs. Distinct organizational forms can bedefined and, as demonstrated, most existing programs canbe classified by form. Furthermore, the programs differ interms of important characteristics other than those whichwere used to define the forms of practice.

Before interpreting the significance of these differencesfor the evaluation of rural health policy, it is worth bearing inmind the limitations of drawing conclusions from the kind ofcross-sectional data presented thus far. First, it cannot beassumed that each organization which was identified ashaving a specific organizational form has had that form sinceits inception. Programs may add or lose personnel, expandor contract their scope of services, affiliate with or becomeindependent of other institutions, or add or drop servicedelivery sites over the course of their development. Conse-quently, a program's present organizational form may differsubstantially from the way it was organized originally or howit was designed by its founders or sponsors. This is animportant limitation in attempts to evaluate the merits ofalternative strategies for health program development. Sec-ond, making inferences from a survey conducted at onepoint in time is difficult because only surviving organizationshave been observed. For example, if the rate of survivalwithin different organizational forms is quite variable, thenknowledge about differences between presently existingpractices will give only a partial and potentially biased view.

Despite these limitations, the CHC typically appears tobe a program with more than one service delivery site,serving a community that has a relatively large populationcomposed of a high proportion of minority or poor people.This type of program has a relatively large and diversemedical staff and a broad scope of non-medical personalhealth services. Almost certainly, the CHC will employsome new health practitioners and often they will approxi-mate or exceed the number of physicians. A broad scope ofservices, dispersed delivery sites, and some reliance on non-

physician providers may all be indicative of an underlyinggoal of increasing access to health care. In terms of itsexternal sources of support, the CHC has invariably hadfederal financial and/or personnel support. Generally thissupport has come from the Department of Health andHuman Services, but often from other federal sources aswell, such as the Departments of Labor and Agriculture,and/or the Office of Economic Opportunity (when it was inoperation). The larger multiple-site CHC is also likely tohave received some state or local public funding as well.

The OGP is an organizational form that is most fre-quently found at a single-service delivery site located in acounty that is not likely to have low per capita income or ahigh proportion of Blacks or Hispanics. The service areapopulation of the OGP is likely to be neither very small (i.e.,under 7,000) nor very large (i.e., over 40,000). The staffing ofthe group is likely to contain a relatively broad array ofmedical specialists, but the organization delivers a relativelynarrow scope of services. New health practitioners areusually not employed in single-site OGPs, but since they areemployed in more than 80 per cent of multi-site grouppractices, it might be inferred that they are used to extendthe physicians' practices to satellite locations and act as asource of referrals for the physicians at the main site.Although 80 per cent of OGPs have been recipients offederalfunds of some type, such organizations appear likely to havealso drawn support from a number of public and privatesources.

The PCC is typically smaller and younger than the CHCor the OGP. The PCC is the most likely of any organizationalform of practice to be located in a smaller community. Overone-half of the single-site PCCs indicate that their servicepopulations are less than 7,000. The PCC appears to belocated in communities which typically are neither far abovenor far below the median for rural counties in terms of percapita income and minority populations. Staffing of PCCs ismore reliant on mid-level practitioners than is the case inCHCs or OGPs. Almost 90 per cent of PCCs receivedsupport from one or more federal sources, and about one infive have received help from private foundations. PCCs arelikely to have received some assistance form local voluntarygroups.

The IEP is the most heterogeneous of the organizationalforms in terms of size and service scope. This is probablydue to the variability in sponsorship of this type of program.Most of the IEPs are satellites of health departments orhospitals, but some are extensions of group practices ormedical schools. The "outside" sponsorship is reflected inthe pattern of support which is typically from state or localpublic sources, with less frequent support from local privateor voluntary community sources. Further understanding ofthe various characteristics of these kinds of programs wouldentail a separate analysis of them according to the kind ofsponsoring agency. For example, the high frequency ofWomen, Infants and Children (WIC) programs, environmen-tal, and maternal and child health service in IEPs reflects theservice configuration characteristic of local health depart-ments, with which many IEPs are affiliated.

The ability to classify the various forms of sponsored


SHEPS, ET AL.

programs for the provision of primary health care services torural communities is an essential step in the evaluation ofalternative strategies for the enhancement of primary medi-cal care in these communities. When these forms of practiceorganization are compared with a number of organizationaland community contextual variables not used in the defini-tion of organizational form, it appears that significant pat-terns of difference among these types of practices exist.These findings confirm the importance of disaggregating theeffects of community and organizational characteristics andtheir covariation in subsequent attempts to measure orevaluate the impact of these programs. Further analyses ofthe National Rural Primary Care Evaluation Project will usethe operational definition of form of practice organizationdescribed above as the basis for evaluating the differentialimpact of alternative types of primary care programs.

REFERENCES1. Sheps CG, Bachar M: Rural areas and personal health services:

current strategies. Am J Public Health 1981; 71(supp):71-81.2. Martin ED: The federal initiative in rural health. Public Health

Rep 1975; 90:291-297.3. Rosenblatt RA, Moscovice I: Establishing new rural family

practices: some lessons from a federal experience. FamilyPractice 1975; 7:755-763.

4. Rosenblatt R, Moscovice I: The growth and evolution of ruralprimary care practice: The National Health Service Corpsexperience in the northwest. Med Care 1978; 16:819-827.

5. Holland CD, Durmaskin BT, Donovan CJ, et al: West VirginiaPrimary Care Clinics: Development, Availability, Utilization,and Service Area Determination for 1977 and 1978: Part I.Morgantown, WV: Office of Health Services Research, January1979, 118 pp.

6. Cordes S, Camasso M, Crawford AH, et al: Progress Report onNonprofit Primary Care Centers: An Analysis of Their FinancialViability and the Quality of Care Provided. Pennsylvania StateUniversity, February 8, 1979, 48pp (mimeograph).

7. Tennessee Association of Primary Health Care Centers. FinalReport of Primary Care Survey. Nashville, TN: February 1981(mimeograph).

8. Kane R, Olsen D, Wright D, et al: Changes in utilizationpatterns in a National Health Service Corps community. MedCare 1978; 16:828-836.

9. Kasteler JM, Hughes CC: The rural health delivery problem.Family & Community Health: J Health Promotion & Mainte-nance 1978; 1:61-70.

10. Blake RJ Jr, Guild PA: Mid-level practitioners in rural healthcare: a three-year experience in Appalachia. J CommunityHealth 1978; 4:15-22.

11. Gordon M, Hartin G: Survey of Health and the Health CareUtilization Patterns of a Small Rural Black Community. Hous-ton, TX: Texas Southern University, June 30, 1975, 178 pp.

12. Voth DE, Whitfield WH, Redfern JM, et al: A Socio-EconomicEvaluation of a Comprehensive Health Care Facility in Arkan-sas: The Jefferson Comprehensive Care Center, Inc. Fayette-ville, AR: Department of Agricultural Economics & RuralSociology, University of Arkansas, March 1979, 48 pp.

13. Sparer G, Dines GB, Smith D: Consumer participation in OEO-assisted neighborhood health centers. Am J Public Health 1970;60:1091-1102.

14. Morehead MA: Evaluating quality of medical care in the neigh-borhood health center program of the Office of EconomicOpportunity. Med Care 1970; 8:118-131.

15. Brooks EF, Graham RM, Warren-Hicks DL, et al: RuralPrimary Care and New Health Practitioners. Chapel Hill, NC:Health Services Research Center. Presented at the 1979 AnnualMeeting of the American Health Planning Association, 31 pp(mimeograph).

16. Coleman S: Physician Distribution and Rural Access to MedicalServices. Santa Monica, CA: Rand Corporation, April 1976, 98pp

17. DHEW: The Health Underserved Rural Areas Program StatusReport as of December 31, 1978. Prepared for the SenateAppropriations Committee by the Department of Health, Edu-cation, and Welfare, HSA, BCHS, ORH, February 26, 1979.

18. Rosenblatt RA, Moscovice I: The National Health ServiceCorps Program: A Review and Discussion of Past Research andEvaluation Efforts. University of Washington, May 1, 1979, 68pp (mimeograph).

19. Walleck SS, Kretz SE: Rural Medicine: Obstacles and Solutionsfor Self-Sufficiency. Lexington, MA and Toronto: LexingtonBooks, D.C. Heath and Company, 1981.

20. Penchansky R, Thomas JW: The concept of access. Med Care1981; 19:127-140.

21. Andersen R: Health status indices and access to medical care.Am J Public Health 1978; 68:458-563.

22. DHEW, Office of the Assistant Secretary for Planning andEvaluation/Health: Directory of Rural Health Care Programs,1979. Washington, DC: OASPE/H, 1979.

23. Davis K, Marshall R: Primary health care services for medicallyunderserved areas. In: Papers on the National Health Guide-lines: The Priorities of Section 1502. Washington USDHEWPub. No. (HRA) 77-64, 1977:1-23.

24. The Primary Care Development Project, Prescription for Pri-mary Care: A Community Casebook. Ithaca: Cornell Universi-ty, 1976.

25. Bernstein JD, Hege FP, Farran CC: Rural Health Centers in theUnited States. Cambridge: Ballinger, 1979.

26. Kane R, Dean M, Solomon M: An evaluation of rural healthcare research. Evaluation Quarterly 1979; 3:139-189.

27. Rockoff ML, Gorin L, Kleinman JC: Positive programming:The use of data in planning for the rural health initiative.Community Health 1979; 4:204-216.

28. National Health Service Corps Scholarship Program-A Reportby the Secretary ofHEW to Congress. DHEW Pub. No. (HRA)79-13. Washington DC: Govt Printing Office, 1978.

29. Rosenblatt R, Moscovice I: Critique of Previous Research andEvaluations of the National Health Service Corps. San Francis-co: Health Services Policy Analysis Center, School of Medi-cine, University of California, 1979.

30. Studt WB, Sorenson JG, Burge B: Medicine in the Intermoun-tain West: A History of Health Care in Rural Areas of the West.Salt Lake City: Olympus, 1976.

31. Kaluzny AK, Konrad TR: Organizational design and the provi-sion of primary care services. In: Bisbee GR (ed): The Manage-ment of Rural Primary Care. Chicago: Hospital Research andEducational Trust, American Hospital Association, 1981.

32. Department of Health, Education, and Welfare, Health Re-sources Administration: The Area Resource File: A ManpowerPlanning and Research Tool. DHEW Pub. No. (HRA) 77-23.Washington DC: DHEW, 1976.

ACKNOWLEDGMENTSThis study was supported by the Robert Wood Johnson Foun-

dation, the US Department of Health and Human Services, and theHealth Services Research Center of the University of North Caroli-na at Chapel Hill.

APPENDIX

Because of the relatively high proportion of programsidentified in Tier I that did not meet the criteria for inclusionin the study, a systematic attempt was made to identify theextent to which qualifying programs were not included in thenational inventory due to non-response. The lists of non-



respondents to the mailings as well as the respondents werecompared with lists of rural primary care programs devel-oped by statewide primary care associations, Health SystemAgencies, State Health Planning and Development Agen-cies, or State Health Departments for five of the stateshaving the largest numbers of rural primary care deliverysites: New Mexico, North Carolina, Pennsylvania, Tennes-see, and West Virginia.

These locally-generated lists were compared with theprograms to which the Tier I questionnaire was sent in thesefive states by the following procedure. An "identificationrate" was calculated. This consisted of the number of sites ina given state identified by the mailing list used by thecontractor who conducted the survey divided by the total

number of sites identified by either the contractor's list orthe locally generated list. The identification rates for the fivestates ranged from a low of 86 per cent in Pennsylvania to ahigh of 100 per cent in North Carolina and Tennessee.

An "adjusted response rate" was also calculated foreach of the five states. This is the number of respondent sitesdivided by the number of sites identified either by thecontractor or the locally generated source. These ratesranged from a low of 64 per cent to a high of 90 per cent.Because data about staffing, starting date, scope of services,and other qualification criteria were not always available fornon-respondents or sites on the locally-generated lists, theseestimates are conservative, i.e., the denominator may beinflated by inappropriate cases.

I Can Polio Be Eradicated?International Symposium on Polio

The persistence of poliomyelitis as a major public health problem in developing areas throughoutthe world, coupled with recent advances in knowledge concerning the disease and methods for itscontrol, make convening a major international symposium on poliomyelitis control both appropriateand timely.

The Forgarty International Center, under it's Program on International Issues in Public Health andRelated Bio-Medical Research, together with the World Health Organization Regional Office of theAmericas will sponsor such a symposium during March 1983 at the Pan American Health OrganizationHeadquarters in Washington, DC. Cosponsors include several Institutes of the National Institutes ofHealth, the Centers for Disease Control, the Food and Drug Administration, the Institute of Medicineof the National Academy of Sciences, the National Foundation for Infectious Diseases, TheRockefeller Foundation, Merieux Institute, Inc., Smith Kline-RIT, Lederle Laboratories, HoechstAktiengesellschaft, the American Public Health Association, and the Agency for InternationalDevelopment.

This conference-workshop meeting will feature invited papers, allowing adequate time fordiscussion by the 200 invited research scientists, epidemiologists, national and international healthadministrators, and vaccine manufacturers.

It is anticipated that the conclusions and recommendations developed during this symposium willhasten the evolution of increasingly effective programs for poliomyelitis control and eradication.

In addition to publication of the proceedings of the symposium, a summary report will be preparedfor submission to a number of widely read clinical and public health journals, to assure an extensiveworldwide dissemination of information to officials responsible for poliomyelitis control.


I

-49

Documents

An Evaluation of Subsidized Rural Primary Care Programs: 1. A