INVESTIGATING THE ISOLATION AND INSULARITY OF PUBLIC

1

INVESTIGATING THE ISOLATION AND INSULARITY OF PUBLIC

ADMINISTRATION AS A FIELD WITH EGO-NETWORK ANALYSES OF THE

CITATIONS TO AND BY ITS PROMINENT JOURNALS

Glenn S. McGuigan, PhD. Candidate, Penn State Harrisburg, [email protected]

Dr. Göktuğ Morçöl, Professor of Public Policy and Administration, School of Public Affairs,

Penn State Harrisburg, [email protected]

ASPA Panel Paper, Denver

March 10, 2018

ABSTRACT

The purpose of this study was to investigate whether public administration is an isolated

and insular field. An earlier study found that research in public administration was largely

isolated from the three disciplines that were believed to be its foundations: law, management,

and political science. Using ego-network analyses with the software UCI Net, we examined the

isolation and insularity of the top scholarly journals of public administration, in comparison to

the top journals of political science and business/management. We obtained the citation data

from the Web of Science in three years: 2005, 2010, and 2015. We computed Index of

Qualitative Variation scores for ego-networks and ratios of citations within one’s field to answer

the isolation and insularity questions. Our results confirm that public administration journals

were isolated, but they were not insular. They are also less prestigious compared to the journals

of political science and business/management.

2

INTRODUCTION

Public administration is a field that emerged and matured relatively later than other

related fields of study, such as political science and management. This latecomer status may have

been the reason for the concerns public administration scholars have had about the status of their

field and its relative standing. Over the last few decades, they investigated the trends in the

scholarly publications in the field (Ni, Sugimoto, and Robin, 2017; Raadschelders, 2011;

Raadschelders & Lee, 2011; Riccucci, 2010; Wright, 2011), the quality and products of public

administration programs (Morgan, Meier, Kearney, Hays, & Birch 1981; Farber Powers, &

Thompson, 1984; McCurdy & Cleary, 1984; White, 1986; Legge & Devore, 1987), and the

quality of research in PhD dissertations in the field (Adams & White, 1994; Cleary, 1992;

Cleary, 2000; McCurdy & Cleary, 1984; White, 1986).

Wright (2011) asked a more pointed question about the standing and status of public

administration: Is public administration an isolated field? In his study, Wright answered this

question by comparing the citations between public administration journals and those in three

related fields, which are considered by other scholars as the foundations of public administration

(e.g., by Waldo, 1984, pp. 24-48): political science, business/management, and law. He reaches

this conclusion based on his analyses of the articles published in public administration journals in

the period from 1977 to 2007. He found that articles published in public administration journals

were cited rarely in political science and management journals. More specifically, he found that

only 0.01 percent of the citations in political science journals in this period (only 73 out of the

total 2935 citations) were citations of the articles published in public administration journals.

Similarly, only 0.02 percent of the citations in management journal in this period (only 55 out of

the total 3,840 citations) were citations of the articles published in public administration journals.

3

Wright concludes that the isolation of the field is an impediment to the development of a

coherent body of knowledge in public administration and consequently the credibility of the

field.

In this study, we aimed to verify Wright’s finding that public administration is an isolated

field. Like Wright, we used journal citations to answer the question of isolation, but we

conducted more advanced analyses in three years: 2005, 2010, and 2015. We defined the concept

of “isolation” more specifically and added the concept of “insularity” to our investigation. We

define isolation in terms of the ratio of the citations of the journals in a field by those in other

fields. We define insularity in terms of the ratios of the citations of other fields by journals in a

particular field. Both isolation and insularity should be defined in relative terms. All fields are

somewhat isolated and insular in the sense that the journals of a field tend to cite the other

journals in the field more frequently, but they cite other fields as well. Then the issue is to what

extent the authors of journal articles in each field find those in other fields important enough

(worthy) to cite. Isolation represent the “prestige” of a field in the sense that if the journals of a

field are cited by others at a relatively higher ratio, that mean that others find the knowledge

produced in the field worth citing. The insularity of journal citations in a field can be interpreted

that the authors in the field are much interested in the knowledge produced in other fields, which

may be because they are content with the knowledge produced in their own field and/or they

think that that the knowledge produced elsewhere is not worth citing.

In this study we aimed to answer the following research questions.

1. To what extent are public administration journals isolated from other fields? To answer

this question, we compared the ego-networks of the citations (in-citations) of the articles

4

published in the top three journals of public administration, with those of political science

and management. These calculations include heterogeneity measures and ratios.

2. To what extent are public administration journals insular in terms of the citations by

public administration journals of the journals in other fields (out-citations)? To answer

this question, we compared the ego-networks of the citations of the articles published in

other academic fields by the articles published in the top three journals of public

administration, with those of the top three journals in political science and management.

These calculations include heterogeneity measures and ratios.

3. Was there a change in the degree of isolation of public administration journals over time?

4. Was there a change in the degree of insularity of public administration journals over

time?

METHODS

Justification for the Analytical Approach

What does it really mean to cite an article? A citation is a connection between the citing

and cited articles. It indicates that the citing article acknowledges the importance of the cited

article. What does this acknowledgement mean? In an earlier study on journal citations, Garfield

(1965, p. 85) observed that authors cite others in positive and negative ways. They may cite

others for positive reasons like paying homage to pioneers, giving credit for related work

(homage to peers), identifying relevant methods for own work, providing background to the

topic of investigation, and correcting one’s own work. They may also cite them to correct the

work of others, criticize them, or dispute their claims. In both kinds of acknowledgment, the

5

authors recognize the importance of previous works for their worth to be praised, emulated, or

criticized. Although Garfield’s pioneering study was criticized by some for the lack of empirical

support for its conclusions (Bornmann & Daniel, 2008, p. 51), its overall implication has been

agreed on: Citing a work is an act of acknowledgement for its importance or prestige. Knoke and

Yang (2008) rephrase this in terms of social network analysis (SNA) and state that since prestige

is the extent to which an actor in a network “receives” or “serves as the object” of relations sent

by others in the network, the citation from the citing journal to the cited journal, is a measure of

prestige (p. 69).

In our study, we conceptualize journals as nodes and the information communicated

to/from that journal as a tie. The citations (or ties) connect the nodes in a network. The “out-

degree” flow of information takes place through the citations from the "citing" journal (or the

ego) to the "cited" journal (or the alter). In terms of measuring the citations, what is being

analyzed is a directional relationship of the citing journal (from) with the cited journal (to).

Journal citations have been one of the major areas in which SNA methods have been

applied. De Solla Price (1965) described an approach to science as a network of interconnected

citations and metaphorically described the process citing articles as “knitting” small segments

and straps into a fabric (p. 515). His work established the notion that tracking and measuring

citations across journals provides a “broad picture” of the research environment and the nature of

using citations as references in papers (p. 510). Since De Solla Price’s pioneering works, others

conducted SNA to examine various dimensions of scholarly publishing. Lin & Liao (2008) used

SNA to examine “word of mouth” research in marketing publications. Burgess & Shaw (2010)

applied SNA to the editorial board membership of the high-ranking journals in business

administration to better understand the relationships between journals based upon editorial board

6

member characteristics (p. 627). Haythornthwaite (1996) called for the use of SNA to study

information exchanges, arguing that this approach reveals information as social networks, with

actors as nodes in the networks and information as the connectors between the nodes (p. 323).

Cooke and Hall (2012) used SNA to examine social media networking to investigate research

network development (p. 786).

In this study, we followed the tradition of SNA applications in journal citations. What is

different in our study is that we used the journal, not the journal article, as our unit of analysis.

There are no other studies that used this unit of analysis, to our knowledge.

Citation Data for Ego-Network Analyses

We obtained the citation data for the ego-network analyses from journal article citations

in the Web of Science database. The term “Web of Science” refers to the collection of databases

delivered through the Web of Science Core Collection, which includes over 12,000 indexed

journals from 10 indexes, including the Social Sciences Citation Index, that provides access to

many of the journal citations used in this study. This multidisciplinary index covers over 2,900

journals across 50 social sciences disciplines (Clarivate Analytics, 2017, par. 1). A related

software that provides citing (out-citation) and cited (in-citation) data from the Web of Science is

InCites Journal Citation Reports. This software allows users to compare citation data from

journals for all of the 12,000 journals indexed in the Web of Science Core Collection.

7

Selection of Journals for Ego Network Analyses

We selected the three “top journals” in public administration, political science, and

business/management for comparisons. In our selection of the fields to compare, we followed

Wright’s lead, except that we dropped law from our list. Our initial analyses indicated that the

citations between public administration journals and law journals were negligible in the period

we studied (2005-2015).

In the selections of the top journals in public administration, political science, and

business/management we used two criteria. First, for political science and public administration,

the journal should be classified with only the single classification as a “public administration”

journal or only a “political science” journal in the Web of Science subject categories as assigned

to the journals in the Social Science Citation Index classifications. We excluded those journals

that are cross-classified (e.g., classified as both public administration and political science) and

placed them in a different category. In the case of business management, we selected the top

journals that had the dual classification of “business” and “management” since this was

necessary to obtain a list of the top management journals. Relying only on the single

classification of “management” would produce a list of journals focused exclusively on supply

chain / operations management.

Second, we identified the “top journals” using the journal impact factor (JIF) metric: we

selected the journals that have the highest JIFs among the journals that are classified only as

public administration journals in 2015. We did the same for those journals that are classified

only as political science journals and business/management journals. We had to exclude some of

the journals from analyses, despite the fact that they have high JIFs. A major example of the

journals we excluded from the analyses was Governance, which was cross-classified as a public

8

administration and political science journal. Annals were also excluded since they are secondary

sources and not primary sources as academic journal publications.

The journals we selected based on the two criteria (sole classification in one field and

highest JIF) are listed below. The details regarding of the categorical classification criteria we

used are presented in Appendix A. A sample of the master-list of the journals from Web of

Science is presented in Appendix B.

Table 1: Top journals in public administration, political science, and business management

based upon 2015 JIF scores

Top Public Administration Journals:

ARPA: American Review of Public Administration

JPART: Journal of Public Administration Research and Theory

PAR: Public Administration Review

Top Political Science Journals:

AJPS: American Journal of Political Science

APSR: American Political Science Review

POL ANAL: Political Analysis

Top Business Management Journals:

AMR: Academy of Management Review

AMJ: Academy of Management Journal

ASQ: Administrative Science Quarterly

The JIFs of these 9 journals in 2005, 2010, and 2015 are presented in Table 2.

9

Table 2

Journal Impact Factor (JIF) for Journals in Public Administration Political Science, and

Management 2005-2015

JIF 2005 JIF 2010 JIF 2015

JPART 1.451 2.086 3.893

PAR 1.099 1.141 2.636

ARPA 0.615 1 1.26

AJPS 1.845 2.588 4.515

APSR 3.233 3.278 3.444

POL ANAL 1 1.864 3.491

AMR 4.254 6.72 7.288

AMJ 2.2 5.25 6.233

ASQ 2.719 3.684 5.316

Categorical Attributes of Journals

A major objective of our analyses was to categorize the journals cited by the public

administration, political science, and management journals (out-citations) and the journals that

are citing these fields (in-citations), as we noted in the introduction. The focus on subject

categories allowed us to get a better understanding of what fields were being cited by journals,

and of what fields were citing the journals.

We used the Web of Science subject classifications for the categories and assigned codes

to them. These categories and codes are presented in Table 3. An important note is that there

were a number of items that were not indexed, such as those journals outside of the Web of

Science universe and book chapters and reports (so-called gray or fugitive literature). We

categorized them as “not indexed.” We categorized the journals with multiple Web of Science

codes as interdisciplinary, but made exceptions to this rule. We coded some of the

10

interdisciplinary journals into their specific interdisciplinary categories. These were the journals

that were more directly related to the fields we studied (e.g., “public administration and other”

and “political science and other”).

Table 3

Coding for subject categories

Public Administration

Public Administration and Other

Public Administration Not Indexed

Interdisciplinary Public Administration and Political

Science

Political Science

Political Science and Other

Political Science Not Indexed

Business Management

Interdisciplinary Business

Business and Other

Business Not indexed

Communication

Computer Science and Information Systems

Criminal Justice

Economics

Education

Engineering

Environmental Studies

Health Care, Occupational Health, and Medical

Interdisciplinary

International Relations

Law

Mathematics and Statistics

Psychology

Sociology and interdisciplinary social sciences

All Others

Not Indexed

11

The rules established for categorizing a journal in this taxonomy based upon the Web of

Science classifications are included in Appendix A.

Assumptions and Limitations

We used the journal impact factor (JIF) as the criterion for selecting the top journals in

each field. In doing so we assumed that JIF is a measure of prestige. The JIF is criticized for a

number of deficiencies, including the two-year citation window it uses, and some statistical

problems (Cameron, 2005; Harzing, 2008; Seglen, 1997). If a journal article is being analyzed

for its JIF score in 2007, for example, there would only be access to literature from 2005 and

2006 to calculate it (Harzing, 2008, note 3). This approach eliminates all of the other citations

from other years in the score. It therefore favors more recently cited literature. The JIF is further

limited by the coverage of journals for each discipline; books and book chapters are excluded;

and language is limited to primarily English (Cameron, 2005, p. 110). The JIF is criticized also

for some statistical problems (Harzing, 2008, par. 24; Seglen, 1997, p. 498); these are beyond the

scope of our discussions.

One must acknowledge the deficiencies of JIFs to establish an author’s or journal’s

importance, but it still is a widely used measurement of the quality of academic journals. While

acknowledging these problems, in this study we used the JIF as a perceived measure of prestige

because of its wide acceptance in academic communities.

It should be noted that there are other, and newer, indicators of journal of prestige. They

include the Scimago Journal Ranking, h-index, 5 year JIF, immediacy index, eigenvector score,

and article influence score (Garcia, Rodriguez-Sanchez, & Fedz-Valdivia, 2012, p. 1017). The

12

Scimago journal ranking is derived from Scopus (Elsevier), while the other measures are based

upon Web of Science data. Since h-index is an author-level measure, it is not addressed here.

In addition to the potential problems with using the JIF as our journal selection criterion,

of our study is limited by the threshold that we established in selecting citations for the analysis.

While we did not exclude any journals based on the timeliness of the citing or cited references

(such as limiting to the most recent two years for example), we had to establish a threshold in

selecting the citations to be included in ego-network analyses (less than five citations from any

source) for practical reasons. This process of setting limits on the number of citations establishes

a measure of selection for those nodes that will be considered “important.” The Web of Science

itself sets a threshold by not listing a citation that is not cited at least twice.

Analytical Approach

We conducted all the analyses using UCI Net. The specific routines we used are

described below.

To what extent are public administration journals isolated from other fields? To what

extent are public administration journals insular in terms of the citations by public administration

journals of the journals in other fields? Since these questions are interlinked, our approach to

answer these questions is to measure the heterogeneity of citations, to see to what extent citing

and cited references are spread across different fields by the journals, and to calculate ratios of

citations, in order to see the difference of citing/cited articles by the journals.

The heterogeneity scores (measures of dispersion) for the journals in our study are

presented in Table 4. We present specifically Agresti’s Index of Qualitative Variation (IQV)

scores in the table. We calculated these scores based on the categories defined in Table 3

13

(previous section). UCINET computes two measures of dispersion for categorical data: Blau's

Measure of Heterogeneity and Index of Qualitative Variation (Borgatti, Everett & Johnson,

2013, p. 271). Both measure how evenly alters are distributed across different categories. Blau's

measure of heterogeneity is 1 minus the sum of the squares of the proportions of each value of

the categorical variable in ego's network.

𝐵𝑙𝑎𝑢′𝑠 𝑚𝑒𝑎𝑠𝑢𝑟𝑒 𝑜𝑓 ℎ𝑒𝑡𝑒𝑟𝑜𝑔𝑒𝑛𝑒𝑖𝑡𝑦: 1 − ( (1/2)^2 + (1/2)^2)

IQV is a normalized version of Blau’s measure; it is equal to the previous column divided

by 1-1/n. We prefer to present the normalized IQV scores here because, as we note later,

political science and management journals receive much higher numbers of citations than public

administration journals, which affect the Blau’s scores.

We used the measures of heterogeneity or tie dispersion to better understand the reach of

journals across fields. In examining the ego-networks based on the citations of the top journals

in public administration, political science, and management, this analysis is an examination of tie

dispersion for the cited and citing journals of each journal as ego. Tie dispersion for categorical

attributes in UCI Net, using valued data, may be assessed through Blau’s H or Agresti’s IQV.

When the characteristic of an alter is measured as a categorical variable, the IQV measures

“egocentric network diversity” (Knoke & Yang, 2008, p. 55), revealing the spread of the ties

throughout categories. As summary measures of dispersion, Blau’s H and Agresti’s Index of

Qualitative Variation (IQV) are related calculations, with the IQV serving as a normalized

version of Blau’s H (Crossley, et al., 2015, 79; Borgatti, Everett, & Freeman, 2002)

14

Both Blau’s and IQV scores vary between 0 and 1. The score of “1” indicates maximum

level of heterogeneity, while “0” indicates the lowest level of heterogeneity (i.e., total

homogeneity). In general, the IQV scores indicate how diverse the “cited journals” (in-citations)

and the “citing journals” (out-citations) are. Higher IQV scores for “cited journals” indicate that

the journal was cited by journal in more diverse fields of study. In other words, the higher scores

indicate that the journal was cited by journals in fields other than public administration, political

science, or management. This could be interpreted as that the journal is more “prestigious” in

fields other than its own (public administration or political science). In other words, the journal is

not isolated (or less isolated). The heterogeneity scores of the in-citation scores of the journals in

the three fields can be compared to determine how isolated the field is. Higher IQV scores for

“citing journals” indicate that the journal cited more fields other than its own field. This could be

interpreted that the journal has a wider “reach.” In other words, the journal is not insular (or it is

less insular). The heterogeneity scores of the out-citation scores of the journals in the three fields

can be compared to determine how insular the field is.

In addition to calculating the IQV scores, we calculated ratios of the IQV scores by

dividing the scores of the Cited Journals (In-Degree) / Citing Journals (Out-Degree). The aim

here was to develop a measure that would represent the heterogeneous orientation of the journal

as a whole by taking into account the in-degree and out-degree measures.

RESULTS

Table 4 displays the dichotomized IQV scores (both in-degree and out-degree) for the 9

journals in three fields for the years 2005, 2010, and 2015. The dichotomized scores are based on

15

dichotomous comparisons between the journal’s own field and other fields (combined). A higher

in-citation score indicates that the journal is cited by other fields at higher rates compared to the

other journals in the table. In other words, the journals with higher rates are less isolated from

other fields. The dichotomized heterogeneity measures for out-citations are measures of the

degree of insularity. A higher out-citation score indicates that the journal cited other fields at

higher rates compared to the other journals in the table. In other words, the journals with higher

rates reaches out more to the fields other than their own and therefore it is less insular.

Table 4

Measures of dispersion (dichotomized IQV) for Cited Journals (In-Degree) and Citing Journals

(Out-Degree): 2005, 2010, and 2015

In-Degree

Out-Degree

2005 2010 2015 2005 2010 2015

JPART 0.38 0.53 0.76 1 1 0.97

PAR 0.79 1 0.93 1 0.91 0.98

ARPA 0.77 0.72 0.39 0.74 0.88 0.83

AJPS 0.81 0.85 0.91 0.75 0.9 0.91

APSR 0.96 0.94 0.98 0.98 0.96 0.95

POL ANAL 0.38 0.56 0.96 0.91 1 0.96

AMR 0.72 0.78 0.8 0.95 0.89 0.83

AMJ 0.7 0.78 0.74 0.81 0.9 0.86

ASQ 0.8 0.83 0.8 0.99 0.95 0.95

The figures in the table show that the public administration journals are the most isolated

(in-degree scores) among the three fields in general. However, PAR is the exception. PAR is

the least isolated in other fields among the public administration journals and it became less

isolated over time, compared to the business/management journals in the table. The isolation of

JPART decreased over time, while that of ARPA increased over time. In general, the journals in

16

political science were the least isolated and their isolation decreased over time. The out-degree

scores of almost all the journals in the table are quite high and stable, meaning that none of the

three fields is insular. Public administration journals are the least insular among the journals of

the three fields. Among the three journals in public administration, ARPA is the most insular

one, but its insularity declined over time.

We can conclude from the results of IQV analyses that public administration is relatively

more isolated compared to the other fields in the sense that its journals are cited less by others,

but its journals reach out more to other fields. Public administration is more isolated than insular,

compared to political science and management.

The IQV index indicates how heterogeneous the incoming or outgoing ties of an ego are,

but an IQV score is an abstract number. To better understand the ties between the citing journals

(“egos”) and the cited journals in in their own fields and those in other fields (“alters”), we

calculated the numbers of citations and their percentages, based upon categorical attributes, for

each of the nine journals we analyzed. These results are presented in multiple tables, including

Table 5 here and in Appendix C. Please note that we included only the tables for 2015 here and

in the appendix. The tables for 2005 and 2010 are not presented in this paper, because they

would add too much volume to the paper; they are available upon request.

The sign “+” in these tables indicates that we combined the journals in cross-listed

categories, if one of the lists was the field of interest to us. For example, the combined category

of “Public Administration +” represents the categories of Public Administration,

Interdisciplinary (Public Administration and Political Science), Public Administration Not

indexed, and Public Administration and Other. The category of “Interdisciplinary Public

Administration and Political Science” was included in “Public Administration +” in the tables

17

about public administration, but not in the other ones (Political Science and

Business/Management).

Table 5

Public Administration Journals In-Degree Citations (Incoming Ties) 2015 (measuring

citations of other journals citing these journals)

JPART PAR ARPA

Incoming ties from Number Percent Number Percent Number Percent

Public Administration + 1527 74.4% 2124 62.9% 410 89.0%

Political Science + 94 4.6% 160 4.7% 5 1.1%

Management &

Business 93 4.5% 271 8.0% 0 0.0%

Interdisciplinary 30 1.5% 50 1.5% 0 0.0%

Psychology 0 0.0% 8 0.2% 0 0.0%

Sociology 97 4.7% 145 4.3% 28 6.1%

Law 6 0.3% 34 1.0% 0 0.0%

Economics 7 0.3% 42 1.2% 0 0.0%

International Relations 10 0.5% 6 0.2% 0 0.0%

Engineering 7 0.3% 10 0.3% 0 0.0%

Computer Science and

Information Systems 27 1.3% 100 3.0% 10 2.2%

Health Care,

Occupational Health,

and Medical

0 0.0% 62 1.8% 0 0.0%

Education 21 1.0% 18 0.5% 0 0.0%

Environmental Studies 40 1.9% 104 3.1% 0 0.0%

Communication 6 0.3% 8 0.2% 0 0.0%

Criminal Justice 5 0.2% 5 0.1% 0 0.0%

Math & Statistics 0 0.0% 5 0.1% 0 0.0%

All Others 22 1.1% 31 0.9% 0 0.0%

Not indexed 48 2.3% 194 5.7% 8 1.7%

Total 2040 99.2% 3377 99.7% 461 100.1%

Total by others 513 1253 51

18

Table 5 shows that PAR had the lowest percentage of citations (62.9%) citations from

other public administration journals and the highest percentage from the journals in other fields

in 2015. This result is consistent with PAR’s highest in-citation heterogeneity score in Table 4.

JPART has 74.4% of its citations from public administration journals. The percentage for ARPA

is 89%. These results are also consistent with their respective dichotomized heterogeneity scores

in Table 4. It should be noted that the total numbers of in-citations and out-citations of PAR are

larger than those of JPART, because of the total numbers of articles published in these journals

(PAR publishes 6 issues a year, whereas JPART publishes only 4). The total numbers of in-

citations and out-citations for ARPA are the lowest. This means that PAR is being cited by more

journals outside of public administration across a broad range of disciplines than JPART or

ARPA, including business management (.08 percent), political science (.047 percent), sociology

(.043 percent), and computer science/information systems (.03 percent).

We calculated the ratios in Table 6 from the scores in Table 5, the tables in Appendix B.

and the tables for 2005 and 2010 that we could not include in this paper. These are the ratios of

in-degree citations to out-degree citations. This ratio is an indicator of the “prestige” of a journal.

If the ratio is higher, it means that the journal is more prestigious in other fields in the sense that

it is recognized by journals in others fields at a higher rate than it recognizes them. This table

shows that the public administration journals have the lowest ratios in general and therefore they

are the least prestigious by this definition. APSR is the most prestigious journal among the 9

included in the table and ASQ is the second most prestigious. Among the public administration

journals, PAR is the most prestigious.

19

Table 6

Ratios of Ties (measuring citations of in-degree/out degree or cited/citing): 2005, 2010,

and 2015

2005 2010 2015

JPART 0.08 0.20 0.48

PAR 0.55 1.48 1.60

ARPA 0.21 0.23 0.19

AJPS 2.92 3.46 3.98

APSR 6.91 6.14 8.62

POL ANAL 0.11 0.42 1.84

AMR 1.95 6.54 9.45

AMJ 2.27 3.35 3.88

ASQ 3.48 13.96 8.20

DISCUSSION AND CONCLUSION

There are two related research questions that we examine in this research. First, is the

literature of public administration isolated from those of other academic fields of study? Second,

is the literature of public administration insular? Wright (2011) demonstrates in his research that

public administration is an isolated field. Although Wright does not ask this question directly, it

is reasonable to ask, is public administration also an insular field: Do public administration

journal articles cite primarily articles in public administration journals, but not the ones in other

fields. As our IQV analyses show, the public administration journals in our study were indeed

more isolated, compared to the journals in business/management, but they were not insular. In

other words, public administration researchers “reached out” to other fields to acquire

knowledge, but the others did not reach back to public administration journals at the same

degree. Our ratio analyses show that there was a “prestige gap” between the public

20

administration journals and the political science and business/management journals: The latter

were more prestigious “among their peers” (in academic fields in general) in the period we

studied.

The IQV analysis results also show that isolation of two public administration journals

(PAR and JPART) diminished between 2005 and 2015 (their in-degree IQV scores increased),

therefore they became less isolated in this period. ARPA’s scores decreased in the same period;

therefore, it became more isolated. The decrease in ARPA’s scores is noteworthy, particularly

given that the scores of the political science and business/management journals either increased

or remained the same in this period. The IQV analyses of out-degree citations also show that

none of the journals in the three fields is insular and their insularity scores did not change

considerably in the period we studied.

We can conclude that public administration researchers reach out to others, but they do

not reciprocate. What are the implications of this relative isolation of public administration?

Certainly, it is an indicator of the prestige of the field. The fact that the top two journals of the

field (PAR and JPART) became less isolated (cited more by journals in other fields) in the period

we studied is encouraging. But the relative prestige gap between public administration and the

other two fields did not diminish in the same period, which indicates that public administration

scholars should continue to be concerned about the standing of their field.

A legitimate question is, why is public administration is relatively more isolated, while it

is not insular? Although this question cannot be answered in this paper, we propose that the

unique nature of the field and its intellectual/identity crisis may have contributed to it. These

explanations have been explored by others (e.g., Ostrom, 1973; Raadschelders, 2011;

21

Raadschelders & Lee, 2011; Riccucci, 2010; Waldo, 1984). Future studies on the topic may find

a good justification for further investigating the problems of public administration as a field.

22

REFERENCES

Adams, G. B., & White, J. D. (1994). Dissertation research in public administration and

cognate fields: An assessment of methods and quality. Public Administration Review, 54(6),

565–576.

Borgatti, S. P., Everett, M. G., & Johnson, J. C. (2013). Analyzing social networks. Los

Angeles; London: SAGE Publications.

Borgatti, S. P., Everett, M.G. and Freeman, L.C. (2002). Ucinet for Windows: Software

for Social Network Analysis. Harvard, MA: Analytic Technologies. Harvard, MA: Analytic

Technologies.

Bornmann, L., & Daniel, H. D. (2008). What do citation-counts measure? A review of

studies on citing behavior. Journal of Documentation, 64(1), 45-80.

Burgess, T. F., & Shaw, N. E. (2010). Editorial board membership of management and

business journals: A social network analysis study of the Financial Times 40. British Journal of

Management, 21(3), 627-648.

Cameron, B. D. (2005). Trends in the usage of ISI bibliometric data: Uses, abuses, and

implications. portal: Libraries and the Academy, 5(1), 105-125.

Clarivate Analytics. (2015). Journal citation reports®. Databases accessible from:

https://jcr.incites.thomsonreuters.com. Retrieved 2/12/18

Cleary, R. E. (1992). Revisiting the doctoral dissertation in public administration: An

examination of the dissertations of 1990. Public Administration Review, 52(1), 55–61.

Cleary, R. E. (2000), The public administration doctoral dissertation reexamined: An

evaluation of the dissertations of 1998. Public Administration Review, 60: 446–455.

Cooke, L., & Hall, H. (2013). Facets of DREaMA social network analysis exploring

network development in the UK LIS research community. Journal of Documentation, 69(6),

786-806.

Crossley, N., Bellotti, E., Edwards, G., Everett, M. G., Koskinen, J., & Tranmer, M.

(2015). Social network analysis for ego-nets. London: SAGE Publications Ltd.

Farber, M., Powers, P., & Thompson, F. (1984). Assessing faculty research productivity

in graduate public policy programs. Policy Sciences, 16(3), 281-289.

García, J. A., Rodriguez-Sánchez, R., & Fdez-Valdivia, J. (2012). Scientific subject

categories of Web of Knowledge ranked according to their multidimensional prestige of

influential journals. Journal of the American Society for Information Science and Technology,

63(5), 1017.

23

Garfield, E. (1965). Can citation indexing be automated? Paper presented at the

Statistical Association Methods for Mechanized Documentation, Symposium Proceedings.,

Washington, DC.

Harzing, A.-W., & van der Wal, R. (2008). Comparing the Google Scholar h-index with

the ISI Journal Impact Factor. Retrieved June 4, 2013, Retrieved

from http://www.harzing.com/h_indexjournals.htm

Haythornthwaite, C. (1996). Social network analysis: an approach and technique for the

study of information exchange, paper presented at the 1996 ALISE Conference (Vol. 18, pp. 323-

342).

Knoke, D., & Yang, S. (2008). Social network analysis (2nd ed.). Los Angeles, [Calif.];

London: SAGE.

Legge, J. S., & Devore, J. (1987). Measuring productivity in U. S. public administration

and public affairs programs 1981-1985. Administration & Society, 19(2), 147-156.

Lin, T. M. Y., & Liao, C.-W. (2008). Knowledge dissemination of word-of-mouth

research: Citation analysis and social network analysis. Libri: International Journal of Libraries

& Information Services, 58(4), 212-223.

McCurdy, H. E., & Cleary, R. E. (1984). A call for 'appropriate methods'. Public

Administration Review, 44(6), 553-554.

McCurdy, H. E., & Cleary, R. E. (1984). Why can’t we resolve the research issue in

public administration? Public Administration Review, 44(1), 49–55.

Morgan, D. R., Meier, K. J., Kearney, R. C., Hays, S. W., & Birch, H. B. (1981).

Reputation and productivity among U. S. public administration and public affairs

programs. Public Administration Review, 41(6), 666-673.

Ni, C., Sugimoto, C. R., & Robbin, A. (2017). Examining the evolution of the field of

public administration through a bibliometric analysis of Public Administration Review. Public


Ostrom, V. (1973). The intellectual crisis in American public administration (Rev. ed.).

University, Ala: University of Alabama Press.

Price, D. J. (1965). Networks of scientific papers. Science, 149(3683), 510-515.

Raadschelders, J. C. N. (2011). The future of the study of public administration:

Embedding research object and methodology in epistemology and ontology. Public


http://www.harzing.com/h_indexjournals.htm

24

Raadschelders, J. C. N., & Lee, K.-H. (2011). Trends in the study of public

administration: Empirical and qualitative observations from Public Administration Review, 2000-

2009. Public Administration Review, 71(1), 19-33.

Riccucci, N. (2010). Public administration: traditions of inquiry and philosophies of

knowledge. Washington, D.C.: Georgetown University Press.

Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating

research. BMJ: British Medical Journal, 314(7079), 498-502.

Waldo, D. (1984). The administrative state: a study of the political theory of American

public administration (2nd ed.). New York: Ronald Press Company.

White, J. D. (1986). Dissertations and publications in public administration. Public


Wright, B. E. (2011). Public administration as an interdisciplinary field: Assessing Its

relationship with the fields of law, management, and political science. Public Administration

Review, 71(1), 96-101.

25

APPENDIX A: Coding and subject taxonomy based upon Web of Science classifications

Coding for subject categories

1: Public Administration

2. Public Administration and Other

3: Public Administration Not Indexed

4. Interdisciplinary Public Administration and Political

Science

5. Political Science

6. Political Science and Other

7. Political Science Not Indexed

8. Management

9. Interdisciplinary Business

10. Business and Other

11. Economics

12. Law

13. Interdisciplinary

14. All Others

15. Not Indexed

16. Sociology

17. Communication

18. International Relations

19. Psychology

20. Engineering

21. Business Not indexed

22. Computer Science and Information Systems

23. Health Care, Occupational Health, and Medical

24. Education

25. Environmental Studies

26. Mathematics and Statistics

27. Criminal Justice

Taxonomy criteria based upon Web of Science Classification (Numbers in parentheses

relate to UCI Net coding)

Public Admin (1-4): Journals classified as Public Administration include those titles with that

single subject classification (coded as “1”). Journals classified as Public Administration and

Other include those titles with the classification of Public Administration, plus one to three other

non-political science classifications, including such subjects as Planning and Development,

Social Work, and Environmental Studies (coded as “2”). Journals classified as Public

Administration Not Indexed includes the journals that are clearly PA journals, but are not

26

indexed within the WOS citations indexes, and therefore do not have an impact factor (coded as

“3”). This does not include books, chapter, or government reports unless they are specifically

public-administration related. Journals classified as Interdisciplinary Public Administration and

Political Science include all journals that have the combined classification, including three

journals, LOCAL GOV STUD; REGUL GOV; and SCI PUBL POLICY, that have three

classifications, but include PA and PS (coded as “4”).

Political Science (5-7): Political science with one subject class is coded as “5.” The subject

classification (6) is Political Science and Other and includes journals that have the PS

classification along with 1-2 other non-PA classifications, such as international relations,

sociology, or communication. While there is another classification for International Relations,

all dual classed journals will be included here, such as INT SECURITY and INT STUD

QUART. The classification (7) Political Science Not Indexed includes journals that are clearly

PS journals, but are not indexed within the WOS citation indexes, and therefore do not have an

impact factor. This does not include books, chapter, or government reports unless they are

specifically political science related.

Management and Business (8-10 & 21): The classification (8) of Management includes just

those journals with the single categorization of that subject, or with the classification of that

subject with Business or Education related subject headings that include Management. It

includes journals that have the subject classification of Business and of Management. It also

includes journals that have those two classifications, or the single classification of Management,

along with one additional business classification such as Finance. The classification (9) of

Interdisciplinary Business includes all journals with a business classification, and another non-

management classification, such as Ethics and Psychology. This includes journals with the dual

subject classification of Business, Finance and also of Economics. It includes the subject

heading of Industrial Relations and Labor. This also includes all journals with a subject

classification of Hospitality, Leisure, Sport & Tourism. The category (10) of Business and Other

includes journals with the single classification of Business, along with other journals that have

the single classification of Business or a related business classification such as Business Finance.

The classification of (21) Business Not Indexed, includes all of the documents that are related to

business but not indexed in the WOS. This does not include books, chapter, or government

reports unless they are specifically business related.

Economics (11): The classification of Economics (coded as “11”) is for those journals with that

single category, or for journals with that category and another unclassified category, such as

Planning and Development and Urban Studies. Any journal with this subject combined with a

business subject, is classified in Business and Other.

Law (12): The classification of Law is just for those journals with that single category.

Interdisciplinary (13): All sources with multiple codes not included in this taxonomy are

identified as Interdisciplinary. The two journals that had both the classes of Law and Economics

are classified here. Any journal that has the subject heading of Interdisciplinary or

Multidisciplinary is included in the classification. Any journal that has two or more subjects not

part of other classifications here is included, such as Environmental Sciences, Energy and Fuels,

27

or Family Studies, and Social Work. Journals with the combined subjects of Management and

Psychology are not classified here but rather in Interdisciplinary Business. Selected subjects

include: Area Studies, Cultural Studies, and Multidisciplinary Sciences.

All Others (14): These journals include the single classifications of all the other journals not part

of the classified items, such as items with the single subject classification of Social Work, Ethics,

and Planning and Evaluation.

Not indexed (15): These journals and other items are not indexed within the WOS citation

indexes, and therefore do not have an impact factor. This includes the item identified as Non-

Traditional that includes books, book chapters, and various types of reports, including

government reports. Non-indexed, conference proceedings, regardless of subject area, are

included here. From this classification, items that were either Business, Public Administration or

Political Science were re-classified into the subject classes described above.

Sociology and Interdisciplinary Social Sciences (16): This code includes all of the journals

with that single subject classification, as well as all journals with that classification, and another

classification or another field outside of the currently classified fields. This classification also

includes Anthropology. For this study, all journals that include the classification of Management

will be in the management category.

Communication and interdisciplinary communication (17): This includes all journals with

the single subject classification as well as all journals with this class and other multiple

classifications.

International relations and interdisciplinary international relations (18): This classification

includes any journal with this single, subject classification.

Psychology (19): This code is used for all of those journals with the single classification of a

discipline in Psychology, such as Social Psychology, Applied Psychology, or journals with

multiple classifications of Psychology and another behavioral science field, or another field

outside of the currently classified fields. All psychology journals with a dual business related

classification with a business classification, such as The Journal of Applied Psychology and

Leadership Quarterly, are classified in the Interdisciplinary Business category.

Engineering (20): This code is used for all journals with that classification, including Industrial

Engineering and all of the other fields that include Engineering and another non-management

subject heading.

Computer Science and Information Systems (22): This code includes all journals with that

subject classification, single or multiple. The code includes all Computer Science and

Information Systems subject classifications.

Health Care Science and Service and Medical Sciences (23): This code includes Public,

Environmental and Occupational Health, Rehabilitation, and Nursing. This classification

includes Healthcare Science and Services, Veterinary Sciences, including all medical sciences.

28

Education (24): This classification includes all journals listed as in the categories of Education

and Educational Research.

Environmental Studies, Natural Sciences, and Science (25): This classification includes all

journals with these subject classifications, including Interdisciplinary Agriculture, Biology,

Ecology, Energy, Fisheries, Microbiology, Nuclear Science and Technology, Oceanography, and

Water Resources. This classification includes journals with dual classifications such as

Environmental Studies and Urban Studies.

Statistics and Mathematics (26): This classification includes all journals with the subjects of

Statistics and Probability.

Criminal Justice (27): This classification includes all journals with the subject of Criminal

Justice.

29

APPENDIX B

This is a two-page sample of the classified journal listings based upon the taxonomy that

was completed from Web of Science. The full listing, which includes 1,756 sources, and is 26

pages in length, is available upon request.

Table B1

Taxonomy of Sources based upon Web of Science Subject Classification

1. Public Administration Public Administration Not Indexed ADMIN SOC ADMIN LAW TREATISE

AM REV PUBLIC ADM ADMIN THEOR PRAXIS

AMME IDARESI DERG AGE DIRECT CITIZEN P

AUST J PUBL ADMIN AGENDAS ALTERNATIVES

CAN PUBLIC ADMIN ALL ORG ARE PUBLIC B

CAN PUBLIC POL ALLIANCE GLOB SUSTAI

CIV SZLE AM COUNTY FRONTIERS

GEST POLIT PUBLICA AM PUBLIC SERVICE RA

INT PUBLIC MANAG J ANAL URBAN SERVICE D

INT REV ADM SCI BIG IDEAS COLLABORAT

J COMP POLICY ANAL BUDG PROC STAT

J HOMEL SECUR EMERG BUREAUCRACY REPRESEN

J PUBL ADM RES THEOR BUREAUCRACY WHAT GOV

POLICY STUD_UK CASE BUREAUCRACY PUB

PUBLIC ADMIN REV CASE STUD CITY CO

PUBLIC MANAG REV CHALLENGING PERFORMA

PUBLIC MONEY MANAGE CHINESE PUBLIC ADM

PUBLIC PERFORM MANAG CITIES

PUBLIC PERS MANAGE CITYSCAPE

REV PUBLIC PERS ADM CIVIL SERVICE REFORM

REV PUBLIC PERSONNEL COLLABORATIVE PUBLIC

TRANSYLV REV ADM SCI COMMONW J LOCAL GOV

CONFL RESOLUT Q

2. Public Administration And Other COUNTY GOV ERA CHAN

CLIM POLICY CREATING PUBLIC VALU

ENVIRON PLANN C CURR TREND PUB SECT

HUM SERV ORG MANAGE DELEGATING POWERS T

J ACCOUNT PUBLIC POL DELIBERATE DISCRETIO

J EUR SOC POLICY DELIBERATIVE DEMOCRA

J POLICY ANAL MANAG DEMOCRACY PUBLIC SER

J SOC POLICY DEV POLICY REV

NONPROFIT MANAG LEAD EMERGENCY MANAGE

POLICY SCI ENTREP REGION DEV

PUBLIC ADMIN DEVELOP FORGING BUREAUCRATIC

SOC POLICY ADMIN GENDER BUDGETS MAKE

GOOD SOC

3. Public Administration Not Indexed GOV FINANCE REV

ADAPTIVE GOVERNANCE GOV ILL EXECUTED DE

ADM BEHAV STUDY DECI GOV RESTRUCTURING C

ADM STATE STUDY POLI GOVERNING

ADMIN BEHAV HANDB ADMIN ETHICS

30

Public Administration Not Indexed (Cont’d) Public Administration Not Indexed (Cont’d) HANDB PUBLIC ADM PUBLIC POLICY

I I CHANGE EC PERFOR PUBLIC POLICY ADM

IMPLEMENTATION PUBLIC POLICY ADMIN

IN PRESS PUBLIC ADM PUBLIC PRODUCTIVITY

INSIDE BUREAUCRACY PUBLIC SERVICE PERFO

INT J MASS EMERGENCI PUBLIC VALUE THEORY

INT J PUBLIC ADMIN PUBLIC POLICY ADM

INT J PUBLIC SECT MA PUBLIC POLICY ADMIN

INT J PUBLIC SECTOR PUBLIC PRODUCTIVITY

INT J URBAN REGIONAL PUBLIC VALUES PUBLIC

J CONTINGENCIES CRIS PURSUIT PERFORMANCE

J HEALTH ORGAN MANAG REINVENTING GOV ENT

J PUBLIC AD IN PRESS RETHINKING DEMOCRATI

JUDICIAL REV BUREAUC Rev Metrop Sustentab

LEADERSHIP ADM SOCIO SELF ORG FEDERALISM

MANAGING COMPLEX NET STAT BUDG PROC

MANAGING NETWORKS AD STATE LOCAL GOVT REV

MEASURING PERFORMANC STATE NONPROFIT AM

METROPOLITAN GOVERNA STREET LEVEL BUREAUC

MOTIVATION PUBLIC MA THEORIES POLICY PROC

NATL CIVIC REV TOOLS GOV

NATL MUNICIPAL REV TOOLS GOV GUIDE NEW

NEW DIRECTIONS PHILA TRANSFORMATION GOVER

NEW I ORG ANAL WHISTLEBLOWING AUST

NONPROFIT MANAGE

NONPROFIT SECTOR RES

4. Interdisciplinary Public Administration and

Political Science OXFORD HDB AM BUREAU GOVERNANCE

OXFORD HDB PUBLIC MA J EUR PUBLIC POLICY

PAP SCI ADM J PUBLIC POLICY

PAP W WILSON LEX LOCALIS

POLIC_J POLICY PRACT LOCAL GOV STUD

POLITICIANS BUREAUCR POLICY POLIT

POLITICS BUREAUCRACY POLICY SOC

PUBLIC ADMIN Q POLICY STUD J

PUBLIC BUDG FINANC POLIT SOC

PUBLIC BUDGETING FIN PUBLIC ADMIN

PUBLIC INTEGRITY REGUL GOV

PUBLIC LAW REV CLAD REFORMA DEM

PUBLIC MANAGE REV POLICY RES

PUBLIC MANAGE OR SCI PUBL POLICY

PUBLIC PERFORMANCE M J EUR PUBLIC POLICY

31

APPENDIX C

Table C1

Public Administration Journals--Outgoing Ties 2015 (measuring citations of these

journals citing other journals)

JPART PAR ARPA

Outgoing ties to Number Percent Number Percent Number Percent



Management &

Business 542 30.0% 282 15.7% 125 13.2%


Psychology 79 4.4% 70 3.9% 39 4.1%

Sociology 77 4.3% 100 5.6% 22 2.3%

Law 29 1.6% 15 0.8% 5 0.5%

Economics 54 3.0% 36 2.0% 6 0.6%


Engineering 0 0.0% 0 0.0% 0 0.0%



Health Care,


and Medical

11 0.6% 0 0.0% 0 0.0%

Education 0 0.0% 0 0.0% 7 0.7%


Communication 0 0.0% 0 0.0% 0 0.0%


Math & Statistics 9 0.5% 10 0.6% 0 0.0%

All Others 6 0.3% 5 0.3% 0 0.0%

Not indexed 15 0.8% 77 4.3% 17 1.8%

Total 1808 99.9% 1790 100.0% 941 99.7%

Total of others 1063 783 275

Ratio 1.13 99.0% 1.89 100.0% 0.49 100.0%

Ratio of others 0.48 1.6 0.19

32

Table C2

Political Science Journals--Incoming Ties 2015 (measuring citations of other journals

citing these journals)

AJPS APSR POL ANAL




Management & Business 39 0.6% 110 1.4% 33 2.7%


Psychology 120 1.7% 116 1.3% 7 0.6%

Sociology 356 4.9% 516 6.0% 42 3.5%

Law 370 5.1% 368 4.3% 44 3.7%

Economics 347 4.8% 740 8.6% 48 4.0%


Engineering 0 0.0% 0 0.0% 0 0.0%



Health Care, Occupational

Health, and Medical 78 1.1% 78 0.9% 65 5.4%

Education 5 0.1% 6 0.1% 5 0.4%


Communication 161 2.2% 145 1.7% 11 0.9%


Math & Statistics 17 0.2% 28 0.3% 16 1.3%

All Others 83 1.1% 153 1.8% 5 0.4%

Not indexed 93 1.3% 133 1.5% 32 2.7%

Total 7221 99.9% 8596 99.9% 1203 99.9%


33

Table C3

Political Science Journals--Outgoing Ties 2015 (measuring citations of these journals

citing other journals)

AJPS APSR POL ANAL


Public Administration

+ 0 0.0% 11 1.0% 0 0.0%


Management &

Business 0 0.0% 0 0.0% 0 0.0%


Psychology 67 3.7% 38 3.4% 0 0.0%

Sociology 41 2.3% 31 2.8% 17 2.7%

Law 33 1.8% 19 1.7% 0 0.0%

Economics 259 14.2% 122 11.0% 60 9.5%


Engineering 0 0.0% 0 0.0% 0 0.0%



Health Care,


and Medical

11 0.6% 0 0.0% 35 5.5%

Education 0 0.0% 0 0.0% 0 0.0%


Communication 18 1.0% 6 0.5% 0 0.0%


Math & Statistics 33 1.8% 0 0.0% 80 12.6%

All Others 0 0.0% 20 1.8% 0 0.0%

Not indexed 87 4.8% 89 8.0% 42 6.6%

Total 1819 100.0% 1109 99.9% 634 100.0%

Total of others 636 430 257

Ratio 3.97 100.0% 7.75 100.0% 1.9 100.0%


34

Table C4

Management Journals--Incoming Ties 2015 (measuring citations of other journals citing

these journals)

AMR AMJ ASQ






Psychology 1120 5.6% 1513 6.6% 501 4.3%

Sociology 234 1.2% 255 1.1% 385 3.3%

Law 10 0.1% 16 0.1% 17 0.1%

Economics 300 1.5% 259 1.1% 136 1.2%


Engineering 483 2.4% 488 2.1% 300 2.6%





Education 115 0.6% 124 0.5% 74 0.6%


Communication 91 0.5% 72 0.3% 52 0.4%


Math & Statistics 32 0.2% 50 0.2% 22 0.2%

All Others 86 0.4% 70 0.3% 67 0.6%

Not indexed 645 3.2% 533 2.3% 267 2.3%

Total 19959 100.0% 22788 99.9% 11714 100.0%


35

Table C5

Management Journals --Outgoing Ties 2015 (measuring citations of these journals citing

other journals)

AMR AMJ ASQ






Psychology 324 16.2% 854 18.6% 143 13.9%

Sociology 125 6.2% 235 5.1% 200 19.4%

Law 0 0.0% 7 0.2% 0 0.0%

Economics 9 0.4% 115 2.5% 24 2.3%


Engineering 0 0.0% 6 0.1% 0 0.0%





Education 0 0.0% 0 0.0% 0 0.0%


Communication 53 2.6% 0 0.0% 0 0.0%


Math & Statistics 0 0.0% 19 0.4% 0 0.0%

All Others 0 0.0% 0 0.0% 0 0.0%

Not indexed 47 2.3% 144 3.1% 8 0.8%

Total 2005 99.8% 4599 100.0% 1029 100.0%


Ratio 9.95 100.0% 4.95 100.0% 11.38 100.0%


Documents

INVESTIGATING THE ISOLATION AND INSULARITY OF PUBLIC