I E E E T R A N S A C T I O N S O N E N G I N E E K I X O M A N A G E M E N T V O L . E M - 1 3 , N O . 2 J U N E , 1966

Sponsorship of Research: A Survey of Scientific Literature, 1920-1960

H E L E N 8. M I L T O N AND E . A . J O H N S O N

Abstract—This paper surveys 116 000 articles, published between 1920 and 1960, in fifty scientific journals. It analyzes each paper by field and by sponsorship. The details of change of sponsorship are followed from the period when 70 percent to 80 percent of sponsor­ship resided in the university and university funds, to the time when this had dropped to below 20 percent. The anatomy with respect to government agencies i s given, together with the field of science. The pattern of sponsorship is changing. The Navy has, in general, provided by far the most generous share of sponsorship. In the period 1949-1960, the average share of sponsorship was : government, 48 percent; university, 29 percent; industry, 15.5 percent; and private research organizations, 7.5 percent. However, the shares of government and industry sponsorship are rising rapidly while those of universities and private research organizations are falling.

INTRODUCTION

Ί H E AUTHORS of this paper, in the 1950^s, set themselves the task of sununarizing the combined

" effects of exponential growth in technology, and of the change since AVorld War I I in sponsorship and management policies on the cost and the effecti\'eness of U. S. research and development. In our previous paper |1] it was shown that the cost of research per technical man year has been changing at a variable rate for the last 40 years. In 1950-1965 this ra te has dropped from about 7 percent to half of that . rate. Increased funds, uncorrected for this inflation in cost, do not give a good measure of support. For example, about one half of the apparent increase in support during the past fifteen years has been real, the other half inflation.

The next step attempted, and reported here, was to relate expenditures and sponsorship to accomplishments in basic research. The question of expenditures turned out to be especially difficult because of the absence, until recently, of e\^en crude accounting procedures on basic research expenditures in either government, colleges and universities, or industry. This paper is concerned with the results of a survey of research literature which does throw light on the rapidly changing pat tern of sponsor­ship of basic research and its effects.

During the nineteenth century and the early p a r t of the twentieth, the United States thrived on technologi­cal advances in industry t h a t were based largely on fundamental research carried on in Europe. In applica-

Manuscript received August 27, 1965. This study was com­pleted several years ago under the sponsorship of The Johns Hopkins University Operations Research Office (ORO) , Baltimore, Md., and the later support of the Case Institute of Technology.

Helen S. Mi l ton is with the Research Anab'sis Corporation, McLean, Va., formerly the ORO, The Johns Hopkins University.

E . A. Johnson is with the Case Institute of Technology, Cleve­land, Ohio.

tion, for example, most of the outstanding World War I I weapons innovations had their early beginnings in the b a s i c research of t h e scientific laboratories of Western Europe. Although the relati\'e status of U. S. and Cana­dian research w a s already becoming comparable to that of EiuOpe, t h e status of U. S. basic research in this century Avas accelerated b y t h e devastations of t h e two world Avars that temporarily disrupted and reduced Europe as a major source of basic information. The ratio of U. S. and Canadian to other-country basic re­search is estimated to have changed by a factor of about 14 between 1850 and 1940. The absolute amounts would have been equal a t about 1950 except for World War I I , which reduced other-country research to almost one t h i r d . The experience of the Second World War estab­lished very clearly the need for balanced research and development within the United States, in view of the imbalance caused by the temporary great loss in achieve­m e n t suffered by t h e rest of the world. Since then, in t h e United States, there has been a concentrated effort by an increasingly greater number of sponsors, within both goA^ernment and industry, to provide more support for LT. S. basic research. Both industry and private agencies have augmented investments in research, and besides expanded research allotments, new agencies have been established within the federal go\^ernment primar­ily to support, rather than conduct, research. In 1965 almost two billion dollars a month, eight or nine times as many dollars as in 1950, were invested in research and development ( R & D ) . But because of the inflation in research costs, current dollars are not an adequate meas­ure since, over the past decade, dollar costs have in­creased about twice as fast as the actual effort ex­pended in R & D [1] .

Unfortunately, as yet there is no real criterion for the value of either the volume or quality of R D to ma­teriel or industrial programs, or national objectives involving application of scientific knowdedge. Policies concerning the management of research, including the allocation of funds and how to conduct research, have been formulated too slowly. I t is, for example, im­possible in basic research to relate the accomplishments in relation to national objectives. All such attempts are fraught with confusion. The pros and cons of sponsor­ship—government sponsorship in particular—arc ar­gued endlessly. ' 'What has all this t ime, money, and ef­fort accomplished? How can we measure our progress? AVhat plans should be made for the future? What kinds of basic research should be supported?" are valid ques-

1066 MILTON AND JOHNSON*. Sl>ONSORSHIP OF RESEARCH

tions in the evaluation of our basic research position. jMeanwhile, the more obvious results of increased

scientific activity are sweUing the output of printed pages. The deluge of articles in all fields has descended on the research investigator in such floods that scientists can only try to be ])oignantly selective. Even if a re­searcher could affoi'd the time to catch up on his reading, it has been facetiously pointed out tha t 'ijetween dinner one day and breakfast the next morning another 600 pai)ors will aj^pear" | 2 | . Such volume is discouraging to those ideological survivors of the philosophy of science current in the early decades of this century, who ex­pected to do their own literature search and to know most of their collaborators and colleagues in both the uni ted States and the rest of the world. But there is encouragement for the nation in the fact that interest in science and scientific output has extended so far and so fast. Solving the problem of scientific communication represents the greatest need in both science and tech­nology [3] , 141. Although the techniques for data process­ing are constantly improving, the prospects tha t litei'a-ture search and its synthesis can ))e solved i)urely by mechanical processes of data storage and retrieval through a supercomputer, or by the excn less jn-actical proposal to breed a race of superlibrarians [5J, is not as yet promising.

In the existing vast store of research publications and its futiu'c great increase, we can find an answer to our question of what is the value of U. vS. basic re­search. Is the amount of scientific ]mblication a real re­flection of our achievements in ]-esearch? Since "the pur­pose of scientific commimication is . . . a compound of the desire to get ahead and the desire to make a contribution to the progress of science and civilization" [6 ] , it is probable that our scientific literature does, at least, reflect the accomplishments of U. S. scientists much better than the financial facts and figures of the organizations with Avhich they are affiliated.

AYilliam Shockley, in 1957, concluded that ^^the loga­rithmic rate of publication is characteristic of the statis­tics of the scientific creative process" [7] , and implied tha t the amount of publication is a measure of reseai'ch progress. Objections have been raised to this Λτew [8] . However, this kind of statistical approach to provide overall measures of research accomplishment has been widely accepted, and several important studies tha t adopted this point of view [9], [10] have been published in recent years.

Can we assume that the amount of basic research being carried on in American laboratories finds pro­portional representation in the pages of the major scien­tific journals? One great difficulty lies in the continued role of the Department of Defense, NASA, and AEC with respect to the major part of U. S. basic research.

In these agencies, security regulations are often still based on World War I I assumptions which are no longer valid [11], [12], [13], i.e., that classified basic

research results can be kept "secret" because of great U. S. superiority in basic research. The overwhelming consensus of scientists today is tha t this is and was a mistaken i)olicy for two reasons: first, because of the renaissance of basic reseai'ch around the rest of the world, which now in the sum exceeds tha t of the U. S. (al­though the U. S. has first place) ; and second, })ecause suppression of basic research results sei'iously slows up internal U. S. progress in the applied and develop­ment fields, as well as in basic research itself, ^ lany U. S. military managers still have the unfortunate illu­sion that with respect to basic research the World War I I situation of overwhelming U. S. monopoly still exists.

Basic research .supi^orted by D O D and other classified agencies is subject to i)ublication lags from the mininunn of about a year, imposed by tedious secui'ity clearance procedures, up to as long as a decade or more. The judg­ment with respect to suppression is not made by skilled r e s e a r c h personnel. ]\Iuch, if not all, of suppressed publi­cations are redone either in unclassified (and probably government sui^ported) places of the U. S. basic research connnunity or elsewhere in the Avorld, and often within t h e classified agencies themselves, .so invidious is the effect of the '^need-to-know^'^ aspects of security regula­tions. The effect of security is thus twofold—first, a certain part of U. S. basic research is not reported or, if at all, with a considerable lag time; and second, the productivity in ϋ . S. basic researcli is I'educed be­cause of useless duplication of suppressed research.

The scientific journal is the logical medium for scien­tific connnunication, and such publications have been increasing exponentially to keep up with new scientific information. According to Price's cuiwe there are now ap­proximately 30 000 scientific journals reporting original articles [14].

Costs of publication, as ν,'εΐΐ as co.sts of research, are a factor restricting communication of insignificant work. Publishers and editors propose and adopt various solu­tions to cost problems, increasing subscri})tion rates (the obvious one), journal mergers, article condensation, world limitation, government support, etc. [15], [16], [17], [18], The cost of research reported in the journals /aries primarily with the research. An analysis of U. S. Air Force basic research projects, and resulting publications, identifies average research costs per pub­lished paper [19]. The highest is $124 000 per technical paper produced in the field of aeromechanics; the lowest is $9900 in mathematics. Between these extremes are research costs of $108 000 per paper in propulsion, $19 200 in physics, including nuclear phj^'sics, $15 900 in solid state physics, $16 100 in the life sciences, and $11 900 in chemistry. At the same time, a study of the 1955-1958 records of six research organizations of vari­ous types and with various sponsorship indicates a range of cost per paper of $22 000 to $47 000 [1] .

The rather overwhelming costs of both research and publication indicate the necessity for careful weighing

O S IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT JUNK

and measuring of material to be jn-esented in the world's scientific literature. One certainty is, however, that selec­tion should not be based on fashions of the moment, but on judgment of worth. Basic research can develo]) new knowledge tha t may vitalize whole new future develop­ments and, by the same token, *'more than one Einstein may appear in the next century as a direct outgrowth of the present activity in ajiplied science" [20].

But what identifies pai)ers in the classification of basic research? This was carefully considered at a sym])os-ium initiated and s])onsored by the Sloan Founda­tion^ for the purpose of analyzing the concepts of basic research.^' In a s})eech at this conference, Dr. TUA'C of the Carnegie Institution of Washington pointed out the pitfalls of liunping all forms of research under ''basic.'' He used the term "mission-directed basic research,'' which considered a ''hyl)rid notion" invented for justifying budget proposals ( [21] , ]). 169). Ho Avas supported by Wooster of the Air Force Office of Scientific Research, who calls such a definition ''i)ractical nominalism," and by Kidd of the National Institutes of Health, who con­siders all definitions of basic research inherently sub­jective [8] , 1221. Both the scientific world and the public a t large are caught on the horns of a semantic dilemma. One can only admit that these i^itfalls of def­inition do exist, and beware.

Among the group at the Sloan Foundation Symposium there was considerable interest in, and support of, the definition of basic research that provided the im]>etus for this survey, which, roughly stated, is: Basic research is fundamental research in science from which the results are publishable in the scientific literature." This study is based on the theory that major scientific journals do in­clude the major basic research findings of scientists fronr all over the country, and tha t quantitative analysis of these journals provides a good yardstick for evaluating both the amount of basic research accomplished and the relative contributions of the national sponsor groups.

MECHANICS OF ORO SURVEY

To the individual who is asked to unco\'er the amount of basic research sponsorship reflected in the articles of 50 scientific journals o\'er the past 40 years, the thought of counting t ha t many pages appears api)alling if not absurd. The task strikes him as easy but deadly dull. I t was neither, really, and the results were highly enlight-

1 Invited cosponsons were tlic National Academy of Sciences and the American Association for the Advancement of Science.

- Basic Science Symposium, held under the auspices of the Alfred P . Sloan Foundation at the Rockefeller Institute Auditor­ium, New York City, May 15-16, 1959.

s i t is regrettable that no reference to Dr. J. W. ΗίΙΓδ proposed wording of such a definition is included in the section of defini­tions in the conference summarj^ ( [21] , pp. 256-57). The Air Force subscribes to this theory, specifically in its regulation A F R 80-33: 1-6 [ 2 3 ] : "The greatest value of research lies in the new information it uncovers and the publishing of these results is an important phase of any research project. . . . In this regard, the technical and scientific journals published in this country offer the Air Force an effective and economical means of publicizing its progress being made in scientific research."

ening. The team that embarked on this enterprise needed new glasses and new tempers a t the end, but the indi­viduals had a vastly enlarged concept of what has been going on in the field of research in the United States.

]\Iany of the problems of such a study were apparent with the opening of the first trial volume, others cropped u]) along the way. Authors often took fiendish delight, it seemed, in concealing the sponsor's identity in the mid­dle of a paragraph buried in the text, rather than in the conventional footnote at the beginning or end of an ar­ticle. Sometimes the organization with which the author was affiliated was hard to place in any category. I t might be considered as belonging equally well within a uni­versity, as par t of an industrial concern, or as a ]:>rivate research laboi-atoiy. Was Project P E N G U I N or MAT-T E R H O R N supported l)y the National Geographic So­ciety or the Atomic Energy Connnission? Guesses Λvere usually wrong, and the identity of all such projects had to be established. Frequently, important items of basic research were reported in the letters-to-the-editor sec­tions, so that nothing on any printed page could be over­looked. If communications of importance appeared in short form or in small print on a large page, the results were endless columns of fractions to be added together. Then, too, there were ahvays articles of a very general nature that described research efforts in broad terms but did not re])ort actual achievements. What should one do with these articles, or the ones whose authors failed to identify themselves beyond giving a street address? Too much time could not be s])ent on individual items in such a lengthy project. Obviously a plan of attack had to be drawn up with a set of rules for the team.

The first problem was to determine which journals should be surveyed.'* The objective was to select journals that carried a significantly high proportion of articles in the area of basic research, and thereby obviate the nec­essity for p]Ofessional discrimination in identifying spe­cific ai'ticles. Proposed lists of journals for each of the various disciplines were submitted by members of the ORO research staff in the individual fields. The culling process was carried out with the aid of senior scientists in the same fields. In the end, exactly 50 were selected that fell roughly into seven broad fields: general sci­ences, physical sciences, life sciences, earth sciences, ap­plied scientific research, mathematics and statistics, and operations research and management.'"' The applied sci­entific research group was allowed in the survey as tan­gent to development areas within the other "basic" cate­gories. The Journal of the Aero/Space Sciences could be classed in the physical sciences, and Radiation Re­search in either the life or the physical sciences, but in view of the specific character of the research represented

'Three characteristics are thought to be essential to a source journal: 1) that it be scholarly in character; 2) that it be widely accepted among practitioners in the field as having excellent ref­erence value; 3) that it be general in its coverage of the field un­der study "[24] .

^ The list of periodicals selected is given in the Appendix.

1966 MILTON AND J O H N S O N ! SPONSORSHH^ OF RESEARCH 99

in 86Λ^0Γα1 of the periodicals, it seemed best to create a special category for them.

Almost all the periodicals chosen had appeared regu­larly from 1949 to 1960/'* and could be located con-A^'eniently in the immediate area; 19 were extant in 1920, the earliest year co\^ered l)y the surA 'ey. Many were or­gans of long-establislied institutions in ])articular fields such as the American chemical and physical societies, or such widely recognized organizations as the American Association for the Advancement of Science and The Frankl in Insti tute. I t was decided tha t all articles ap­pearing in the selected periodicals would be considered reflections of basic research except those of an obviously historical or descripti\'e character. Foreign articles''' as Λνοΐΐ as book re\dews, abstracts, and proceedings of meetings were to be omitted.

The next problem was to define categories and to find a com'cnient method for recording the number of articles and i)ages. An initial tabulation sheet was devised with columns for the major groups sponsoring basic research —federal go\'ernment, u n i v e r s i t i e s , industry, and p r i v a t e research organizations. The government classification in­cluded the Depar tment of Defense (DOD)—w'uh four subsections: Army, Navy, Air Force, and Interservice; the Atomic Energy Commission; The National Science Foundation; and all other government groups. As the authors ' interest lay particularly with the amount of re­search sponsored by D O D , it was determined tha t an article sponsored in par t by a D O D agency and in part by other groups would be tallied as an article under the D O D agenc3% Λvith the number of pages in the article d i \Tded equally among all the sponsoring agencies. There was ample justification for this ruling in the fact t ha t a project Avith any government sponsorship is usu­ally pret ty \vell covered by this subsidy, (It is of interest to note tha t the final totals showed only minute differ­ences in the trends based on the a r t i c l e count and the page count.) In some cases, an agency like the National Insti tutes of Heal th sponsored the work reported in a large proportion of the articles in a journal. Articles un­i d e n t i f i e d as to sponsorship or author affiliation were few^ in number, appearing generally prior to 1949, and were excluded from the totals. The difficulty of dealing with too many fractions for easy tabulation Avas o b v i ­a t e d by considering any article smaller than half a page as a half page, and anything larger than half a page as a wdiole page. This method s o m e t i m e s o v e r e s t i m a t e d the pagination quantitatively, but had little effect propor­tionately.

The research team was comprised of five persons with completely different backgrounds: a librarian, an econ-

6 According to Burton, material published from 1946 to 1959 accounted for three-fourths of all citations in American engineer­ing journals ( [24 ] , p. 137).

^ Seventy-nine percent of the citations in Burton's study of en­gineering journals were to materials published in the U. S. Prior to 1926, foreign journals were dominant ( [24 ] , p. 137),

omist with business experience, a technical aide with knowledge of computing, a retired Colonel formerly on the staft' of the Adjutant General, and a research assist­ant with a background in geography and currently studying mathematics. The comlnnation ]:)roved very ef­fective for unsnarling the tangle of government vs. pri­vate sponsors and detecting all sorts of subtle ti'ends. A name unfamiliar to one w\as often innnediately recog­nized l)y another.

Tlic individual approaches created some problems in achieving uniformity in classification. Decision making was bound to play some i)art in sponsorship identifica­tion, a n d individiml interpretation of the established guides could be formulated only througli actual experi­ence in making the talmlations. Although some divergent concei)tions were easily dissipated, continuing discrep­ancies in the early period l)etween summaries of volumes of the same journals called for a complete review. With almost one tenth of the material surveyed, the study grouj) of full-time surveyors suspended article and page counting to discuss and a]*ri\'e at a meeting of minds on category coverage and rules of procedure, with this r e ­sult: ] iar t - t ime help was largely discontinued; the initial segment of work was redone; the procedure for one indi­vidual to survey the entire series of one jouriml was es­tablished and adhered to whenever practical.

Even so, i t is accepted tha t the human element played a part in the tal)ulation of data. To discoΛ^er what devi­ation might occur in tabulating identical material , either between individuals or from one count t o the next by the same indiv^idual, tΛvo i)ersons were asked to do a single volume and one person Λvas asked to recount two vol­umes after a n interval of time. Of the two people, the second person totaled two less articles than the first and 50 pages more, but, as the A'olume recounted was from Science^ which has many very short reports and many foreign contributors, the difference in the number of articles was not unduly large. The difference in the page count in a volume of 786 pages could be accounted for by the fact tha t one person sometimes called a portion of a page a whole page, whereas the other Avas more precise in dividing the pages into exact halves. The individual who recounted a specific A'olume of the Aniericaii Journal of Physiology a n d one volume of the Astrophysical Journal came out with t h e same total of articles for the first and one less for the second. The total in one volume was nine pages more the first than the second time around, and nine pages less in the next series. This small sample proved the impossibility of complete consistency, but in ­dicated tha t in the overall picture the differences tend to cancel out. The estimated accuracy of the survey is Avithin about five percent.

Collecting and tabulating the data required long hours of concentrated work and continuous rechecking. The material was gathered on sheaves of tabulat ion sheets. The survey coverage (1920-1960) totaled 115 872 ar­ticles and 699 184 pages. For t h e period 1949-1960

100 IKKE TRANSACTIONS ON E X G l N E E R l N d Μ Α Χ Α ( ί Ε λ Ι Ε Ν Τ J U N E

alone, there were 100 549 articles and 559 103 jiages. Interpretation of these data rests scpiarely on an un­

derstanding of the methodology outlined a l )0 \ ' e . A glance at the figures in this tabulation and those in the Navy study 19] for the identical voliune of the Physical Kc-vicw shows tha t the results ai'rived at can Λ'ary con­siderably according to the method ai)plied to the search for data and the classification thereof. The latitude of choice in both methodology and interpretation is enor­mous, and choices there must be.

Two i)ossible weaknesses in this study appear in the method applied to counting joint articles and pages. As indicated previously, articles were comited as units and pages di\^ided among si)onsors. For instance, instead of dividing an article in the case of NaA-y-Atomic Energy Commission joint sponsorship, the article \vas given to the Navy and the jxiges di\'ided l)etween the two agen­cies. Thus the advantage of the Navy shows up in num-l)er of articles, not in pages. In the case of the ΟΙΓΚΌ of Secretary of Defense (OSD) and Interservice, under DOD, the distinction between Anny-Navy, Navy-Air Force, and Air Force-Army sponsorshij) was not made. Often, of course, all three services were sponsors. The joint services account for only five percent of the articles and six percent of the pages in the whole D O D picture, so that the misrepresentation is not serious.

INDICATED TRENDS

This survey covers the ]:)criod 1920-1960 with em­phasis on the last 12 years. Annual publications \vere scanned for the period 1949-1960; the count was made at five-year intervals for the period 1920-1945, i.e., 1920, 1925, 1930, etc. Three periods are significant in this dis­cussion: 1920-1960; pre-World War II 1920-1945;^ and post-AVorld AVar I I , 1949-1960.

The 50 journals smweyed \vcre selected on the basis of their importance in the past decade and, as pointed out l)reviously, many did not start piil)lication until well after 1920. As a result, the count begins with only 22 journals, swells to 41 by 1949, and rises to the full 50 in the last 12 years. Of the total coverage, 53 percent of the articles and 43 percent of the pages were recorded from five periodicals. The massive size of such sets as the Journal oj the American Chemical Society and the Phys­ical Review provides the explanation. Init ial publica­tion of the two journals noted was in 1879 and 1893, i*especti\'ely.

As might be expected, there is a definite, albeit fluctu­ating, rise in the total of journal articles and pages from 1920 to 1960. Between 1949 and 1960, however, the number of articles published increased by 65 percent and the number of pages by 64 percent. The survey totals for both articles and pages are shown by years in Fig. 1. Although the general trend for the entire period is up-

s Publication of World War II research results occurred largely after 1945.

ward, obvious lulls can l)e noted for the depression years of the thiities and the A\Orld War I I years of the forties, Avith a slight dij^ in 1952 during the Korean War. If we extrapolate t he 1920-1930 trends and the 1945-1960 trends, they meet at about 1980. We might speculate that the combined efi'ect of tlie great dejiression and of World AA'ar I I was to delay scientific progress by twenty years.

In general, data on articles are utilized in this sum­mary with occasional page data presented for compara­tive purposes. There are, as jiointed out in the preceding section, slight vai'iances in the two sets of data , but the trends indicated are consistent. Each breakdown in the data coverage was charted for both articles and pages, but examples of the comparability are sufficient. Figure 2 is repi-esentative. I t shows the proportion by years of national groups in the sponsorship of all articles and pages for the [>ei'iod of the survey 1920-1960.

The trend in s])onsoi'ship of the basic research re-poi'ted in the increasing number of }.)ublished articles is identified in Fig. 3, which shows for 1920-1960 the ac­tual number of articles suppoi'ted by funds from govern­ment agencies, universities and colleges, industry, and private research organizations, Ta})lc I details the in­formation thi'ough the presentation of total number of articles by years and the ])iOportionate percentages of the s])onsoring groups. Along with the decided upswing in volume of publication in the ])ast decade is a definite shift in relative volume of research funds provided by s])onsoring groups. This is observable in Fig. 4, Avhich l)resents the i)ercentage data on articles tha t appeared in Fig. 2. From 1920 to 1940, colleges and uniΛ'ersities were ])i'edominant not only in their historic role of per­forming basic research but in financial support of the re­search. In the ])re-AVorld War II ])eriod, the universities, reprc^senting both academic institutions and individual faculty researchers, funded the efforts reported in more than 60 percent of the articles appearing in the journals surveyed. Indust ry , private research organizations, and goΛτrnment sponsored the remainder, and each fell con­sistently below the 20-pcrcent mark.

The technological demands of World War I I provided the impetus for increased support of basic research on the par t of the federal government. With the upswing in government sponsorship, the relative proportion of uni­versity support started dropping. I n Figs. 3 and 4 the university and gOΛx·rnment lines cross in 1950. From then on the proportion of go\'ernment sponsorship in­creased and uniΛ^ersity support decreased. Meanwhile, industry and private research increased support suffici­ently to maintain a relatively steady proportion of the enlarging annual totals of research publications with the indications of an upward swing for industry in 1957-1959. Figure 5 is an enlargement of the last period of Fig. 3, 1949-1960, with the steady rise of government sponsorship standing out very clearly beside the more gradual decline of the universities.

1966 MILTON AND J O H N S O N : S P O N S O R S H H ' OF RESEARCH 101

Fig. 1. Basic r o T a r c h , 1920-1960. by niiinbcr of articles and pages. Survey of 50 s c i e n t i f i c journals. J^iges. Articles.

T A B L E I

N A T I O N A L Spoxsofisnip O F B A S I C R E S E A I I C I I B Y

P E R C E N T A G E O F A R T I C L E S F O R 1920-1960

Total Private Number Clovern- Uni­ Organ i

of Total ment versity Industry zation? Articles Percent Percent Percent Percent Percen

11)20 ii:r2 100 17.5 6 7 . 6 4 . 2 10.7 1925 1775 12.9 6 9 . 9 6 .7 10 .5 ]D:^0 3041 12.1 6 9 . 1 11 .0 7 . 8 1D:̂ ") :U95 10.3 7 4 . 6 6 . 6 8 . 5 1D40 3666 7 .4 7 4 . 6 9 . 5 8 . 5 1945 2514 15.4 5 9 . 9 17 .0 7 .7 1949 6040 :i2.5 4 3 . 3 15 .6 8 . 6 1950 6395 39.2 3 9 . 0 14.1 7 .7 1951 7738 38 .9 3 7 . 4 14 .2 7 . 5 1952 7370 41 .0 3 7 . 1 13 .6 8 . 3 J953 8183 43 .4 :33.δ 13 .9 9 . 2 1954 8375 48 .5 3 2 . 7 12 .4 6 . 4 1955 8709 50 .0 28 9 14 .2 6 . 9 1956 8754 53.7 2 5 . 9 13 .5 6 . 9 1957 9201 52 .3 2 5 . 7 15.7 6 . 3 19.5S 9576 52.5 2 2 . 1 17 .9 7 . 5 1959 10217 55.8 1 8 . 3 19 .4 6 . 5 1960 9991 55.7 1 9 . 1 18 .5 6 . 7

80

60

•z. υ

40

20

- 1 Γ

,^^^^^ P r i v a t e r e s e a r c h o r g a n i z a t i o n s

1960

Fig. 2. National sponsorship of basic research, 1920-1960, by per­centage of articles and pages. Survey of 50 scientific journals.

Pages. Articles. Fig. 4. National sponsorship of basic research, 1920-1960, centage of articles. Survey of 50 scientific journals. — ernment. University. Industry Priv ganizations.

1960

by per-Gov-

r'ate or-

6000

4000 U

2000 μ

1960

Fig. 3. Nat ional sponsorship of basic research, 1920-1960, by number of articles. Survey of 50 scientific journals. ^— Gov­ernment. University. Industry Private or­ganizations.

6000

5000 μ

4000 h

3000 μ

1000 L

1960

Fig. 5. National sponsorship of basic research, 1949-1960, by number of articles. Survey of 50 scientific journals. Gov­ernment University. Industry Private or­ganizations.

1 0 2 IEEE TRANSACTIONS ON ENGINEERING M A N A G E M E N T

T A B L E 11

S i M M A R v O F N A T I O N A L S P O N S O R S H I P O F I I E S K A R C H l o u 1 1 ) 2 0 - 1 0 4 5 A N D 1 0 4 0 - 1 9 0 0 T I M E F R A M E S

J U N E

1920-1945 1949-1900

Source

1 Sponsorship Percent of total Spo i.sorship Percent of total

Source 1 Articles Pages Articles Pages Articles 1 Pages

Articles Pages

Total 1 15 :V23 140 021 100.0 100.0 100 549 1 559 1G3 100.0 100.0 Cioverumeut ! 1 7S:J IS 727 11 .t) 13.4 4S 270 1 200 004 48.0 47. G l^niversity 10 731 94 V2S 70.0 t)7.2 29 253 ICS 549 29.0 30.2 Industry ^ i 1 4S5 12 565 9.7 9.0 15 529 75 029 15.5 13.5 Private researcli

organizations \ 1 :]24 14 001 S.7 10.4 7 491 I 48 981 1

7.5 S.7

An o\'erall suininary of national sponsoj'ship, with a breakdoAvn for the 1920-1945 and 1949-1960 periods, is presented in Table I I . The emphasis is on proportion­ate s])onsorship for giOU])s. The actual totals for the two l)eriods cannot be compared since only one year out of each five is included in the 1920-1945 period.*' The go\-e r n m e n t , a latecomer in research suj)port, sponsored only 12 i^ercent of the total articles surveyed in tlie earlier j^eriod and captured 48 i)ercent of the total in tlie postwar decade. The universities, with 70 i^ercent of the prewar total, drojiped to 29 percent in t h e 1949-1960 survey. Industry accounted for 10 and 16 percent for the two iH'i'iods, respectiA'cly, and private organizations, 9 and 7 percent of the total articles.

It is obvious from the t rends that tlie role of the fed­era l goA'ernment as a si)onsor of basic research will be increasingly important. By 1953-1954 t h e federal gov­ernment supported over two thirds of the basic research carried on at the nation's colleges and universities |25]. By 1957-1958 the federal government was providing the funds for over half the nation's basic research [26]. One industrial representative, for example, has stated "major effort Λνϋΐ continue to be focused on the concept of basic scientific research as a vital national resource, eventually publicly funded on its own merits alone'' [27]. Based on the trends from this sample, it can be anticipated tha t the federal total λνίΐΐ represent some 90 percent of the national support of basic research Avithin the next 20 years.

AVithin the federal government there has been a shift in emphasis in research support comparable to the changing pattern Avithin national support. The story of the growth of sponsorship within the government is told in Table I I I and Figs. 6 and 7. Table I I I identifies the total number of articles and the relative proportions sponsored by the D O D , the Atomic Energy Commission, the three research agencies (the National Research Council, the National Academy of Science, and the Na­tional Science Foundat ion) , and all other government

9 Based on the curves, total articles and pages approximate 69 000 and 630 000 for all the years of the 1920-1945 period.

agencies from 1920 to 1960. Figure 6 indicates the num­ber of articles for each of the sponsoring groups. Figure 7 shows the ])ercentage pat tern for these grou]:)S in r e ­lation to the government total .

FiOin 1920 to 1945, the government agencies that often worked hand in hand with the universities—the Geological Sur\'ey, the Coast and Geodetic Survey, the special branches of the Depar tment of Agriculture, the Bureau of •Mines, the National Bureau of Standards, etc.—were the sponsors of approximately three-fourths of the articles i)roi)ared under federal sponsorship. With the outbreak of World War I I , the ^^other government'^ agencies stai'ted slipping into a lesser position in federal sponsorship of research, a trend similar to that of the colleges and universities in the national incture. At the same time the mili tary branches began their climb as a result of war demands on R &· D and, soon after the Avar, DOD was t he major financial supporter of the federally sponsored research re])orted in this sample of scientific literature. Maintaining a top position throughout the productive 1950's, the mil i tary departments' sponsor­ship in the postwar period represented 46 percent of the goA^ernment-sponsored articles against 27 percent for the ^'other" agencies that had predominated in the 1920 -1945 period. The Atomic Energy Commission, which came into existence on December 31, 1947, made the spectacular contribution of 21 percent of all government-sponsored articles between 1949 and 1960, plus its ad­ditional joint sponsorship with the NaAy. The dramatic rise of the Atomic Energy Commission in the research field is illustrated by its survey totals of sponsored ar­ticles and pages a t the beginning and end of the decade: 324 articles and 1457 pages in 1949, and 1056 articles and 6586 pages in 1960. The National Science Founda­tion, another latecomer, was established in 1950 with the stated purpose of promoting the progress of science largely through grants to individuals and educational institutions. The upsAving of the joint National Science Foundation, National Research Council, and National Academy of Science curve during the 1950's in Fig. 6 reflects the Nat ional Science Foundation's investment of

1966 MILTON AND J O H N S O N I SPONSOKSHH' OF RESEARCH 103

T A B L E I I I

CioVEUNMENT Sl'ONSORSlHI' OF B A S I C RESEARCil B Y

P E R C E N T A G E O F A R T I C L E S F O R 1920-1960

Total Other Number N S F - Govern­

of 1)01) Ai : c N R C - N A S ment, Articles Percent l^erceul Percent Percent

1920 198 17.2 L 5 81 .3 1925 229 7 .0 3 1 . 4 61 .6 1930 367 8 .4 15.8 75 .8 1935 328 4 . 3 26 .5 69 .2 1940 273 4 . 8 10.2 S5.0 1945 388 30 .7 9 . 8 59 .5 1949 1961 52 .8 16.5 1.4 29 .3 1950 2508 51 .2 19.5 1.7 27 .6 1951 3012 48 .5 2 3 . 2 0 .6 27 .7 1952 3025 4S.6 2 0 . 6 0 . 4 30 .4 1953 3554 48 .9 21 .7 1.9 27 .5 li)54 4065 50 .8 22 .*3 2 .7 24 .2 1955 4357 49 .4 21 .2 4 .7 24 .7 1956 4700 46 .2 23 .5 ('). 6 23.7 1957 480S 4 4 . 4 20 .7 8 . 4 26 .5 1958 5029 41 .6 20 .6 10.3 27 .5 1959 5()94 40 3 19.9 11.8 28 .0 1960 5563 39 .0 19.0 13.0 2 9 . 0

2400

1500 h

>300 μ

Pig. 6. Gov(»rnment si)onsorshin of l)asic r(\<c^arch, 1920-1960, by number of articles. Surv(^' of 50 scientific journals. Other iiovcrnmont, N R C ^ N A S , XSF D O D A E C .

100

1960

Fig. 7. Government sponsorshi]) of basic research, 1920-1960, by percentage of articles. Survev of 50 s(iientific journals. Other government. N P C . NAS, N S F D O D AEC.

funds in research studies. These gains in sponsorship are reflected in the slightly decreasing performance of DOD when com|nired i)ercentagewise witli the govern­ment total in Fig. 8 .

The pattern of D O D sponsorship indicated by the lit­erature survey is shown in Table IV and Fig. 9 . As noted abo \T , the mili tary services ])layed a small role in basic research prior to AVorld War IL The Λvar-promoted be­ginning flowered for tlie Navy with the establishment by act of Congress of the Office of Naval Research on August 1, 1 9 4 6 . This was an initial step in the general effort to obtain fedei'al support for basic research that subserpiently resulted in the establishment of the Na­tional Science Foundation, an independent federal agency.

In the D O D total of sponsored articles during the })e]"iod 1 9 4 9 - 1 9 6 0 . the Navy accounts for 5 4 percent, the Air Force 2 1 ]iercent, and the Army 1 9 percent, with OSD and intei-service enterpi'ises making u]) the rest— approximately 6 i)ercent. In 1 9 4 9 , Navy-sponsored re­search was reported in almost 8 0 0 articles. The Navy's ])eak year was 1 9 5 4 , in which 1 1 6 2 sponsored articles (Fig. 1 0 ) and 6 0 4 6 i)ages (Fig. 1 1 ) are recorded in this literatui'c survey. Since that year the other seiwices have made relative gains vis-a-vis the Navy. The Army reached its peak Λvit l l ovei" 5 0 0 articles and more than 3 0 0 0 Images in 1 9 5 5 . The Air Force has had a consistent increase in recent years. ]\Iost of its basic research is let out on contract, and the service is not only interested in the publislied results from that research Init keeps a running recoi'd of the publ icat ions.Sponsorsl i ip by OSD and through joint arrangements of the services has in­creased in the past decade.

The seven categories of subject coverage into which the 5 0 journals HIWE been grouped for this study appear in Tables and YI and Figs. 1 2 and 13 . Table V ])re-sents the totals for articles in selected years during the period 1 9 2 0 - 1 9 4 5 . Table VI presents comparable annual figures for the period 1 9 4 9 - 1 9 6 0 . Figure 1 2 identifies the number of articles and images recorded for the period 1 9 4 9 - 1 9 6 0 . The physical sciences exceed each of the other groups by a wide margin. The actual weighting for articles is 6 1 percent of the D O D total in the physical sciences^ 1 1 and 9 percent in the applied and the life sciences, and ranging down from 8 to 1 percent in the other groups. The lowest representation is the operations research and management category, a significant field in relation to research, but obviously a more nebulous and a newer one in the scientific research area than any of the other classes represented. This Aveighting in quanti ty of periodicals per category, as ΛΥΟΙΙ as the difference in bulk between certain serials, rules out the possibility of

^0 The Air Force Office of Scientific Research is responsible for the detailed canvass of publications residting from Air Force contract funds for basic research.

104 IEEE TRANSACTIONS ON E N G I N E E R I N G MANAGEMENT J U N E

making direct comjiarisons between categories. For in­stance, in this survey i t is impossible to compare pub­lished output in the physical sciences with that of the life sciences. Such unequal samples only permit a con­sideration of trends irithin categories.

Figure 13 traces the number of articles in each cate­gory on a five-year basis from 1920 to 1945 and points up the relative yearly increases among the groups fol­lowing AVorld War I I . The federal government and the universities are the leading sponsors within the seven research categories (Tabk' VII ) . In the period 1949-1960, the government led with 47 to 54 percent of the total articles in the physical sciences, life sciences, general sciences, and applied sciences; uniA'crsities and colleges led with 55 jiercent of the articles in mathematics and statistics. Although the government is credited with the highest percentage of articles in the eartii sciences, and imi\^ersities the highest percentage of articles in operations research and management, the leading roles are re\^ersed in the i3age counts. These instances repre­sent two of the few cases in Avhich the article and page counts deviate sufficiently to affect general comparisons. In these cases, then, the goA^ernment and universities share honors.

Figure 14 identifies the seven science categories of major military emphasis in recent years. In five of the seven categories, the Navy holds its long-established publication lead among the armed services.

1950 1952 1954 1956 1958 1960

Fig. 8. Percentage of government basic research sponsored bv D O D , 1949-1960, by articles and pages. Survey of oO scientific journals. Pages. Articles.

1930 1950 1960 1940 Y E A R

Fig. 9. D O D sponsorship of basic research, 1920-1960, by service and number of articles. Survey of 50 scdentific journals. N a v y Air Force. Army OSD, Interservice.

Figures 15 and 16 indicate the trends in university and private sponsorship in specific areas. The interest of the universities in the ])hysical sciences has predomi­nated from 1920 on whereas, as far back as 1922, the j)ri\'ate research giOU])s concentrated more heavily on tlie life sciences. Figure 16 reflects the infiuence of such large organizations as the Hockefellcr Institute, the Sloan-Kettering Foundation, and the Commonwealth Fund, which have given theii' chief sup])ort to medicine and allied fields.

Compared to the line in Fig. 17, representing overall government sponsorship, the number of articles in the life sciences sjionsored by the military services fFig. 18) is not particularly large. The rajnd rise of the '''go\̂ern-nient' ' line is brought al)out by the articles published under the vast research programs of the National In­stitutes of Health and tlie Pui)lic Health Service. Only in earth sciences (Fig. 19) and in operations research and management (Fig. 20) does the NaA^y fall below the Air Force in 1949-1960 totals, and only slightly below in those groups. Figure 21 shows the recent lead of the Air Force in applied scientific research, with the Navy hold­ing its margin in inathematics and statistics (Fig. 22).

T A B L E I V

DOl.) S P O N S O R S H U * O F B A S I C R E S E A R C H B Y S E R V I C E A N D

N U M B E R O F A R T I C L E S F O R 1049-1960

Total Number of Articles Army Navy

Air Force

OSD a I η terser

1949 1035 117 789 67 62 1950 1283 138 990 101 54 1951 1460 122 1140 143 55 1952 1469 179 1054 186 50 1953 1737 251 1140 280 66 1954 2064 414 1162 375 113 1955 2152 530 1068 454 100 1956 2172 515 1047 465 145 1957 2133 481 974 537 141 1958 2093 479 847 625 142 1959 2294 519 881 715 179 1960 2169 439 782 757 191

1200

1000 μ

800

y 600 μ

400 μ

200 μ

1950 1952 1954 1956 1958 1960

Fig. 10. D O D sponsorship of basic research, 1949-1960, by sendee and number of articles. Survey of 50 scientific journals. N a v y Air Force. Army OSD, Interservice.

1966 MILTON AND J O H N S O N : SPONSORSHIP OF RESEARCH 105

1950 19S2 1954 1960

Γίμ;. 11. DOD r.poiisor.ship of basic rcscarcli, 1940-1960, })y service and number of pas:cs. Survey of 50 scientific journals. Navy.

Air Force. Army OSD, Intcr.scrvico.

T A B L E V

S u R V K Y O F S C I E N T I F I C Ι ^ Π Έ Ι Ι Λ Τ Ι Ί Ι Ε F O R 1920-1945 T O T A L A R T J C L K S B Y C A T E ( ; O R Y A N D Y E A H

1920-1945 1920 1925 1030 1940 1945

Total L'):32:̂ 1132 1775 :U)41 3195 2514

Applied Scientific Resear(rh 8H) — — 240 174 225 168

Physical Sciences 7280 442 802 14GS 15:50 1898 1140 Mathematics and

Statistics l3:u 88 117 190 321 326 283 General Science 1788 204 293 383 345 323 241 lOarth Sciences 49 70 01 87 79 46 Life Sciences 3716 349 49:i 685 738 815 636

T A B L E V I

S r u v E Y O F S C I E N T I F I C L I T E R A T T R E F O R 1949-1960 T O T A L A R T I C L E S B Y C A T E G O R Y A N D Y E A R

Total 1949-1960 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960

Total 100549 6040 ()395 7738 7370 8183 8375 8709 8754 9201 9576 10217 9991

Applied Scientific Research 8799 513 503 561 605 776 709 745 781 820 894 1070 822 5636 Physical Sciences 56790 3317 3697 4586 4230 4560 4881 49()2 4854 5j;j6 5236 5695

822 5636

Mathematics and Statistics 5797 :>:)7 326 413 401 431 473 551 541 575 494 594 661 General Science 8877 676 695 691 701 719 717 693 742 818 771 831 823 Earth Sciences 2314 146 142 163 186 187 211 225 199 211 214 229 201 Life Sciences 1697t) 1043 1014 1309 1187 1470 1316 1450 15:̂ 9 1520 I8:u 1629 1659 Operations Research

169 189 and INIanagement 1002 8 18 15 60 40 68 83 98 121 133 169 189

OR & Mgt,

CATEGORY

Fig. 12. Total number of pages and articles, 1049-1960, by cate­gories of research. Survey of 50 scientific journals. E l l Pages, •i Articles.

6000

4000

2000

1920 1930 1940 YEAR

1950 1960

Fig. 13. Articles on basic research, 1920-1960, b y year and sub­ject categor>^ Survey of 50 scientific journals. —; Physical sciences. Life sciences. Applied sciences General sciences. Mathematics and statistics. Earth sciences. Operations research and management.

1 0 6 IEEE TRANSACTIONS ON ENGINEERING M A N A G E M E N T J U N E

T A B L E V I I

S U M M A R Y O F B A S I C R E S E A R C H P U B L I C A T I O N S H Y C A T E G O R Y A N D S P O N S O R S H I P

Percent of totnl

Government Universitv Inclustrv Private

organizations

Category Articles Pages Articles Pages Articles Pages Articles Pages Articles Pages

1 9 2 0 - 1 9 4 5 * Physical sciences 7 2 8 0 4 7 3 7 6 9 1 0 7 4 6 8 1 1 1 3 6 9 Life sciences 3 7 1 6 4 5 2 5 2 1 0 7 7 0 7 5 4 3 1 6 1 5 Earth sciences 3 9 2 6 8 2 0 2 0 1 7 7 0 71 1 1 9 1 1 General sciences 1 7 8 8 2 0 0 7 9 2 9 4 1 5 6 3 9 6 9 9 1 1 Mathematics and

statistics 1331 1 4 0 8 9 8 7 8 9 9 0 1 1 2 2 Applied scientific

research 8 1 6 6 4 0 5 8 8 4 0 3 S 4 9 5 2 3 2 Operations research (Not a])plicable to

and management, this period)

1 9 4 9 - 1 9 6 0 Ph^'sical sciences 5 6 7 9 0 2 4 7 7 1 4 4 7 4 9 2 8 2 7 2 0 1 9 5 5 Life sciences 1 6 9 7 0 1 1 7 2 2 4 5 4 4 9 2 2 2 3 5 5 1 9 2 3 Earth sciences 2 3 1 4 2 5 0 7 0 4 9 4 2 4 0 4 7 3 3 8 8 General sciences S 8 7 7 4 1 6 8 1 5 1 5 1 3 3 3 4 5 6 1 1 9 Mathematics and

statistics 5 7 9 7 6 7 5 6 0 3 9 4 3 5 5 5 1 3 3 3 .>

Applied scientific research 8 7 9 9 4 9 4 6 4 4 7 4 6 · 2 3 2 3 2 7 2 8 3 3

Operations research and management 1 0 0 2 1 0 4 5 0 3 1 3 6 3 8 3 5 2 4 2 0 7 9

= Survey covered one year in each five-year period: 1 9 2 0 , 1 9 2 5 , 1 9 3 0 , 1 9 3 5 , 1 9 4 0 , and 1 9 4 5 .

4 2500

^ 2000

500

P h y s i c o l S c i e n c e s

M a t h & G c n e r o l E o r t l i A p p l i e d S c i . L i f e S t o t . S c i e n c e s S r i f n c e s R e s . A r e a s S c i e n c e s

O p e r . Rc^s & M g t . . S r i e n c e s R e s . A r e a s S c i i

C A T E G O R Y

Fig. 1 4 . Total number of articles spon.sored by D O D , 1 9 4 9 - 1 9 6 0 , by subject categories of basic research. Survey of 5 0 scientific journals. B | ^^^^^3'· 111 Air Force. [ | Army. ^ OSD and Interservice.

1920 1930 1940 YEAR

1950 1960

Fig. 15. Universit}' sponsorship of basic research, 1920-1960, by subject categoiy. Survey of 50 scientific jounials. Phys­ical sciences. Life sciences. Applied sciences General sciences. Mathematics and statistics, Earth sciences.

MILTOX AND J O H N S O N I S P O N S O R S H I P OF RESEARCH

900 μ

107

1960

Fig. 16. Private research organization sponsorsliip of basic re­search, 1920-1960, by subject category. Sun^oy of 50 scientific journals. Life sciences. Physical sciences General sciences. Mathematics and statistics. Earth sciences.

1960

Fig. 18. D O D sponsorship of basic research in life sciences, 1949-1960. Survey of seven .scientific journals. N a v y . Air Force. — Army OSD, Interservice.

I960

Fig. 17. Sponsorship of basic research in life sciences, 1920-1960. Survey of 50 scientific journals. Government. uni­versity. Industry Private organizations.

1 i 1 Γ

1950 1952 1954 1956 1958 1960

Fig. 19. D O D sponsorship of basic research in earth sciences, 1949-1960. Survey of seven scientific journals. Navy. Air Force. Army O S D . Interservice.

30

25

20 « Λ U i - I

^ 15

<

10

5

/ A \

1950 1952 1954 YEAR

1956 1958 1960

Fig. 20. D O D sponsorship of basic research in operations research and management, 1949-1960. Survey of four scientific journals.

Navy . Air Force. Army OSD, Inter­service.

I960

Fig. 21. D O D sponsorship of basic research in applied scientific research, 1949-1960. Survey of six scientific journals. Na\'y,

Air Force. Army OSD, Interservice.

100

60

40

20

1950 1952 1954 1956 YEAR

1958 1960

Fig. 22, D O D sponsorship of basic research in mathematics and sta­tistics, 1949-1960. survey of nine scientific journals. Navy .

Air Force. Army. OSD, Interservice.

1200

1 0 8 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT J U N E

CONCLVSIONS

The publication trends identified in tliis survey cor­respond to the general pat tern that is known to exist in the financing of research. This tends, thereby, to su])-port the thesis tliat the amount of basic research could be identified quantitatiλ'ely through a careful and thor­ough survey of the entire scientific literature. If adequate data on basic research expenditures Avere available for the period of the survey, or even for the past decade, the logical next step v;ould be to determine the relation be­tween funds expended and the survey results. Similarly, if program data were aΛ'ailable, lags in pubhcation could be identified through the survey findings. Unfortunately, there are few such ])recise figures obtainable except for those in the Air Force study cited previously [23]. How­ever, with the competition for research funds now in full swing, more and more data on funding are being tabu­lated and released. The continuing efforts of the Xa-tional Science Foundation to provide pertinent da ta in this field will greatly facilitate the evaluation of research costs and productivity in the future.

The economic growth of any country today is closely integrated with the progression from research to devel­opment to production. There is, as well, an important positive feedback effect, i.e., economic growth is an in-centiA^e and permits iiwestment in research, wdiich in turn spurs development based on the discoveries of pure research. Following World War I I , industrial competi­tion for markets stimulated industrial interest in fun­damental research. Independently, government support of basic research reflects its responsibility for national defense and such broad national requirements as health and Avelfare. All these are mutually supporting. The type of survey conducted here is one means of estimating the actual volume of basic research productivity.

I t is becoming increasingly recognized that research administration is an in t eg ra l part of fundamental re­search and is critically important. Meanwhile, the United States is gaining experience in the many facets of research expansion. D a t a a r e needed t h a t will he lp t he g o v e r n m e n t in choosing and establishing incentiA-es and procedures tha t wdll further improve basic research in the United States. I t is hoped t h a t the data p r o v i d e d in this survey will be valuable as a p a r t of this effort.

I t is the opinion of the authors that it would be worth­while and practical to initiate a survey of U. S. research literature to be conducted by scientists capable of criti­cally examining and properly categorizing the research articles in their respective fields. The objective would be to develop a quanti tat ive and qualitative index of per­formance in both research and development. Such an index would be a most A^aluable guide in the expenditure of funds devoted to the support of basic research.

APPENDIX

JOURNALS INCLUDED IN SURVEY OF SCIENTIFIC LITERATURE

Physical Sciences

American Chemical Society Journal Astrophysical Journal Journal of Colloid Science Journal of Api)lied Physics Journal of Organic Chemistry Journal of Physical Chemistry Journal of Polymer Science Journal of the Acoustical Society Journal of the Electrochemical Society'' Journal of the Optical Society Physical Review Re\Tews of Alodern Physics

λΓα^ιοηιαΗ^^ί and Statistics

American Journal of iMathematics American INIathematical ^Monthly Annals of ^Mathematical Statistics Communications on Pure and Apjdied Mathematics Duke Alathematical Journal Journal of ^Mathematics and Physics Journal of the American Statistical Association Pacific Journal of jMathematics Transactions of the American Mathematical Society

General Science

American Journal of Science Franklin Institute Journal National Academy of Science Proceedings Journal of Research of the National Bureau of

Standards Science

Applied Scientific Research

Journal of Applied Mechanics Journal of the Aero/Space Sciences Journal of the Audio-Engineering Society Radiation Research Re\dew of Scientific Instruments Transactions of the American Society of Mechanical

Engineers

Earth Sciences and Oceaiiography

Annals of the Association of American Geographers Geographical Review Journal of Geology Journal of Marine Research Seismological Bulletin Journal of Atmospheric and Terrestial Physics Journal of Meteorology

1966 MILTON AND JOHNSON! SPONSORSHIP OF RESEARCH 109

Life Scieucefi

American Journal of Clinical Nutrition American Journal of Physiology Journal of Biological Chemistry Jom-nal of Comj^arativc^ and Physiological

Psychology Journal of Comparative Neurology Journal of Ex|)crimental Psychology Metabolism

Operations liesearch and Management

Econometrica Journal of Industrial Engineering Journal of the Operations Kesearch Society ]\ianagement Science

A C κ Ν ο ΝΝ' L ΕDCi λ IΕ Ν Ύ

This paper is based on a survey covering approxi­mately 116 000 articles and 700 000 pages. This cov­erage, though small as a sample of scientific literature, is vast in itself and rej^resents the work of H. H. Green and the large group of individuals who helped review the oO selected journals. Long-term team members in­cluded E. H. Huggler, ϊνΐ. G. Page, Λ'. W. McGuri, and C. B. Murphy.

The periodicals not aΛ\ailable in the Operations Re­search Office collection were scanned in the Library of Congress, the Army Library, the libraries of the Na­tional Institutes of Health, the Carnegie Institution of Washington, the National Bureau of Standards, and the National Geographic Society. Some gaps were filled by interlibrary loans from other institutions. The librarians of all these organizations were most hospitable and helpful.

This study was not published in the open literature at the time it was completed in hopes it could be appreci­ably expanded. However, it has been in such wide demand because of its basic nature tha t publication ap­pears to be highly desirable. This is accomplished here­with.

REFERENCES

HI K. A. Johnson and H. S. Milton, "A proposed cost-of-re-search index," IRE Trans, on Engineenng Management, vol. EM-8, pp. 172-176, December 1961, and cunent 1965-1966 updating.

[2] C. Zirkle, 'Our splintered learning and the status of sci-entist.s," Science, vol. 121, p. 513-519, April 15 1955.

[3] C. β . Bourne, Bibliographi/ on the Μechanization of Information Retrieval. Menlo Park, Calif.: Stanford Re­search Institute, 1958.

[4] J. H. Shera, A. Kent, and J. W. Perry, Injormation Re-.sources—.4 Challenge to American Science and industry. N e w York: Intenscience, 1958, ba.sed on proceedings of a si)ccial meeting of the Committee on Documentation Re­search, Western Reserve university, Cleveland, Ohio, Feb-ruaiy 1958.

[5.1 R. A. Fairt.horne, '•Automatic retrieval of recorded infor­mation,'' The Computer Journal, vol. 1, pp. 36-41, April 1958.

161 R. E. Cleland, "The use of material," Science, vol. 121, pp. 519-523, April 15 1955.

171 B. F. Mcissner and W. Shockley, ' O n the stati.^tics of in­dividual variations in ])roductivity in research laboratories," Proc. IRE (Correspondence), vol.* 45, pp. 1409-1410, October 1957.

[8] C. V. Kidd, "Basic research—description versus definition," Science, vol. 129, pp. 368-371, Febniary 13 1959.

[91 Nava l Research^ Advisory Committee, Dept . of Navy , A Report, to the Secretary of the Navy on Basic Research in the Navy, vols. 1 and 2, prepared by Arthur D . Little, Inc., Cambridge, Mass., 1959.

[10] J. C. Fisher, ''Basic research in industrv," Science, vol. 129, PI). 1653-1657, June 19 1959.

[11] W. S. WhitpJiead, "High cost of government secrecv," The Reporter, vol. 18, ιηχ 22-25, Janu^uy 9 1958.

[12] N . Candelaria, "Security and the echtor," Science, vol. 121, pp. 528-530, April 15 1955.

[13] L. Y. Berkner, "Security and scientific progress," Nev) Republic, vol 131, pp. 6-10, July 12 1954.

[14] D . J. de Solla Price, Little Science, Big Science. New York: Columbia University Press, 1962, pp. 8-9.

[15] Editorial. "Note on publication co.< t̂," / . Geophys. Res., vol. 58, pp. 428-429, 8e])tember 1953.

[16] M. A. Jurgens, "Research publication: a federal responsi­bility?," Science, vol. 110, pp. 209-212, August 26 1949.

[17] R. Tumbleson and Π. L. Brownson, "Survey of operations and finances of scientific journals," Science, Vol. 119, pp. 357-359, March 19 1954.

[18] M . 0. Lee, "Problems in financial management of scientific journals," Scioice, vol. 119, pp. 530-532, April 23 1954.

[19] Brig. Gen. B. G. Holzman, Cmdr., U S A F Ofiiice of Sci­entific Research, private connnunication, September 4 1959.

[20] R. Stevens, "Research builds America's future," Atlantic Monthly, vol. 200, pp. 147-150, October 1957.

[21] M . A. Tuve , "Basic research in private research institu­tions," in Symposiuni on Basic Research. Washington, D . C : American Association for the Advancement of Sci­ence, pub. 56, 1959.

[22] H. Wooster, "Basic research," Science (Letter to the Edi­tor), vol. 130, p. 126, July 17 1959.

[23] Dept . of Air Force, "Publication of researcli results in pri­vate technical and scientific journals," A F R 80-33, sec. 2, l)p. 1-6, February 3 1956.

[24] R. E. Burton, "Citations in American engineering jour­nals," Am. Document, ])p. 135-137, April 10 1959.

[25] National Science Foundation, "Basic re.^'earch, a national resource," N S F 57-35, p. 34, October 1957.

[26] , "Funds for the performance of basic research in the United States, 1953-1958," Reviews of Data on Research and Development, N F S 60^3, no. 22, pp. 1-11, August 1960.

127] W. W. Bender, "The annual report of the Martin Com­pany's Institute for advanced study, 1958," Research Insti­tute for Advanced Studies, ]>. 6.