media.nature.com...In some cases, it was challenging to distinguish whether an individual was ’unemployed,’ or whether their career outcome was unknown (Based on anecdotal evidence,

Methods

Obtaining and Processing Alumni Data

We identified all NIEHS postdoctoral scholars within the January 2000-December 2014 time-frame by ex-amining NIEHS administrative records, and filtering based on when individuals started and ended theirtraining fellowships. Filtering these records resulted in 891 postdoctoral alumni within the fifteen-year timeframe specified. We note that the completeness of our dataset relies on how robustly records are main-tained. Minimal clerical errors are essential to obtaining a complete list of alumni. After obtaining thislist, we identified career outcomes by searching publicly available information. It should be noted that atleast one other identifying factor besides “name” had to be present to ensure we had found the correctperson. For example, if a past NIEHS affiliation was not obvious, we searched for other identifying factorssuch as degree type and year, which was known from NIEHS records. After identifying alumni, we alsodetermined the fields in which they obtained advanced degrees, and classified these degrees according tothe taxonomy developed by the National Center for Education Statistics (NCES) for standard comparison(https://nces.ed.gov/ipeds/cipcode/). The following protocol describes the methodology weemployed for finding alumni:

1. Search LinkedIn by alumni name and obtain career outcomes if there are matching results. Other-wise, proceed to the next step.

2. Search ResearchGate by alumni names and obtain career outcomes if there are matching results.Otherwise, proceed to the next step.

3. Search PubMed by both alumni names and NIEHS mentor names, and trace forward to their latestcareer positions. If there is no matching result, proceed to the next step.

4. Use Google to search alumni names plus “NIEHS”, and/or plus “PhD” (or the appropriate degree).Filter Google results with additional alumni information as mentioned before. If there is no matchingresult, proceed to the next step.

5. If online searches failed to produce results, we examined past NIEHS ‘Board of Scientific Counselor’laboratory review packages (similar to NIH grant progress reports) which include career outcomesfor laboratory alumni.

6. When all strategies were exhausted, their career outcome is dubbed “Unknown”.

Creating a Career Outcome Taxonomy and Classifying Individuals

In 2013, we began a pilot effort to identify the careers that NIEHS alumni were engaged in. Initially, weclassified their careers using the methodology within the 2012 NIH BWF, but we ultimately decided thatreporting outcomes in this manner yielded results that were too vague. In 2014, the NIH Intramural Re-search Program requested information on alumni career outcomes from each NIH Institute. In responseto this request, we developed a three-tiered, hierarchical career outcome taxonomy that followed a pro-gression from broad to increasingly more specific—thus classifying alumni by ‘job sector,’ ‘job type’ and‘job specifics;’ our system was adopted by the NIH Division of Intramural Research in July 2014. In de-veloping our system, we began with minimal preconceptions about how to classify the data and followed a‘bottom-up’ approach to decide how to best logically parse the career outcomes that we already knew fromthe 2013 pilot study. After developing categories for these outcomes, we crafted standard definitions foreach categorical component to ensure consistency when classifying individuals. The standard definitions(see Supplementary Tables 1-3) were created ‘in parallel’ with categorizing alumni—this process was madepossible by researching their career details in order to methodically and consistently assign individuals to

Nature Biotechnology: doi:10.1038/nbt.4059

https://nces.ed.gov/ipeds/cipcode/

a group within each tier while simultaneously using the ‘career detail’ information to hone standard defi-nitions. Using this methodology provides an advantage over prior studies, which have partially relied onclassifying individuals based primarily on their job titles. Our work revealed that relying on job titles alonedoes not paint a full picture of the true nature of an individual’s work. For example, some individuals havea job title of ‘Health Scientist,’ while their day-to-day work encompasses directing a training program forscientists. In this case, classifying by the job title alone may have led to the incorrect assumption that anindividual was conducting bench research, when in fact, they were engaged in science administration andproject management. Therefore, delving beyond job titles in order to research career details helped us toconcomitantly develop a taxonomy and classify individuals within it through trial-and-error. Researchingthese details included tasks such as reading individuals’ publication abstracts, CVs (when publicly avail-able online), LinkedIn summaries, grant application summaries from NIH RePORTER, and/or examiningthe overall nature of the work being conducted by employers in order to gain a clearer understanding ofalumni careers. In many cases, we found that individuals were engaged in multiple areas—for example,conducting both applied, basic, and clinical research. In those cases, we examined both the quantity ofpapers and the position of authorship on papers in order to estimate the type of research a given alumwas most involved in. As a word on the feasibility of reproducing this effort—characterization was largelystraightforward, and only a few required extensive research to appropriately classify them. We acknowl-edge, though, that classifying individuals by hand involves making assumptions—especially regarding jobtypes and job specifics. However, now that the leg-work required to create a set of standard definitions hasbeen accomplished here, these definitions could be displayed within surveys if institutions were to chooseto ascertain career outcomes in this manner. Surveys that include defined terms would result in more ro-bust data due to the combined effect of coupling additional clarity with a reduction in assumptions.

Benefits: A key strength of this taxonomy lies in how each tier captures distinct, largely non-overlapping aspects of careers, allowing for more finely-tuned information to be determined as one moves from the ‘job sectors’ all the way to the ‘job specifics.’ In this taxonomy, we dissect not only the proportion of individuals in a particular job type (for example, tenure-track) within a given sector, but we analyze the specifics of the research (or other job function) they are performing (ex: basic, applied, clinical, etc.) within these positions. We feel there is value in ascertaining these specifics, especially given that we see dramatic, statistically significant career outcome differences associated with them.

Analysis and Visualization of Career Outcomes

To analyze alumni career data, we used open source R language (http://www.R-project.org/)and RStudio (http://www.rstudio.com/) to visualize data. Job location data were plotted as a circu-lar migration map (https://CRAN.R-project.org/package=migest) and state distribution map(https://CRAN.R-project.org/package=choroplethr). The connections between the threejob categories were plotted as a Sankey diagram using the networkD3 R package (https://CRAN.R-project.org/package=networkD3). Demographic data was plotted as diverging stacked barcharts (https://cran.r-project.org/web/packages/HH/), and the remaining plots were gen-erated with the R package ggplot2 (https://cran.r-project.org/web/packages/ggplot2/).All plots were annotated and formatted with Adobe Illustrator.

Statistical Analysis of Career Outcomes

Statistical tests were performed using open source R language. A Chi-square test was performed to com-pare career outcome differences between two groups, such as between international scholars vs. USscholars, or between NIEHS alumni vs. postdocs within the 2012 NIH BWF report. When multiple compar-isons were performed, p-values were adjusted using the Bonferroni correction.


http://www.R-project.org/

http://www.rstudio.com/

https://CRAN.R-project.org/package=migest

https://CRAN.R-project.org/package=choroplethr

https://CRAN.R-project.org/package=networkD3

https://CRAN.R-project.org/package=networkD3

https://cran.r-project.org/web/packages/HH/

https://cran.r-project.org/web/packages/ggplot2/

Supplementary Tables

Supplementary Table S1. Description of Job Sectors

Supplementary Table S2. Description of Job Types

Job Sector Category DefinitionAcademic Institution Universities, colleges, and primary or secondary education schools. Hospitals with a clear university/medical

school affiliation are also included in this category.Government Agency Includes federal, state, county, municipal, and international government.For-profit Company Commonly referred to as ’Industry’, includes any company/organization designed to make a profit.Non-profit Organization Includes organizations which do not make a profit. Research institutes and hospitals functioning outside of a

university or medical school are included within this group.Independent/Self-Employed Includes individuals unaffiliated with a primary outside organization; those owning their own company would

be included within this category. Individuals within this group are often employed as freelance writers or theyhold temporary independent contract positions.

Unknown or Undecided Either the career outcome is unknown, or the job sector could not be determined. This category also includesindividuals who are unemployed. In some cases, it was challenging to distinguish whether an individual was’unemployed,’ or whether their career outcome was unknown (Based on anecdotal evidence, we estimate that2% are ’unemployed,’ which also matches the estimates within the 2012 NIH Biomedical Workforce Report).

Job Type Category DefinitionTenure-Track Faculty Faculty-like, tenured or tenure-eligible positions primarily found within academia, government, and nonprofit

organizations, typically holding titles of ’Assistant’, ’Associate’ or ’Full Professor’ (within academia), or other titles such as ’Principal Investigator’, ’Group Leader’, or ’Principal Scientist’ (within other sectors).

Non-Tenure-Track Faculty Faculty-like positions that are ineligible for tenure. Individuals classified into this job type frequently conductindependent research or teach, mostly within the academic sector. The most common job title was ’ResearchAssistant Professor.’ Examples of other job titles included ’Assistant Professor’ or ’Assistant Research Profes-sor.’

Professional Staff Broad term describing non-faculty positions; this category includes those conducting independent work butnot supervising a large team. Examples of individuals within this category may include staff scientists workingwithin government, staff working within for-profit companies in a range of areas such as basic and appliedresearch, product development, business development, intellectual property, etc. Those teaching outside of auniversity or college (high school, for example) are classified within this group.

Support Staff Refers to those not leading independent efforts but primarily supporting the work of others. Examples ofthose included within this job type include laboratory technicians, those working within core facilities, andthose providing technical or customer support.

Management Refers to those leading or managing large teams or efforts, often with the word ’Director’ as part of their jobtitle. These positions are frequently held by those with more years of experience, who have had the chanceto rise within the ranks of their field.

Trainee Conducting additional training. Examples include those conducting additional postdoctoral training, thosepursing additional fellowships (such as the FDA Commissioner’s Fellowships), or those pursuing an additionaldegree, such as a JD or DDS.


A three-tiered, hierarchical taxonomy was developed–accompanied by definitions–to classify alumni career outcomes by job sectors (Supplementary Table S1), job types (Supplementary Table S2), and job specifics (Supplementary Table S3). Supplementary Table S4 shows job examples classified by this taxonomy.


Supplementary Table S3. Description of Job Specifics

Job Specifics Category DefinitionAdditional Postdoctoral Training Conducting an additional postdoctoral fellowship.Additional Advanced Degree* Pursuing an additional advanced degree. Examples of additional degrees pursued by our fellows include a

JD and a DDS.Primarily Teaching Individuals were defined as ’primarily teaching’ if their teaching load was greater than their research load. To

estimate load, we examined lists of classes taught alongside lists of publications. Examples of job titles in thiscategory include ’Lecturer,’ ’Instructor,’ and ’Assistant/Associate/Full Professor’.

Primarily Applied Research Conducting research involving the practical application of science to solve a specific problem. Examplesof research classified as ’applied’ in this study included the development of assays for use in screeningpharmaceutical targets, the toxicological screening of compounds to identify whether they may cause adversehealth effects, epidemiological research to determine whether particular substances may be associated withnegative health effects, etc.

Primarily Basic Research Conducting research to understand fundamental, ’basic,’ natural processes or phenomena. Examples ofresearch defined as ’basic’ include understanding how DNA replication works, understanding the biologicalpathways responsible for the immune response, etc. While individuals may have been conducting a mix ofbasic and other types of research, they were defined as ’primarily basic research’ if more than 50% of theirrecent publications reflected this definition.

Primarily Computational Research/Informatics Primarily involved in developing new computational/statistical methods for analyzing complex data to solveproblems, or specializing in the overall advancement of data analytics and visualization. Individuals within thiscategory frequently had backgrounds in mathematics, statistics, bioinformatics, or computer science. Exam-ples of job titles within this category may include ’Assistant Professor,’ ’Biostatistician,’ ’Research Scientist,’etc.

Primarily Clinical Research Conducting research aimed at developing and determining the safety and effectiveness of biologics, drugs,and devices intended for human use. We limited the definition of ’clinical research’ to only include researchthat had reached the stage of including human subjects.

Primarily Clinical Practice* Primarily involved in using established treatments to diagnose and/or care for patients. Examples includephysicians, nurses, pharmacists, veterinarians, acupuncturists, etc.

Regulatory Affairs Primarily involved in either developing or enforcing regulations (such as those working for the FDA), or ensur-ing that regulated industries comply with regulations pertaining to their business. Examples of job titles withinthis category may include terms such as ’Regulatory Scientist,’ ’Regulatory Affairs Manager,’ ’Regulatory Tox-icologist,’ etc.

Science Administration/Project Management Primarily involved in the administration/management of scientific programs or projects. This category specif-ically excludes those in grant administration, who are categorized separately within this study due to theirvolume. Examples of job titles within this category include ’Project Manager,’ ’Director of Career Develop-ment,’ ’Scientific Program Analyst,’ etc.

Intellectual Property/Licensing and Patenting* Primarily involved in patenting creations/inventions of the mind (ideas, concepts, etc.) and/or licensing use ofthese creations. Individuals in this category often have the term ’Patent Agent’ within their job title.

Consulting* Often including the term ’Consultant’ within their job title, these individuals primarily provide expert advice intheir specialty area, which is used to guide others in solving problems, developing strategic plans, providinga specific service, etc. Examples in this study include those providing toxicological advice, veterinary advice,business advice, etc. Medical Science Liaisons, because of their role as healthcare consulting professionals,were also classified within this group.

Science/Public Policy* Primarily involved at the interface between science and policy. Individuals often communicate science topolicymakers to help them make informed decisions regarding, for example, allocation of funds for a particularscientific endeavor. An example of a job title within this field includes ’Policy Analyst.’

Science Writing or Communications Primarily involved in writing/editing and improving scientific communication. Examples of job titles within thiscategory may include terms such as ’Medical Writer, ’Science Writer,’ ’Editor,’ ’Proposal Developer,’ ’Translator,’etc.

Grants Management* Primarily involved in the management of grants, whether in the initial stages of determining priority areas forfunding, or in other stages, such as grant review, or grant administration after a positive funding decision.Examples of job titles within this category may include ’Health Scientist Administrator,’ and ’Program Director.’

Business Development or Operations* Primarily involved in developing or implementing strategies for expanding business opportunities (businessdevelopment), or involved in optimizing the efficiency with which a business operates (business operations).Examples of job titles within this field may include ’Business Development Manager,’ and ’Business OperationsManager.’

Sales/Marketing* Primarily involved in marketing or selling a product or service. Examples of job titles within this category mayinclude the terms ’Account Manager,’ or ’Sales Representative.’

Technical/Customer Support Primarily involved in supporting a customer’s ability to use a product or service, or in supporting the functionsof a laboratory. Examples of job titles within this category may include ’Field Applications Specialist,’ ’LabManager,’ and ’Instrument Scientist.’

Other Involved in a field not readily specified by those above, typically in a non-science-related area. Examples ofindividuals at NIEHS within this field include a ’Textile Artist,’ ’Wine-Maker,’ etc.


* = less than 2.5% of fellows entered into each of these categories, and they are shown together and referred to as the “REST COMBINED” when visualizing outcomes.


Supplementary Table S4. Job Classification Examples

Alumni Example Job Sector Classification Job Type Classification Job Specifics ClassificationA Research Associate at Duke University Academic Institution Trainee Additional Postdoctoral TrainingA Dental Student at Case Western Reserve University Academic Institution Trainee Additional Advanced DegreeAn Instructor at Koc University Academic Institution Non Tenure-Track Faculty Primarily TeachingAn Assistant Professor at Saint Mary’s College thatteaches undergraduate courses and manages under-graduate research

Academic Institution Tenure-Track faculty Primarily Teaching

A Scientist I at KBI Biopharma who purifies proteins foruse in clinical trials

For-Profit Company Professional Staff Primarily Applied Research

An Assistant Professor at Fukushima Medical Univer-sity conducting epidemiological research

Academic Institution Tenure-Track faculty Primarily Applied Research

An Assistant Professor at the Hebrew University ofJerusalem studying the basis of genomic instability

Academic Institution Tenure-Track faculty Primarily Basic Research

A Staff Scientist at NIEHS who crystallizes proteins todetermine their substrate specificity

Government Agency Professional Staff Primarily Basic Research

An Assistant Professor of Applied Statistics at Chung-Ang University who develops new mathematical mod-els for analyzing data

Academic Institution Tenure-Track faculty Primarily Computational Re-search/Informatics

A Lead Medical Director at Novartis that is responsiblefor directing phase I-IV clinical activities for compounds

For-Profit Company Management Primarily Clinical Research

A Neurologist at Kaiser Permanente involved in patientcare

Non-profit Organization Professional Staff Primarily Clinical Practice

A Review Chief at the FDA who oversees reviewal ofnew drug applications

Government Agency Management Regulatory Affairs

An Assistant Provost of Research at the University ofMiami who creates research development programs

Academic Institution Management Science Administration/ProjectManagement

A Patent Agent at Bayer Corporation For-Profit Company Professional Staff Intellectual Property/Licensingand Patenting

A Consultant at AVOS Consulting that helps compa-nies make strategic decisions regarding which com-pounds to pursue for clinical development

For-Profit Company Professional Staff Consulting

A Policy Analyst at FASEB Non-Profit Organization Professional Staff Science/Public PolicyA Freelance Science Writer Independent/Self-Employed Professional Staff Science Writing or Communica-

tionsA Program Manager at the Department of HomelandSecurity that oversees grant portfolios

Government Agency Professional Staff Grants Management

A Director of Search & Evaluation-Business Develop-ment at Bristol Myers Squibb

For-Profit Company Management Business Development or Oper-ations

A Technical Sales Representative at STEMCELL Tech-nologies that sells products and advises customers re-garding custom cocktails and density gradient products

For-Profit Company Support Staff Sales/Marketing

A Laboratory Manager at the University of Arizona thatmaintains the daily operation of the lab and carries outtechnical laboratory support

Academic Institution Support Staff Technical/Customer Support

A Mass Spectrometry Facility Director at the Universityof Wisconsin who maintains equipment, trains individ-uals, analyzes samples and assists with grant prepara-tion

Academic Institution Non Tenure-Track Faculty Technical/Customer Support

A self-employed Textile Artist Independent/Self-Employed Professional Staff Other


Supplementary Figures

Change in Gender Distribution Over Time

Percent

year

sN

um

ber o

f Alu

mn

i

Female

Change in US/International Scholar Distribution Over Time

Percent

yea

rs

US International

2010−2014

2005−2009

2000−2004

70 60 50 40 30 20 10 0 10 20 30 40 50 60 70

282

318

291

2010−2014

2005−2009

2000−2004

70 60 50 40 30 20 10 0 10 20 30 40 50 60 70

282

318

291

Male

Nu

mb

er of A

lum

ni

Supplementary Figure S1. Demographic Changes Amongst NIEHS Alumni Upper Panel Relativepercentages of male and female NIEHS alumni were binned into five-year increments: 2000-2004; 2005-2009; 2010-2014. Within the 2000-2004 time period, the percentage of male postdoc alumni was greaterthan 60%. Since that time, the gender distribution has become almost equally balanced, and with a slightlyhigher percentage of females within the 2010-2014 time period. Lower Panel Relative percentages of U.S.and international scholars were binned into five-year increments: 2000-2004; 2005-2009; 2010-2014. Thedistribution of U.S. versus international alumni has remained relatively even over the past fifteen years, withevidence of a trend towards slightly fewer international fellows (45%) within the most recent time period(2010-2014).


ALAZ AR

CA

CO

CT

DE

FL

GA

ID

IL IN

IA

KSKY

LA

ME

MD

MAMI

MN

MS

MO

MT

NE

NV

NH

NJ

NM

NY

NC

ND

OH

OK

OR

PA RI

SC

SD

TN

TX

UT

VT

VA

WA

WV

WIWY

Number ofalumni

0

[1, 11)

[11, 20)

[20, 30)

30

294

Supplementary Figure S2. NIEHS Alumni Job Location in the United States. One-third of ALL alumniwork in North Carolina, and when only examining those working within the United States, the proportion inNorth Carolina is nearly 50%. The next highest concentration of alumni includes those in the Maryland/DCmetro area. The remaining alumni are distributed across the United States in a manner approximatelyproportional to state populations.


Other academicsector (19%)

Computation/informatics (1%)

Applied research (4%)

Basic research (17%)

Clinical

Teaching (3%)

Tenure-track faculty (29%)

research (4%)

Supplementary Figure S3. Highlight of Job Specifics within Academic Tenure-Track Positions. Of those who enter into academic tenure-track positions, 17% enter into basic research, 4% applied research, 1% in computation/informatics research, and the remainder are either teaching or in clinical research positions.


InternationalScholar(61.4%)

U.S. Scholar(38.6%)

Additional Postdoctoral Training(N=50)

U.S. Scholarworking abroad

(9.1%)

U.S. Scholarworking in U.S.

(29.5%)

International Scholarworking in U.S.

(50.0%)

International Scholarworking abroad

(11.4%)

Tenure-track Investigators (N=270)

InternationalScholar(64.5%)

U.S. Scholar(35.5%)

U.S. Scholarworking abroad

(2.9%)

U.S. Scholarworking in U.S.

(32.6%)

International Scholarworking in U.S.

(19.7%)

International Scholarworking abroad

(44.8%)

Supplementary Figure S4. A Closer Look at All Tenure-Track Positions and All Individuals En-gaged in Additional Postdoctoral Training. Left We examined the relative proportion of U.S. versus international scholars in tenure-track positions, and found that nearly two-thirds of tenure-track faculty positions are held by international scholars (center circle, light magenta). However, most hold these positions abroad (45% of all tenure-track positions held by international scholars abroad (outer circle, dark purple)). Of the U.S. scholars, most hold tenure-track positions in the United States (outer circle, dark green), with only 3% of tenure-track positions being held by U.S. scholars abroad (outer circle, light green). Right We examined everyone that was engaged in additional postdoctoral training and found that over 60% were international scholars (center circle, light magenta). Most fellows conducted additional postdoctoral training within the United States, regardless of whether they were international or U.S. (outer circle, dark magenta & dark green, respectively).


6%11%

9%19%

9%

104

38

81

35

455

China

India

Japan

South Korea

United States

0% 20% 40% 60% 80% 100%Percent

Cou

ntry

Orig

in

Legend (Degree)DVMDVM;PhDMDMD;PhDPhDPhD;JDPhD;MPHYR

Num

ber of Alum

ni

Supplementary Figure S5. Distribution of Degree Types Among NIEHS Alumni. Most NIEHS alumni possess a PhD (green). A higher percentage of alumni from Japan, China and South Korea hold MD (purple) or MD/PhD (pink) degrees.


45.7%

43.0%

11.3%

5.5%

16.9%

17.6%

1.0%

13.3%

18.0%

18.8%

2.0%

2.0%

5.1%

0.0%

***

***

Academic Researchor Teaching

Government Research

Industrial Research

Non−Science Related

Science RelatedNon−Research

Unemployed

Unknown

0% 10% 20% 30% 40% 50%Percent

Legend

NIH BiomedicalWorkforce 2012

NIEHS

Supplementary Figure S6. NIEHS Alumni Career Outcomes Compared to Career Outcomes of PhD-Holders Examined in the 2012 NIH Biomedical Workforce Report. Scholars were re-classified to match the categories in the NIH Biomedical Workforce report. For example, to compare our results with the Gov-ernment Research category of the 2012 NIH report, we included the following: 1) Job Sector: Government;2) Job Types: All; 3) Job Specifics: Primarily Basic Research, Primarily Applied Research, Primarily ClinicalResearch, Primarily Computational Research/Informatics. Similarities are apparent, with significantly moreNIEHS alumni entering into government research compared with the national average, and significantlyless entering into non-science related careers compared with those in the NIH report. (*** = p < 0.001)


Documents

media.nature.com...In some cases, it was challenging to distinguish whether an individual was ’unemployed,’ or whether their career outcome was unknown (Based on anecdotal evidence,