Upload
bunme
View
64
Download
0
Embed Size (px)
DESCRIPTION
The Productive Postdoc: Do Working Conditions Affect Outcomes?. Geoff Davis Visiting Scholar and Survey Principal Investigator Sigma Xi, The Scientific Research Society [email protected]. Improving the Postdoctoral Experience. Many calls for changes to the postdoc - PowerPoint PPT Presentation
Citation preview
The Productive Postdoc:Do Working Conditions Affect Outcomes?
Geoff Davis
Visiting Scholar and Survey Principal Investigator
Sigma Xi, The Scientific Research Society
Improving the Postdoctoral Experience
• Many calls for changes to the postdoc– National Academies, AAU, NPA, etc
• Big question: What, if anything, works?
What Works? Changes have costs (money, time)
Do benefits justify investments? What should priorities be?
What gives the biggest bang for the buck?
These are empirical questions
Our “Experiment” Postdoc administration takes place largely at the
level of the PI Tremendous variability in conditions from lab to lab Recent, limited introduction of new practices
Natural experiment Ask postdocs about their working conditions Ask about how well they are doing Find conditions associated with positive outcomes
Sigma Xi Postdoc Survey Ran a big web survey
Contacted 22,400 postdocs at 47 institutions ~40% of all postdocs in US
Overall response rate: 38%* (*See tech report for details)
Our Sponsor
The Alfred P. Sloan Foundation
Alfred P. Sloan Michael Teitelbaum
Additional Support
Werthheim Fellowship, Harvard University
Partner Organizations
National Postdoc Association
Science’s Next Wave
NBER/Sloan Scientific Workforce Group
Sketch of Our Analysis
• Create measures of inputs (working conditions, demographics, etc) and outcomes
• Build linear models to test hypothesis that inputs have an impact, gauge magnitude of impact (if any)
How Do We Determine Success?• Ideal: track people down in 10 years, see what
they are doing / have done• Problems:
– Very expensive– Takes 10 years to learn anything
• Driving via the rear view mirror
• Instead, look at immediate proxies for longitudinal data
Outcomes• What makes for a “good” experience?
• No single “best” measure– Different people want different things
• Create collection of outcome measures– Look at impact of inputs on each
Subjective Outcome Measures• Subjective success measure
– Overall satisfaction, preparation for independent research, quality of training in research / teaching / management
• Advisor relations measure– How is your advisor doing? Is s/he a mentor? How
would s/he say you are doing?
• Generate numerical scores by summing Likert scored answers
Objective Outcome Measures• Absence of Conflict/Misconduct
– Has postdoc had a conflict with advisor? Has s/he seen misconduct in the lab?
• Productivity– Rate at which papers submitted to peer
reviewed journals
Outcome Measure Distributions
Outcome Measure Details• Correlations all fairly low
– Subjective success and advisor relations ~0.45– Other pairwise correlations all < 0.2
Our Explanatory Variables• Model outcomes as function of explanatory
variables– Field of research– Institution– Basic demographic variables
• Sex• Citizenship• Minority/Majority Status• Type of degree (MD vs PhD)
– Total time as a postdoc– “Working Conditions”
“Working Conditions”• How do we measure working conditions?
• Inspiration comes from various calls for changes– Look at rate of implementation
Recommended Changes• 5 broad classes of recommended changes
– Pay people more– Fellowships rather than assistantships– Better benefits– More structured oversight– Transferable skills training
Measures of Working Conditions
• Salary measure– log(annual salary), full-time people only
• Independent Funding measure– Dummy variable, 1 if fellowship, 0 otherwise
• Benefits measure– Count of different benefits received (health
insurance, retirement plan, etc)
Structured Oversight• Structured Oversight measure
– Count of administrative measures in place• Individual development plans
• Formal reviews
• Policies (authorship / misconduct / IP / etc)
• Letters of appointment
– High values = lots of structure, low = little
Training• Transferable Skills Training measure
– Count of areas in which postdoc reports receiving training
– Grant writing, project/lab management, exposure to non-academic careers, negotiation, conflict resolution, English language, etc
– High values = training in lots of areas– Low values = no training in lots of areas
Working Conditions Distributions
Working Conditions Details• Again, correlations all fairly low
– Structured oversight and skills training ~0.30– Other pairwise correlations all < 0.15
What Has Biggest Impact?
• Who is most satisfied, most productive, etc?
• People with– Independent funding?– High salaries?– Lots of benefits?– Lots of structured oversight?– Lots of types of transferable training?
Simple Analysis• Crude analysis: compare satisfaction,
productivity, etc for people in appointments with – Fellowships / other funding– High / low salaries – High / low benefits– High / low structure– High / low training
Independent FundingFellowship Other
% satisfied 74% 70%
Advisor grade (0=F, 4=A)
3.0 3.1
% reporting conflicts
14% 14%
Papers submitted / year
1.1 1.2
Salary
Highest 25% Lowest 25%
% satisfied 71% 68%
Advisor grade (0=F, 4=A)
3.0 3.1
% reporting conflicts
16% 13%
Papers submitted / year
1.2 1.2
Benefits
Highest 25% Lowest 25%
% satisfied 76% 62%
Advisor grade (0=F, 4=A)
3.2 2.9
% reporting conflicts
11% 18%
Papers submitted / year
1.3 1.2
Structured OversightHigh structure Low structure
% satisfied 80% 60%
Advisor grade (0=F, 4=A)
3.4 2.7
% reporting conflicts
9% 21%
Papers submitted / year
1.4 1.0
Transferable Skills TrainingHigh training Low training
% satisfied 83% 56%
Advisor grade (0=F, 4=A)
3.4 2.7
% reporting conflicts
10% 17%
Papers submitted / year
1.3 1.1
Regression Coefficients Subjective
Success Advisor
Relations Absence of
Conflict Productivity
total structure 0.158 *** 0.159 *** 0.283 *** 0.045 *** total training 0.455 *** 0.247 *** 0.120 *** 0.050 *** total benefits 0.094 *** -0.000 0.125 *** -0.033 * log(salary) 0.024 0.112 *** -0.036 0.031 . funding 0.178 *** 0.048 0.131 0.015 male 0.089 ** 0.015 0.138 0.081 ** citizen 0.081 ** 0.035 0.077 -0.058 * underrepresented 0.051 0.013 0.017 -0.019 medical degree -0.178 *** -0.107 * -0.452 *** -0.032 months total -0.004 *** -0.003 *** 0.018 *** 0.001
Take Home Message #1
• Structured oversight and transferable skills training
make a big difference
Causality?• We have correlation. Is there causation?
– Psych literature gives reasons to believe in causation
• Alternative explanations1. Structure and training attract people who are
intrinsically more satisfied / productive / successful
2. Structure / training correlate with some other unobserved factor– Advisors are effective managers / have more resources– Postdocs take more initiative / are better organized / etc
Causality?• 2 classes of explanation
1. Structure/training attract intrinsically more productive people
2. Structure/training directly cause productivity or are indicators for some causal mechanism
(Some combination of 1 & 2 also possible)
• Should be able to differentiate between 1 & 2 by looking at people with multiple appointments
Intrinsic vs. Time-Localized
Causality?• Add in terms that allow for change in slope of
papers(t) curve starting at beginning of most recent postdoc
• Equivalent to adding interactions with ratio (months in current postdoc / total months as postdoc) to regression model
• Training appears to have a time-localized effect• Other inputs ambiguous
Don’t Pay Postdocs?• Not saying postdocs shouldn’t be paid!
– Hard to attract US students to science if you don’t pay them
• Maslow’s hierarchy of needs– Must meet basic physical security needs first– Living wage, basic benefits
• More nuanced interpretation of data: beyond a certain threshold, structure and training matter more than compensation
• Institutional “postdoc tax” to support service provision?
More Details
• Look at individual components of structure and training measure
• What specific measures have the greatest impact?
Impact
• One measure appears to have significant impact all 4 outcomes:– Research / career plans
• Written plans
• Plans that spell out what both postdoc and PI will do
• Advocated by FASEB, National Academies
Plans• Compare those with such a plan to those
without:– Much less likely (~40%) to be dissatisfied– Much less likely (~30%) to have conflicts
• After controlling for field, institution, demographics:– Submitted ~14% more papers for publication
Why?• Plans:
– Expectation setting device• Postdocs without plans were much more likely to report PI had
not lived up to expectations– Contract
• Research shows that people are more likely to live up to explicit (esp. written) commitments
– Forces postdocs to take responsibility for their careers early
• More time to take advantage of training opportunities– Time management device
• Mechanism for focusing effort
Take Home Message #2
• Individual development plans make a big difference
Additional Measures Several other measures show concrete
benefits: Teaching experience Exposure to non-academic careers Training in proposal writing Training in project management Training in ethics
Policy Implications For postdocs, more effective to invest
additional dollars in management than in salaries
Management at all levels: Infrastructure for institutional oversight /
training Management training for PIs Management training for postdocs
Further information More information at
http://postdoc.sigmaxi.org
Workshop (with NPA) in January 2006
Contacts Geoff Davis, PI, [email protected] Jenny Zilaro, Project Manager, [email protected]
Extra Material
End Products Sigma Xi:
Highlights in May/June issue of American Scientist Tech reports (2 out now, more to come) Scholarly paper this fall
NPA: Analyses of various topics
NBER SEWP
Workshop in January 2006
Aside: Postdoc Definition• Half a dozen different definitions
– AAMC, AAU, FASEB, NAS, NSF
• BUT if you read and compare them, they all say the same thing– Only substantive difference is that FASEB includes
narrow subset of clinical fellows
– (We excluded them from this analysis)
• Most people don’t fully satisfy definition anyway
Postdoc Definition• The appointee has a PhD or equivalent degree,• the degree was received recently,• the appointment is temporary,• the purpose of the appointment is training for a research
career,• the appointment involves substantially full-time research
or scholarship,• the appointee is expected to publish the results of his or
her research, and• the appointee works under the supervision of a senior
scholar or a department in a university or research institution.
Survey Non-Response 30-second summary of non-response
analysis: Non-citizens and African Americans appear to
be slightly under-represented No evidence of bias based on level of
satisfaction (respondents not overly disgruntled)
Survey Non-Response• Survey respondents atypical in one
important way– Participating institutions all had PDO, PDA, or
administrator interested in postdoc affairs
• Participating institutions probably better off than average
Salaries
• Median salary: $38,000
• Up from $28,000 in 1995
Inflation
• A 10% increase above inflation since 1995– ($28,000 in 1995 = $34,700 in 2004)
• NIH budget doubled over the same period(in inflation-adjusted dollars)
Experience
• Salaries increase at about 2.9% per year of experience
Field• Overall average = $39,300
• Average salary in most common fields ranges from $37,500 to $40,000
• Higher: – Electrical engineering ($45,000)– Physics ($42,600)– Oncology ($41,400)– Materials science ($41,200)
• Lower:– Ecology ($35,600)
Institution Type
• Govt labs pay 20% more than average
• Public universities pay 9% less than average
Taxes• Tax loophole: some postdocs don’t have to pay FICA
(7.65% of income)– 23% benefit– New IRS rules affect this
• Tax penalty: some postdocs pay extra self-employment tax (also 7.65% of income)– 12% pay– Independent contractor status carries hidden tax penalty!
• Potential $6,000 impact on salary
Part-time
• 3% report part-time status
• Average hours worked previous week: 45
Hours
• 51 hours/week median
• Postdoc hourly wage ~ $14.90
Hours
• 51 hours/week median
• Postdoc hourly wages = $14.90/hour
• Harvard janitors = $14.00/hour
Foreign Postdocs
• International Men and Womenof Mystery
Basic Demographics Citizenship:
Citizens: 40% Permanent residents: 6% Temporary visa holders: 54%
PhD: US PhD: 53% Non-US: 47%
Non-US PhDs Where PhD earned:
Almost 80% of postdocs on temporary visas earned their PhDs outside the US
Non-US PhDs invisible in NSF stats
All US citizens (41%)
Permanent residents (6%)
Temporary (53%)
US 53% 97% 51% 21%
Elsewhere 47% 3% 49% 79%
Non-US PhDs Where non-US PhDs were earned:
Country of citizenship 86% Different country, same continent 7% Different continent 7%
Temporary Visa HoldersCitizenship
China 24%
India 11%
Germany 6%
South Korea 6%
Japan 6%
Canada 5%
France 5%
United Kingdom 4%
Spain 3%
Italy 3%
Top 10 73%
Source of PhD
China 18%
India 10%
Japan 8%
UK 8%
Germany 8%
France 6%
Canada 5%
South Korea 4%
Israel 3%
Spain 3%
Top 10 73%
Non-US Postdocs and PhDs China and India dominate
Market share of postdocs comparable to share of doctorates (China = 23%, India = 10%)
Next largest LDC is Argentina, #16 for both citizenship and PhDs, with 1% of each
Temporary Visa Holders by FieldElectrical engineering 72%
Physics 67%
Chemistry 61%
Molecular biology 58%
Biochemistry 57%
Cell biology 57%
Earth sciences 52%
Ecology 36%
Psychology 21%
Broad Field
Temporary visas Non-US PhDs
Life/health sciences
52% 47%
Physical sciences / engineering
63% 44%
Social sciences 23% 18%
Other Characteristics
US postdocs: 49% men/51% women
69% married 33% have children Median age: 33
International postdocs: 65% men/35% women
69% married 35% have children Median age: 33
Other Characteristics One notable difference for married postdocs
US postdocs: 15% have non-working spouse Non-citizen postdocs: 44% have non-working spouse
Some visas (e.g. H) don’t have provision for spouse to work
Domestic vs International: Papers
International postdocs publish more Average peer-reviewed publications as a postdoc
Citizens/PR 2.6 Temporary 3.3 (27% more)
Difference is smaller (.1 papers/year) after we control for time as a postdoc, field, institution, sex, but statistically significant
Domestic vs International: Hours Non-citizens work longer hours Average weekly hours worked
Citizens/PR 50 Temporary 52 (4% more)
Difference is smaller (1.3 hours/week) after we control for time as a postdoc, field, institution, sex, but still statistically significant
Domestic vs International: Salary BUT non-citizens are paid substantially less Median annual salary
Domestic $40,000 International $37,000 (8% less)
Domestic postdocs earn $2,200/year more than international postdocs after controlling for field, institution, sex, time as a postdoc, and funding mechanism
Domestic vs International: Grants
Citizens write more grant proposals (results suggest mostly fellowship applications)
Grant proposals written while a postdoc Citizens 1.6 Non-citizens 1.1 (31% fewer)
International postdocs write fewer grant proposals even after controlling for field, institution, sex
Domestic vs International: Satisfaction Non-citizens report slightly lower levels of
satisfaction with the postdoc experience Average satisfaction
(-2 = dissatisfied / 2 = satisfied) Citizens/PR 0.8 Temporary 0.6
Difference disappears when one controls for salary, discipline, institution, sex, and time as a postdoc
Security Problems To what extent have US national security regulations
affected your ability to do the following:(% responding “Some” or “A lot”)
Conduct your research in the US: 30% Travel outside the US to conduct your research: 40% Visit your country of citizenship: 55% Re-enter the US after leaving the country: 57% Bring your immediate family members to the US: 36%
Free-text comments express considerable frustration
More information More information at
http://postdoc.sigmaxi.org
Contacts Geoff Davis, PI, [email protected] Jenny Zilaro, Project Manager,
Survey Responders Difficult to obtain ground truth for
assessing results Plan: compare results of pilot survey to known
values for one institution with good records Reality: survey revealed that the institution in
question was missing lots of postdocs (~10% of the local population)
Survey Responders Fortunately we found an alternative with better
records Differences in response rates consistent with
levels of variation in a random sample for Sex Citizenship Minority status
No strong evidence of non-response bias
Further Non-response Analysis Survey literature: propensity to respond is a
continuous variable Early responders: high propensity Late responders: lower Non-responders: lowest
Idea is that non-responders are more similar to late responders than early responders
Compare early and late responders. Differences suggest potential non-response bias.
Non-response Bias? Who are missing 66% of postdocs? No significant difference between early and late
responders by Sex Overall satisfaction
Significant but small difference by citizenship (p ~0.04) Early responders: ~49% citizens Late responders: ~45% citizens
Non-citizen postdocs are probably slightly underrepresented
Domestic vs International: Satisfaction Non-citizens report slightly lower levels of
satisfaction with the postdoc experience Average satisfaction
(-2 = dissatisfied / 2 = satisfied) Citizens/PR 0.8 Temporary 0.6
Difference disappears when one controls for salary, discipline, institution, sex, and time as a postdoc
Settlement Interests
Level of interest (0=None, 2=High) in settling in various regions (ignoring visa issues)*
US Europe Asia
US citizens 2.0 0.8 0.2
European citizens
1.4 1.8 0.3
Asian citizens 1.6 1.2 1.3
Settlement Interests Level of interest (0=None, 2=High) in settling in
various regions (ignoring visa issues)*
US Europe Asia
US citizen, US PhD 1.97 0.75 0.20
US citizen, non-US PhD 1.67 1.50 0.25
European citizen, US PhD 1.64 1.43 0.21
European citizen, non-US PhD 1.35 1.86 0.28
Asian citizen, US PhD 1.73 1.04 1.33
Asian citizen, non-US PhD 1.58 1.20 1.26