Upload
abraham-hodges
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
1 December 2006 SSP Core Facility
Department of Statistics
What Researchers Need to Know about the
Statistical Consulting Processor
Working with the Statistician as a
Research Colleague
Walt StroupProfessor & Chair, UNL Department of Statistics
1 December 2006 SSP Core Facility
Department of Statistics
Outline for Talk
I. Role of a Statistician
II. Statistical View of Research Process From planning to finished product
III. Essential features of an effective partnership What is expected of you; what you should expect
IV. What can go wrong & what to do about it
1 December 2006 SSP Core Facility
Department of Statistics
Cool Quote – Source Unknown
Writers tell stories with words
Painters tell stories with pictures
Musicians tell stories with music
Statisticians tell stories with numbers
HoweverThere must be a story to tellor the words, pictures, music,numbers --- and sophisticated methodology ---are “...full of sound and fury, signifying nothing...”
1 December 2006 SSP Core Facility
Department of Statistics
So...Statisticians:
The object of consulting is
The object of consultingconsulting collaborationisThe Sciencenotthe math nor the statistical techniques
1 December 2006 SSP Core Facility
Department of Statistics
Cool Quote – Source Unknown
There are three subjects everybody thinks they can be an expert in with no formal training:−Law
−Medicine
−Statistics
NOT !!!
1 December 2006 SSP Core Facility
Department of Statistics
Role of Statistics in Research
Another cool quote, source also unknown: “Think of ’Statistics’ as ‘the rules of evidence
for scientific inquiry’” Webster: “Statistics is a branch of
mathematics...”
Better: Oxford English Dictionary – “Statistics is the science of data collection, analysis, and reporting for the purposes of characterizing a population, testing a theory, or making a decision
1 December 2006 SSP Core Facility
Department of Statistics
Some Influences
My Major Professor: R.L. Anderson “This is the worst data set I ever saw! Why didn’t they talk to me first? Unbelievable!”
GeraldHahn
90% of the value of statistics is in what happens before any data are collectedWhen people see me after the fact I often feel thatthe only thing I can do is write the death certificate
“Don’t argue – take data”
“You need to learn to think of yourself not as a statistical consultant but as a research consultant”
1 December 2006 SSP Core Facility
Department of Statistics
E. O. Wilson
not the exact quote, but true to spirit:
What sets good research apart is
usually not one’s cleverness in finding
the answer but one’s skill in•asking the right question, and•phrasing it carefully and well
to which I would add:“...and collecting the right data...”
Theme?Planning!
1 December 2006 SSP Core Facility
Department of Statistics
Steps in Research Planning Protocol Description
What is the Research Question?
Relevant Evidenceto answer question(response variables
How will evidence beused to answer(hypothesis test?)
How will evidencebe collected – design?(clinical trial model? sample survey? )
How much evidence?(sample size?)
Collect data
Data analysis ConclusionsNew questions
Answer
?
1 December 2006 SSP Core Facility
Department of Statistics
When is this Planning Applicable?
Grant writing
Preparing a dissertation proposal Journal Publication Informing Policy Decisions
1 December 2006 SSP Core Facility
Department of Statistics
late Senator Daniel Patrick Moynihan
we in this nation face a choice
we can inform public policywithsound research
or
we can continue to make policy on the basis of data-free ideology
1 December 2006 SSP Core Facility
Department of Statistics
Another of “Walt’s mentors”: Dal Kratzer
defined statistics as “experimentation science” Protocol description form
Intended for pharmaceutical research & development
With word-smithing and tweaking, readily applicable as a template for research planning
1 December 2006 SSP Core Facility
Department of Statistics
Protocol Description FormWhat are the objectives – these must be translated into specific questions that can be answered with data Key: operating definitions
What is the relevant evidence? Distinguish between essential variables and auxilliary variables
What are the measurement units? This implied defining the target population & how you intends to sample / observe it
What is the treatment design?
Experiment design?
Power analysis – how many observations do I need
Murphy’s Law – what can go wrong? what will I do?
How will I analyze the data? How does this relate to objectives?
Plan for reporting? Target audience? Intended purpose
1 December 2006 SSP Core Facility
Department of Statistics
Working with Statistician
Haste makes waste Build a relationship Plan to educate statistician about your research
−Two way street
−You’re not a statistical expert; the statistician is not an expert in your discipline
“De-jargonization”
1 December 2006 SSP Core Facility
Department of Statistics
Types of statistical investigations
“Data mining”−characterization, “fishing expedition”
− looking for patterns that “might” be there
− latent class, factor analysis, other multivariate methods
Associations−“cross-tabs”, log-linear models, path analysis
−often based on survey or retrospective data
Treatment or factor effect−based on experiment design or quasi-design
−“clinical trial” model; closest thing to cause & effect
1 December 2006 SSP Core Facility
Department of Statistics
Essential Parts of the Process - I
What is the research question?− It is easy to be vague
− It is easy to be vague when you think you aren’t
−Operating definitions are essential
Is “Math in the Middle” effective?−what do you mean by “effective”?
−compared to what?
−once “effective” has an operating definition, can you measure it? how?
− if you can’t, you don’t have an operating definition yet
1 December 2006 SSP Core Facility
Department of Statistics
Essential Parts of the Process - II Power First
−what is the population?−what elements represent the population?−how do you (ethically) observe the elements?
Then−what are the relevant sources of variation among the
elements?−what is the magnitude of variation assoc with these
sources?−how big an effect is considered important?
1 December 2006 SSP Core Facility
Department of Statistics
Essential Parts of the Process - III
Power Should be a dialog At its best
−mutual learning
−dress rehearsal for data collection and analysis
INCLUDE: what can go wrong & what is my plan?−missing data happen
−missing data are inevitable
1 December 2006 SSP Core Facility
Department of Statistics
Essential Parts of the Process - IV
Stay away from jargon and “discipline-speak”−applies to you−applies to statistician
Ask questions−don’t let “stat-speak” go by− it’s not your ignorance, it’s the statistician’s language skills
Answer questions− if a good statistician asks you a question, it’s because she is
curious & trying to get herself in the best position to be helpful
−questions are not a hidden agenda to find fault or embarrass you
1 December 2006 SSP Core Facility
Department of Statistics
Essential Parts of the Process - V
If possible, face-to-face meetings are better Haste makes waste
− last minute, rush-job consulting invites trouble
Another “mentor quote”−“we don’t have time to do it right, but we have plenty of
time to do it over”
EGO
1 December 2006 SSP Core Facility
Department of Statistics
Analysis
Vastly easier in well-designed study Vastly better if statistician has been involved all
along Parsimony
− if two analyses show essentially same results and plausibly satisfy assumptions, the simpler analysis is better
−90% rule
However− just because it is traditional does not make it right
−computer has had huge impact on statistical methods
1 December 2006 SSP Core Facility
Department of Statistics
Analysis & Reporting Part of statistician’s job is to help write report
−material & methods: statistical design analysis
− result and discussion help explain and interpret
Statistical science itself is a work in progress−your project may motivate statistical research
−methods may be unfamiliar to reviewers
−statistician may need to educate referees / editors
1 December 2006 SSP Core Facility
Department of Statistics
Etiquette & Ethics Authorship
−would research / grant have been possible without statistician’s efforts?
−NO means a co-authorship should be offered
Funding−external funding is lifeblood of research university
−statistician’s dilemma in academia:
−“do I close my door to colleagues in allied disciplines to write my own grants, or do I participate in interdisciplinary teams?”
− for latter, co-PI grant arrangements with real $$$ essential
1 December 2006 SSP Core Facility
Department of Statistics
More Etiquette & Ethics Special Note on Graduate (Dissertation) Research
Common Unfortunate Situation: Graduate Student as Ping-Pong Ball
When statistician advises graduate student on design and/or analysis (ideally, “and” – ed.) student, student’s major professor, and statistician should be on same page
Ideal – statistician on PhD committee But at least – meet face to face periodically
1 December 2006 SSP Core Facility
Department of Statistics
Final Thought
Most Statistics departments are seriously understaffed – UNL no exception
Not a complaint – just a reality Conflicting demands
− teaching
−research
−consulting
Applied statistics need interdisciplinary work – just as quality research needs engaged statistics
BUT – you may have to be a nag