Upload
oral
View
46
Download
0
Tags:
Embed Size (px)
DESCRIPTION
The Human Side of Statistical Consulting*. Bert Gunter Genentech April 2005. *With Apologies to Doug Zahn and James Boen, who authored a book with this title. Disclaimer. - PowerPoint PPT Presentation
Citation preview
11
Bert GunterBert GunterGenentechGenentech
April 2005April 2005
The Human Side of The Human Side of Statistical Consulting*Statistical Consulting*
*With Apologies to Doug Zahn and James Boen, who authored a book with this title.
22
DisclaimerDisclaimer
These are my personal views and These are my personal views and do not necessarily reflect the do not necessarily reflect the opinions or policies of my opinions or policies of my department, my colleagues, department, my colleagues, Genentech or your local sponsorsGenentech or your local sponsors
33
OutlineOutline
Psychology and SociologyPsychology and Sociology
StatisticsStatistics
PracticePractice
44
-ology-ology
There are There are nono statistical problems statistical problems• Only engineering, scientific, medical, Only engineering, scientific, medical,
finance, ... problems that require finance, ... problems that require statistical design and data analysisstatistical design and data analysis
• Someone else always owns the data Someone else always owns the data or is responsible for doing the or is responsible for doing the experiment or studyexperiment or study
• If you don’t like science, you shouldn’t If you don’t like science, you shouldn’t be doing statisticsbe doing statistics
55
ConsequencesConsequences
Learn more about the subject Learn more about the subject mattermatter• It is It is your your responsibility to understand responsibility to understand
what the important subject matter what the important subject matter issues are and apply appropriate issues are and apply appropriate “statistical” approaches to them “statistical” approaches to them (which could and often does mean a (which could and often does mean a good graph or two, as Bill Forrest also good graph or two, as Bill Forrest also emphasizes).emphasizes).
66
Success depends on:Success depends on:• Collaboration and communicationCollaboration and communication
To define the essential issuesTo define the essential issues
To determine what data might shed light on themTo determine what data might shed light on them
To determine useful analyses of the dataTo determine useful analyses of the data
To communicate the results of those analysesTo communicate the results of those analyses
• ImaginationImaginationBeware of the obvious solutionBeware of the obvious solution
But also beware of reinventing wheelsBut also beware of reinventing wheels
These are at least as important as mere These are at least as important as mere technical knowledgetechnical knowledge
77
The most important type of The most important type of statistical error is not I or II, but III:statistical error is not I or II, but III:• Right answer – Right answer – wrong questionwrong question
John Tukey: “An approximate answer to the right John Tukey: “An approximate answer to the right question is worth a great deal more than a precise question is worth a great deal more than a precise answer to the wrong question”answer to the wrong question”
George Box: “All models are wrong; but some are George Box: “All models are wrong; but some are useful.”useful.”
88
HR HR
Everything is personalEverything is personal• Subject matter experts/investigators Subject matter experts/investigators
rarely fully understand the statistical rarely fully understand the statistical issuesissues
• Hence, their acceptance of Hence, their acceptance of youryour methods for dealing with methods for dealing with theirtheir problem and data is based on problem and data is based on trusttrust
99
You should ...You should ...
CareCare• What is the context?What is the context?• Can we do it better than the way it is Can we do it better than the way it is
“usually” done?“usually” done?
TeachTeach• At right levelAt right level• Mostly informallyMostly informally
SellSell• the value of statistical methodsthe value of statistical methods
Share your enthusiasm. Statistics is not Share your enthusiasm. Statistics is not a spectator sport – get involved!a spectator sport – get involved!
1010
... (suggested by David Giltinan)... (suggested by David Giltinan)
Be wise – compromise!Be wise – compromise!
But if you must dig in, choose your But if you must dig in, choose your battles wiselybattles wisely
Find good people to work with – Find good people to work with – and nourish the working and nourish the working relationshiprelationship
1111
-ics-ics
AllAll problems are statistical problems are statistical• Experimental design and data Experimental design and data
analysis are part of the warp and weft analysis are part of the warp and weft of scienceof science
All experiments are designed; the All experiments are designed; the only question is whether well or only question is whether well or poorly.poorly.
1212
Consequences ... Consequences ...
You are a professional, so ...You are a professional, so ...• Be proactive in your collaborationBe proactive in your collaboration• Help the investigator Help the investigator askask the right question the right question• Emphasize importance of good designEmphasize importance of good design• Never accept data at face valueNever accept data at face value
How were they obtained?How were they obtained?
What systematic sources of variability might What systematic sources of variability might mislead?mislead?
• Never assumeNever assumerandomizationrandomization
replicationreplication– at what levels of the variability hierarchyat what levels of the variability hierarchy
relevance of past experiencerelevance of past experience
1313
In sum, In sum,
Do not undervalue what you can Do not undervalue what you can contributecontribute
Statisticians are trained to understand Statisticians are trained to understand and anticipate how variability, which is and anticipate how variability, which is an inherent part of all natural an inherent part of all natural phenomena, can affect observations of phenomena, can affect observations of reality. This gives us powerful insight that reality. This gives us powerful insight that many scientists do not possess. We need many scientists do not possess. We need to use that insight to “help catalyze the to use that insight to “help catalyze the scientific learning process.” (George Box)scientific learning process.” (George Box)
1414
But ... But ...
Do not overvalue it eitherDo not overvalue it either• Prior knowledge and experience Prior knowledge and experience
matter even if they cannot be easily matter even if they cannot be easily captured and quantifiedcaptured and quantified
• All relevant information does not All relevant information does not reside in the data at handreside in the data at hand
Frequently, very little does.Frequently, very little does.
1515
PracticePractice
Collaborators, colleagues, co-Collaborators, colleagues, co-workersworkers• Not “clients”Not “clients”• Attitude makes a differenceAttitude makes a difference
1616
Turf:Turf:
Whenever possible, meet on theirs, Whenever possible, meet on theirs, not yoursnot yours• CBWA: “Collaborating by Walking CBWA: “Collaborating by Walking
Around”Around”• Touch the equipment, meet the Touch the equipment, meet the
subjects, observe the critters, try out subjects, observe the critters, try out the product, ...the product, ...
1717
Rules to Practice ByRules to Practice By
1818
Rule 1:Rule 1:
NEVERNEVER give advice over the give advice over the phone.phone.• It will always be badIt will always be bad
1919
Rule 2Rule 2
NEVERNEVER answer when asked how answer when asked how many.many.• It is almost always the wrong questionIt is almost always the wrong question• The right question has to do with The right question has to do with
defining the goals of the experimentdefining the goals of the experimentTypically, you will have to help the Typically, you will have to help the investigator figure this out.investigator figure this out.
2020
Rule 3Rule 3
Always interrogate the Always interrogate the measurementmeasurement• Systematic sources of measurement Systematic sources of measurement
variability aboundvariability abound• Ivestigators rarely know how to Ivestigators rarely know how to
quantify themquantify themBut they often can tell you what could be But they often can tell you what could be therethere
2121
Rule 4 (suggested by Lisa Bernstein)Rule 4 (suggested by Lisa Bernstein)
Get the raw dataGet the raw data• Many instruments, vendor-supplied/user-Many instruments, vendor-supplied/user-
created software “preprocess” the data created software “preprocess” the data in ad hoc, crazy ways devised by folks in ad hoc, crazy ways devised by folks with no statistical trainingwith no statistical training
Producing irretrievable junk that no Producing irretrievable junk that no subsequent analysis can redeemsubsequent analysis can redeem
• Often difficult to get “raw data” and Often difficult to get “raw data” and difficult to deal with when you have difficult to deal with when you have themthem
e.g. imagese.g. images
2222
Rule 5Rule 5
Thermodynamics: Disorder rules Thermodynamics: Disorder rules unless you work hard to defeat it. unless you work hard to defeat it. So...So...• Take nothing for grantedTake nothing for granted• Provide explicit step by step Provide explicit step by step
instructions, data format instructions, data format specifications, ...specifications, ...
• Remember MurphyRemember Murphy
2323
Some useful things I’ve learnedSome useful things I’ve learned
Most science is about hypothesis Most science is about hypothesis generation, not hypothesis testinggeneration, not hypothesis testing
All scientists have strong priorsAll scientists have strong priors
All variation is causedAll variation is caused
Beware of the data that aren’t there Beware of the data that aren’t there (suggested by Bill Forrest)(suggested by Bill Forrest)
All replicates are not created equalAll replicates are not created equal
You can never know too much You can never know too much statisticsstatistics• But most of what you learn is sillyBut most of what you learn is silly
2424
A few more ...A few more ...
Listen more, talk lessListen more, talk lessAlways provide an “executive summary” Always provide an “executive summary” of your results in a graph or two and a of your results in a graph or two and a paragraph or so of textparagraph or so of textA little paranoia can be good thingA little paranoia can be good thingIt’s better to lead the parade than It’s better to lead the parade than sweep up after the elephants (good sweep up after the elephants (good design is more important than fancy design is more important than fancy analysis)analysis)If you don’t think it’s ethical, don’t do itIf you don’t think it’s ethical, don’t do it• Your job is to speak for the data – integrity is Your job is to speak for the data – integrity is
everything!everything!
2525
And most important ...And most important ...
HAVE FUN !HAVE FUN !• (else why do it?)(else why do it?)