Embed Size (px)
Citation preview
Dr Max L. Wilson [email protected]
RepliCHIHow to tackle replication of HCI research.
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
Thinking forward.
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
Why discuss replication?• Replication is a corner-stone of good science
• We should facilitate it - but we do not (do we?)
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
We’re not all scientists tho!
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
Why discuss replication?• We live in a global community - but run studies locally
• Does all research apply cross-culturally?
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
Why discuss replication?• Technology moves fast - duh.
• Do all findings still apply?
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
But its a problem
• Data, software, andpeople might beproprietary
• Minor differences in study methods or tools might create an effect
• Methods and tools are hard to reproduce exactly
• There may be no benefit / outcome
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
But its a problem
• Can you imaging publishing a paper thats not novel work?
• Can you imagine getting a review saying: incremental
• Novelty and significance are key review criteria
• TOCHI “publishes only original and significant research papers.”
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
It’s a bit scary too
• Can create conflict, and doubt
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
We should talk about it
• So we did - we are - and hopefully we will
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
The 2011 panel
• We had an excellent audience and lots of questions
• but not enough discussion
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
- Most Importantly -
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
TodayCHI2011
Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
• We want to talk to you more
• And we’ve got some important people to listen
• CHI2012 Program co-chair
• CHI2013 Conference Chair
• TOCHI Information Director
• CHI2014 Conference co-chair
• SIGCHI VP Publications
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
The future?CHI2011
Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
• A route to publication - extended abstract
• A venue to talk about replication - openly
• Motivation / Reason / Reward for replicating research
• A mature community that checks / validates itself
• A place where people look to for updates
RepliCHI
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
RepliCHI
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
• Invites Extended Abstracts
• About the attempt (more so than the research)
• Submitted around time of alt.chi
• Presented discussed in a whole day event (like DocCon)
• Some portion presenting in the main proceedings
RepliCHI
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
• Invites Extended Abstracts
• About the attempt (more so than the research)
• Submitted around time of alt.chi
• Presented discussed in a whole day event (like DocCon)
• Some portion presenting in the main proceedings
RepliCHI
Now: we break/improve the idea
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
Questions?CHI2011
Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
Tuesday, 29 May 12
Dr Max L. Wilson [email protected]
Today then
• ~10 mins - an intro
• ~10 mins - the questions
• ~30 mins - in working groups
• ~20 mins - feedback/discussion
• ~10 mins - wrap up
CHI2011 Panel
CHI2012 SIG
CHI2013 ?
CHI2014 ?
CHI2015 ?
Tuesday, 29 May 12
