Tales from the Trenches - OR –
Replicating things that have already been said at this meeting
Robert R. Hoffman, Ph.D.
© 2011 Robert R. Hoffman All rights reserved
“Data Coding, Analysis, Archiving, and Sharing for Open Collaboration: From OpenSHAPA to Open Data Sharing,”
© 2011 Robert R. Hoffman All rights reserved
Experimental Psychology (Cognitive, Psycholinguistics)
U of Cinti, U of Minn
Expertise Studies Cognitive Systems Engineering
Human Factors
Me
© 2011 Robert R. Hoffman All rights reserved
Klein et al 2002
IEEE Intelligent Systems
Mental simulation Problem detection
Coordination Re-planning
Expertise development
(2 seconds ?)
Cacciabue & Hollnagel 1995
Macrocognition as a paradigm in
Cognitive Systems Engineering
Gunnar Johansson 1980
© 2011 Robert R. Hoffman All rights reserved
© 2011 Robert R. Hoffman All rights reserved
Communities of Practice
“Behavior” Versus Activity
Activity Theory
Work Analysis
Sociotechnics
© 2011 Robert R. Hoffman All rights reserved
Ancient History
“How fast was the car going when it
(bumped, crashed) into the other car?”
Barbara Tversky, et al.
© 2011 Robert R. Hoffman All rights reserved
That was then, this is….
then. . . . . .
Myth of the Normal Curve
Examples: Sampling under a stopping rule Traffic delays (lots of brief ones, rare long ones) Achieving progressive criteria in pole vaulting Errors in motor coordination tasks
Patil, G. P. (1960). On the evaluation of the negative binomial distribution with examples. Technometrics, 2, 501-505.
Sichel, H. S. (1951). The estimation of parameters of a negative binomial distribution with special reference to psychological data. Psychometrika, 16, 107-127.
© 2011 Robert R. Hoffman All rights reserved
© 2011 Robert R. Hoffman All rights reserved
"Call For Data"
• Usability/Learnability analysis
• Performance at the very first trials
of learning any task; any DVs
• Exact modeling of discrete
non-Gaussian distributions
Learning geometrical patterns Learning to use a cell phone by the elderly Learning to operate an automotive GPS (route-finding) Learning to recognize voices in auditory localization Learning to control an avatar in a virtual world Learning of the structure of biological categories Learning to fly a cockpit simulator
9 data sets on hand, 9 more pending
© 2011 Robert R. Hoffman All rights reserved
© 2011 Robert R. Hoffman All rights reserved
?
© 2011 Robert R. Hoffman All rights reserved
?
© 2011 Robert R. Hoffman All rights reserved
?
© 2011 Robert R. Hoffman All rights reserved
?
Retabbing
Fonts
Column widths
Etc.
Rule #1
Clean-up is always necessary
© 2011 Robert R. Hoffman All rights reserved
What does "No OT" mean?
What did you really do?
Did I fix the tab delineations correctly?
Rule #2
You always have to go back and talk to the researcher
Worse. . . .
People forget things about their own data, even short-term
© 2011 Robert R. Hoffman All rights reserved
How do you cope with the consequences of these Rules?
Do you impose constraints?
- OR -
Do you acknowledge that clean-up will always be necessary, and figure out ways to make it easier.
The Control Challenge
© 2011 Robert R. Hoffman All rights reserved
© Robert R. Hoffman All rights reserved
Requirements v. "Desirements"
("help" versus "impose")
Designing for kluges and work-arounds
Hoffman, R. R. & Elm, W. C. (2006, January/February). HCC implications for the procurement process. IEEE: Intelligent Systems, pp. 74-81.
Koopman, P. & Hoffman, R. R., (November/December 2003). Work-Arounds, Make-Work, and Kludges. IEEE: Intelligent Systems, pp. 70-75.
© 2011 Robert R. Hoffman All rights reserved
Challenge #1
Finding data by data constraint (e.g., data type,
meaning)
Challenge #2
Finding data by design constraint (e.g., between v. within, etc.)
The Search Challenges
© 2011 Robert R. Hoffman All rights reserved
Challenge #3
Frame problems (a priori categories v. search categories)
Challenge #4
Practicalities of representation - cryptograms, acronyms, abbreviations
The Representation Challenges
© 2011 Robert R. Hoffman All rights reserved
Challenge #5
Chiding resarchers re: design limitations, methodological questions
Challenge #6
Topical research data v. Statistics itself as an area of research
The Purpose Challenges
© 2011 Robert R. Hoffman All rights reserved
Usefulness
Usability
Understandability
Learnability
Having seen all the Tools & Systems. . .
© 2011 Robert R. Hoffman All rights reserved
These features are measurable
e.g., Data on Researcher time, effort, resources
“labor intensive” “efficient data mining”
© 2011 Robert R. Hoffman All rights reserved
www.ihmc.us/users/rhoffman/main www.ihmc.us
Thank you!
© 2011 Robert R. Hoffman All rights reserved