Upload
harvey-wilson
View
221
Download
0
Tags:
Embed Size (px)
Citation preview
Anchoring and Adjustment in SQL Query Reuse
A Little Help can be a Bad Thing:
Gove Allen November 10, 2006 University of Houston
Gove AllenTulane University
Jeffrey ParsonsMemorial University of
Newfoundland
Code Reuse in Information Systems
• Code reuse (Cox 1990)
• Study of IS reuse (Frakes & Terry 1996)
• Query Reuse
• SQL is: A standard, portable across development environments, portable across DBMS, easily learned
Anchoring and Adjustment
• Tversky and Kahanman 1973
• Heuristic used to reduce cognitive burden
• Bias arises due to insufficient adjustment
• Example from Judgment
• Example from Problem Solving
Anchoring and Adjustment in Query Reuse
• A query is a candidate for reuse if it satisfies a similar information request
• Anchoring is promoted by the fact query executes and produces results (plausibility)
• Adjustment bias:
Surface level vs. Deep structure• Also consider role of domain familiarity
Hypotheses
• More errors H1
• Shorter time H2
• Higher confidence H3
• Worse prediction H4
When users reuse queries:
• Less reuse H5 • Better adjustment H6
Domain familiarity will lead to:
• Adjust surface anchors better than deep ones H7
In general, users will:
Domain Familiarity
• Difficult to study empirically
• Virtually impossible to assign subjects to different levels of domain familiarity
• Parallel database schemata
• Within subject assignment
• Implications for problem-solving in general
Gen
e O
nto
log
y D
om
ain
University
Un
iver
sity
Do
mai
n
Gen
e O
nto
log
y D
om
ain
Parallel Questions
How many times have professors employed students from Arizona (state='AZ')?
How many sponsors are the primary sponsors for courses that can be taken for credits ranging from 1 to 3 (credit_range='1-3')?
How many definitions are there for obsolete terms (is_obsolete=1)?
How many species are the primary species for gene products with a symbol of ADH2 (symbol='ADH2')?
Experimental Instrument
• Implemented as website
• Allows for presentation of example query
• Collects confidence information
• Allows dynamic query evaluation
• Automatic scoring
Question without Anchor
Question with Anchor
Subjects
• 157 Subjects
• 6 universities in United States
• Introduction to database management
Results for Anchoring
H1 More Errors P < .0001
Our sample: Half as likely to produce correct result
H2 Shorter Time P < .0001
Our sample: Less than half the time (40%)
H3 Higher Confidence P = 0.6981
Our sample: Only 0.6% difference
H4 Worse Prediction P < .0001
Our sample: More than 50% more overconfident
Results for Domain Familiarity
H5 Less Reuse P = 0.8875
Our sample: Only 0.5% difference
H6 Better Adjustment P = 0.7757
H6a Surface Anchors P < .0001
Our sample: 25% more likely for low familiarity
H6b Deep Anchors P = . 0148
Our sample: 7% more likely for high familiarity
Counter Significance
Surface vs. Deep Anchors
H7 Users will adjust surface anchors better than they will deep
anchorsP < .0001
Our sample: Subject almost three time as likely to correctly adjust surface-level anchors than deep-structure anchors
Implications• Given an opportunity, query writers will
reuse existing queries• Reusing queries can lead to higher
error rate• Reusing queries can lead to more
overconfidence• Adjustment bias is not entirely
mitigated by domain familiarity• Query writers focus on surface-level
changes when modifying queries
Questions and
Discussion