Upload
giselle-barrera
View
27
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Bayesian Networks for Sketch Understanding. Christine Alvarado MIT Student Oxygen Workshop 12 September 2003. Sketching in Design. Mechanical Engineering. Software. A Challenge In Sketch Understanding. Noisy Input. - PowerPoint PPT Presentation
Citation preview
Bayesian Networks for Sketch Understanding
Christine AlvaradoMIT Student Oxygen Workshop12 September 2003
Sketching in Design
Mechanical Engineering
Software
A Challenge In Sketch Understanding
Noisy Input
There is no one threshold for shapes or constraintsInterpretation depends on context
Naïve Approach
Why not just try all possibilities?
Naïve Approach
Why not just try all possibilities?
Arrow?
Naïve Approach
Why not just try all possibilities?
Arrow?
Naïve Approach
Why not just try all possibilities?
Arrow?
Naïve Approach
Why not just try all possibilities?
Arrow?
Naïve Approach
Why not just try all possibilities?
Arrow?
Naïve Approach
Why not just try all possibilities?
Si ik
nMust consider interpretations
n = number of strokes/segments
S = set of shapes
ki = subcomponents in
shape Si
Naïve Approach
Why not just try all possibilities?
Si ik
nMust consider interpretations
n = number of strokes/segments
S = set of shapes
ki = subcomponents in
shape Si
And this only considers shapes independently
Previous Approaches Use Rigid Segmentation
Single stroke shapes Palm Pilot Graffiti Long et. al. [1999]
Explicit Segmentation Quickset: Cohen et. al. [2001]
Pause between strokes
Recognition Using Partial Interpretations
Recognition: Build partial interpretations (PIs) as the
user draws based on easily recognizable low-level shapes
Prune unlikely PIs and use likely PIs to find misrecognized low-level shapes
Evaluating PIs Graphical Models: Missing data = unobserved nodes Interpretation influenced by top-down
and bottom-up information
BN fragments [similar to PRMs, Getoor et. al. 1999]
(Define Arrow (Components (Line shaft) (Line head1) (Line head2)) (Constraints (connects shaft.p1 head1.p1) (connects shaft.p1 head2.p1) (= head1.length head2.length) (< head1.length shaft.length) (< (angle head1 shaft) 90) (< (angle shaft head2) 90) (> (angle head1 shaft) 0) (> (angle shaft head2) 0)))
L1:L2:L3:
C1:C2:C3:C4:C5:C6:C7:C8:
Arrow
L1 L2 L3 C1 C2 C3 C8…
Instantiated and linked together as recognition proceeds[Hammond and Davis, 2003]
Primitive shapes/Constraints
Observation node added when primitive linked to stroke
P(Obs|Prim) determined through data collection
L1
Obs
Example
Sq. error(Stroke a)
Sq. error(Stroke b)
Line(l1)Connects
l1 l2
Arrow
Line(l2) Line(l3)
Quad
LineLine
RemainingArrow
Constraints
Force(F)
Force-pushes-body
Body(B)Touches F B
0.99
0.95
0.95
0.970.99
0.5
0.59
Observation
Example
Sq. error(Stroke a)
Sq. error(Stroke b)
Sq. error(Stroke c)
Line(l1)Connects
l1 l2
Arrow
Line(l2) Line(l3)
Quad
LineLine
RemainingArrow
Constraints
Force(F)
Force-pushes-body
Body(B)Touches F B
0.99
0.95
0.95
0.970.99
0.5
0.59
Observation
Example
Sq. error(Stroke a)
Sq. error(Stroke b)
Sq. error(Stroke c)
Line(l1)Connects
l1 l2
Arrow
Line(l2)Line(l3)
Quad
LineLine
RemainingArrow
Constraints
Force(F)
Force-pushes-body
Body(B)Touches F B
1
1
1
0.971
0.47
0.61
0.95
Observation
Example
Sq. error(Stroke a)
Sq. error(Stroke b)
Sq. error(Stroke c)
Line(l1)Connects
l1 l2
Arrow
Line(l2)Line(l3)
Quad
LineLine
RemainingArrow
Constraints
Force(F)
Force-pushes-body
Body(B)Touches F B
Observation
1
1
1
11
0.47
0.95
1
Sq. error(Stroke d)
Ellipse
0.99
0.970.99
Observation
Current/Future Work
Expand domain/include other domains
Gather sketches from users
Conclusion
Graphical models evaluation Partial Interpretations Context-guided search
More drawing freedom + More robust recognition =
More natural interfaces (i.e. The goal of OXYGEN)