View
214
Download
0
Tags:
Embed Size (px)
Citation preview
1 Ellen L. Walker
Recognizing Objects in Computer Images
Ellen L. WalkerMathematical Sciences Dept
Hiram CollegeHiram, OH 44234
[email protected]://hirame.hiram.edu/~walkerel
2 Ellen L. Walker
Why Recognize Objects?
“Put the clothing on the bed”
“Turn left at the apartment building”
? ??
?
3 Ellen L. Walker
Steps in Object Recognition
BLOCK
CYLINDER
MODELS
IMAGE FEATURES GROUPS
INTERPRETATION
segmentation grouping
matchingmatching
CYLINDER
BLOCK
4 Ellen L. Walker
Formation of a 2D Image
camera
image plane
image (2D on the image plane)
scene (3D)
5 Ellen L. Walker
Images are Composed of Pixels
Image has 138 x 255 pixels (35190 total)
Pixel values range from 0 (black) to 255 (white)
6 Ellen L. Walker
Edge Detection
• Use a technique called convolution to detect edges
• The 3x3 convolution of a pixel is the sum of the products of the pixel and its 8 neighbors with a 3x3 mask
• The convolution of the shaded cell with the mask isAbs(4x-1 + 3x0 + 2x1+ 4x-2 + 3x0 + 2x2 + 4x-1 + 3x0 + 2x1)
= 8
4 3 2 2
4 3 2 2
4 3 2 2
4 3 2 2
-1 0 12 0 2
-1 0 1
Image
Mask
7 Ellen L. Walker
Edge Detection with Convolution
0 0 0 0 0 0 0 0
0 0 1 1 1 1 1 1
0 1 4 3 2 2 2 2
0 1 4 3 2 2 3 2
0 1 4 3 2 2 3 3
0 1 4 3 2 2 3 2
0 1 4 3 2 2 3 2
0 0 0 0 0 0 0 0
0 6 12 12 9 8 8 0
0 5 9 8 5 5 6 0
0 0 0 0 0 1 3 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 6 4 2 1 0 0 0
0 13 7 6 3 1 0 0
0 16 8 8 4 3 1 0
0 16 8 8 4 4 2 0
0 16 8 8 4 4 1 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 12 16 14 10 8 8 0
0 18 16 14 8 6 6 0
0 16 8 8 4 4 4 0
0 16 8 8 4 4 2 0
0 16 8 8 4 4 2 0
0 0 0 0 0 0 0 0
horiz.conv.
vert.conv.
bothconvolutions
-1 0 12 0 2
-1 0 1
-1 2 -10 0 01 2 1
8 Ellen L. Walker
Edges from Test Image
After convolution and thresholding
This is still an image …
next find endpoints of straight segments
9 Ellen L. Walker
Fitting Line Segments to Edge Pixels
For each path of connected edge pixels:
draw a segment between the endpoints of the path
find the point farthest from the line
if it’s too far, then make that point an endpoint and recurse
10 Ellen L. Walker
Segments from Test Image
Circles denote endpoints
Note “broken” segments
11 Ellen L. Walker
Grouping Collinear Line Segments
Line segments are collinear if:
angle between them is small enough ()
distance between them is small enough (PD)
“gap” between them is small enough (G)
B
PD
G
collinearity
exact
not at all0 180
12 Ellen L. Walker
Grouped Segments from Test Image
Long vertical and horizontal segments were found
13 Ellen L. Walker
Higher Levels of Grouping
Junctions
Curves & Polygons (no T-Junctions allowed)L-Junction L-Junction
Both interior angles must be inthe same direction
IPDAB
A
B
IPDE = 0AB
L-Junction
A
B
IPDEAB
IPDAB
T-Junction
14 Ellen L. Walker
Groups from the Test Image?
After grouping curves and polygons, and combining polygons that share an edge and no T-junctions
Finally, we’re ready to recognize!
15 Ellen L. Walker
Types of Models
Exact Models — Describe single objects
e.g. CAD models for factory parts
the object is needed to construct the model!
Generic Models —Describe classes of objects
Exact models with parameters (e.g length, width, radius)
Structural constraints describe how the object is constructed
Functional constraints describe how the object is used
16 Ellen L. Walker
Matching Method Depends on Model Level
Image
when objects are hard to describe (e.g. faces)
match by correlation (like convolution)
Features, Groups
good for geometric objects
match by finding correspondences between image and model features (or groups)
Complete Objects
good for classes of objects
match by applying constraints from model to objects derived from the image
17 Ellen L. Walker
Sample Object Model (Box)
Constraints Adjacent faces are perpendicular (12 constraints) Adjacent edges are perpendicular (8 constraints) Opposite faces are parallel (3 constraints) Opposite edges are parallel (6 constraints)
top
left
front
top left back
top left
left front
left back
bottom left back
bottom left front bottom right front
right front
top right front
top right back
top right
top back
bottom front
bottom left
18 Ellen L. Walker
Finding Correspondences
Search the interpretation tree
each node is a correspondence
each path to a leaf is an interpretation
“prune” the tree based on constraints
B2 B6 B8 B1 B7 B3
A1 A2 A3 A4 A5 A6
…
C3 C7
D8 D4
AB
C
DE
F
Image Corners
Box Model
1 2
3
45
6 7
8
19 Ellen L. Walker
Structural Constraints
Buildings — structural constraints
Buildings have rectangular walls and flat horizontal roofs
Each outer wall of a building intersects exactly two other walls, forming a closed ring
Each edge of the roof is the perpendicular intersection of the roof and a wall
20 Ellen L. Walker
Functional Constraints
Beds — functional constraints
A bed has a surface to sleep on that is relatively flat.
A bed has physical means of supporting the sleeping surface at a comfortable height.
A bed has clearance above and to at least one side of the sleeping surface.
21 Ellen L. Walker
Considering Occlusion in the Interpretation
Include names & parameters of objects
Include hypotheses for hidden parts when feasible
Specify only what is guaranteed!
Image Interpretation
22 Ellen L. Walker
Toward Real World Solutions
Increasingly complex models
Large number of alternatives to choose from
Uncertainty at all levels
Recognizing partially-visible objects
Recognition must be fast enough
Generating models automatically (learning!)