Low-Level Vision

Low Level Vision--outline

• Problem to be solved

• Example of one computation—lines

• How simple computations yield more complex information

Problems to be solved

Problem 1: IndeterminaciesProblem 2: The input to resolve these indeterminacies is impoverished

Indeterminacies

Many of the qualities of objects that we would like to know about trade off with other qualities.

shape/orientationreflectance/light source/shadowsize/distance

Shape/Orientation

Reflectance/Light Source/Shadow

This joke turns on the assumption that you will see a shadow, not a difference in reflectance of the object (moon) across its face.

Size/Distance

Problems

So problem 1 is that the types of information that we want trade off

with one another

Problem 2 is that the initial information the visual system has is extremely impoverished

This is the input

You end up with the #of objects, their sizes,shapes, distances,textures, motions.

How do you get from one to the other?

Researchers divide this question into two parts:Low-level vision: we assume that we can’t get much information out of this array of intensity values.There must be algorithms that summarize this info.High-level vision: taking the output of the low-level processes and transforming it to get objects & their properties.

Simple computation

35 35 35 5 5 535 35 35 5 5 535 35 35 5 5 535 35 35 5 5 535 35 35 5 5 535 35 35 5 5 5

You saw this before. . . . Can you tell what this is?

Crucial summary--find edges

An edge is a sudden discontinuity in intensity.

35 35 35 5 5 535 35 35 5 5 535 35 35 5 5 535 35 35 5 5 535 35 35 5 5 535 35 35 5 5 5

Why edges?Edges frequently correspond to the boundaries of objects; a map of edges is a good start to identifying objects.

Edges are invariant to lighting conditions.

How to find edges?

Computationallyeasy to find discontinuities

Compare means of adjacent columns, rows, diagonals

What about textures?

Why don’t you see a million objects when you see a hat with many “edges” (Herringbone pattern)?

Assess at more than one scale

Assess neighboring columns: yields five edges

Assess every three columns (i.e., take the mean) yields one edge

Assess at more than one scale

Biological evidence

Retina

Ganglion cells: center-surround

On-off can combine to form line detectors

Or an edge detector

Hubel & Wiesel’s experiments

Biological evidence

It does seem that some of the cells relatively early in the visual processing stream care about edges.

All of this was about lines.

Now how do you get distance, shapes, etc.

Shape/orientation indeterminacy

Perkin’s laws--conjunctions of lines assumed to correspond to different 3D shapes.

Perkins’ Laws

Possibly built into early visual processing.

Pop-out with perkins’ laws type angles, but not with other angles.

That’s it for lines

Focus on other assumptions

Light source/reflectance/shadow

What’s this?

Assumption 1: surfaces are uniformly colored. (That’swhy shading gives the impression of 3 dimensions. Shading is assumed to be due to hills & valleys.

Light/reflectance/shadow

Shadow: light is assumed to be coming from above.

Reflectance/Light Source/Shadow

How is constancy figured out?

Obviously, absolute constancy is not calculated

Local contrast

Assumption 3: the brightest thing around is white; the darkest thing around is black.

Distance/size

Isn’t it the case that we frequently just know the size an object should be?

This is familiar size and it’s actually not that powerful a cue.

Familiar size

When you remove the cue of height in the picture planethe person looks tiny.

Cues to distanceConvergence--not very effective

More effective are a range of cues that can be evaluatedin a picture plane, and so are often called pictorial cues.

Occlusion

Texture Gradient

Linear perspective

Height in picture plane

Atmospheric Perspective

Stereopsis

Very important cue. This is NOT a pictorial cue.

Based on the fact that the two eyes get slightly different views of the world.

StereopsisDifference between what the left eye and right eye sees is called retinal disparity.

Farther object, less difference

Stereopsis

Problem: how do you match up the views of the two retinas if objects are similar?This is called the correspondence problem.(Consider that highlights differ because of different reflectionsand there are geometric distortions due to seeing things froma different angle.)

StereopsisSolutions to the correspondence problem

1. Uniqueness constraint: an object in the left eye can be matched to only one item in the right eye.2. Epipolar line constraint: because the eyes don’t move independently in vertical dimension, there is a limited number of places that an object on the left retina can be on the right retina.

Where will the boat be in the right retina?

It can’t be just anywhere. It must be somewhere on the horizon line

More generally. . . .

Summary• Problems

– Indeterminacies– Impoverished Input

• Lines– Computations– Biology

• Solutions– Shape/orientation (Perkins’s laws)– reflectance/light source/shadow (uniform color, local

contrast)– Size/distance (familiar size, convergence, occlusion,

texture gradient, linear perspective, relative height, atmospheric perspective, stereopsis).