04/04/2007 1

Image Understanding Architecture: Exploiting Potential Parallelism in

Machine Vision

04/04/2007 2

Overview

• Heterogeneous parallel processor– Three distinct layers

• Meet the real-time computational requirements of computer vision systems

• Exploit the various forms of parallelism within a computer vision algorithm suite– Data parallelism– Control parallelism

• Collaborative effort– University of Massachusetts, Amhurst (UMASS)– Hughes Research Laboratory

• Circa 1980’s– Part of the DARPA image understanding research initiative

04/04/2007 3

Background

• What is computer vision?• Process images (or streams of images) with the

intent of – Object recognition– Vehicle guidance– Manufacturing– etc.

• Create a system that extracts information (not just data) from a picture

04/04/2007 4

An Image(as we see it)

04/04/2007 5

Mechanization of Processing

04/04/2007 6

156 145 137 146 163 151

150 173 171 177 174 171

104 132 133 142 164 165

86 116 164 162 155 152

95 121 150 137 123 136

123 128 141 129 126 129

An Image(as the computer sees it)

177 167 153 141 145 126

144 153 160 172 173 158

116 138 138 140 152 151

65 96 131 140 148 145

71 98 147 127 120 131

99 105 144 116 117 123

238 237 234 227 223 216

229 227 224 220 225 221

205 212 221 220 225 220

177 192 213 207 212 217

164 180 211 208 209 215

190 194 220 212 210 219

04/04/2007 7

What gets processed?

• Gradients (edges) • Color• Transformations

– Rotation– Translation– Stretching

• Texture • Shading• Shape• Context

All of these are based on searching for patterns in the image colors

04/04/2007 8

How should a computer process visual scenes?

• As it turns out, mimicking a biological system in circuits and software is extremely difficult– Cameras are not as sophisticated as the eye– Processors/software are not as sophisticated as the

brain

04/04/2007 9

How should a computer process visual scenes?

• Preprocessing– Image conditioning (number crunching)

• Low Level Vision– Feature extraction (number crunching)

• Mid Level Vision– Feature description (symbolic processing)

• High Level Vision– Object recognition (advanced data structures)

04/04/2007 10

Three distinct “levels”

• Data intensive– Preprocessing– Low Level Vision

• Semi-data intensive, semi-control intensive– Mid Level Vision

• Control intensive– High Level Vision

04/04/2007 11

How do we process visual scenes?

• Preprocessing– Image conditioning

• Low Level Vision– Feature extraction

• Mid Level Vision– Feature description

• High Level Vision– Object recognition

R1

R2

R3

G1 Intersects(70)

Intersects(70)

Intersects(40)

Bounds

Bounds

Bounds

04/04/2007 12

Basic Philosophy

• Create a computer comprised of three architectures, each suited to one of the levels of a computer vision application

• Heterogeneous architecture– Multiple types of processing elements– Multiple interconnect topologies– Multiple programming languages

04/04/2007 13

The Architecture

04/04/2007 14

The Architecture

• Content Addressable Array Parallel Processor (CAAPP)– SIMD– Configurable into separate groups

• Intermediate Communication Associative Processor (ICAP)– MIMD– SPMD (Single Program Multiple Data)

• All PEs have the same program but each has it’s own program counter (Asynchronous SIMD)

• Symbolic Processing Array (SPA)– MIMD

04/04/2007 15

CAAPP

• Bit-serial processors• ALU• 320 bits of cache• 32KBits of main memory• Instructions come “from above”

– Array Control Unit (ACU)

• Communication– Configurable mesh – coterie network– A “coterie” is a group of PEs that work [somewhat]

independently (still only 1 instruction stream)

04/04/2007 16

ICAP

• Digital Signal Processor (DSP)– Specialty architecture for performing numerical

transforms

• 320 bits of cache

• 256KBytes of main memory (128K program, 128K data)

• Communication– Cross-bar switch

04/04/2007 17

SPA

• Not fully specified– Commercially available multi-processor– Networked workstations

04/04/2007 18

CAAPP to ICAP Communication

• One ICAP PE is responsible for (communicates with) 64 CAAPP PEs (8 x 8 mesh)

• Communication is via a dual-port [shared] memory

04/04/2007 19

ICAP to SPA Communication

• One SPA PE is responsible for (communicates with) 64 ICAP PEs (8 x 8 mesh)

• Communication is via a dual-port [shared] memory

04/04/2007 20

Full System Specification

• 64 SPA PEs– MIMD– RISC processing architecture

• 4K ICAP PEs (64 x 64)– MIMD/SPMD– Digital Signal Processor (DSP)

• 256K CAAPP PEs (512 x 512)– SIMD– 1-bit processing architecture

04/04/2007 21

1st Generation

• Proof of concept– 4096 CAAPP processors (64 x 64)– 64 ICAP processors– 1 SPA processor

CAAPP chip

ICAP board

System chasis

04/04/2007 22

2nd Generation

• 1/16th of a full scale system– 16K CAAPP processors– 64 ICAP processors

• Commercial chips – TI TMS320C40 32-bit processor

• Communication is token-ring over 2 x 2 meshes plus inter-processor DMA channels

– 4 SPA processors• Networked workstations

04/04/2007 23

2nd Generation

CAAPP chip

ICAP board

ICAP communcationtopology

04/04/2007 24

Programming

• Required writing separate code for each level• In the beginning there were 3 different

programming languages involved– Forth, C, Assembly

• Plan was to move to C/C++/Lisp with parallel extensions (class libraries)– Ada was planned

• Goal was to develop a single language compiler for the entire system

04/04/2007 25

Interesting Bits

• This group actually started from the problem specification and set out to build an architecture to support it– Contrary to other parallel processor developments

of the time

• Programming proved very difficult– Require intimate knowledge of architecture

(especially the coterie network) and algorithms– A simulator of the full system existed for program

development, research

04/04/2007 26

End Notes

• Emphasis was on– Proof of concept– Mapping algorithms to the architecture– Fabricating chips

• Cancelled 1995– Various chips/board fabricated and tested– Various software components developed and

tested