Smart Camera as Embedded System

ADAMS ENGINEERING COLLEGE; M.Tech VLSI; 12651D5710

Smart Cameras as Embedded Systems

B. SwarnalathaM.Tech VLSI12651D5710

Adams Engineering CollegePaloncha.


Image

Video

Digital Camera & Smart Camera

Detection and recognition algorithms

Low level processing

High level processing

Towards and Embedded System

Requirements

Components

Core Processor TriMedia

Advancements in VLSI required to implement

Embedded Smart camera

Overview of Seminar


IMAGE

10 12 35 54 34 23 201 2 10 124 5 6 7 8 9 9 9 0 087 6 8 0 7 68 8 9 09 65 87 88 7 9 9 8 8 8 88 8 8 8 8 89 9 90 0 000 5 54 4 55 6 76 7 43 65 7 7 89 7 6 6 8 997 6 6 6 78 9 166 6 77 64 44 4 5 55 43 2 54 87 545 6 54 67 45 7 3 7 98 54

An image may be defined as a two-dimensional function f(x, y) where x and y are spatial (plane) coordinates, and the amplitude of f at any pair of coordinates (x, y) is called the intensity or gray level of the image at that point. A small region in the image is shown in matrix.


VIDEO

Video signal is a representation of a scene with time as a third parameter.

V = (x, y, t)

• The parameters of a video signal are:

• Frame size: 560x480 & frame rate: 30 fps

• Video signal format:Analog & Digital

Video capture devices are Sony Canon samsung


Digital Camera and Smart Camera

Digital Cameras capture only images as digital files that users can upload to their Computer , manipulate with software and distribute electronically. Eg. Nikon

Smart Cameras capture high-level descriptions of the scene and analyze

What they see.

These devices could support a wide variety of applications including

human and animal detection,

surveillance,

motion analysis, and

facial identification

Eg. Windows for KINECT Digital CameraSMART Digital Camera


Detection and Recognition Algorithms

Low Level Processing

• Region Extraction

• Contour Following

• Ellipse fitting

• Graph Matching

High Level Processing

• Hidden Markov Models

• Classifiers


DETECTION AND RECOGNITION

ALGORITHMS

Low level processingRegion Extraction

The subject to be identified for gesture recognition in the video frames are extracted. Fig2. (b)•Contour following

Grouping of pixels into contours that geometrically define the regions. Fig2. (c)•Ellipse fitting

To correct for deformations in image processing caused by clothing, objects in the frame, or some body parts blocking others, an algorithm fits ellipses to the pixel regions to provide simplified part attributes. Fig2. (d)•Graph matching

Meaningful feature vectors are extracted from the modeled body parts with ellipses. Figure2


High level Processing

The high level processing component , which can be adopted to different

Applications, compares the motion pattern of each body part.

Example, a pointing gesture could be recognized as a command

to “go to the next slide” in a smart meeting room

or “open the window” in a smart car ,

Where as a smart security camera might interpret the

gesture as suspicious or threatening.


Human Detection and Activity Recognition Algorithm


Towards an Embedded Smart Camera

RequirementsFrame rate: The embedded smart camera system must process a certain amount of frames per second to properly analyze motion and provide useful resultsLatency: The amount of time takes to produce a result from the processed frames

Components•100 MHz Philips TriMedia TM-1000

as video processor.•Hi8 Cameras•Shared memory interface•Host computer•Debugging algorithms and

programs•Networked system for connecting

multiple cameras

Smart Meeting Room


TriMediaProcessor TM-1000

The Philips TriMedia TM1000 family of devices have a higher performance Very Long Instruction word (VLIW ) core.


TriMediaProcessor TM-1000

•a 5-issue VLIW architecture with a 32-bit word size;

•27 functional units, offering a choice of operation types

in each slot in the instruction;

•any operation can be guarded to provide conditional

execution without branching;

•instruction set and functional units optimized with

respect to media processing;

•a single multi-ported register file with bypass network,

allowing 1-cycle latency operations;

•32 kB, 8-way instruction cache;

•16kB, 8-way, quasi-dual ported, data cache;

•a variable-length (compressed) instruction set design

Features


Advancements in VLSI required to implement a smart camera embedded system

Algorithmic challengesChoosing best computationally efficient algorithm with minimum memory usage.

Library FunctionsThe library function of the processor should provide special function to provide Instruction level-parallelism

Eg1. INONZERO: This instruction takes two input operands. If the first is non zero, the destination is set to the value of the second operand; otherwise it is set to zero.

Eg2. IABS: This instruction can provide absolute values


Advancements in VLSI required to implement a smart camera embedded system

Control-to-data transformationThe data transformation can be controlled using a processor with more functional units.

Designing SINGLE-INSTRUCTION MULTIPLE-DATA (SIMD) Processors to achieve real time performance.

Existing processors like (80xx, INTEL ) are pixel plane processors. These processor can perform arithmetic and logical operations with limited no of operands.

Embedding single-instruction multiple-data (SIMD) processors into sensors is critical to improve the real time performance.

Processor architectures are to be designed to take multiple data in one instruction to increase the computation power required to meet the real time challenges.


Reference: Wayne Wolf, Burak Ozer, Tiehan Lv “Smart Camera as EmbeddedSystems”. IEEE Trans. Vol. 45, 2002.

Copy right IEEE


THANK YOU

Education

Smart Camera as Embedded System