Course outline (rough): 1. Image Processing Algorithms 2. …kolesnik.leute.server.de/lectures/Lecture1.pdf · signal is broken down into a set of picture elements (pixels). Quantization–

Lecture 1Lecture 1

Video Data Analysis, B-IT Marina Kolesnik http://www.fit.fraunhofer.de/~kolesnik/

Course outline (rough):Course outline (rough):1.1.

Image Processing AlgorithmsImage Processing Algorithms

2.2.

ApplicationsApplications3.3.

Motion tracking in Video Motion tracking in Video SequencesSequences

4.4.

Geometry of Image FormationGeometry of Image Formation

Video Data Analysis Video Data Analysis

Marina KolesnikFhG Institute for Applied Information Technologies

Office: C5-228

Lecture 1Lecture 1


Video Data AnalysisVideo Data Analysis

1.1.

Digitized images and their propertiesDigitized images and their properties2.2.

Image processing operatorsImage processing operators

3.3.

Image transformsImage transforms4.4.

Image filtering in frequency domainImage filtering in frequency domain

5.5.

Image segmentationImage segmentation6.6.

Image matchingImage matching

7.7.

Classification using neural networks, HMM, active learning Classification using neural networks, HMM, active learning 8.8.

Optical flowOptical flow

9.9.

Motion trackingMotion tracking10.10.

Applications, etcApplications, etc……

11.11.

Image geometry, camera modelingImage geometry, camera modeling

Lecture 1Lecture 1


Video Data Analysis: Video Data Analysis: ExercisesExercises

Solving of problemsSolving of problems

Designing of algorithmsDesigning of algorithms

Formal implementation of algorithmic stepsFormal implementation of algorithmic steps

etcetc....

Lecture 1Lecture 1


LiteratureLiterature

Reading:Reading:

1.1.

Digital Image Processing. Rafael C. Gonzalez and Richard E. WoodDigital Image Processing. Rafael C. Gonzalez and Richard E. Woods. s. ISBN: 0ISBN: 0--201201--

5080350803--6, 1993.6, 1993.

2.2.

Image Processing, Analysis, and Machine Vision. Milan Image Processing, Analysis, and Machine Vision. Milan SonkaSonka, Vaclav Hlavac, Roger , Vaclav Hlavac, Roger Boyle. ISBN: 0Boyle. ISBN: 0--534534--9539395393--X, 1999.X, 1999.

3.3.

Handbook of image processing operators. Handbook of image processing operators. Reinhard Klette and Piero Reinhard Klette and Piero ZamperoniZamperoni. ISBN: . ISBN: 04710471--956422, 1996.956422, 1996.

4.4.

Machine Learning and Statistical Modelling Approaches to Image RMachine Learning and Statistical Modelling Approaches to Image Retrieval. etrieval. YixinYixin

Chen Li, Chen Li, JiaJia Li, James Z. Wang. ISBN: 1Li, James Z. Wang. ISBN: 1--40204020--80348034--4, 2004. 4, 2004.

5.5.

Introductory techniques for 3Introductory techniques for 3--D Computer Vision. D Computer Vision. Emanuele Emanuele TruccoTrucco, Alessandro , Alessandro VerriVerri. ISBN: 0. ISBN: 0--1313--261108261108--2. 1998.2. 1998.

Web Site: http://www.fit.fraunhofer.de/~kolesnik/lectures.html

Lecture 1Lecture 1


What is Video Data What is Video Data

Analysis? Analysis? --

An Historical IntroductionAn Historical Introduction

What is the actual problem?What is the actual problem?

--

Start: Enhancing and correcting of NASA spacecraft imagery. LunaStart: Enhancing and correcting of NASA spacecraft imagery. Lunar mission. r mission. --

Remote sensing Remote sensing ––

enhancement and analysis of aircraft imagery of Earth surface. enhancement and analysis of aircraft imagery of Earth surface. -- Artificial Intelligence & Visual IntelligenceArtificial Intelligence & Visual Intelligence

Lecture 1Lecture 1




Examples of ApplicationsExamples of Applications

Categorization of images according to their source (electromagneCategorization of images according to their source (electromagnetic spectrum)tic spectrum)helps systematizing image processing applicationshelps systematizing image processing applications

Reading: 1:1.3

Lecture 1Lecture 1





A long, long list (not full!):A long, long list (not full!):

GammaGamma--Ray imaging in medicine:Ray imaging in medicine:

-

Patient is injected with a radioactive isotope;

-

Emission is registered by a gamma- ray detector;

Helps identifying bone pathology such Helps identifying bone pathology such as infections or as infections or tumorstumors..

Reading: 1:1.3

Bone scan (left) and Positron Emission Tomography (PET) image (right).

Lecture 1Lecture 1



--

XX--Ray imaging in medicine, Ray imaging in medicine, industrial manufacturing, and industrial manufacturing, and astronomyastronomy

Anticlockwise from top left:- Chest X-ray.- Aortic angiogram.- Head CT.- A star in the constellation of Cygnus.- Electronic circuit board .

Reading: 1:1.3




Lecture 1Lecture 1



--

Imaging in the Visible in Infrared Imaging in the Visible in Infrared Bands in light microscopyBands in light microscopy

Reading: 1:1.3




Lecture 1Lecture 1



--

Imaging in the Visible in Infrared Imaging in the Visible in Infrared Bands in remote sensing which Bands in remote sensing which includes several bands.includes several bands.

Reading: 1:1.3




Lecture 1Lecture 1



Imaging in the Visible in Infrared Bands in remote sensing whichImaging in the Visible in Infrared Bands in remote sensing which

includes several bands.includes several bands.

Reading: 1:1.3




Lecture 1Lecture 1



Imaging in the Visible in Infrared Bands in remote sensing whichImaging in the Visible in Infrared Bands in remote sensing which

includes several bands.includes several bands.

Reading: 1:1.3




Lecture 1Lecture 1



Imaging in the Infrared Band: Imaging in the Infrared Band: Night Time Lights of the WorldNight Time Lights of the World

Reading: 1:1.3




Lecture 1Lecture 1



Quality control:Quality control:

Automated visual Automated visual inspection of inspection of manufactured goodsmanufactured goods

Reading: 1:1.3




Lecture 1Lecture 1



More application in the More application in the visual spectrum:visual spectrum:

--

matching of thumb printmatching of thumb print--automated money countingautomated money counting--

Automated license plate Automated license plate readingreading

Reading: 1:1.3




Lecture 1Lecture 1


Reading: 1:1.3



Examples of ApplicationsExamples of ApplicationsA long, long list A long, long list (not full!):(not full!):

More application More application in the visual in the visual spectrum:spectrum:

--DetectingDetecting//ReadingReading

BarcodesBarcodes

Lecture 1Lecture 1


Reading: 1:1.3



Examples of ApplicationsExamples of ApplicationsA long, long list A long, long list (not full!):(not full!):

More application More application in the visual in the visual spectrum:spectrum:

--DetectingDetecting//ReadingReading

BarcodesBarcodes

Lecture 1Lecture 1



Imaging in the radio band Imaging in the radio band –– MRI images in medicineMRI images in medicine

Reading: 1:1.3




Lecture 1Lecture 1



DNA chips DNA chips ––

reshaping the reshaping the molecular biology:molecular biology:

--

probes are tagged by probes are tagged by fluorescent molecules;fluorescent molecules;

--

probes are stimulated by laser;probes are stimulated by laser;

--

the emitted light is captured by the emitted light is captured by a detector.a detector.

Two applications:Two applications:

1. Identification of gene 1. Identification of gene sequences.sequences.

2. Determination of expression 2. Determination of expression level (abundance) of geneslevel (abundance) of genes

Reading: 1:1.3

The intensity and color of each spot encode information on a specific gene from the tested sample.




Lecture 1Lecture 1



Cell segmentation:Cell segmentation:

-- probes are tagged by a dice;probes are tagged by a dice;

--

magnified probe images are magnified probe images are acquired using a microscope;acquired using a microscope;

Applications:Applications:

-- Detection of viral diseases.Detection of viral diseases.

Reading: 1:1.3

Cells absorbed red dice indicate the presence of a viral infection




Lecture 1Lecture 1



Cell segmentation:Cell segmentation:

--

neural tissue is marked with neural tissue is marked with several dices;several dices;

--

magnified probe images are magnified probe images are acquired using a microscope;acquired using a microscope;

Applications:Applications:

-- Development of new drugs.Development of new drugs.

Reading: 1:1.3

Task: count different cell / neuron types.




Tasks:- number of neurons (blue)- length, width and an extent of bifurcations of axons (green)

Lecture 1Lecture 1


List of contemporary application List of contemporary application domains (not full!):domains (not full!):

-- Biological research

- Defense/Intelligence

- Document Processing

- Industrial Visual Inspection

- Law Enforcement Forensics

- Material Research

- Medical Diagnostic Imaging

- Computer-Assisted Surgery

- Photography

- Publishing/Prepress

Reading: 1:1.3

Continuation:Continuation:

-- Remote Sensing/Earth Resources Remote Sensing/Earth Resources

- Space Exploration/Astronomy

- Document Processing

- Video/Film Special Effects

- Video Archiving/Transmission

- Security/Real Time Monitoring

- Advertisement/Augmented Reality

- Content-based image retrieval

-

Security, biometrical imaging: iris, fingerprints, genetic code;




Lecture 1Lecture 1


Formation of Digital Video DataFormation of Digital Video DataReading: 1: 2.3.3

Lecture 1Lecture 1


Reading: 1: 2.4.1

ChargeCharge--Coupled Device Coupled Device (CCD)(CCD)

Imaging ChipImaging Chip

SamplingSampling ––

is a process by which a continuous 2is a process by which a continuous 2--D D signal is broken down into a set of picture signal is broken down into a set of picture elements (elements (pixelspixels). ).

QuantizationQuantization ––

is the assignment of values to is the assignment of values to pixels according to the strength of the signal pixels according to the strength of the signal they representthey represent

Formation of Digital ImagesFormation of Digital Images

Lecture 1Lecture 1


Reading: 1: 2.4.1


Lecture 1Lecture 1


Reading: 1: 2.4.1

TerminologyTerminology::Spatial coordinates Spatial coordinates --> Spatial resolution> Spatial resolution

––> > image resolutionimage resolution

Grayscale Grayscale ––> gray> gray--levels levels ––> radiometric resolution> radiometric resolution


Lecture 1Lecture 1


Reading: 1: 2.4.2

The quality of video data is defined by:1.

Spatial resolution –

the proximity of image samples.

2.

Spectral resolution-

bandwidth of the light frequencies captured by the camera.

3.

Radiometric resolution –

number of gray-levels.

4.

Time resolution –

interval between subsequent image frames.

Formation of Digital Video DataFormation of Digital Video Data

Lecture 1Lecture 1


Reading: 1: 2.4.3

The effect of varying the number of samples in a digital image.The effect of varying the number of samples in a digital image.


Lecture 1Lecture 1


Reading: 1: 2.4.2

The effect of reducing the The effect of reducing the number of greynumber of grey--levels in a levels in a digital image.digital image.

The grayscale is quantized into k equal intervals. Usually, 8 bits are used to express brightness values, than the number of gray-

levels is k=28 =256


Lecture 1Lecture 1


Reading: 1: 2.4.3

The effect of reducing the The effect of reducing the number of greynumber of grey--levels in a levels in a digital image.digital image.


Lecture 1Lecture 1


Reading: 1: 2.4.2

The coordinate conventionThe coordinate convention

The The M x NM x N

digital image in digital image in a matrix forma matrix form

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−−

−−

=

)1,1(...)1,1()0,1(

)1,1(...)1,1()0,1()1,0(...)1,0()0,0(

),(

NMfMfMf

NfffNfff

yxfMMM

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

−−−−

−

−

1,11,10,1

1,11,10,1

1,01,00,0

...

...

...

NMMM

N

N

aaa

aaaaaa

MMMA


Lecture 1Lecture 1


Reading: 1: 2.4.2

Implementation example: Implementation example: ““NegativeNegative””


Lecture 1Lecture 1


Lecture 1Lecture 1


Representation of digital Representation of digital imagesimages

The domain of the image function f(x,y) is a region R in the plane:

{ }mm yyxxyxR ≤≤≤≤= 1,1),,(

The “sifting property of the Dirac

function: it gives the value of the function f(x,y) at the point (a,b):

0,0),(1),(

:),(

),(),(),(

≠==

=−−

∫∫

∫∫

∞

∞−

∞

∞−

∞

∞−

∞

∞−

yxallforyxanddxdyyx

yxondistributiDiracthewhere

bafdxdybyaxyxf

δδ

δ

δ

Reading: 2: 2.1.2

The sampling process of a continuous image function is a linear combination of Dirac

pulses located at the points (a,b) that cover the whole image plane:

),(),(),( yxfdadbybxabaf =−−∫∫∞

∞−

∞

∞−δ

Lecture 1Lecture 1


Image sampling & quantizationImage sampling & quantization

The sampled image is the product: ),(),(),( yxsyxfyxf s =

Image digitization means that the continuous function f(x,y) is sampled into a matrix with M rows and N columns:

∑∑= =

Δ−Δ−=

==Δ=Δ=M

j

N

k

ykyxjxyxs

NkMjforykyxjx

1 1

),(),(

,..,1,,..,1,

δ

The continuous values of the image function are usually quantized

into L equal intervals. The number of L is usually an integer power of 2: L=2b.

The number, B,

of bits required to store a digitized image is:

B = M x N x b

Lecture 1Lecture 1


ExercisesExercises

1.

Write a program to sub-sample gradually the given image from its original size of 1024x1024

pixels down to size

32x32

pixels.

2.

Given an image, write a program to reduce gradually the number of gray levels from 256 to

2

in integer powers of 2.

How to display resulting images?

Documents

Course outline (rough): 1. Image Processing Algorithms 2. …kolesnik.leute.server.de/lectures/Lecture1.pdf · signal is broken down into a set of picture elements (pixels). Quantization–