Upload
elizabeth-webb
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
Multimedia Specification Design and Production
2012 / Semester 1 / L2Lecturer: Dr. Nikos [email protected]
2
Digital Media
In computers, audio, image and video are stored as files just like other text files.
For images, these files can have an extension like: BMP, JPG, GIF, TIFF, PNG, etc.
For audios, the file extensions include: WAV, MP3, …
The videos files usually have extensions: AVI, MOV, MKV
Digital Audio, Image & Video
3
An Example of Digital Image
Let’s open an image file is its “raw” format:
Digital Audio, Image & Video
* P6: (this is a ppm image)* Resolution: 512x512* Depth: 255 (8bits per pixel in each channel)
4
An Example of Digital Image (cont.)
Digital Audio, Image & Video
Color depth or Bit depth is the number of bits used to indicate the color of a single pixel in a bitmapped image or video frame buffer.
1-bit color (21 = 2 colors): monochrome 2-bit color (22 = 4 colors): CGA, gray-scale 3-bit color (23 = 8 colors): many early home computers with TV displays 4-bit color (24 = 16 colors): least common denominator VGA 5-bit color (25 = 32 colors) 6-bit color (26 = 64 colors) 8-bit color (28 = 256 colors): most early color Unix workstations, VGA at low-res 12-bit color (212 = 4096 colors): some Silicon Graphics systems
The HDMI 1.3 specification defines bit depths of 30 bits (1.073 billion colors), 36 bits (68.71 billion colors), and 48 bits (281.5 trillion colors) …& transparency
5
An Example of Digital Image (cont.)
Digital Audio, Image & Video
1 bit (2 colors) 2 bits (4 colors) 4 bits (16 colors)
8 bits (256 colors) 24 bits (16,777,216 colors, "truecolor")
6
An Example of Digital Image (cont.)
Digital Audio, Image & Video
8 bits
16 bits
24 bits
7
An Example of Digital Image (cont.)
Digital Audio, Image & Video
An image containsa header anda bunch of (integer) numbers.
8
Digital Media Capturing
Digital Audio, Image & Video
To get a digital image, an audio or a video clip, we need some media capturing device such as:
a digital camera or a scanner, a digital audio recorder, or a digital camcorder.
All these devices have to complete tasks:
Sampling: To convert a continuous media into discrete formats. Digitization: To convert continuous samples into finite number of digital numbers. There are probably some further compression process.
9
An Audio Signal
Digital Audio, Image & Video
10
Sampling for an Audio Signal
Digital Audio, Image & Video
0 10 20 30 40 50 60 70 80 90 100-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
t
Am
plit
ud
e
Sampling period Ts,fs =1/Ts
Signal Period T, f = 1/T
The sampling frequency or sampling rate fs is defined as:
the number of samples obtained in one second, or fs = 1/T
…and is measured in Hertz
11
Sampling for an Audio Signal
Digital Audio, Image & Video
fs = 2.5f
fs = 1.67f
0 10 20 30 40 50 60 70 80 90 100-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
t
Am
plit
ud
e
Original signal
A new component is added
This is denotedas aliasing.
0 10 20 30 40 50 60 70 80 90 100-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
t
Am
plitu
de
12
Sampling for an Audio Signal
Digital Audio, Image & Video
0 10 20 30 40 50 60 70 80 90 100-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
t
Am
plit
ude
0 10 20 30 40 50 60 70 80 90 100-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
t
Am
plitu
de
fs = 2f
There are infinite numberof possible sinwaves going throughthe sampling points
13
Frequency Decomposition
Digital Audio, Image & Video
We can use “Fourier Transform” to compute these frequency components.
Nyquist Theorem
14
Image Sampling
Digital Audio, Image & Video
The sampling theorem applies to 2D signal (images) too Nearest point interpolation.
15
Image Sampling (original image)
Digital Audio, Image & Video
16
Image Sampling (Aliasing due to sampling)
Digital Audio, Image & Video
17
Digitization
Digital Audio, Image & Video
The samples are continuous and have infinite number of possible values.
The digitization process approximates these values with a fixed number of numbers.
To represent N numbers, we need log2N bits.
18
Digital Audio
Digital Audio, Image & Video
You often hear that an audio is 16bits at 44kHz.
44KHz is the sampling frequency. Music has more high frequency components than speech. 8kHz sampling is good enough for telephone quality speech.
16bits means each sample is represented as a 16bit integer.
Digital audio could have more than one channels.
19
Digital Images
Digital Audio, Image & Video
An image contains 2D samples of a surface, which can be represented as matrices. Each sample in an image is called a pixel.
20
Types of Digital Images
Digital Audio, Image & Video
Grayscale image
Usually we use 256 levels for each pixel. Thus we need 8bits to represent each pixel (2^8 == 256)
Some images use more bits per pixel, for example MRI (Magnetic resonance imaging) images could use 16bits per pixel.
A 8bit grayscale Image.
21
Binary Image
Digital Audio, Image & Video
A binary image has only two values (0 or 1).
Binary image is quite important in image analysis and objectdetection applications.
22
Binary Image
Digital Audio, Image & Video
[ b7 b6 b5 b4 b3 b2 b1 b0]
Each bit plane is a binary image.
Most Significance Bitplane - MSB
Less Significance Bitplane - LSB
b7
23
Dithering
Digital Audio, Image & Video
A technique to represent a grayscale image with a binary one.
Convert image to4 levels: I’ = floor(I/64)
0 1
2 3
24
Dithering Matrix
Digital Audio, Image & Video
0 1
2 3
0 23 1The dithering matrix is:
Definition
A square matrix of threshold values that is repeated as a regular array to provide a threshold pattern for an entire image in the dither method of image representation
25
Color Image
Digital Audio, Image & Video
r
g
b
There are other color spaces: YUV, HSV etc.
24 bit image
26
Color Table
Digital Audio, Image & Video
Image with 256 colors
r
g
b
Clusters of colors
It is possible touse much less colorsTo represent a color imagewithout much degradation.
27
Human Vision
Digital Audio, Image & Video
Human eye has two kinds of light sensitive cells.1.The rods and2.The cones Rods response curve
(black and white vision)
Cones response curve(color vision)
R = s E() Sr()dG = s E() Sg()dB = s E() Sb()d
28
Colors
Digital Audio, Image & Video
Colorimeter experiment
Color matching function
29
CIE Color Matching Functions
Digital Audio, Image & Video
The amounts of R, G, B lighting sources to form single wavelength light forms the color matching curves.
CIE color matching curves CIE standard color matching functions
CIE = International Commission on Illumination
30
Gamma Correction
Digital Audio, Image & Video
Linearly increasing intensityWithout gamma correction
Linearly increasing intensitywith gamma correction
31
Video (Analog Video)
Digital Audio, Image & Video
Even frameOdd Frame
Definition
Analog video is a video signal transferred by an analog signal.
An analog color video signal contains luminance, brightness (Y) and chrominance (C) of an analog television image.
When combined in to one channel, it is called composite video as is the case, among others with NTSC, PAL and SECAM.
Analog video may be carried in separate channels, as in two channel S-Video (YC) and multi-channel component video formats.
32
Color System in Video
Digital Audio, Image & Video
YUV was used in PAL (an analog video standard) and also used for digital video.
Y is the luminance component (brightness) Y = 0.299 R + 0.587 G + 0.144 B
U and V are color components U = B – Y V = R - Y
33
PAL vs NTSC vs SECAM
Digital Audio, Image & Video
PAL (Phase Alternating Line), is an analogue television colour encoding system used in broadcast television systems in many countries.
NTSC (National Television System Committee) receivers have a tint control to perform colour correction manually. If this is not adjusted correctly, the colours may be faulty. The PAL standard automatically cancels hue errors by phase reversal, so a tint control is unnecessary.
SECAM (Sequential Color with Memory) is an earlier attempt at compatible colour television which also tries to resolve the NTSC hue problem. It does so by applying a different method to colour transmission, namely alternate transmission of the U and V vectors and frequency modulation, while PAL attempts to improve on the NTSC method. SECAM transmissions are more robust over longer distances than NTSC or PAL.
34
PAL vs NTSC vs SECAM
Digital Audio, Image & Video