Introduction to Multimedia 1
ICS 218 -Multimedia Systems and Applications
Lecture 1 - Introduction to MultimediaProf. Nalini [email protected]
Introduction to Multimedia 2
Course logistics and details
Course Web page - http://www.ics.uci.edu/~ics218
Lectures - MW 2:00-3:20p.m, ELH 110Course Laboratories - machines on 3rd floor
CS labs ICS 218 Textbook:
Multimedia: Computing, Communications and Applications Ralf Steinmetz and Klara Nahrstedt, Prentice-Hall Inc.
Other reading materialTechnical papers and reports
Introduction to Multimedia 3
Course logistics and details
Homeworks and Assignments 3 homeworks in the quarter of which 1 or 2 may
be programming assignments.
Tests Final Exam - as per UCI course catalog
Course Project Maybe done individually, in groups of 2 or 3(max) Potential projects on webpage
Data representation, multimedia systems and networks, multimedia applications
Introduction to Multimedia 4
ICS 218 Grading Policy
Homeworks - 30% • (3 homeworks each worth 10% of the final grade).
Class Project - 40% of the final gradeFinal exam - 30% of the final gradeFinal assignment of grades will be based
on a curve.
Introduction to Multimedia 5
Lecture Schedule
Weeks 1 and 2:Multimedia Representation
• Introduction to Multimedia• Audio/Image/Video Representation
Weeks 3 and 4: Multimedia Compression
• Encoding and Compression Techniques • Image Compression (JPEG) • Video Compression (MPEG/MPEG2/MPEG4)
Week 5 and 6: Multimedia Resource Management
• Multimedia Quality of Service and Server Design • Multimedia Operating Systems - Process Management,
Filesystems and buffer management
Introduction to Multimedia 6
Course Schedule
Weeks 7 and 8: Multimedia Communication Systems
• MM Networking• MM Communication
Week 9: Multimedia Synchronization
• Multimedia Synchronization Concepts • Multimedia Synchronization Enforcement
Week 10: Multimedia Applications and Services
• Multimedia Video Conferencing• Multimedia Entertainment Applications
Introduction to Multimedia 7
Introduction
Multimedia DescriptionWhy multimedia systems?Classification of MediaMultimedia SystemsData Stream Characteristics
Introduction to Multimedia 8
Multimedia Description
Multimediais an integration of continuous media (e.g. audio,
video) and discrete media (e.g. text, graphics, images) through which digital information can be conveyed to the user in an appropriate way.
Multimany, much, multiple
MediumAn interleaving substance through which something
is transmitted or carried on
Introduction to Multimedia 9
Why Multimedia Computing?
Application drivene.g. medicine, sports, entertainment, education
Information can often be better represented using audio/video/animation rather than using text, images and graphics alone.
Information is distributed using computer and telecommunication networks.
Integration of multiple media places demands oncomputation powerstorage requirementsnetworking requirements
Introduction to Multimedia 10
Multimedia Information Systems
Technical challenges Sheer volume of data
Need to manage huge volumes of data
Timing requirementsamong components of data computation and
communication.Must work internally with given timing constraints - real-
time performance is required.
Integration requirementsneed to process traditional media (text, images) as well as
continuous media (audio/video).Media are not always independent of each other -
synchronization among the media may be required.
Introduction to Multimedia 11
High Data Volume of Multimedia Information
Speech 8000 samples/s 8Kbytes/s
CD Audio 44,100 samples/s, 2 bytes/sample
176Kbytes/s
Satellite Imagery
180X180 km 2̂ 30m 2̂ resolution
600MB/image (60MB compressed)
NTSC Video 30fps, 640X480 pixels, 3bytes/pixel
30Mbytes/s (2-8 Mbits/s compressed)
Introduction to Multimedia 12
Technology Incentive
Growth in computational capacityMM workstations with audio/video processing capabilityDramatic increase in CPU processing power Dedicated compression engines for audio, video etc.
Rise in storage capacityLarge capacity disks (several gigabytes)Increase in storage bandwidth,e.g. disk array
technology
Surge in available network bandwidthhigh speed fiber optic networks - gigabit networksfast packet switching technology
Introduction to Multimedia 13
Application Areas
Residential Servicesvideo-on-demandvideo phone/conferencing systemsmultimedia home shopping (MM catalogs, product
demos and presentation)self-paced education
Business ServicesCorporate trainingDesktop MM conferencing, MM e-mail
Introduction to Multimedia 14
Application Areas
EducationDistance education - MM repository of class videosAccess to digital MM libraries over high speed
networks
Science and Technologycomputational visualization and prototypingastronomy, environmental science
MedicineDiagnosis and treatment - e.g. MM databases that
provide support for queries on scanned images, X-rays, assessments, response etc.
Introduction to Multimedia 15
Classification of Media
Perception MediumHow do humans perceive information in a computer?
• Through seeing - text, images, video • Through hearing - music, noise, speech
Representation MediumHow is the computer information encoded?
• Using formats for representing and information• ASCII(text), JPEG(image), MPEG(video)
Presentation MediumThrough which medium is information delivered by
the computer or introduced into the computer?• Via I/O tools and devices• paper, screen, speakers (output media)• keyboard, mouse, camera, microphone (input media)
Introduction to Multimedia 16
Classification of Media (cont.)
Storage Medium• Where will the information be stored?• Storage media - floppy disk, hard disk, tape, CD-ROM
etc. Transmission Medium
• Over what medium will the information be transmitted?• Using information carriers that enable continuous data
transmission - networks• wire, coaxial cable, fiber optics
Information Exchange Medium• Which information carrier will be used for information
exchange between different places?• Direct transmission using computer networks• Combined use of storage and transmission media (e.g.
electronic mail).
Introduction to Multimedia 17
Media Concepts
Each medium definesRepresentation values - determine the information
representation of different media• Continuous representation values (e.g. electro-
magnetic waves)• Discrete representation values(e.g. text characters in
digital form)Representation space determines the surrounding
where the media are presented.• Visual representation space (e.g. paper, screen)• Acoustic representation space (e.g. stereo)
Introduction to Multimedia 18
Media Concepts (cont.)
Representation dimensions of a representation space are: Spatial dimensions:
two dimensional (2D graphics)three dimensional (holography)
Temporal dimensions:Time independent (document) - Discrete media
• Information consists of a sequence of individual elements without a time component.
Time dependent (movie) - Continuous media• Information is expressed not only by its individual value
but also by its time of occurrence.
Introduction to Multimedia 19
Multimedia Systems
Qualitative and quantitative evaluation of multimedia systems Combination of media
continuous and discrete.
Levels of media-independencesome media types (audio/video) may be tightly
coupled, others may not.
Computer supported integrationtiming, spatial and semantic synchronization
Communication capability
Introduction to Multimedia 20
Data Streams
Distributed multimedia communication systems
data of discrete and continuous media are broken into individual units (packets) and transmitted.
Data Streamsequence of individual packets that are transmitted in
a time-dependant fashion.Transmission of information carrying different media
leads to data streams with varying features• Asynchronous• Synchronous • Isochronous
Introduction to Multimedia 21
Data Stream Characteristics
Asynchronous transmission mode • provides for communication with no time restriction• Packets reach receiver as quickly as possible, e.g.
protocols for email transmissionSynchronous transmission mode
• defines a maximum end-to-end delay for each packet of a data stream.
• May require intermediate storage• E.g. audio connection established over a network.
Isochronous transmission mode• defines a maximum and a minimum end-to-end delay
for each packet of a data stream. Delay jitter of individual packets is bounded.
• E.g. transmission of video over a network.• Intermediate storage requirements reduced.
Introduction to Multimedia 22
Data Stream Characteristics
Data Stream characteristics for continuous media can be based on
Time intervals between complete transmission of consecutive packets
• Strongly periodic data streams - constant time interval• Weakly periodic data streams - periodic function with finite
period.• Aperiodic data streams
Data size - amount of consecutive packets• Strongly regular data streams - constant amount of data• Weakly regular data streams - varies periodically with time• Irregular data streams
Continuity• Continuous data streams• Discrete data streams
Introduction to Multimedia 23
Classification based on time intervals
Strongly periodic data stream
Weakly periodic data stream
Aperiodic data stream
T
T
T1 T3T2
T1 T2
T
Introduction to Multimedia 24
Classification based on packet size
TD1
D1
TD1D2D3D1D2D3
D1D2D3
Dn
Strongly regular data stream
Weakly regular data stream
Irregular data stream
t
t
t
Introduction to Multimedia 25
Classification based on continuity
Continuous data stream
Discrete data stream
D
D1 D2 D3 D4
D
D1 D2 D3 D4
Introduction to Multimedia 26
Logical Data Units
Continuous media consist of a time-dependent sequence of individual information units called Logical Data Units (LDU).
• a symphony consists of independent sentences• a sentence consists of notes• notes are sequences of samples
Granularity of LDUs• symphony, sentence, individual notes, grouped samples,
individual samples• film, clip, frame, raster, pixel
Duration of LDU:• open LDU - duration not known in advance • closed LDU - predefined duration
Introduction to Multimedia 27
Granularity of Logical Data Units
Film
Clip
Frame
Blocks
Pixels