Upload
judith-waters
View
259
Download
1
Embed Size (px)
Citation preview
Introduction to MultimediaIntroduction to Multimedia
Introduction to MultimediaIntroduction to Multimedia
1- What is Multimedia?2- Multimedia and Hypermedia3- World Wide Web4- Overview of Multimedia Software Tools
1- What is Multimedia?2- Multimedia and Hypermedia3- World Wide Web4- Overview of Multimedia Software Tools
What Is Multimedia ?What Is Multimedia ? What Is Multimedia ?What Is Multimedia ?
Multi Multi refer to many or multiplerefer to many or multiple
Media Media Vehicle or agent to convey for somethingVehicle or agent to convey for something Example: air for sound, writing for wordsExample: air for sound, writing for words Computer is a medium for ideas and information Computer is a medium for ideas and information
- Express and communicate these ideas- Express and communicate these ideas
- Store and manipulate information in many different forms- Store and manipulate information in many different forms
MultimediaMultimediaMultiMultiMultiMulti MediaMediaMediaMedia
The big picture The big picture
ImagImagee
TextText
SpeeSpeechch
AudiAudioo
VideVideoo
MultimediaMultimedia
Virtual worldsVirtual worldsStreaming videoStreaming video
Web pagesWeb pagesStreaming audioStreaming audio Client-serverClient-server
MultimediaMultimediaSystemsSystems
AuthoringAuthoring
Media asset Media asset managementmanagement
ToolsTools
FrameworksFrameworks
NetworkNetwork
DistributedDistributedMultimediaMultimediaSystemsSystems
GLOBAL STRUCTURE OF MultimediaGLOBAL STRUCTURE OF Multimedia
GLOBAL STRUCTURE OF MultimediaGLOBAL STRUCTURE OF MultimediaApplication domain : provides functions to the user to develop and present multimedia projects. This includes Software tools, andmultimedia projects development methodology.
System domain : including all supports for using the functions of the device domain, e.g., operating systems, communicationsystems (networking) and database systems.
Device domain : basic concepts and skill for processing various multimedia elements and for handling physical device.
Application domain : provides functions to the user to develop and present multimedia projects. This includes Software tools, andmultimedia projects development methodology.
System domain : including all supports for using the functions of the device domain, e.g., operating systems, communicationsystems (networking) and database systems.
Device domain : basic concepts and skill for processing various multimedia elements and for handling physical device.
What Is Multimedia ? What Is Multimedia ?
MultimediaMultimedia Presentation of a computer application Presentation of a computer application Incorporating media elements such as text, graphics, video, Incorporating media elements such as text, graphics, video,
animation and sound on a computer.animation and sound on a computer.
Multimedia SystemsMultimedia Systems the computer and software systems that are used in the computer and software systems that are used in
multimedia environments multimedia environments Interactive control Interactive control
Distributed multimedia systemsDistributed multimedia systems operate over some form of network infrastructureoperate over some form of network infrastructure
When different people mention the term multimedia, theyWhen different people mention the term multimedia, they
often have quite different, or even opposing, viewpoints.often have quite different, or even opposing, viewpoints.
• A PC vendor: a PC that has sound capability, a DVD-ROM A PC vendor: a PC that has sound capability, a DVD-ROM drive , and perhaps the superiority of multimedia-enabled drive , and perhaps the superiority of multimedia-enabled microprocessors that understand additional multimedia microprocessors that understand additional multimedia instructions.instructions.
• A consumer entertainment vendor: interactive cable TV A consumer entertainment vendor: interactive cable TV with hundreds of digital channels available, or a cable TV-with hundreds of digital channels available, or a cable TV-like service delivered over a high-speed Internet like service delivered over a high-speed Internet connection.connection.
• A Computer Science (CS) student: applications that use A Computer Science (CS) student: applications that use multiple modalities, including text, images, drawings multiple modalities, including text, images, drawings (graphics), animation, video, sound including speech, and (graphics), animation, video, sound including speech, and interactivity.interactivity.
What Is Multimedia ? What Is Multimedia ?
Multimedia and Computer Science:Multimedia and Computer Science: Graphics, visualization, computer vision, data Graphics, visualization, computer vision, data
compression, graph theory, networking, database systems.compression, graph theory, networking, database systems.
• Components(elements) of MultimediaComponents(elements) of Multimedia
Multimedia involves multiple modalities of text, audio, Multimedia involves multiple modalities of text, audio,
ImImages, drawings, animation, and videoages, drawings, animation, and video
A brief history of multimedia A brief history of multimedia
1970-an
1980
1990
Future
8 bit processor
CD-ROM, Local and Wide Area Networking16 bit processorsDesktop Computer
32 bit
World Wide Web
Digital Audio and Video
Graphical Operating Systems
Modern multimedia PCs Modern multimedia PCs 3D graphics cards with 64 MB of graphics memory3D graphics cards with 64 MB of graphics memory
64 note sound cards with wavetable synthesis64 note sound cards with wavetable synthesis
256 MB of main memory256 MB of main memory
60+ GB hard disk space60+ GB hard disk space
50+ speed CD readers50+ speed CD readers
56 Kb/s modem or ISDN connection56 Kb/s modem or ISDN connection
Content(Media)
Applications (Software)
Platforms(Hardware)
Networking
Level 1Level 1
Level 2Level 2
Level 3Level 3
Level 4Level 4
Multimedia Industry Structure
Level 1Level 1 - Consists of MM content providers, for example the
news industry, the television industry and the entertainment industry.
- Level 2Level 2- Consists of multimedia application developers.- Applications include distance learning, desktop
videoconferencing, multimedia kiosks, entertainment and imaging.
- Level 3 Level 3 - Consists of MM platform builders.- These builders include Silicon Graphics, Sun, Intel,
Apple, and other hardware vendors.
Multimedia Industry Structure
Level 4Level 4 - The network infrastructure. - There are two very different networking
environments that will use MM applications: - business networks and - the emerging public networks.
Multimedia Industry Structure
Why Apply Multimedia TechnologyWhy Apply Multimedia Technology
Ease of useEase of use Intuitive InterfaceIntuitive Interface Immersive ExperienceImmersive Experience Self-Paced Interaction and Better RetentionSelf-Paced Interaction and Better Retention Better Understanding of the ContentBetter Understanding of the Content Cost EffectivenessCost Effectiveness More Fun = Greater EfficiencyMore Fun = Greater Efficiency
Multimedia DevelopmentMultimedia Development
ProcessProcess
Capture from device
Read from a file
Receive over the network
Apply effect filters and transforms
Compress or decompress
Convert between formats
Present
Save to a file
Send across the network
InputInput OutputOutput
Examples of multimedia systems Examples of multimedia systems
Content delivery system examples: Television/(radio?) (old, basic system) Web/Internet type Mobile (radio networks)Content distribution type examples: Music Movies (audio/visual content Games Virtual worlds (Check Second life on the Web) eBooks? eMagazines? eNewspapers?
Multimedia Application
A Multimedia Application is an application which uses a collection of multiple media sources e.g. text, graphics, images, sound/audio, animation and/or video.
Multimedia System Desirable Features
Speci Very High Processing Power al Hardware/Software needed Efficient and High I/O Large Storage and Memory High Speed Network Support
Components of a Multimedia System
Capture devices
• e.g. Video Camera, Microphone, Digitising/Sampling Hardware, etc.
Storage Devices
• e.g. Hard disks, CD-ROMs, DVD, etc Communication Networks
• e.g. Ethernet, Internets, etc. Computer Systems
• e.g. Multimedia Desktop machines, Workstations Display Devices
• e.g. CD-quality speakers, HDTV, Hi-Res monitors, Color printers etc.
Multimedia Data: Text
Text • Input: keyboard• Stored and input character by character.• Storage of text is 1 or 2 bytes per character.• Other forms of data (e.g. Spreadsheet files,
XML) may store format as text (with formatting).
• Format: Raw text or formatted text e.g HTML, Rich Text Format (RTF), Word or a program language source
Multimedia Data: Text
Text (cont.)• Not temporal — BUT may have natural implied
sequence e.g. HTML format sequence, Sequence of Java program statements.
• Size Not significant compared with other multimedia data.
Formatted Text
Multimedia Data: Images
Images• Still pictures which (uncompressed) are
represented as a bitmap (a grid of pixels).• Input: Digital camera, scanner or generated by
graphics editor programs (e.g. Paint)• Analog sources will require digitising.• Stored at 1 bit per pixel (Black and White), 8
Bits per pixel (Grey Scale, Color Map) or 24 Bits per pixel (True Color)
Multimedia Data: Images
Images (Cont.)• Size: a 512x512 Grey scale image takes up 1/4
Mb, a 512x512 24 bit image takes 3/4 Mb with no compression.
• Storage increases with image size• Compression is commonly applied
Image : PNG File Format
Multimedia Data: Vector Graphics
Vector Graphics• Format: constructed by the composition of
primitive objects such as lines, polygons, circles, curves and arcs.
• Input: usually generated by a graphics editor program (e.g. Freehand)
• Vector Graphics are usually editable or revisable (unlike Images).
• Standards : SVG, OpenGL, WMF, etc.• Do not take up a very high storage overhead.
Multimedia Data: Vector Graphics
Vector Graphics (Cont.)• Advantages: reduction of data size, meaningful
representation, easier manipulation, resizable without losing quality .
• Disadvantages: not suitable for reproducing photographs or paintings, take a long time to display complex image.
Vector Graphics vs Images
Graphic (WMF) : 10262 bytes
Image (BMP) : 51054 bytes
enlarge
enlarge
Vector Graphics Examples
Multimedia Data: Audio
Audio• Audio signals are continuous analog signals.• Input: microphones and then digitised and
stored • usually compressed.• CD Quality Audio requires 16-bit sampling at
44.1 KHz• 1 Minute of Mono CD quality audio requires 5
Mb.
Multimedia Data: Video
Video• Input: usually captured by a video camera.• There are a variety of video (analog and digital)
formats• Raw video can be regarded as being a series
of single images.• There are typically 25, 30 or 50 frames per
second.• a 512x512 size monochrome video images take
25*0.25 = 6.25Mb for a minute to store uncompressed.
• Digital video clearly needs to be compressed.
Multimedia Delivery
Offline • Use removable storage as medium delivery• Examples : CD-ROM, PC, DVD
Online• Use a network to send information• Examples : WWW• Enables delivery live multimedia content. • Examples : Video Conferencing, broadcast
Multimedia
Data Transfer over the Internet
kbits/s 6KB Text File
100KB Image
4MB Movie
GSM Data 9.6 4.5 84 Seconds 56 Minutes
Slow Modem 14.4 3 Seconds 56 Seconds 37 Minutes
GPRS 36.6 1.25 Seconds 22 Seconds 15 Minutes
Fast Modem 56 1 Second 14 Seconds 9 Minutes
ISDN 128 <1 Second 6 Seconds 4.3 Minutes
T1 Leased Line
1544 <1 Second 1 Second 21 Seconds
ADSL (Typical)
6000 <1 Second <1 Second 5 Seconds
22 Multimedia and HypermediaMultimedia and Hypermedia
A hypertext system: A hypertext system:
meant to be read nonlinearly, by followingmeant to be read nonlinearly, by following links that point links that point to other parts of the document, or to other documents.to other parts of the document, or to other documents.
HyperMediaHyperMedia::
not constrained to be text-based, can include not constrained to be text-based, can include other media, other media, e.g., graphics, images, and especially the continuous e.g., graphics, images, and especially the continuous media | sound and video.media | sound and video.
- The World Wide Web (WWW) | the best example of - The World Wide Web (WWW) | the best example of ahypermedia application.ahypermedia application.
Multimedia Multimedia
means that computer information can be repremeans that computer information can be represented sented through audio, graphics, images, video, and animationthrough audio, graphics, images, video, and animation
in addition to traditional media.in addition to traditional media.
• Typical present multimedia applicationsTypical present multimedia applications
include(Examples of )include(Examples of ):: - Digital video editing and production systems.- Digital video editing and production systems.
- Electronic newspapers/magazines.- Electronic newspapers/magazines.
- World Wide Web.- World Wide Web.
- On-line reference works: e.g. encyclopedias, games, etc.- On-line reference works: e.g. encyclopedias, games, etc.
- Home shopping.- Home shopping.
- Interactive TV.- Interactive TV.
- Multimedia courseware.- Multimedia courseware.
- Video conferencing.- Video conferencing.
- Video-on-demand.- Video-on-demand.
- Interactive movies.- Interactive movies.
3 World Wide Web3 World Wide Web
The W3C has listed the following goals for the WWW:The W3C has listed the following goals for the WWW: 1. Universal access of web resources (by everyone every-1. Universal access of web resources (by everyone every-
where).where).
2. Eeffctiveness of navigating available information.2. Eeffctiveness of navigating available information.
3. Responsible use of posted material.3. Responsible use of posted material.
History of the WWWHistory of the WWW1960 - Charles Goldfarb et al. developed the Generalized1960 - Charles Goldfarb et al. developed the Generalized
Markup Language (GML) for IBM.Markup Language (GML) for IBM.
1986 - The ISO released a nal version of the Standard 1986 - The ISO released a nal version of the Standard Generalized Markup Language (SGML).Generalized Markup Language (SGML).
1990 - Tim Berners-Lee invented the HyperText Markup1990 - Tim Berners-Lee invented the HyperText Markup
Language (HTML), and the HyperText Transfer Language (HTML), and the HyperText Transfer
Protocol (HTTP).Protocol (HTTP).
1993 - NCSA released an alpha version of Mosaic based on1993 - NCSA released an alpha version of Mosaic based on
the version by Marc Andreessen for X-Windows |the version by Marc Andreessen for X-Windows |
the first popular browser.the first popular browser.
1994 - Marc Andreessen et al. formed Mosaic Communica1994 - Marc Andreessen et al. formed Mosaic Communications tions Corporation | later the Netscape Communications Corporation | later the Netscape Communications Corporation.Corporation.
1998 - The W3C accepted XML version 1.0 specifications1998 - The W3C accepted XML version 1.0 specifications
as a Recommendation | the main focus of the W3Cas a Recommendation | the main focus of the W3C
and supersedes HTML.and supersedes HTML.
HTTP (HyperText Transfer Protocol)HTTP (HyperText Transfer Protocol)
HTTP:HTTP: a protocol that was originally designed for transmit- a protocol that was originally designed for transmit-
ting hypermedia, but can also support the transmission ofting hypermedia, but can also support the transmission of
any le type.any le type.
HTTP is a stateless request/response protocol: HTTP is a stateless request/response protocol: no inforno information mation carried over for the next request.carried over for the next request.
The basic request format:The basic request format:
Method URI VersionMethod URI Version
Additional-Headers:Additional-Headers:
Message-bodyMessage-body
The URI (Uniform Resource Identifier): an identifier for theThe URI (Uniform Resource Identifier): an identifier for the
resource accessed, e.g. the host name, always preceded byresource accessed, e.g. the host name, always preceded by
the token \http://".the token \http://".
Two popular methodsTwo popular methods: GET and POST.: GET and POST.
HTML (Hyper Text Markup Language)HTML (Hyper Text Markup Language)
HTML:HTML: a language for publishing Hypermedia on the World a language for publishing Hypermedia on the World
Wide Web | defined using SGML:Wide Web | defined using SGML:
1. HTML uses ASCII, it is portable to all dierent (possibly1. HTML uses ASCII, it is portable to all dierent (possibly
binary incompatible) computer hardware.binary incompatible) computer hardware.
2. The current version of HTML is version 4.01.2. The current version of HTML is version 4.01.
3. The next generation of HTML is XHTML | a reformulation 3. The next generation of HTML is XHTML | a reformulation of HTML using XML.of HTML using XML.
HTML uses tags to describe document elements:HTML uses tags to describe document elements:
- <token params> | defining a starting point,- <token params> | defining a starting point,
- </token> | the ending point of the element.- </token> | the ending point of the element.
- Some elements have no ending tags.- Some elements have no ending tags.
Naturally:Naturally:
HTML has more complex structures and can be HTML has more complex structures and can be mixed in with other standards.mixed in with other standards.
XML (Extensible Markup Language)XML (Extensible Markup Language)XML:XML: a markup language for the WWW in which there is a markup language for the WWW in which there is
modularity of data, structure and view so that user or modularity of data, structure and view so that user or application can be able to define the tags (structure).application can be able to define the tags (structure).
Example of using XML to retrieve stock information from aExample of using XML to retrieve stock information from a
database according to a user query:database according to a user query:
1. First use a global Document Type Definition (DTD) that1. First use a global Document Type Definition (DTD) that
is already defined.is already defined.
2. The server side script will abide by the DTD rules to2. The server side script will abide by the DTD rules to
generate an XML document according to the query usinggenerate an XML document according to the query using
data from your database.data from your database.
3. Finally send user the 3. Finally send user the XML Style Sheet (XSL) dependingXML Style Sheet (XSL) depending
on the type of device used to display the information.on the type of device used to display the information.
The following XML related specications are also standardized:The following XML related specications are also standardized:
- XML Protocol: XML Protocol: used to exchange XML information between used to exchange XML information between processes.processes.
- - XML Schema: XML Schema: a more structured and powerful languagea more structured and powerful language
for defining XML data types (tags).for defining XML data types (tags).
- - XSL:XSL: basically CSS for XML. basically CSS for XML.
- - SMIL:SMIL: synchronized Multimedia Integration Language, synchronized Multimedia Integration Language, pronounced \smile"|a particular application of XML (glob-pronounced \smile"|a particular application of XML (glob-
ally predefined DTD) that allows for specification of ally predefined DTD) that allows for specification of interaction among any media types and user input, in ainteraction among any media types and user input, in a
temporally scripted manner.temporally scripted manner.
SMIL (Synchronized Multimedia SMIL (Synchronized Multimedia IntegrationLanguageIntegrationLanguage))
Purpose of SMIL: Purpose of SMIL: it is also desirable to be able to publishit is also desirable to be able to publish
multimedia presentations using a markup language.multimedia presentations using a markup language.
- A multimedia markup language needs to enable scheduling- A multimedia markup language needs to enable scheduling
and synchronization of different multimedia elements, andand synchronization of different multimedia elements, and
define their interactivity with the user.define their interactivity with the user.
- The W3C established a Working Group in 1997 to come up- The W3C established a Working Group in 1997 to come up
with specifications for a multimedia synchronization with specifications for a multimedia synchronization language | SMIL 2.0 was accepted in August 2001.language | SMIL 2.0 was accepted in August 2001.
SMIL 2.0 is specified in XML using a SMIL 2.0 is specified in XML using a modularization modularization apapproach similar to the one used in xhtml:proach similar to the one used in xhtml:
1. All SMIL elements are divided into modules | sets of1. All SMIL elements are divided into modules | sets of
XML elements, attributes and values that define one XML elements, attributes and values that define one conceptual functionality.conceptual functionality.
2. In the interest of modularization, not all available modules2. In the interest of modularization, not all available modules
need to be included for all applications.need to be included for all applications.
3. Language Profiles: species a particular grouping of3. Language Profiles: species a particular grouping of
modules, and particular modules may have integration modules, and particular modules may have integration requirements that a profile must follow.requirements that a profile must follow.
- SMIL 2.0 has a main language profile that includes almost all - SMIL 2.0 has a main language profile that includes almost all SMIL modules.SMIL modules.
44 Overview of Multimedia Software Overview of Multimedia Software ToolsTools
The categories of software tools briefy The categories of software tools briefy examined here are:examined here are:
1. Music Sequencing and Notation1. Music Sequencing and Notation
2. Digital Audio2. Digital Audio
3. Graphics and Image Editing3. Graphics and Image Editing
4. Video Editing4. Video Editing
5. Animation5. Animation
6. Multimedia Authoring6. Multimedia Authoring
Music Sequencing and NotationMusic Sequencing and Notation
Cakewalk: Cakewalk: now called Pro Audio.now called Pro Audio.
- The term sequencer comes from older devices that stored- The term sequencer comes from older devices that stored
sequences of notes (\events", in MIDI).sequences of notes (\events", in MIDI).
- It is also possible to insert WAV les and Windows MCI- It is also possible to insert WAV les and Windows MCI
commands (for animation and video) into music trackscommands (for animation and video) into music tracks
(MCI is a ubiquitous component of the Windows API.)(MCI is a ubiquitous component of the Windows API.)
CubaseCubase: another sequencing/editing program, with capabil: another sequencing/editing program, with capabil ities ities similar to those of Cakewalk. It includes some digitalsimilar to those of Cakewalk. It includes some digital
audio editing tools.audio editing tools.
MacromediaMacromedia SoundeditSoundedit: mature program for creating au: mature program for creating audio for dio for multimedia projects and the web that integrates wellmultimedia projects and the web that integrates well
with other Macromedia products such as Flash and Director.with other Macromedia products such as Flash and Director.
DigitalDigital AudioAudio
DigitalDigital AudioAudio toolstools deal with accessing and editing the actual deal with accessing and editing the actual sampled sounds that make up audio:sampled sounds that make up audio:
- - CoolCool EditEdit: a very powerful and popular digital audio: a very powerful and popular digital audio
toolkit; emulates a professional audio studio |multi tracktoolkit; emulates a professional audio studio |multi track
productions and sound le editing including digital signalproductions and sound le editing including digital signal
processing effects.processing effects.
- - Sound ForgeSound Forge: a sophisticated PC-based program for: a sophisticated PC-based program for
editing audio WAV files.editing audio WAV files.
- - Pro ToolsPro Tools: a high-end integrated audio production and: a high-end integrated audio production and
editing environment | MIDI creation and manipulation;editing environment | MIDI creation and manipulation;
powerful audio mixing, recording, and editing software.powerful audio mixing, recording, and editing software.
Graphics and Image EditingGraphics and Image Editing
Adobe IllustratorAdobe Illustrator: a powerful publishing tool from Adobe.: a powerful publishing tool from Adobe.
Uses vector graphics; graphics can be exported to Web.Uses vector graphics; graphics can be exported to Web.
Adobe PhotoshopAdobe Photoshop: the standard in a graphics, image : the standard in a graphics, image proprocessing and manipulation tool.cessing and manipulation tool.
- Allows layers of images, graphics, and text that can be - Allows layers of images, graphics, and text that can be separatelyseparately
manipulated for maximum flexibility.manipulated for maximum flexibility.
- Filter factory permits creation of sophisticated lighting-- Filter factory permits creation of sophisticated lighting-effects file-ters.effects file-ters.
Macromedia FireworksMacromedia Fireworks: software for making graphics : software for making graphics specifspecifically for the web.ically for the web.
Macromedia FreehandMacromedia Freehand: a text and web graphics editing: a text and web graphics editing
tool that supports many bitmap formats such as GIF, PNG,tool that supports many bitmap formats such as GIF, PNG,
and JPEG.and JPEG.
Video EditingVideo Editing
Adobe PremiereAdobe Premiere: an intuitive, simple video editing tool for: an intuitive, simple video editing tool for
nonlinear editing, i.e., putting video clips into any order:nonlinear editing, i.e., putting video clips into any order:
- Video and audio are arranged in \tracks".- Video and audio are arranged in \tracks".
- Provides a large number of video and audio tracks, super-- Provides a large number of video and audio tracks, super-
impositions and virtual clips.impositions and virtual clips.
- A large library of built-in transitions, filters and motions- A large library of built-in transitions, filters and motions
for clips for clips ) effective multimedia productions with little) effective multimedia productions with little
effort.effort.
Adobe After EectsAdobe After Eects: a powerful video editing tool that: a powerful video editing tool that
enables users to add and change existing movies. Can addenables users to add and change existing movies. Can add
many effects: lighting, shadows, motion blurring; layers.many effects: lighting, shadows, motion blurring; layers.
Final Cut ProFinal Cut Pro: a video editing tool by Apple; Macintosh: a video editing tool by Apple; Macintosh
only.only.
AnimationAnimationMultimedia APIsMultimedia APIs::
Java3D:Java3D: API used by Java to construct and render 3D API used by Java to construct and render 3D
graphics, similar to the way in which the Java Mediagraphics, similar to the way in which the Java Media
Framework is used for handling media files.Framework is used for handling media files.
1. Provides a basic set of object primitives (cube, splines,1. Provides a basic set of object primitives (cube, splines,
etc.) for building scenes.etc.) for building scenes.
2. It is an abstraction layer built on top of OpenGL or2. It is an abstraction layer built on top of OpenGL or
DirectX (the user can select which).DirectX (the user can select which).
DirectX : DirectX : Windows API that supports video, images, audio and Windows API that supports video, images, audio and 3-D animation3-D animation
OpenGL:OpenGL: the highly portable, most popular 3-D API. the highly portable, most popular 3-D API.
3D Studio Max: 3D Studio Max: rendering tool that includes a number ofrendering tool that includes a number of
very high-end professional tools for character animation,very high-end professional tools for character animation,
game development, and visual effects production.game development, and visual effects production.
Multimedia AuthoringMultimedia Authoring
Macromedia FlashMacromedia Flash: allows users to create interactive movies: allows users to create interactive movies
by using the score metaphor, i.e., a timeline arranged in by using the score metaphor, i.e., a timeline arranged in parallel event sequences.parallel event sequences.
Macromedia DirectorMacromedia Director: uses a movie metaphor to create : uses a movie metaphor to create ininteractive presentations | very powerful and includes a teractive presentations | very powerful and includes a built in scripting language, Lingo, that allows creation of built in scripting language, Lingo, that allows creation of complex interactive movies.complex interactive movies.
AuthorwareAuthorware: a mature, well-supported authoring product: a mature, well-supported authoring product
based on the Iconic/Flow-control metaphor.based on the Iconic/Flow-control metaphor.
Quest:Quest: similar to Authorware in many ways, uses a type similar to Authorware in many ways, uses a type
of flowcharting metaphor. However, the flowchart nodesof flowcharting metaphor. However, the flowchart nodes
can encapsulate information in a more abstract way (calledcan encapsulate information in a more abstract way (called
frames) than simply subroutine levels.frames) than simply subroutine levels.
SET OF TOOLSSET OF TOOLS
QuestionsQuestions
Q1) When designing multimedia systems, what two Q1) When designing multimedia systems, what two levels of functionality need to be considered? levels of functionality need to be considered? Briefly define these levels.Briefly define these levels.
There are two levels of description for an application: There are two levels of description for an application:
** Functional levelFunctional level -- what the application does and source and -- what the application does and source and display data formats. display data formats. E.g.E.g. An application may deal with the An application may deal with the retrieval and display of movies. It retrieves movies and retrieval and display of movies. It retrieves movies and displays them in a window. displays them in a window.
** System LevelSystem Level -- The entities that the application deals with and -- The entities that the application deals with and how it deals with them. In the above example we deal with how it deals with them. In the above example we deal with video frames and voice samples and need to process these video frames and voice samples and need to process these accordingly. accordingly.
Q2) For each common multimedia data type discuss what common Q2) For each common multimedia data type discuss what common functionalities should be supported by a multimedia systemfunctionalities should be supported by a multimedia system..
The following functionality should be provided by a multimedia system: The following functionality should be provided by a multimedia system:
Digital Representation of MediaDigital Representation of Media -- --
Many standardised formats for many media Many standardised formats for many media
Capture: Digitisation of Media Capture: Digitisation of Media -- --
special Hardware/Software special Hardware/Software
Creation and editing Creation and editing ––
assemble media and alter it assemble media and alter it
Storage Requirements Storage Requirements ––
significant for multimedia significant for multimedia
CompressionCompression -- --
related to above and below, ie can save on storage but can hinder retrieval related to above and below, ie can save on storage but can hinder retrieval
Structuring and retrieval methods of media Structuring and retrieval methods of media – –
simple to advanced DataBase Storage simple to advanced DataBase Storage
Display or Playback methods Display or Playback methods -- --
effect of retrieval must view data effect of retrieval must view data
Media SynchronisationMedia Synchronisation -- --
display multimedia as it is intendeddisplay multimedia as it is intended
Typical Range of operations required for common media include:Typical Range of operations required for common media include:
Text Text
Editing Formatting Sorting Indexing Searching ncrypting Editing Formatting Sorting Indexing Searching ncrypting
The above operations involve basic processing such as: The above operations involve basic processing such as:
Character Manipulation Character Manipulation
String Manipulation String Manipulation
AudioAudio
Audio Editing Synchronisation Conversion/Translation Audio Editing Synchronisation Conversion/Translation
Filtering/ Sound Enhancing Operators Compression Searching Filtering/ Sound Enhancing Operators Compression Searching Indexing Indexing
The above operations involve basic processing such as: The above operations involve basic processing such as:
Sample Manipulation Sample Manipulation
Waveform ManipulationWaveform Manipulation
Graphics Graphics
Graphic primitive Editing Shading Mapping lighting Viewing Graphic primitive Editing Shading Mapping lighting Viewing
Rendering Searching Indexing Rendering Searching Indexing
The above operations involve basic processing such as: The above operations involve basic processing such as:
Primitive Manipulation Primitive Manipulation
Structural/Group Manipulation Structural/Group Manipulation
Image Image
Pixel operations Geometric Operations Filtering Conversion Pixel operations Geometric Operations Filtering Conversion
Indexing Compression Searching Indexing Compression Searching
The above operations involve basic processing such as: The above operations involve basic processing such as:
Pixel Manipulation Pixel Manipulation
Area Manipulation, copy image areas, delete area, process areas of Area Manipulation, copy image areas, delete area, process areas of interest interest
Animation Animation
-- --
Primitive/Group Editing Structural Editing Image Overlaying Primitive/Group Editing Structural Editing Image Overlaying
Rendering Synchronisation Searching Indexing Rendering Synchronisation Searching Indexing
The above operations involve basic processing such as: The above operations involve basic processing such as:
Pixel Manipulation Pixel Manipulation
Area Manipulation, copy image areas, delete area, process areas of Area Manipulation, copy image areas, delete area, process areas of interest interest
Frame Manipulation, Copy frame(s) to frame(s), Process frame(s) Frame Manipulation, Copy frame(s) to frame(s), Process frame(s)
Video Video
-- --
Pixel Operations Frame Operations Editing Synchronisation Pixel Operations Frame Operations Editing Synchronisation
Conversion Mixing Indexing Searching Video Effects/Filtering Conversion Mixing Indexing Searching Video Effects/Filtering
The above operations involve basic processing such as: The above operations involve basic processing such as:
Pixel Manipulation Pixel Manipulation
Area Manipulation, copy image areas, delete area, process areas of Area Manipulation, copy image areas, delete area, process areas of interest binterest b
Frame Manipulation, Copy frame(s) to frame(s), Process frame(s)Frame Manipulation, Copy frame(s) to frame(s), Process frame(s)
q3) q3) You have been commissioned to produce a Multimedia mail You have been commissioned to produce a Multimedia mail system. What media should be supported in such a mail system. What media should be supported in such a mail system and how should an application facilitate assembly, system and how should an application facilitate assembly, delivery and reading of the mail?delivery and reading of the mail?
SKECTH OF SOULTIOIN REALISATION OF BANDWIDTH/MESSAGE SKECTH OF SOULTIOIN REALISATION OF BANDWIDTH/MESSAGE SIZE and how deal with it important. Also how the application SIZE and how deal with it important. Also how the application deals with media is important.deals with media is important.
Multimedia mail an extension of test-only mail:Multimedia mail an extension of test-only mail:
Basic content still testBasic content still test
Annotated with richer mediaAnnotated with richer media::
ImageImage
Speech/AudioSpeech/Audio
Video ClipVideo Clip
Possible problemsPossible problems
Send all data at once ---- large mail messageSend all data at once ---- large mail message
High BandwidthHigh Bandwidth
High Server OverheadsHigh Server Overheads
Mailboxes need large storageMailboxes need large storage
Mail apps need large memoryMail apps need large memory
Low quality audio and images not a problem if small in Low quality audio and images not a problem if small in number?number?
Terminal being used the read mail supports all media?Terminal being used the read mail supports all media?
Printing etc may lose their MessagePrinting etc may lose their Message
Solution Annotations (tags/links) in main bodySolution Annotations (tags/links) in main body
Annotations can be sent with main message (Explicit Media Annotations can be sent with main message (Explicit Media Inclusion)ORInclusion)OR
Get requested specifically by recipient when message is read or Get requested specifically by recipient when message is read or indeed if media is required.(Implicit Media Inclusion)indeed if media is required.(Implicit Media Inclusion)
Latter method allows reading of mail on more platforms etcLatter method allows reading of mail on more platforms etc
Could send previews of data (i.e. lower resolution audio, images, Could send previews of data (i.e. lower resolution audio, images, image snapshots of video, thumbprints of larger media?)image snapshots of video, thumbprints of larger media?)
Application RequirementsApplication Requirements
Multimedia enabledMultimedia enabled
Good user friendly InterfaceGood user friendly Interface
Good network connectionGood network connection
Good media support of formatsGood media support of formats
Input ---- support for Multimedia AuthoringInput ---- support for Multimedia Authoring
Input ---- support for Graphics/Image/Audio/Video input devicesInput ---- support for Graphics/Image/Audio/Video input devices
Q4) Q4) What is meant by the terms Multimedia and Hypermedia? What is meant by the terms Multimedia and Hypermedia? Distinguish between these two concepts.Distinguish between these two concepts.
Multimedia Multimedia ---- An Application which uses a collection of ---- An Application which uses a collection of multiple media sources e.g. text, graphics, images, multiple media sources e.g. text, graphics, images, sound/audio, animation and/or video.sound/audio, animation and/or video.
HypermediaHypermedia --- An application which uses associative --- An application which uses associative relationships among information contained within multiple relationships among information contained within multiple media data for the purpose of facilitating access to, and media data for the purpose of facilitating access to, and manipulation of, the information encapsulated by the data.manipulation of, the information encapsulated by the data.
Q5) What is meant by the terms static media and dynamic Q5) What is meant by the terms static media and dynamic media? Give two examples of each type of media.media? Give two examples of each type of media.
Static Media – does not change over time, e.g. text, graphicsStatic Media – does not change over time, e.g. text, graphics
Dynamic Media --- Time dependent (Temporal), e.g. Video, Dynamic Media --- Time dependent (Temporal), e.g. Video, sound, animation.sound, animation.
q6) q6) Give a definition of multimedia and a multimedia system?Give a definition of multimedia and a multimedia system?
Multimedia Multimedia
is the field concerned with the computer-controlled is the field concerned with the computer-controlled integration of text, graphics, drawings, still and moving images integration of text, graphics, drawings, still and moving images (Video), animation, audio, and any other media where every (Video), animation, audio, and any other media where every type of information can be represented, stored, transmitted type of information can be represented, stored, transmitted and processed digitally.and processed digitally.
A Multimedia System A Multimedia System
is a system capable of processing multimedia data and is a system capable of processing multimedia data and applications.applications.
Q7) What are the key distinctions between multimedia data and Q7) What are the key distinctions between multimedia data and more conventional types of media?more conventional types of media?
Multimedia systems deal with the generation, manipulation, Multimedia systems deal with the generation, manipulation, storage, presentation, and communication of information in storage, presentation, and communication of information in digital form. digital form.
- The data may be in a variety of formats: text, graphics, images, - The data may be in a variety of formats: text, graphics, images, audio, video. audio, video.
- A majority of this data is large and the different media may need - A majority of this data is large and the different media may need synchronisation -- the data may have temporal relationships synchronisation -- the data may have temporal relationships as an integral property. as an integral property.
- Some media is time independent or static or discrete media: - Some media is time independent or static or discrete media: normal data, text, single images, graphics are examples. normal data, text, single images, graphics are examples.
- Video, animation and audio are examples of continuous media- Video, animation and audio are examples of continuous media
Q8) What key issues or problems does a multimedia system have Q8) What key issues or problems does a multimedia system have deal with when handling multimedia data?deal with when handling multimedia data?
A Multimedia system has four basic characteristics: A Multimedia system has four basic characteristics:
Multimedia systems must be computer controlled. Multimedia systems must be computer controlled.
Multimedia systems are integrated. Multimedia systems are integrated.
The information they handle must be represented digitally. The information they handle must be represented digitally.
The interface to the final presentation of media is usually The interface to the final presentation of media is usually interactive. interactive.
Multimedia systems may have to render a variety of media at the Multimedia systems may have to render a variety of media at the same instant -- a distinction from normal applications. There is same instant -- a distinction from normal applications. There is a temporal relationship between many forms of media (e.g. a temporal relationship between many forms of media (e.g. Video and Audio. There 2 are forms of problems here Video and Audio. There 2 are forms of problems here
Sequencing within the media -- playing frames in correct order/time Sequencing within the media -- playing frames in correct order/time frame in video frame in video
Synchronisation -- inter-media scheduling (e.g. Video and Audio). Synchronisation -- inter-media scheduling (e.g. Video and Audio). Lip synchronisation is clearly important for humans to watch Lip synchronisation is clearly important for humans to watch playback of video and audio and even animation and audio. playback of video and audio and even animation and audio. Ever tried watching an out of (lip) sync film for a long time? Ever tried watching an out of (lip) sync film for a long time?
The key issues multimedia systems need to deal with here are: The key issues multimedia systems need to deal with here are:
How to represent and store temporal information. How to represent and store temporal information.
How to strictly maintain the temporal relationships on play How to strictly maintain the temporal relationships on play back/retrieval back/retrieval
What process are involved in the above. What process are involved in the above.
Data has to represented digitally so many initial source of data Data has to represented digitally so many initial source of data needs to be digitise -- translated from analog source to digital needs to be digitise -- translated from analog source to digital representation. The will involve scanning (graphics, still representation. The will involve scanning (graphics, still images), sampling (audio/video) although digital cameras now images), sampling (audio/video) although digital cameras now exist for direct scene to digital capture of images and video. exist for direct scene to digital capture of images and video.
The data is large several Mb easily for audio and video -- therefore The data is large several Mb easily for audio and video -- therefore storage, transfer (bandwidth) and processing overheads are storage, transfer (bandwidth) and processing overheads are high. Data compression techniques very common.high. Data compression techniques very common.
Q9) Briefly describe eight hardware or software features that a Q9) Briefly describe eight hardware or software features that a Multimedia System should possess.Multimedia System should possess.
The following features should be present:The following features should be present:
o o Very High Processing PowerVery High Processing Power
needed to deal with large data processing and real time delivery needed to deal with large data processing and real time delivery of media. of media.
Special hardware commonplace.Special hardware commonplace.
o o Multimedia Capable File SystemMultimedia Capable File System
needed to deliver real-time media -- needed to deliver real-time media -- e.g. Video/Audio Streaming.e.g. Video/Audio Streaming.
Special Hardware/Software needed Special Hardware/Software needed e.g RAID technology.e.g RAID technology.
o o Data Representations/File Formats that support multimediaData Representations/File Formats that support multimedia
Data representations/file formats should be easy to handle yet Data representations/file formats should be easy to handle yet allowallow
for compression/decompression in real-time.for compression/decompression in real-time.
Efficient and High I/OEfficient and High I/O
input and output to the file subsystem needs to be efficient and input and output to the file subsystem needs to be efficient and fast.fast.
Needs to allow for real-time recording as well as playback of data.Needs to allow for real-time recording as well as playback of data.
e.g. Direct to Disk recording systems.e.g. Direct to Disk recording systems.
o o Special Operating SystemSpecial Operating System
to allow access to file system and process data efficiently andto allow access to file system and process data efficiently and
quickly. Needs to support direct transfers to disk, real-timequickly. Needs to support direct transfers to disk, real-time
scheduling, fast interrupt processing, I/O streaming scheduling, fast interrupt processing, I/O streaming etc.etc.
o o Storage and MemoryStorage and Memory
large storage units (of the order of 50 -100 Gb or more) and largelarge storage units (of the order of 50 -100 Gb or more) and large
memory (50 -100 Mb or more). Large Caches also required andmemory (50 -100 Mb or more). Large Caches also required and
frequently of Level 2 and 3 hierarchy for efficient management.frequently of Level 2 and 3 hierarchy for efficient management.
o o Network SupportNetwork Support
Client-server systems common as distributed systems common.Client-server systems common as distributed systems common.
o o Software ToolsSoftware Tools
user friendly tools needed to handle media, design and developuser friendly tools needed to handle media, design and develop
applications, deliver media.applications, deliver media.
q10) The main types of multimedia data are: graphics, images, q10) The main types of multimedia data are: graphics, images, audio,audio,
s and video. What technical issues are associateds and video. What technical issues are associated
when these data types are integrated in a Multimedia System?when these data types are integrated in a Multimedia System?
For each media type briefly relate to the issues involved in For each media type briefly relate to the issues involved in generating,generating,
capturing, storing and transmitting the respective media capturing, storing and transmitting the respective media components.components.
Multimedia issues :Multimedia issues :
o Sequencing within the media o Sequencing within the media -- playing frames in correct -- playing frames in correct order/time frame in videoorder/time frame in video
o Synchronisation -- inter-media schedulingo Synchronisation -- inter-media scheduling
o How to represent and store temporal information.o How to represent and store temporal information.
Data Types:Data Types:
GraphicsGraphics
Source: Graphics input devices include: keyboard (for text and Source: Graphics input devices include: keyboard (for text and cursor control), mouse, cursor control), mouse,
trackball or graphics tablet.trackball or graphics tablet.
Storage: Graphics are usually constructed by the composition of Storage: Graphics are usually constructed by the composition of primitive objects such asprimitive objects such as
lines, polygons, circles, curves and arcs. Graphics are usually lines, polygons, circles, curves and arcs. Graphics are usually generated by a graphicsgenerated by a graphics
editor program ( editor program ( e.g. Freehand) or automatically by a program ( e.g. e.g. Freehand) or automatically by a program ( e.g. Postscript usuallyPostscript usually
generated this way). Graphics are usually editable or revisable generated this way). Graphics are usually editable or revisable (unlike Images).(unlike Images).
Transmission: Graphics files usually store the primitive assembly Transmission: Graphics files usually store the primitive assembly and do not take up aand do not take up a
very high overhead for bandwidth (e.g. Flash Vector Graphics)very high overhead for bandwidth (e.g. Flash Vector Graphics)
ImagesImages
Source: Images may be generated by programs similar to graphics Source: Images may be generated by programs similar to graphics or animationor animation
programs. But images may be scanned for photographs or pictures programs. But images may be scanned for photographs or pictures using a digital scannerusing a digital scanner
or from a digital camera. Some Video cameras allow for still image or from a digital camera. Some Video cameras allow for still image capture also. Analogcapture also. Analog
sources will require digitising.sources will require digitising.
Storage: Images may be stored at 1 bit per pixel (Black and White), Storage: Images may be stored at 1 bit per pixel (Black and White), 8 Bits per pixel8 Bits per pixel
(Grey Scale, Colour Map) or 24 Bits per pixel (True Colour). Thus a (Grey Scale, Colour Map) or 24 Bits per pixel (True Colour). Thus a 512x512 Grey scale512x512 Grey scale
image takes up 1/4 Mb, a 512x512 24 bit image takes 3/4 Mb with no image takes up 1/4 Mb, a 512x512 24 bit image takes 3/4 Mb with no compression.compression.
Transmission: The storage overhead increases with image size so Transmission: The storage overhead increases with image size so compression iscompression is
CMP632 Multimedia Systems SOLUTIONSCMP632 Multimedia Systems SOLUTIONS
commonly applied at source for bandwidth/storage e.g. JPEGcommonly applied at source for bandwidth/storage e.g. JPEG
AudioAudio
Source: They are first captured by a microphones and then Source: They are first captured by a microphones and then digitised and stored ondigitised and stored on
diskdisk
Storage: usually compressed as CD quality audio requires 16-bit Storage: usually compressed as CD quality audio requires 16-bit sampling at 44.1sampling at 44.1
KHz. So 1 Minute of Mono CD quality audio requires 60*44100*2 KHz. So 1 Minute of Mono CD quality audio requires 60*44100*2 BytesBytes
which is approximately 5 Mb.which is approximately 5 Mb.
Transmission/Bandwidth: Maybe be compressed (MPEG) perhapsTransmission/Bandwidth: Maybe be compressed (MPEG) perhaps
streamed/copressed (Realaudio). Wav files may not be compressedstreamed/copressed (Realaudio). Wav files may not be compressed
VideoVideo
Source:Analog Video is usually captured by a video camera and Source:Analog Video is usually captured by a video camera and then digitised.then digitised.
May be digitised at source (Digital Video Camera)May be digitised at source (Digital Video Camera)
Storage: Raw video can be regarded as being a series of single Storage: Raw video can be regarded as being a series of single images. There areimages. There are
typically 25, 30 or 50 frames per second. Therefore a 512x512 sizetypically 25, 30 or 50 frames per second. Therefore a 512x512 size
monochrome video images take 25*0.25 = 6.25Mb for a minute to monochrome video images take 25*0.25 = 6.25Mb for a minute to storestore
uncompressed. Digital video clearly needs to be compressed.uncompressed. Digital video clearly needs to be compressed.
Transmission/Bandwidth: Maybe be compressed (MPEG) perhapsTransmission/Bandwidth: Maybe be compressed (MPEG) perhaps
streamed/copressed (Realvideo/Quicktime).streamed/copressed (Realvideo/Quicktime).
14 Marks Total: Unseen --- Assimilation/Extended reasoning of a 14 Marks Total: Unseen --- Assimilation/Extended reasoning of a few parts offew parts of
the coursethe course
Q11) Briefly describe five ways in which content can be Q11) Briefly describe five ways in which content can be formatted and delivered in a Multimedia Authoring formatted and delivered in a Multimedia Authoring System.System.
1. Scripting (writing)1. Scripting (writing)
Standard Text --- say what you want with wordStandard Text --- say what you want with word
2. Graphics (illustrating)2. Graphics (illustrating)
“ “A picture is worth a thousand words” say what you want with a A picture is worth a thousand words” say what you want with a graphic illustrationsgraphic illustrations
3. Animation (wiggling)3. Animation (wiggling)
Now we approach multimedia --- say what you want with a Now we approach multimedia --- say what you want with a graphic animation or videographic animation or video
4. Audio (hearing)4. Audio (hearing)
Sounds can convey alerts, ambience and contents say what Sounds can convey alerts, ambience and contents say what you want with a narrationyou want with a narration
5. Interactivity (interacting)5. Interactivity (interacting)
True mulitedia immerse yourself in am interactive True mulitedia immerse yourself in am interactive presentation, possibly more instructive. Interactive actions presentation, possibly more instructive. Interactive actions can start animations, audio, move to new parts of can start animations, audio, move to new parts of presentation, control simulations etc.presentation, control simulations etc.