ITNP80: Course Organization€¦ · University of Stirling 2 ITNP80: Course Organization • Lecturers: – Prof. Leslie Smith: room 4B85, ext 7435, email [email protected]

1 Spring 2015 © University of Stirling

2

ITNP80: Course Organization •  Lecturers:

– Prof. Leslie Smith: room 4B85, ext 7435, email [email protected]

•  Teaching (see Schedule on Course Website – for what happens when, updated regularly!) – two lectures per week (most weeks)

Tue 1200 4W3, Thur 1100 4W3 also Tue 1600 4W6 in week 2.

– one practical in most weeks (first one on 27 Jan, 4X5) – one tutorial in certain weeks

• Assessment: – assignment 50% – examination 50%

•  Course Website: www.cs.stir.ac.uk/courses/ITNP80 Spring 2015 © University of Stirling

3

What is Multimedia? Multi = �many�, Media = plural of medium

A medium is a means of communicating information

Spring 2015 © University of Stirling


4

What is Multimedia? (2)

Multimedia on computer is combining

text, graphics, sound, animation, video...

to enhance the user experience.

5

�Multimedia is the seamless integration of text, sound, images of all kinds and control software within a single digital information environment.�

Tony Feldman, multimedia consultant (quoted in England and Finney, “Managing Multimedia”)


6

Overview of IT80

•  Aim: To give you a guided tour of Multimedia, and to provide a foundation for whatever further work you may do in this area

•  Course contents span a large range of topics, so the course has more breadth than depth

•  Tour more of the form of a rollercoaster than a sedate walking tour - blink and you will miss something!


7

Topics in ITNP80 We will consider multimedia only on

computers. •  Multimedia:

–  Multimedia design –  Multimedia authoring

•  Graphics: –  Still images –  Animation and video

•  Sound –  Digitization –  Use

Software: Adobe Photoshop, Flash, Audacity


8

Multimedia Books •  Stephen McGloughlin: �Multimedia: Concepts and

Practice�, Prentice Hall, 2001 •  Nigel Chapman and Jenny Chapman:

•  �Digital Multimedia�, John Wiley, (2ndEd. 2004 or 3rd Ed. 2009) and Digital Media Tools, 3rd Ed. 2007

•  Cunliffe & Elliott, Multimedia Computing. Lexden, 2005

•  Y Rogers, H Sharp and J Preece, Interaction Design: Beyond Human-Computer Interaction, Wiley, 2011, ISBN: 978-0470665763

You are not expected to buy every book. Recommendation: buy whichever will be most helpful to you in the future & use library & on-line resources for the rest. On-line resources •  Many of them, available from the course website:

http://www.cs.stir.ac.uk/courses/ITNP80/


9

Graphics

ITNP80 (Multimedia) , Spring 2015

© University of Stirling Spring 2015 © University of Stirling

10

Overview •  Chapman & Chapman: Chapters 3, 4, 5 •  Richardson: Chapter 6 •  Representing Graphical Data (today’s lecture) •  Digitising Pictures and Text (next lecture) •  Graphical File Formats (To be covered after

the Colour Topic) •  Compressing Graphical Data (To be covered

later)


11

Representing Graphical Data Chapman & Chapman, chapters 3,4,5 Richardson


12

Overview •  In this module:

–  input, output and representation of graphical data –  Java for graphics programming

•  Wider coverage of computer graphics: –  2-D images –  3-D scenes –  virtual worlds –  general graphics programming (e.g. OpenGL)


13

Graphics in ITNP80 •  ITNP80 has a practically-oriented style:

–  Richardson’s ‘’Practical Computer Graphics’ is one of the few books to take this approach

•  The aim is not to make you a graphic designer, but instead: –  give you an understanding of issues concerning

graphics input, output and representation –  equip you for practical situations where you might

need to use graphics –  allow you to use graphics in web pages, icon design,

reports, talks


14

Representing Graphical Data •  Logical and physical aspects need defined •  Use of colour:

–  pixels –  colours –  transparency –  palettes

•  Types of representation: –  bitmaps –  vector data –  other formats


15

Physical / Logical Representation •  Differing representations of graphical data:

–  physical representation of graphical data is how it actually appears on devices

–  Virtual representation of graphical data may be in a graphics file, or internally in a program

•  We may also have a logical representation of the data’s structure in our minds –  These are often not the same: –  the differences vary from slight to very large –  converting from a virtual representation to an actual

display on a physical device is called rendering


16

Picture elements: Pixels •  Computer graphics deals with screen or display

device properties •  Displays have many small dots called pixels

(smallest screen display element) –  a pixel is the smallest logical unit of display on the

screen –  can be monochrome (black and one colour) or

(multi-) coloured –  Generally:

•  arranged logically in a 2D grid •  All the same size

–  e.g. a 3 x 3 array of coloured pixels:


17

Pixels •  Physical display may not be a perfect 2D grid:

•  Colours specified by Red (R), Green (G), Blue (B) values •  To display an image on the monitor, a graphics display

program renders coloured areas using appropriate pixel settings i.e. set each pixel to an appropriate colour or shade of grey, so that pattern of pixels on the screen produces the desired image. Spring 2015 ©

University of Stirling 18

Colours of Pixels •  Black or white pixels need a single bit

–  1 on/white; 0=off/black •  Colours are specified (or converted) using

colour code values (e.g. RGB values) in some way

•  A typical format (RGB) uses 24 bits format: 8 bits: (R,G,B) takes up 1 byte for each colour

•  A common feature these days is to also have an alpha channel , specifying a level of transparency. e.g. in Java 2 –  (R,G,B,α) takes up 4 bytes –  Examples. See also Chapman & Chapman p 135


19

Transparency


20

Colour Terminology •  Lots of confusion!

–  ‘black and white’ is often not a good terminology to use –  black and white photographs are not black or white, but

grayscale –  grayscale refers to shades of gray, where the RGB values

are all the same –  monochrome refers not to one single colour, but

historically to one colour with black, so monochrome really means two colours, usually black and white (but may mean grayscale)

–  monochromatic in colour blindness refers to grayscale!


21

Palettes •  A palette is a mapping from a small set of numbers to

specifically chosen colours from a wide range (224 typically): •  Many images need only few specific colours:

–  a palette is a subset of colours from a larger range –  an image file can have its own palette (saved as part of image

file) –  ‘indexed images’ use palettes (as lookup tables) –  Used in various file formats, monitor displays

•  Example: A web-safe palette – 216 colours reproduced by all web-browsers on any system (using 8-bit colour display) –  not very good for photo quality images

•  Less important in these days of huge memories and high bandwidths

•  Chapman & Chapman: Pages 161-165 Spring 2015 © University of Stirling

22

Representing Image Data: Bitmap Images

•  Bitmapped formats: –  model an image as an array of pixel values –  Bitmap data is (logically) a 2-D array of pixels –  A bitmap gives the colours of the picture, pixel-by-

pixel (bit-by-bit), in this example: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 1 1 1 0 0 1 1 1 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

each pixel is represented by one bit (on or off) Spring 2015 © University of Stirling

23

Bitmaps •  A Bitmap was also known as a raster maps (the

term is still in use in some circumstances- define lines of dots (pixels)): –  when there were just monochrome monitors, bitmaps

really did code with single bits per pixel –  when colour was introduced, the term pixelmap was

used for coloured images •  Nowadays, bitmaps refer to 2-D arrays of bits or

colours •  Each pixel is mapped to graphics/video memory – one

or more bits per pixel •  Logically bitmaps are 2-D arrays, but may be stored

in other ways (e.g. Java 2 used a 1-D int array) Spring 2015 © University of Stirling

24

Graphical Data Representation •  Bitmaps have a fixed resolution (level of detail) in an

image: –  limited by number of pixels on display screen

•  There are other ways of representing image data which do not have a fixed resolution – the resolution varies dependent upon the content of the image: –  Some are general purpose –  Some are program-specific –  Some are application-specific

•  State-of-the-art graphics programs have proprietary formats (images are represented internally in an application-specific way), but then exported to bitmap formats for display

•  Richardson: Section 1.4 Spring 2015 © University of Stirling

25

Vectors: Another way to represent image data

•  Vector-based formats contain descriptions of one or more objects, rather than pixels

•  Uses a �draw-then-edit� method of image creation •  Often the objects are mathematically based/specified

–  eg line segments, polygons, circles, splines –  Image is stored as a mathematical description of objects

e.g. collection of lines, curves & shapes making up the image


26

Vector Images •  Vector formats are well suited to 2-D images:

–  line drawings, graphs, architectural/engineering drawings, …


27

Bitmap vs. Vector Format •  Bitmap format:

–  fixed resolution –  every logical pixel is stored –  scaling leads to loss of quality

•  Vector format: –  description of component shapes –  can be displayed at any scale



28

Scaling Bitmaps •  Resampling •  Increase in size: add new pixels

–  upsampling

•  Decrease in size: throw pixels away –  downsampling

(Images © MacAvon Media Productions)


29

Scaling Bitmaps (2) •  How best to calculate new pixel

colours? –  Pixel interpolation

•  Nearest neighbour –  Choose colour of pixel with largest

overlap •  Bilinear interpolation

–  Average colours of surrounding pixels –  Weight by their level of overlap

•  More complex mappings –  Bicubic interpolation (Image ©

MacAvon Media Productions) 30

Scaling Bitmaps (3)

(Image © MacAvon Media Productions)

Nearest neighbour

Bilinear interpolation

Bicubic interpolation


31

Bitmaps vs Vector Files Editing a vector file (objects can be selected & transformed using mathematical operations)

…can�t in a bitmap? (as these

are stored as

collection of bits, editing can only take place at level of pixels!) Spring 2015 © University of Stirling

32

Bitmap vs. Vector Format •  Advantages of vector format:

–  good for storing images composed of line-based or 3-D objects (e.g. wire-frame models)

–  easy to convert to bitmap format –  Can be very compact

•  Disadvantages of vector format: –  not good for storing complex images (e.g.

photographs) –  Appearance of image can vary widely, depending

upon the Application –  rendering the image may take significantly longer

than for bitmaps Spring 2015 © University of Stirling

33

From Vectors to Bitmaps… •  Historically, vector data was widely used:

–  Pen plotters used pens to draw on paper (an early form of graphics printer) – these were cheap and produced line-based drawings

•  Mostly diagrams, graphs, rather than photographs! –  storage of high-volume bitmap files was expensive

•  With advent of cheap storage and high-resolution output, many images are now bitmaps. Bitmaps are everywhere! –  digital cameras –  Just look at the web use of images and graphics (GIFs,

PNGs, JPEGs everywhere!)


34

…and back again •  Trends are shifting towards a greater use of vector

data – the bitmap trend may not last. Size is again an issue... –  big bitmaps take more memory, storage and transmission time

over the internet •  High resolution screens: 3840x2160 = 25Mbytes/image

–  vector formats are better for 3-D imaging, and 3D imaging is growing more important

•  Fuelled by such concerns as film-making, entertainments industry

•  Other Graphics Representations: –  hybrid, e.g. metafile formats and fractal representation

techniques –  Animation formats including Special purpose 3-D formats


35

Metafile Formats •  A metafile can store both vector and bitmap data:

–  usually for vector files that might have some bitmapped data added

–  a bitmap is typically regarded as one type of ‘vector’ object

–  typically most elements in the file are vectors, with occasional bitmaps (e.g. file contains vector instruction to draw square then fill with bitmap pattern)

•  Richardson: Section 6.9 Spring 2015 © University of Stirling

36

Fractal Image Representation: Sierpinski Triangle

•  Same structure at all levels of detail, i.e. self-similar at all scales

•  Can use any repeated shape


37

Fractal Image Compression •  A recent development in image compression:

–  An image (or part of an image) is represented by a mathematical formula which represents a pattern that repeats at different scale (aim is to describe underlying patterns/self-similarity in an image)

–  to display an image on a device, the formula is repeatedly applied to (perhaps) a blank ‘seed’ image of required size

–  a resolution-independent way of storing images (takes less space than a bitmap)

•  Really more a form of image encoding than compression for its own sake –  It is compression because the formula takes up less space

than a bitmap would. –  Many real-world scenes (e.g. mountains, clouds, landscapes)

can be described fractally Spring 2015 © University of Stirling

38

Fractal Image Compression •  Typically an image would start in bitmap form •  The compressed image has an optimal

resolution close to that of the original image –  result is softer, not pixellated –  sometimes used for image enhancement

•  Self-similar images compress well: –  works for many photos, e.g. of forest, sea or sky

•  Compression/encoding takes time, but decompression/decoding is very fast

•  Chapman & Chapman: Pages 109-110



39

A Fractal Mountain


40

Fractal Clouds


41

Bitmap versus Fractals Original bitmap x2 Fractal version

Fractals are particularly good when generating synthetic images!


42

Practical Graphics Issues

43

Digitising Pictures and Text


44

Overview •  Chapman & Chapman: Chapters 4 & 7 •  Digitizing and Rendering

–  From image to bitmap •  Anti-aliasing

–  What is it and how to do it? •  Lines •  Shapes

•  Fonts –  Anatomy and types of a fonts –  How to make fonts look nice –  Font sources


45

Digitising Pictures •  How do we go from a continuous image to a bitmap? •  A picture is scanned/sampled and the bits of the bitmap are set

according to what colour �wins� for that pixel - resulting in a bitmap for the ‘best’ colour in each pixel: –  results in jagged edges


46

Digitising Pictures •  The same principle applies to rendering text

(text reproduction), and to converting vector to bitmap formats (which is first step in rendering an image!)


47

Digitising Pictures •  Each bit/pixel set to one of two colours

because screens were originally binary: –  results are not often good

•  Later monochrome images: e.g. gray levels –  how to achieve better results with colour monitors

that can also display grayscale? •  Monochrome bitmaps are still useful for high-

contrast images: –  e.g. dark text on a light background

•  Non-colour printing involves just one colour (generally black) on white paper


48

Anti-Aliasing •  Previous examples for text use only two colours •  The results can be more visually appealing if a range of colours is

used at the edges of an object. Anti-aliasing effectively blurs the edges to create a smoother edge. •  - use different shades of colour or gray to reduce jaggedness

•  Richardson: Sections 1.5, 7.3


49

Anti-Aliasing (a close-up)


50

Anti-Aliasing Shapes •  Specify gray level or opaqueness in proportion

to coverage of shape


51

Anti-Aliasing Lines •  Opaqueness in proportion to how many subpixels the line passes

through •  Sub-divide each pixel into sub-grids •  Count how many sub-pixels the line passes through to determine

gray level


52

Fonts •  Historically, computer did not have fonts

(styles of text) as such •  Characters were just characters, displayed as

dots: –  on screen (fixed number of columns and rows) –  on printer (dot matrix, daisywheel)

•  Chapman & Chapman: Chapter 7 •  Richardson: Chapter 5


53

Anatomy of A Font

One ‘point’ is 1/72 of an inch: an old measurement

descent!sans serif

serif

ascent

leading

baseline

point size descent

Point size is a measure of the body size (the distance from the top of the highest character to the bottom of the lowest character) in 72nds of an inch



54

Fonts •  Not to be confused with character sets! •  Fonts are different ways (styles) of representing

characters •  e.g. to get the Greek letter alpha,

–  �a� in �Symbol� font is displayed as �α� –  α can also be obtained as character 03B1x of the

Unicode character set •  Not a 1-1 correspondence between characters and

font shapes (glyphs) –  some characters consist of more than one glyph –  some glyphs represent a combination of characters

(ligatures): N.B. A glyph is *not* a character. It is a shape for one (or more) characters.

55

Some Common Fonts For: normal writing, display, script, symbol, fixed-width, fractional widths early

development typewriter

•  Arial (sans serif, TrueType) •  Times New Roman (serif, TrueType) •  Lucida Console (TrueType, fixed-width)

•  Courier New (serif, TrueType, fixed-width)

• Courier (serif) • Script (Script, a script font)•  Σψνβολ (Symbol, a symbol font) �  !"#$%"#$& (Wingdings, a symbol font) Spring 2015 © University of Stirling

56

Bitmapped Fonts •  Each font shape/glyph is represented as a

bitmap so scaling doesn’t work well •  Characters do not scale

•  Entire font families are needed at different

resolutions (how fonts were used in early word processors), e.g.

8 point 10 point 12 point 14 point 16 point 18 point

20 point 24 point 36 point

72 point Spring 2015 © University of Stirling

57

Vector-Based Fonts •  Nowadays, vector-based fonts have taken over

–  referred to as scalable or outline fonts

•  These are defined in terms of: –  a set of component shapes (straight lines and curves) –  hints (see later)

•  Scalable fonts are converted to bitmapped glyphs as needed –  for rendering on a screen or a printer

•  Aside: –  Java�s fonts in Java2D are scalable fonts –  These use Unicode as the character set


58

Displaying Fonts Nicely •  Solutions from printing industry •  Solutions specific to computer-based fonts •  Ways to do this include:

–  Non-fixed-width –  Anti-aliasing –  Hinting –  Kerning (spacing)


59

Anti-Aliasing Example


60

Anti-Aliasing Text •  The result may not always be better:

–  e.g. here’s a 8pt plain ‘A’ in Times New Roman •  normal anti-aliased


61

Problems with Small Text •  Digitisation may result in uneven display of

small text: –  consider the results for ‘h’ and ‘m’



62

Hinting •  Hinting is the provision of hints as to how to

display fonts in certain circumstances •  A font may be defined without hints, but

having hinting results in a better quality font A Microsoft hint to leave vertical and horizontal strokes solid!

hinted!

63

Kerning •  Improves text appearance by changing the

spacing of adjacent text (even to the point of overlapping)

•  Done by use of kern pairs •  A high quality font (such as TrueType fonts) have

suitable kern pair settings for different combinations of characters

•  TrueType fonts may have anything from 100 to 500 kern pairs (150 is typical) depending upon potential character overlaps.



64

Without kerning:

With kerning:

65

Font Foundries/Types The rivals! •  Adobe Postscript (industry standard):

–  a page description language (PDL) for printers –  often directly hardware-implemented in printer –  provides many fonts (more than 20,000) –  expensive licence

•  Apple TrueType (licensed by Microsoft): –  A collection of software-implemented, high quality vector-

based fonts –  Suitable for non-printing uses –  Converted to bitmaps which are then sent to a printer –  collection of high quality fonts

•  Both are scalable and use hinting Font foundries •  Other fonts required can be purchased! •  NEXT LECTURE ON “COLOUR” Spring 2015 © University of Stirling

Documents

ITNP80: Course Organization€¦ · University of Stirling 2 ITNP80: Course Organization • Lecturers: – Prof. Leslie Smith: room 4B85, ext 7435, email [email protected]