14
Multimodal Semiotics. The Importance of Visual Communication and Interaction. Dienstag, 10. August 2010

Multimodal Semiotics

  • Upload
    svngl

  • View
    2.263

  • Download
    18

Embed Size (px)

DESCRIPTION

 

Citation preview

Page 1: Multimodal Semiotics

Multimodal Semiotics.The Importance of Visual Communication and Interaction.

Dienstag, 10. August 2010

Page 2: Multimodal Semiotics

Outline

1. Context

2. Thesis

3. Reading Multimodal Texts

1. Modalities

2. Approaches of Analysis

4. Case studies

1. BooneOakley

2. The Guardian: War Logs

3. Driving Game (NYT)

4. Alice in wonderland

5. Sources

Dienstag, 10. August 2010

Page 3: Multimodal Semiotics

Current tendencies influencing digital content production

Rethinking Browser and Data Storage

Desktop-Application-Flow of Web Services (AJAX, HTML5)

Supply, Policies and Control (App-Store)

Advertising (iAds) and monetarizing (Flattr)

Application Web,Multimodal Content and Interactions, APIs

Dienstag, 10. August 2010

Page 4: Multimodal Semiotics

Thesis

The production of content and information architecture is not exhausted in structuring information.

The meaning of texts is determined by more than verbal, visual and interactive elements.

Visuality ≠ Decoration

Digital Content > Visuality + Verbality

Dienstag, 10. August 2010

Page 5: Multimodal Semiotics

Modalities of (current) Online and Mobile Media

VISION: Fonts, Images, Colors, Layout, Brightness...

AUDITION: Sound, Speech, Music ...

TACTITION: Vibrating, Pressure, Gestures

EQUILIBRIOCEPTION: Balance

Dienstag, 10. August 2010

Page 6: Multimodal Semiotics

Meaning of visual texts

tied to the context of the recipient

production of knowledge

grammar is defined by culture and social life

heavily uncodified and often subtile

analyzed in a bunch disciplines

Dienstag, 10. August 2010

Page 7: Multimodal Semiotics

Concepts

Discouse

“a set of statements that articulate a particular way of thinking, feeling and being in the world” (Cranny-Francis, 2005, P. 41)

Genre

“it enables us to understand the kinds of meanings users do generate from them [the texts]” (Cranny-Francis, 2005, P. 41)

Intertexuality

“enables us to understand how the web site relates to other visual genres - not simply formal, but also in terms of (potential) meanings.” (Cranny-Francis, 2005, P. 43)

Dienstag, 10. August 2010

Page 8: Multimodal Semiotics

CASE STUDIES

Dienstag, 10. August 2010

Page 9: Multimodal Semiotics

BooneOakleyhttp://www.booneoakley.com

Dienstag, 10. August 2010

Page 10: Multimodal Semiotics

BooneOakley

multiple

drawing style + childish storytelling

message: min. efforts, max. effect

navigating through timeline

embedding and other youtube-features

Dienstag, 10. August 2010

Page 11: Multimodal Semiotics

Afganistan War Logshttp://www.guardian.co.uk/world/the-war-logs

Dienstag, 10. August 2010

Page 12: Multimodal Semiotics

War Logs

Access to Source-Data / Documentation

Interactive Mapping

Timeline

Datajournalism: Presentation of Statistics

Meaning of Statistics: Non-emotional, rational approach to war events. Subtile Opinionmakings.

Dienstag, 10. August 2010

Page 13: Multimodal Semiotics

Ludic Journalism/Storytelling

(1) Driven to Distruction (NJT) http://www.nytimes.com/interactive/2009/07/19/technology/20090719-driving-game.html

(2) Alice in Wonderland (iPad-App)http://www.youtube.com/watch?v=gew68Qj5kxw

Dienstag, 10. August 2010

Page 14: Multimodal Semiotics

Sources

Cranny-Francis, A. (2005): Multimedia: texts and contexts. London: SAGE.

Foucault, M. (1969) The Archaeology of Knowledge. London: Routledge.

Halliday, M. A. K. (1978) Language as social semiotics: The social interpretation of language and meaning. Maryland: University Park Press.

Kress, G., and T. van Leeuwen (1996) Reading Images: The Grammar of Visual Design, London: Routledge.

Kress, G., and T. van Leeuwen (2001) Mulitmodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.

Mitchell, W. J. T. (2005) What do Pictures Want? The Lives and Loves of Images. Chicago: University of Chicago Press.

Morville, P. / Rosenfeld, L. (2006) Information Architechture for the World Wide Web 3rd Ed., Sebastopol: O’Reilly.

Dienstag, 10. August 2010