Taking HTML5<video>

a step further

Silvia PfeifferMozilla / W3C / Xiph /Annodex / Vquence

Storyboard

» Capabilities of <video>

» Aspects of video usability and accessibility

» Designing solutions

Capabilities of<video>

<video> is simple, sexy

<videosrc="elephants_dream.ogv"poster="elephants_dream.png"controls>

</video>

Compare to Flash<OBJECT

classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000"codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" WIDTH="550" HEIGHT="400"id="myMovieName">

<PARAM NAME=movie VALUE="myFlashMovie.swf"><EMBED href="myFlashMovie.swf" WIDTH="550"

HEIGHT="400" NAME="myMovieName" ALIGN=""TYPE="application/x-shockwave-flash"PLUGINSPAGE="http://www.macromedia.com/go/getflashplayer"></EMBED>

</OBJECT>

Video and CSS

http://www.zachstronaut.com/lab/isocube.html

.left {-webkit-transform: rotate(15deg) skew(15deg, 15deg);

-moz-transform: rotate(15deg) skew(15deg, 15deg);

}

Video, Canvas, CSS + js

https://developer.mozilla.org/samples/video/chroma-key/index.xhtml

#c2 { background-image: url(foo.png); background-repeat: no-repeat; }

Video and Face Detection

http://hacks.mozilla.org/2009/06/connecting-html5-video/comment-page-1/

1:16m : twitter and video2:00m : face recognitionFull-screen

Dynamic Content Injection

http://people.mozilla.com/~prouget/demos/DynamicContentInjection/play.xhtml

<canvas id="mirrorVideo"/><svg id="playButton"

xmlns="http://www.w3.org/2000/svg" width="100"height="100">

<path d="M 40 70 L 70 50 L 40 30"/> <circlecx="50" cy="50" r="40"onclick="processor.playVideo()"/>

</svg>

Video in SVG

http://www.double.co.nz/video_test/video.svg

<foreignObject> <div xmlns="http://www.w3.org/1999/xhtml"> <video src="myvideo.ogg"/> </div></foreignObject>

Video and Physics

http://people.mozilla.com/~blizzard/launch/

Javascript and video for the win!

Aspects ofVideo Usability

Keyboard Control

» @controls space bar → / ← CTRL + → / ← HOME + → / ← ↑ / ↓ on volume button

» Exposure to screen readers

Deep Access to Video

http://www.annodex.net/~silvia/itext/mediafrag.html#t=30

Media Fragment URIs:• Time offset browser bar• Time offset <video> @src

Aspects ofVideo Accessability

Screen Reader ARIA

» @aria-label

<video class="v" src="elephant.ogv"poster="elephant.png”controlsaria-label="Elephants Dream video"title="Elephants Dream video">

</video>

Sign Language

http://www.ehow.com/video_4403510_sign-language-movies.html

Signing by humans

http://www.vcom3d.com/vault_files/forest_asl/

Or by avatars in the future

Audio Descriptions

http://www.youtube.com/watch?gl=AU&hl=en-GB&v=i2VXp0s0BLw

Spoken by a human

What can be donewith text

Captions / Subtitles / AudioDescriptions

Intro video

http://localhost/~silvia/mozilla/itext/webjam.html

Other time-aligned text

» Lyrics» Karaoke» Ticker Text» Annotations/active regions» Transcripts» Linguistic markup» Timed Metadata

Advantages

» Search engines» Cross-site use» Linking between videos» Advertising -> Monetisation

A structuredapproach:

out-of-band text

SRT example100:00:15,000 --> 00:00:17,951At the left we can see...

200:00:18,166 --> 00:00:20,083At the right we can see the...

300:00:20,119 --> 00:00:21,962...the head-snarlers

LRC example

[ti:Let's Twist Again][ar:Chubby Checker oppure Beatles, The][au:Written by Kal Mann / Dave Appell, 1961][al:Hits Of The 60's - Vol. 2 – Oldies]

[00:12.00]Lyrics beginning ...[00:15.30]Some more lyrics ...

Text Abstraction» Start time» End time» Text» Onenter event» Onleave event» [Styling]» [Hyperlink]https://wiki.mozilla.org/Accessibility/HTML5_

captions_v2

Associate with <video>

<video src="elephants_dream.ogv controls> <itextlist category=”SUB"> <itext src=”subtitle_en.srt" lang="en"/> <itext src=”subtitle_de.srt" lang="de"/> <itext src=”subtitle_fr.srt" lang="fr"/> <itext src=”subtitle_jp.srt" lang="jp"/> </itextlist></video>

Capabilities itextlist

» API: @category @onenter @active @onleave @name

» Default display in menu inaudio/video controls

» Default styling for categories

Capabilities itext

» API: @src @charset @lang @delay @type @stretch

» JavaScript API: currentText allText langName

» Default choice by browser language

Actual Examples

http://www.annodex.net/~silvia/itext/

SubtitlesCaptionsChapter MarkersTextual Audio DescriptionsLyricsStyling

A structuredapproach:in-line text

Multitrack video

» Video: Sign language

» Audio: Dubbed tracks Audio descriptions

» Text: Captions / subtitles / etc

Aim

» Identical Interface» JavaScript» CSS» Media-format

State ofAffairs

Implementations

» JavaScript prototype Firefox 3.5 / Chromium / Safari 4.03 /

Opera 9.62» Firefox

Keyboard control Deep linking () Out-of-band patch In-line patch

Standardisation

» WHATWG discussions» W3C discussions

HTML WAI-XTECH

» In-principle Google, Opera» 1st November, Stanford Uni,

W3C Workshop on Video A11y

Silvia Pfeiffer

silvia.pfeiffer1@gmail.comhttp://blog.gingertech.netTwiter: @silviapfeiffer