HTML5 multimedia

Patrick H. Lauke / JSDay / Verona / 17 Maggio 2012

BROWSER-NATIVE VIDEO AND AUDIO

<video>

Adobe Flash currently most commonvideo/audio delivery mechanism

<object width="425" height="344"><param name="movie"

value="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&fs=1&"></param>

<param name="allowFullScreen" value="true"></param>

<param name="allowscriptaccess" value="always"></param>

<embed src="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&fs=1&" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="425" height="344"></embed></object>

plug-in is a black box

<video src="video.webm"></video>

originally proposed by Opera in 2007dev.opera.com/articles/view/labs-a-call-for-video-on-the-web

<video src="video.webm" controls autoplay muted loop preload="none|metadata|auto" poster="poster.jpg" width="320" height="240"> <a href="video.webm">Download movie</a></video>

video as native object

people.opera.com/patrickl/experiments/webm/hover+transition

powerful (simple) API

<video> and JavaScript

var v = document.getElementById('player');

v.play()v.pause()

v.volume = … v.currentTime = …

v.addEventListener('loadeddata', function() { … }, true)v.addEventListener('play', function() { … }, true)v.addEventListener('pause', function() { … }, true)v.addEventListener('timeupdate', function() { … }, true)v.addEventListener('ended', function() { … }, true)

www.w3.org/2010/05/video/mediaevents.html

people.opera.com/patrickl/experiments/webm/basic-controls

HTML5 does not specifyhow browser controls should look

people.opera.com/patrickl/experiments/webm/fancy-controls

people.opera.com/patrickl/experiments/webm/fancy-swap

video formatsthe big debate?

HTML5 does not specifyvideo container/codec(same as with images in HTML 4.01)

MP4/H.264or

Ogg Theoraor

WebM/VP8

MP4 / H.264

ubiquitous, licensing/royalties

Ogg Theora

free and open, no licensing feesnot many tools for it, not web optimised

WebM / VP8

open and royalty-free, web-optimisedsupport by hardware and software vendors

http://tools.google.com/dlpage/webmmf

Browsers vs codecs

mobile and devices slightly better...

Chome Firefox IE Opera Safari

Ogg Theora ✔ ✔ ✔

WebM / VP8 ✔ ✔ ? ✔

MP4 / H.264 ✔ ✔ ✔

providing multiple sources<video controls width="…" height="…">

<source src="movie.mp4" type="video/mp4"><source src="movie.webm" type="video/webm"><source src="movie.ogv" type="video/ogg"><!-- fallback content -->

</video>

specifying codecs<video controls width="…" height="…">

<source … type='video/mp4; codecs="avc1.42E01E,mp4a.40.2"'><source … type='video/webm; codecs="vorbis,vp8"'><source … type='video/ogg; codecs="theora,vorbis"'><!-- fallback content -->

</video>

understanding fallback<video controls width="…" height="…">

<source … type='video/mp4; codecs="avc1.42E01E,mp4a.40.2"'><!-- fallback content →<p>Shown to browsers that don't support video element,and NOT just browsers that don't support MP4/H.264</p>

</video>

people.opera.com/patrickl/experiments/webm/wrongsource

feature-detection for codecs

v.canPlayType('video/webm;codecs="vp8,vorbis"')

"probably" | "maybe" | "" < WAT?

video and Responsive Web Design<source … media="(min-width:1000px)"><source … media="(max-width:1000px)">

people.opera.com/patrickl/experiments/webm/mediaquery

people.opera.com/patrickl/experiments/webm/mediaquery

flash fallback for older browsershttp://camendesign.com/code/video_for_everybody

<video controls width="…" height="…"><source src="movie.mp4" type="video/mp4" /><source src="movie.webm" type="video/webm" /><source src="movie.ogv" type="video/ogg" />

<object width="…" height="…" type="application/x-shockwave-flash" data="player.swf">

<param name="movie" value="player.swf" /><param name="flashvars" value=" … file=movie.mp4" /><!-- fallback content -->

</object>

</video>

<audio>

audio...exactly the same as video

<audio src=”music.mp3” controls autoplay … ></audio>

<audio controls autoplay><source src="music.mp3" type="audio/mpeg" /><source src="music.oga" type="audio/ogg" /><!-- fallback content -->

</audio>

formats: MP3 vs Ogg Vorbis (vs WAV)

people.opera.com/patrickl/experiments/audio/windchime

no need for <audio> or <video> in your DOM

var s = document.createElement('audio');s.src = 'aaargh.oga'; s.play();

audio “sound sprites” to counterlatency and overhead

people.opera.com/~emoller/audio-demo

<canvas>

canvas drawing ready-made images

ctx = canvas.getContext("2d");

var logo = new Image();logo.src = 'logo.png';

ctx.drawImage(logo,x1,y1,w1,h1,x2,y2,w2,h2);

or call in an existing image already on the page

canvas access to image data array

ctx = canvas.getContext("2d");canvasData = ctx.getImageData(x,y,w,h);

[R,G,B,A,R,G,B,A,R,G,B,A,R,G,B,A, … ]

canvas also works with video

ctx = canvas.getContext("2d");v = document.getElementById('player');

ctx.drawImage(v,x1,y1,w1,h2,x2,y2,w2,h2);

grab currently displayed frame (update as appropriate)

html5doctor.com/video-canvas-magic

media.chikuyonok.ru/ambilight

is it all safe to use, right now?

www.youtube.com/html5

don't do browser sniffing

http://www.flickr.com/photos/timdorr/2096272747/

feature-detectionprogressive enhancement, graceful degradation

diveintohtml5.info/everything.html

feature-detection for audio/video

if (!!document.createElement('video').canPlayType;) { … }

if (!!document.createElement('audio').canPlayType;) { … }

sublimevideo.net

www.jplayer.org

www.textfiles.com/underconstruction

dev.w3.org/2011/webrtc/editor/webrtc.html

dev.w3.org/2011/webrtc/editor/getusermedia.html

camera access// Operanavigator.getUserMedia({video:true},success,error);[…]video.src = stream;

// WebKitnavigator.webkitGetUserMedia("video",success,error);[…]video.src = window.webkitURL.createObjectURL(stream);

gUM Shield: http://html5doctor.com/getusermedia/

www.opera.com/browser/next

neave.com/webcam/html5

shinydemos.com/qr-code

www.opera.com/developerpatrick.lauke@opera.com