The Immobile Web

!e Immobile Web

Jason Grigsby • @grigs • cloudfour.comSlides: bit.ly/immobilism

Follow along

Tweets:@grigsSlides: bit.ly/immobilism

http://www.flickr.com/photos/stevegarfield/4247757731/

http://www.flickr.com/photos/alphachimpstudio/4990357031/

Photo by Alan Light http://www.flickr.com/photos/alan-light/216012860/

Image source: http://gigaom.com/video/99-dollar-logitech-revue-worth-buying/

Vimeo on Google TV

Woah… it’s just a web page?

View in Couch Mode

If there is a coming zombie apocalypse of devices…

http://www.flickr.com/photos/adactio/6301799843

then TVs are likely the next wave.http://www.flickr.com/photos/athena1970/3935208655/

consumes 32.7% of peak downstream traffic in U.S.

Big enough to get its own button.

88%tablet

86%smartphone

People often use mobile while watching tv.

owners say they use their device while watching TV at least once a month.

http://blog.nielsen.com/nielsenwire/online_mobile/double-vision-global-trends-in-tablet-and-smartphone-use-while-watching-tv/

http://www.flickr.com/photos/acaben/541334636/

“I "nally cracked it.”

http://www.flickr.com/photos/yamagatacamille/4267887034/

http://allthingsd.com/20091111/nokia-apple/

#e current Apple TV doesn’t have a browser…

http://www.flickr.com/photos/acaben/541334636/

[People] “don’t want a computer on their TV,” Apple CEO Steve Jobs said today. “#ey have computers. #ey go to their wide-screen TVs for entertainment. Not to have another computer. #is is a hard one for people in the computer industry to understand, but it's really easy for consumers to understand. #ey get it.”

http://www.flickr.com/photos/acaben/541334636/

And so, it turns out people want keyboards. I mean, when I started in this business one of the biggest challenges was that people couldn’t type.…

And if you do email of any volume, you gotta have a keyboard. So we look at the tablet and we think it’s gonna fail.

—Steve Jobs, 2003

http://www.flickr.com/photos/acaben/541334636/

Big opportunity is an App Store on Apple TV.

Major Brands Using Combo of Web and Native

Even Apple uses embedded webviews

Apps =

http://www.flickr.com/photos/34818713@N00/1314251273/

Apps =

http://www.flickr.com/photos/34818713@N00/1314251273/

Embedded Web Views =

Apps =

http://www.flickr.com/photos/34818713@N00/1314251273/

Embedded Web Views = 3rd Party Browsers

Apps =

If that is true, don’t you think Apple will ship Safari?http://www.flickr.com/photos/34818713@N00/1314251273/

Embedded Web Views = 3rd Party Browsers

“By the summer of 2012, the majority of the televisions you see in stores will have Google TV embedded in it”

Photo by JD Lasica/Socialmedia.bizhttp://www.flickr.com/photos/jdlasica/5181380514/

?!?!

Samsung 23.6%

Vizio 15.4%

LG Electronics 12.4%

Sony 8.0%

Toshiba 7.8%

Others 32.8%

Q4 2011 US LCD TV Market Share

Samsung 23.6%

Vizio 15.4%

LG Electronics 12.4%

Sony 8.0%

Toshiba 7.8%

Others 32.8%

Q4 2011 US LCD TV Market Share

Samsung 23.6%

Vizio 15.4%

LG Electronics 12.4%

Sony 8.0%

Toshiba 7.8%

Others 32.8%

Q4 2011 US LCD TV Market Share

Samsung 23.6%

Vizio 15.4%

LG Electronics 12.4%

Sony 8.0%

Toshiba 7.8%

Others 32.8%

Q4 2011 US LCD TV Market Share

?

=

=

http://www.unwiredview.com/2011/12/21/andy-rubin-%E2%80%9Candroid-daily-activations-top-700k-a-day%E2%80%9D-on-the-way-to-1-million-a-day-in-q2-2012/

Will Google TV follow Android’s path?

Considering TVs helps inform how we build for mobile.

http://www.flickr.com/photos/revdancatt/3789612273/

And can help us avoid short-sighted solutions

http://www.flickr.com/photos/pss/4876189045/

So let’s dig in shall we?

First, the good news.

Surprisingly good browsers in newer TVs.

Year Brand Model Score Bonus Points Notes

2011 LG 60PZ950 221 6

2011 Sony KDL55HX729 222 6

2011 Sony NSZ-GT1 343 8 Google TV v3.2

2012 LG 47LS5700 302 8

2012 LG 55LM62BND1 302 8

2012 Sharp LC80LE844 281 0

2012 Samsung UN55ES6100 229 2

2012 Samsung UN55ES8000 244 2

2012 Sony KDL55HX750 267 6

2012 iPhone 4S 305 9 Running iOS 5.1

2012 Google Chrome 378 13 Chrome 18 on Mac

html5test.com

Total possible points: 475

• backgroundsize

• borderimage

• borderradius

• boxshadow

• canvas

• canvastext

• csstransforms

• csstransitions

• fontface

• generatedcontent

• hashchange

• hsla

• input:autofocus

• input:max

• input:min

• input:pattern

• input:required

• input:step

• multiplebgs

• opacity

• postmessage

• rgba

• sessionstorage

• smil

• svg

• svgclippaths

• textshadow

100% of Smart TVs tested support

• localstorage

• applicationcache

• csscolumns

• cssgradients

• history

• input:multiple

• input:placeholder

• inputtypes:email

• inputtypes:number

• inputtypes:search

• inputtypes:tel

• inputtypes:url

• webworkers

• audio:m4a

• audio:mp3

• cssanimations

• cssre!ections

• draganddrop

• !exbox-legacy

• inputtypes:range

• video:h264

• websqldatabase

70% or more support

• websockets

• inlinesvg

• inputtypes:date

• inputtypes:datetime

• inputtypes:datetime-local

• inputtypes:month

• inputtypes:time

• inputtypes:week

• input:autocomplete

• input:list

• geolocation

• indexeddb

• inputtypes:color

• audio:wav

• webgl

• audio:ogg

• csstransforms3d

• !exbox

• touch

• video:ogg

• video:webm

• audio*

• video*

Poor support for the following

*!is has to be a mistake.

More importantly, the 2012 models are much faster.

More importantly, the 2012 models are much faster.

(How fast remains a secret. No one publishes CPU speeds.)

So what’s it like to use these TVs?

http://www.flickr.com/photos/nathaninsandiego/4829858186/

#e biggest problem is input.

Welcome back T9. We’ve missed you.

Which is why you see remotes like this

Apple hasn’t solved this problem either

Apple’s iOS Remote App Makes Life Bearable

Nearly every manufacturer has their own apps now.

Nearly every manufacturer has their own apps now.

Nearly every manufacturer has their own apps now.

One of the core lessons:Multi-screen world.

It’s a multi-screen world.

http://www.flickr.com/photos/marcof/5139160297/

How does the web play in this new world?

Two main methods of input on these TVs.(Other than linked apps)

D-pads

Pointers

http://thecoolgadgets.com/lg-magic-motion-remote-control-updating-with-3d-button-gesture-and-voice-recognition/

When done properly, D-Pad works very well.

(Free Opera TV SDK Emulator!)

When done properly, D-Pad works very well.

(Free Opera TV SDK Emulator!)

But when D-pads control on-screen pointers… Yuck!

Pointers work well with on screen cursors, but are harder to use

precisely. Easy to miss your target.

Well implemented D-pad is faster.

Plenty of experiments.

Touchpads integrated into remotes

Voice control?

Gesture Controls?

Eventually something will stick, right?

#e TV Context

http://www.flickr.com/photos/imnohero/2330548144

#e TV Context

Yes, I said “Context.”http://www.flickr.com/photos/imnohero/2330548144

http://www.flickr.com/photos/brunauto/5062644167/

Our vision of mobile context is often wrong.

80% during misc downtime

76% while waiting in lines

62% while watching TV

69% for point of sale research

http://www.flickr.com/photos/missmeng/5327470961

TMI: 39% use phone on the toilet.

Yes, we can’t know the mobile context.

http://www.flickr.com/photos/timcaynes/158599960/

But when it comes to TVs,

context seems to matter again.http://www.flickr.com/photos/imnohero/2330548144

Two great sources for information on designing for TVs.

http://dev.opera.com/articles/view/creating-web-content-for-tv/

https://developers.google.com/tv/web/docs/design_for_tv

http://www.flickr.com/photos/chrisbartow/5835428673

Designing for a 10-foot UI

• Limit paragraphs to 90 words

• Break into small chunks

• Line length: 5-7 words

• Body text around 21pt on 720p and 28pt on 1080p

• Add more leading

• Minimum font size of 22px

• Line length: 10 words or less

• Generous leading

Making text easy to read

Google Opera

“A good rule of thumb is to increase the size of an element (such as an image or font) 1.5x for 720p and 2.0x for 1080p relative to the size of that element in a normal PC browser experience.”

—Google TV Guide

• When designing a web page for TV, the viewable area should display less information overall, and what's there should focus on a con"ned set of tasks (even consider performing their desired task automatically or select by default).

• Primary activity often revolves around quick information look-up (for instance, cast and crew details for a particular movie, weather reports, TV listings) and quick access to services. Web content for TV should therefore be optimised — in terms of overall presentation, navigation and functionality — and task-focused, giving quick and clear access to all relevant features and information.

Optimize for tasks

Google Opera

Hover is back!

“!e main interface of Google TV encourages the use of the D-pad on the remote to make selections on a screen -- it's likely that users will keep this habit even on the web.”

/* CSS */#copyright { nav-down: #logo;}

CSS3 Basic User Interface speci"cation for directional focus navigation

http://dev.opera.com/articles/view/tweaking-spatial-navigation-for-tv-browsing/

button#b1 { nav-index:1; nav-right:#b2; nav-left:#b4; nav-down:#b2; nav-up:#b4;}

button#b2 { nav-index:2; nav-right:#b3; nav-left:#b1; nav-down:#b3; nav-up:#b1;}

W3C’s example of four buttons

button#b3 { nav-index:3; nav-right:#b4; nav-left:#b2; nav-down:#b4; nav-up:#b2;}button#b4 { nav-index:4; nav-right:#b1; nav-left:#b3; nav-down:#b1; nav-up:#b3;}

http://www.w3.org/TR/css3-ui/#nav-dir

Unfortunately, only 2 of the 10 TVs I tested supported spatial navigation.

Google TV jQuery UI Library

http://code.google.com/p/gtv-resources/

Unfortunately, I had trouble getting the Google TV jQuery UI library to work on non-Google TVs.

Needs more testing.

Both solutions require adding a layer of CSS or JS speci"cally to support TV interaction.

• Google TV may not be able to render a page as quickly as your workstation.

Performance Challenges

Google Opera• Modest hardware. Somewhere

between high-end smart phones and low end laptops.

• Avoid overly heavy and complex JavaScript.

• Avoid layering and opacity.

• Low limit on caching. Cannot assume assets cached. Cannot rely on cookies for subsequent session.

http://thereisnofold.com/

Uh oh… scrolling sucks on many TVs.

Horizontal paging is preferred

http://gtv-resources.googlecode.com/svn/trunk/gtv-jquery-demo/index.html

• Only HDTVs.

• 720p and 1080i/p

• "e exact pixel dimensions of the display varies by TV manufacturer.

• Provides an auto-zoom feature which you need to design for or around.

Supporting different screen resolutions

Google Opera• Most modern web-enabled

TVs support 1280×720 as a minimum resolution.

• 720p content is usually upscaled

• Virtual resolutions — as an example, the Nintendo Wii has a virtual width of 800 pixels. Height varies based on the type of TV (4:3 or 16:9 aspect ratio) and user settings.

Browserscope report screen sizes

Actual resolutions differ greatly.

No problem. We’ll adapt to the screen resolution.

http://bradfrostweb.com/blog/web/mobile-first-responsive-web-design/

No problem. We’ll adapt to the screen resolution.

http://bradfrostweb.com/blog/web/mobile-first-responsive-web-design/

HDTV!

No worries. We’ll use media type.<link rel="stylesheet" media="screen" href="..."><link rel="stylesheet" media="tv" href="...">

None of the TVs supported the TV media type.

https://twitter.com/#!/patrick_h_lauke/status/190078528568569856https://twitter.com/#!/patrick_h_lauke/status/190078688287653889

Media types are useless except for screen and print.

Web developers are litter bugs.

http://www.flickr.com/photos/jpdaigle/3393858438/

Sympathy for the Browser Makers

We need to be careful of the impact we have on the world.

http://www.flickr.com/photos/samout3/3411361042/

Detect a TV with JavaScript?

Detect a TV with JavaScript?

Nope.

http://www.flickr.com/photos/77799978@N00/5351372848/

Alright "ne.We’ll use device

detection.

Mozilla/5.0 (DirectFB; Linux; ko-KR) AppleWebKit/534.26+ (KHTML, like Gecko) Version/5.0 Safari/534.26+

User Agent String for a 2012 LG Smart TV

Mozilla/5.0 (DirectFB; Linux; ko-KR) AppleWebKit/534.26+ (KHTML, like Gecko) Version/5.0 Safari/534.26+

User Agent String for a 2012 LG Smart TV

Nothing we can use in that string!

#at sucks.

http://www.flickr.com/photos/plunkmasterknows/357836855/

What’s the solution?I don’t see many people actively working on it.

Are Smart TVs the equivalent of phones before the iPhone was released?

http://www.flickr.com/photos/rhodes/6297487639/

Are Smart TVs the equivalent of phones before the iPhone was released?

#is gives me hope.http://www.flickr.com/photos/rhodes/6297487639/

Responsive web design still makes sense.

Choosing responsiveness, as a characteristic shouldn’t necessarily de"ne the wider implementation approach. Device Experiences (i.e. standalone sites, aimed at a group of devices) can also be responsive, providing the %exibility to support a much wider range of devices.

—Stephanie Rieger

ARTICLES TOPICS ABOUT CONTACT CONTRIBUTE FEED

D E C E M B E R 14, 2010

Smartphone Browser Landscapeby P E T E R - P A U L K O C H

Published in: User Interface Design, Mobile, Mobile Design, Mobile Development

Discuss this article » | Share this article »

Users expect websites to work on their mobile phones. In two to three years, mobile support

will become standard for any site. Web developers must add mobile web development to their

skill set or risk losing clients.

How do you make websites mobile compatible? The answer is obvious: By testing them on all

mobile phones, and by solving the problems you encounter. But, that’s a useless answer. It’s

impossible to test your designs on every mobile phone out there. Within the mobile phone

landscape, there are at least ten operating systems (OSs) and fifteen browsers that require

consideration. Mobile devices are expensive, and not every web developer can afford to buy

five to ten of them. Testing “on all mobile phones” is impossible for most web developers.

In this article, I’ll give you an overview of the mobile web market, as well as phone platforms

and their browsers, so that you can decide which mobile devices to test on. Then, we’ll look at

how to set up a mobile test bed.

Search ALA

include discussions

TopicsCode

Content

Culture

Design

Mobile

Process

User Science

SnapshotMost web designers and

developers (not to

mention the entire

blogosphere) fall

squarely in the high-end

market. A cultural bias

exists against OSs aimed

at any other market. As

a result, most people

focus on the struggle

between iOS and

Android, and ignore the

rest. This has to change.

Stay in better touch withcustomers with

No.

320

“Testing on as many devices as possible is a great idea in theory, but in practice it is untenable. Even if we buy a few devices to try to cover more ground, they will be outdated in just a few months or a year at most. So are we supposed to buy multiple devices per year?”posted at 11:32 am on December 14, 2010 by klayon

“If that’s the mobile landscape, I want no part of it.”posted at 07:22 am on December 15, 2010 by Polsonby

http://www.alistapart.com/comments/smartphone-browser-landscape/

If you thought phones were bad,You ain’t seen nothing yet!

Most stores have no remotes and no wi-"

http://www.flickr.com/photos/elmada/1431918753/

Bring your phone for tethering to TVs

http://www.flickr.com/photos/bendodson/3367856091/

Don’t be surprised if your data gets used…

What are people actually expecting when they buy a smart tv?

http://www.flickr.com/photos/presta/623444414/

Storage capacity not listed as a spec.

Is there a Smart TV market?

Opera TV Emulator

http://www.opera.com/business/tv/emulator/

Google TV Emulator

https://developers.google.com/tv/android/docs/gtv_emulator

Google TV Spotlight in Chrome Fullscreen

http://www.google.com/tv/spotlight-gallery.html

Google TV Spotlight in Chrome Fullscreen

http://www.google.com/tv/spotlight-gallery.html

What lessons can we take away from TVs?

320 px

Too much focus on the smaller screen.

Why do we need to look at our phones to get directions?

One vibration for left.Two for right.

http://www.flickr.com/photos/williamhook/4225307113

TV resolution reinforces this idea1980 px

1080

px

This is HDTV

It may be multiple screens interacting together.

Need to think about multiple screens.

http://www.apple.com/ipad/features/

It may be multiple screens interacting together.

Need to think about multiple screens.

http://www.apple.com/ipad/features/

What does TV tell us about context?

No context Context?

What does TV tell us about context?

Or is TV context different simply because we’re still in the feature phone era of Smart TVs?

http://www.flickr.com/photos/rhodes/6297487639/

Net%ix: posture de"ning different experienceshttp://www.%ickr.com/photos/ezu/66815736/

“Some people at Net%ix have been arguing for a single experience across all devices. #is has never born out in any kind of testing. Instead, Net%ix has a variety of experiences on different devices and even regions.”

http://www.lukew.com/ff/entry.asp?1339

• User posture: Stationary, Lean back, on-the-go, shared

• Input capabilities: pointer/keyboard, LRUD/OSK, Gesture/OSK

• Navigation style: controls & windows, panes

• Display capabilities: Hi-Res, near, far away, small, medium, large

•"ese constraints are really powerful. You need to embrace them to get to appropriate designs.

http://www.lukew.com/ff/entry.asp?1339

When we need more control to craft an experience for a given device, how can we do so in a sustainable manner?

http://www.flickr.com/photos/nathaninsandiego/4829858186/

Yes, Smart TVs suck right now.

But it would be dangerous to dismiss them.

Fundamental Truth:We can’t predict future behavior from

a current experience that sucks.

So even if we don’t have to design for

TVs today…

It behooves us to start thinking about and planning for what it will be like to do so…

Flickr photo by Drift Words: http://www.flickr.com/photos/44124413076@N01/11846265/

So we won’t build solutions for today’s problems and

then "nd ourselves surprised by what comes next.

Jason Grigsby @grigs • cloudfour.comSlides: bit.ly/immobilism

!ank You!Special thanks to Patrick H. Lauke, the Google TV team, Flickr users sharing under creative commons & the kind

folks at Beaverton Video Only.

http://www.flickr.com/photos/sualk61/4083223760/

Get 40% off of the print and 50% off of ebook version using code AUTHD at oreilly.com.

OR Amazon link (no code): http://bit.ly/hf-mw