The 5 Challenges to Image Delivery on the Web Today 5 CHallanges to... · 2 Samy Makki, Senior Solutions Engineer Akamai Technologies GmbH @SamyMakki [email protected]

Samy Makki

The 5 Challenges to

Image Delivery on the Web Today

UX/Frontend

2

Samy Makki, Senior Solutions Engineer

Akamai Technologies GmbH

@[email protected]

1. Why we need Images2. The Images Problem 3. 5 Challenges to Image Delivery Today

Why we need Images

©2018 AKAMAI | FASTER FORWARDTM

63% OF MOBILE & TABLETSHOPPERS RANK

PRODUCT IMAGES FIRST OVER OTHERCONTENT ON THE PAGE94%

ARTICLES WITH IMAGES GET

M O R ETOTAL VIEWS

A picture is worth a thousand words… & better engagement

Fast websites make more money too!!!


High Quality Images Lead to Overweight Pages

• Images makeup on average 53% of page weight

• Avg. page weight surpassed 2MB in 2015

• Images alone in 2016comprise 2.3 x more page weight than an entire page 2010!

The Images Problem


The Images Problem

108 variants per image!

3 sizes

x 4 formats

x 3 resolutions

x 3 layouts

5 Challenges to Image Delivery Today

1 2 3 4 5

ResizingWeb

FormatsCompression Code Automation


5 Challenges to Web Image Delivery

1

Resizing

Mobile Data Plans

Less is More

Don’t deliver more than you need

Device ConstraintsMobile devices

Network PerformanceCellularWi-Fi

Why Serve the right sized image?

£ $ ¥ €

Size matters

Size on screen: 315 x 420900 x 1200PNG

1.2MB

Resized: 185KB85% saving

How do you paint an image

Request DecodeCopy to

GPUDisplay

Store in memory

520

1280

What does an image REALLY look like?

1280 x 520 x 4 =

2.5 MB

Images contain A LOT of information


“...25% of new Android phones have only 512MB of RAM.”

Jen FitzpatrickVP of Product Management

Google Maps

Not all devices are created equal

1.0s 2.0s 2.6s

334ms

1003ms

1180ms

222ms

494ms

Decode times

0.7s

103ms

Resizing in the browser

600 X 600 200 X 200

Reduce each image by 50px

550 X 550 150 X 150

What break points?

550px X 550px (600 X 600 – 550 X 550) X 4 = 230,000 wasted bytes

150px X 150px(200 X 200 – 150 X 150) X 4 = 70,000 wasted bytes

200px 700px 1200px

1x and 2x – Twice as many pixels/inch

Width

CSS: 320px

Physical: 320px

DPR: 1x

Width

CSS: 320px

Physical: 640px

DPR: 2x

Size isn’t everything

What happens if you don’t bother?

Original Image Didn’t Bother Art Direction

2

Web Formats

Most used images on the web today

Alexa top 1000August 2016

New Flavours of JPEGFormat Size vs

JPEGProgressive Transparency

SupportHardware Decoding

Encoder Browser Support

JPEG N/A N/A No No jpegtran Everybody

WebP -35% -35% Yes No cwebp

JPEG XR -30% N/AIn Spec, not

BrowsersMaybe jxrlib

JPEG 2000 -30% N/AIn Spec, not

BrowsersMaybe OpenJPEG

10+

12.1+4+23+

6+ 6+

http://jpegclub.org/jpegtran/

https://developers.google.com/speed/webp/download

https://jxrlib.codeplex.com/

http://www.openjpeg.org/

Grow revenue opportunities with fast, personalized

web experiences and manage complexity from peak

demand, mobile devices and data collection.The right format image

Right formatZoom image is on a transparent overlay

Resized PNG : 185KB

JPG Version: 81KB56% saving

But JPEG isn’t transparent?

WebP Version: 42KB77% saving

Source: https://developers.google.com/speed/webp/gallery2

https://developers.google.com/speed/webp/gallery2


We can detect approximately 10 millionunique colours.

The Eye can be tricked

We notices changes to lightmore than changes to colour

Chroma Subsampling: 4:4:4

4

4

4

Chroma Luma

+



2

2

4

Luma

+

Chroma

4 Samples50% Saving over 4:4:4



2

0

4

Luma

+

2 Samples75% Saving over 4:4:4

Chroma


Chroma Subsampling - The result

Chroma Subsampling - The result - And why you should care!

Formula to calculate the memory needed for an image

(W x H x 3) - (W x H x Subsample_level x 2)

1300px x 1024px

RGBA 5,324,800 4:4:4 3,993,6004:2:2 2,662,4004:2:0 1,996,800 62.5% memory saving


Which Format supports Chroma Subsampling?

Jpeg Yes

WebP (lossy) 4:2:0 only

Jpeg2000 Yes

JPegXR Yes

PNG No

GIF No


When to use Chroma-Subsampling?

Use Example Comments

Icons / Logos 4:4:4 Alternative: Use lossless

formats such as PNG, GIF

or SVG for superior quality

Images with

text

4:4:4 Alternative: Overlay text

using CSS. This will enable

translation and accessibility

for your images.

Images 4:2:0

3

Compression

Lossy v Lossless

“lossy” images can never

be uncompressed to an

exact 1:1 copy of the

original source image

What is image quality?

Q5: 5KB Q10: 9KB Q15: 13KB

Q50: 31KB Q75: 46KB Q90: 99KB

Quality by size and scale

0

50

100

150

200

250

300

350

400

450

10 20 30 40 50 60 70 80 90 100

File

siz

e in

KB

JPEG Quality Rating

File size v’s Jpeg Quality Ratings

300x400 450x600 600x800 750x1000 900x1200

Large reduction between 100-90

After 60 limited returns

Quality by user experience

0

50

100

150

200

250

300

350

400

450

10 20 30 40 50 60 70 80 90 100

File

siz

e in

KB

JPEG Quality Rating

File size v’s Jpeg Quality Ratings

300x400 450x600 600x800 750x1000 900x1200

• Poor Engagement• Good Performance• Un-engaged users

• Quality• Poor

Performance• Annoyed

users

• Happy users

Perceptual Quality & Structural Similarity

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli

What is Structural Similarity?

An algorithm that uses

Brightness,

Contrast,

and Structure

to compare images.

What is Structural Similarity?

How do we use Structural Similarity for optimization?

Structural Similarity gives us a

human benchmark

to compare images of different qualities

to an original.

How do we use Structural Similarity for optimization?

Q10DSSIM: 0.0566

Q50DSSIM: 0.0251

Q80DSSIM: 0.0179

Original quality: 99Size: 345.26 KB

Original format: JPEG

Original width: 1031px

Byte savings: 91.97% (27.73 KB vs. 345.26 KB)

Quality: 75

Content-Type: image/webpWidth: 1031px

Example : SSIM Perceptual Medium

Byte savings: 88.21% (40.72 KB vs. 345.26 KB)

Quality: 60

Content-Type: image/jp2Width: 1031px

4

Code

Delivery

1. Browser Solutions – Responsive images2. Client side - JavaScript3. Server side – Cookies & Headers4. 3rd Party tools


Srcset, Sizes & Client Hints

Browser Decides - Why not use Media Queries?

Known by the developer when

writing code

Known by the browser when

rendering

Viewport dimension

no yes

image size relative to the

viewportyes no

screen density no yes

source file dimensions

yes no

Srcset - let the browser choose

<img src=”small.jpg”

srcset=”small.jpg 300w, large.jpg 500w”

sizes=”(max-width:400px) 300px, 500px”

alt=”selfie picture”>

ALTERNATIVELY USE RELATIVE WIDTHS



sizes=”(max-width:400px) 100vw, 50vw”


src for fall back

comma-separated list of available versions of the

image; each image’s width is specified

using the w descriptor

Sizes help tell the browser when to select each image

Sizes can also use viewport percentage to inform the decision



sizes=”(max-width:400px) 300px, 500px”


ALTERNATIVELY USE RELATIVE WIDTHS



sizes=”(max-width:400px) 100vw, 50vw”


What the browser now knows

Known by the developer when

writing code

Known by the browser when

rendering

Viewport dimension

no yes

image size relative to the

viewportyes yes (sizes)

screen density no yes

source file dimensions

yes Yes (srscet)

What if the browser told the server?

Client Hints

<meta http-equiv=”Accept-CH”

content=”DPR, Viewport-Width, Width”>

Srcset OK for simple switching, what about the other challenges (image Formats, Art Direction)?


<picture> & srcset

The Picture Element Part I

<picture>

<source media="(max-width : 400px)"

srcset="uploads/ex2_hotel_400.jpg">



<source media="(min-width : 801px)"


<img src="uploads/ex2_hotel_400.jpg"/>

</picture>

New element

<img> for fall back

Multiple sources

The Picture Element Part IIAdding format selection

<picture>

<source type="image/webp" and media="(max-width : 400px)"

srcset="uploads/ex2_hotel_400.webp">

<source type="image/webp" and media="(max-width : 800px)"

type="image/webp" and srcset="uploads/ex2_hotel_500.webp">

<source type="image/webp" and media="(min-width : 801px)"

srcset="uploads/ex2_hotel_800.webp">







<img src="uploads/ex2_hotel_400.jpg" />

</picture>

Additional Check for type


resolution switching with srcset& <picture>

<img src="retail_files/accessories.jpg"

srcset="retail_files/accessories.jpg?width=300 1x,

retail_files/accessories.jpg?width=600 2x,

retail_files/accessories.jpg?width=900 3x"

width="95%“

/>

Resolution switching with srcset

Simple density descriptor

Adding resolution to the <picture>

<picture>


srcset="retail_files/mens_accessories.jpg?width=300 1x,

retail_files/mens_accessories.jpg?width=600 2x,

retail_files/mens_accessories.jpg?width=900 3x">


srcset="retail_files/mens_accessories.jpg?width=350 350w,

retail_files/mens_accessories.jpg?width=700 700w,

retail_files/mens_accessories.jpg?width=1050 1050w"

sizes="50vw">


srcset="retail_files/mens_accessories.jpg?width=600 1x,

retail_files/mens_accessories.jpg?width=1200 2x">

<img src="retail_files/mens_accessories.jpg?width=400" width="95%" />

</picture>

Simple multipliers

Or, tell the browser for screens under 1024px, the image will take up 50% of the viewport and there are 3 sizes available

Art direction & the Picture Element

<picture>


srcset="retail_files/leather_jacket.jpg?imwidth=300&impolicy=cropped">


srcset="retail_files/leather_jacket.jpg?imwidth=350&impolicy=cropped">


srcset="retail_files/leather_jacket.jpg?imwidth=600">

<img src="retail_files/leather_jacket.jpg?imwidth=300" width="95%" />

</picture>

Cropped image for mobile & tablet

Non cropped for desktop

5

Automation

What could be automatedby an Image Service?

• Rendition (size & transformations) creation and storage based on Policies, driven with Picture & Srcset + hints (i.e. width and

crop parameter)

• Choose correct image format and compression (Optimize FileSize) SSIM Analysis & User Agent No need for „type“ attribute anymore

Network awareness

Original Image

Quality: 100%Size: 60KB

Network Conditions

Average

Adapted Image

Throughput: HighSize: 54KB (90%)

Throughput: MedSize: 40KB (67%)

Throughput: LowSize: 24KB (40%)

What could be automated byan Image Service?

• Rendition (sizes & transformations) creation and storagebased on Policies, driven with Picture & Srcset + hints (i.e. width

and crop parameter)

• Choose correct image format and compression (Optimize FileSize) SSIM Analysis & User AgentNo need for „type“ attribute anymore

• Let Image Service choose compression (SSIM value) based on Network Conditions

How do I know if I’m doing a good job?

Designing an image strategy

Summary: simplify where you can but don’t compromise on quality or performance

76

Samy Makki, Senior Solutions Engineer

Akamai Technologies GmbH

@[email protected]

https://bit.ly/2JFu1kz

Thu, April 26, 2018Schedule: 12:00PM - 5:00PM- Kick Off + Lunch- Akamai for DevOps Overview + Roadmap- Getting to Know the Akamai API / CLI- Deeper Dive into Third-Party Integrations(like Terraform) and Cloudlets- Q&ASocial Hours: 5:00PM - 8:00PM

Documents

The 5 Challenges to Image Delivery on the Web Today 5 CHallanges to... · 2 Samy Makki, Senior Solutions Engineer Akamai Technologies GmbH @SamyMakki [email protected]