FITC - Exploring Art-Directed Responsive Images

Rami Sayar - @ramisayar

Senior Technical Evangelist

Microsoft Canada

Sharing my experience attempting to perform art-direction and finally attempting to automate art-direction… This talk will give you an overview of the most popular tools.

• What are Art-Directed Responsive Images

• How to Create Art-Directed Responsive Images

• Using Smartcrop.js, Imgix and Other Tools

• Dealing with Web Backends

• Building Your Own System with Cognitive Services

• Automating with Webpack

• Working knowledge of HTML5, CSS3, Responsive Web Design and JavaScript.

Note: Slides will be made available on SlideShare.net/RamiSayar. Follow @ramisayar.

• “Responsive Web Design” is now just called “Web Design”

• Various techniques and polyfills were tried until <picture>, sizes andsrcset became standard.

• srcset allows us to specify a comma-separated list of image URLs.• The x descriptor allows us to specify a device-pixel-ratio e.g. 2x

• The w descriptor allows us to specify an image for a width-constraint

• sizes allows us to describe how many pixels an image needs.• Can be relative to the viewport e.g. described in ems or fixed e.g. in px.

• Can pair media queries to specify how many pixels for a specific width

• <picture> & <source> allows you to do art-direction and select images-based on breakpoints and media queries.• media is an attribute in <source> executed when true

<picture>

<source media="(min-width: 36em)" srcset="large.jpg 1024w, medium.jpg 640w, small.jpg 320w" sizes="33.3vw" />

<source srcset="cropped-large.jpg 2x, cropped-small.jpg 1x" />

<img src="small.jpg“ />

</picture>

Source: Smashing Magazine

In a responsive design, we want to target images to fit with the device.• On a large screen, we want to

show a large image with plenty of details.

• On a small screen, we want to shrink or crop an image to make it more relevant, useful and beautiful

Source: Use Cases Responsive Images

• Full Control - Manually select the resize and crop or even an entirely different image for various sizes or aspect ratios.• Pros: Full Artistic Control. Best Visual Effect.

• Cons: Manual & Tedious.

• Best Guess – Use an algorithm to resize and crop or even an entirely different image past certain sizes or aspect ratios.• Pros: Fully automatable.

• Cons: Algorithms might miss.

And not just different image sizes…

1. Find edges using laplace

2. Find regions with a color like skin

3. Find regions high in saturation

4. Generate a set of candidate crops using a sliding window

5. Rank them using an importance function to focus the detail in the center and avoid it in the edges.

6. Output the candidate crop with the highest rank

npm install -g smartcrop-cli

node ./smartcrop-cli.js --width 100 --height 100 photo.jpg square-thumbnail.jpg

Fairly complicated on Win10, tweet @ramisayar for instructions or help.

• Imgix can do both face-based cropping, entropy-cropping and mixing the two.

• Many different options for resizing and cropping images:• Scale• Fit• Limit• Fill• Pad• Crop• Thumb• Imagga crop & scale <- area of interest cropping• Aspect ratio based cropping• Control gravity

Show Imagga Crop Mode

• Provides a web UI and allows for manual art-direction

• responsive.io is a service that optimizes and delivers perfect images for every screen.

• Can automatically do Art Direction using Focal-point cropping and Smart zoom.

• Smartcrop.js• Sketchy algorithm at best• Doesn’t figure out responsive image breakpoints

• Imgix• Have to host images on S3?• Not free

• Cloudinary• Does everything but you sorta have to program an API to do responsive image

breakpoints + art-direction

• Sizzlepig• No automation, all manual

• Responsive.io• Amazing but need $$$ money.

Using existing tools.

• Image thumbnailing middleware for Connect.js/Express.js that integrates with content-aware cropping provided by Smartcrop.js

• Dependencies: imagemagick (or graphicsmagick), Cairo.

var thumbs = require('connect-thumbs');app.use(thumbs());

yourdomain.com/thumbs/medium/images/aHR0plci5qcGc.jpg

• github.com/inadarei/connect-thumbs

• Responsive Images is part of the WordPress Core as of 4.4.

• Art-Direction Not Really Possible without paying for a service like Cloudinary.

• wordpress.org/plugins/cloudinary-image-management-and-manipulation-in-the-cloud-cdn/

• Smartcrop-rails gem is the integration of smartcrop.js JavaScript library for your Rails 4 application.

• Not many solutions out there

To Automate Offline & To Be Cheapish

It’s also open sourced at github.com/cloudinary/responsive_breakpoints_generator

Cognitive Services let you build apps with powerful Machine Learning algorithms using just a few lines of code.

• Computer Vision API

Thumbnail API: Generate a high quality storage-efficient thumbnail based on any input image. Apply smart cropping to generate thumbnails that differ from the aspect ratio of your original image, yet preserve the region of interest.

Didn’t have time to write the plugins for you but here are the steps.

Objective 1: Automate Art-Direction at Build Time

Objective 2: Make This Transparent.

Steps:

• Run require(“image”) through the Responsive Breakpoints Generator.

• Smartcrop with Cognitive Services

• Output Images to Build Folder

• Replace tags with <picture> in HTML

Objective 1: Automate Art-Direction at Upload Time

Objective 2: Make This Transparent.

Steps:

• Write filter that runs generated images by WordPress Core through Cognitive Services

• Replace generated images with art-directed versions

• Serve images

• Art-Directed Responsive Images Primer

• Using Smartcrop.js, Imgix, Cloudinary and Other Tools

• Dealing with Web Backends• Node.js• WordPress• Ruby on Rails

• Building Your Own System with Cognitive Services

• Automating with Gulp – YOUR HOMEWORK!

tw: @ramisayar | gh: @sayar

• Automatically Art-Directed Responsive Images? Here You Go.• Responsive Image Breakpoints Generator, A New Open Source Tool

• Responsive Images Done Right: A Guide To <picture> And srcset• Picturefill• Smartcrop.js

• Automatically art-directed responsive images• Responsive Images: Use Cases and Documented Code Snippets to Get

You Started• Retina revolution

• Automatic Point-of-Interest Cropping with imgix• Automate Your Responsive Images With Mobify.js• Addressing the Responsive Images Performance Problem: A Case Study

• Responsive Images Done Right: A Guide To <picture> And srcset

• https://responsive.io/

©2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Office, Azure, System Center, Dynamics and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.