Motion & Gesture Interactions in the digital age

Motion & Gesture Interactions

in the digital ageAntonio De Pasquale

Senior Interaction Designer at frog@myinteraction

Simone LippolisDesign technology II at frog

@leepolis

My name is Antonio De PasqualeI'm a Senior Interaction Designer at frog Milan

A little about me

I'm specialized in digital interfaces & user experience

I'm passionate about the "aesthetics" of movement

I'm from Sicily and I love the sea.

@myinteraction

My name is Simone LippolisI'm a Design Technologist II at frog Milan

A little about me

Simone Lippolis is a Design Technologist at frog design

with over 10 years of experience in software and web

development.

@leepolis

Motion & Gestures interactions in the digital age

Motion, what?

MotionGraphicsDesignInteractionsMotion graphics are graphics that use video footage and/or animation technology to

create the illusion of motion or rotation, and are usually combined with audio for use

in multimedia projects. Motion graphics are usually displayed via electronic media

technology, but may be displayed via manual powered technology

* Wikipedia

MotionGraphicsDesignInteractionsMotion Design is a subset of graphic design in that it uses graphic design principles in

a filmmaking or video production context (or other temporally evolving visual medium)

through the use of animation or filmic techniques. Although this art form has been

around for decades, it has taken quantum leaps forward in recent years in terms of

technical sophistication.

* Wikipedia

MotionGraphicsDesignInteractionsObjects that don’t move don’t interact. An interaction is some sort of communication,

and communication is about movement: our vocal cords vibrating as we speak, our

hands and arms writing or typing as we send email or instant messages, sound and

data moving between two entities. -

Dan Saffer - The element of Interaction Design

Without motion,there can be no interaction.You press a key, and an email window closes.

There is motion on your screen.

Dan Saffer - The element of Interaction Design

Static interface and physical controlsFrom the first personal computing with

Dynamic interface and Motion controlTo an incredible new powerful devices with

Motion & GestureInteraction

in the Digital Age

Introduction

Motion Design & Digital ArtifactsAnimation Design Principles

Code (and) Motion

Technology Vs MotionDesign with code

INTRODUCTION

I was giving the demo to someone a little while ago, and I finished the demo and I said what do you think?

They said, ‘You had me at scrolling.’

Motion Design & Digital products

STEVE JOBS, 2007 (on the iPhone iner+al scroll feature)

Introduction Motion Design & Digital products

The beginningGames used motion for the first time to visualize the effect

of a specific interaction in the digital space

Web 1.0: Java & Gif

Introduction

Applet Java

At the beginning of the Web, few technologies allowed

motion in web pages must mostly just for eye-candy

Animated Gif

Introduction Motion Design & Digital products

The Golden Age of FlashFrom a simple animation tool, Flash changed the way

designers experiences with motion & interaction

2Advacend, 2000Screen transition, Loading screenhttp://v2.2a-archive.com/flashindex.htm

TheVoid, 1998Animated menu, interactive objects

http://www.thevoid.co.uk/index.html

LeoBurnet, 20063d navigation, motion detection

http://v2.2a-archive.com/flashindex.htm

Introduction

With the explosion of mobile apps, motion started to

became a core part of the interaction models

The App World

Card 3d parallaxRevealing hidden controls Top-down bouncing menu

Introduction

With the introduction of the new w3c specifications, html &

css added new life to motion & dynamic on the web

The Web Reloaded

Introduction

Gesture & MotionGesture recognition became a common pattern on console and motion

interactions was the right answer to show a new kind of affordance.

Physical Motion patterns becomes virtual interactions.

Introduction

The FutureNew desktop physical control, css4 html6, smartphone

gesture recognition, hands & fingers tracking...

Native animationin app gesture

1993 2005 2007 Today

Interactive controls

Full animationengine

3d, video, cam & motion tracking

Pong, 1978Gif, 1980

Applet Java, 1989

1978 1980 1989

Flash, 1993 Flash 3d, 2005

Leapmotion

The Evolution of Motion Interaction

iOs, 2007

Introduction

HTML5 + CSS3

Simple animation UI elements

InteractiveUI elements

WebkitTechnology

Motion

iOs 7, 2013

Web Nativeanimation

JavascriptSafari + Webkit

Jquery

Flash AS.20

Natural UI

Xbox Kinect

Gestural interaction

Advanced motion patterns

What are the UX principles that helps technologist & designers

in shaping the next generation experience?

Introduction

Disney animation principles

Introduction

Disney's Twelve Basic Principles of Animation is a set of rules that defines a realistic

impression of a motion with the basic laws of physics. Some of these principles have

great relevance for the animations in the user Interface.

* Panop Koonwat - https://vimeo.com/65815545

Animation Design Principles

Introduction

Many of this principles can be applied to 6 macro UX categoriesto improve functionality and to increase the affordance of the actual patterns

Introduction

Extension

Orientation

Feedforward

Feedback

Highlight

Awaking

12 principles 6 categories

* Johannes Tonollo, Meaningful Transitions - http://www.ui-transitions.com

Introduction

Spatial Extension reduces the complexity of the user interface

by extending the virtual space.

Extension

Space extensionGood for Mobile

Extra space for info3d as an additional info layer

Introduction

Orientation describes the way in which a logical connection can

be created and visualized between objects and object's states.

Orientation

DirectionsContent position

Explaining IAHelps exploration

Introduction

Feedforward functions as a conveyor of possible interactions.

An illustrating animation is used to prepare the user for the outcome of an action.

Feedforward

Increase affordanceAnticipate hidden interactions

Tutorial & helpexplain new pattern

Introduction

Feedback comprehensively indicates what the result

was of the user's interaction.

Feedback

Start/End of a processObject activation

Accepted interactions

Introduction

Highlighting is the guidance of attention by using animations.

This category draws the user's attention and clarifies changes of states

in the user interface, which otherwise might be missed.

Highlight

Focus on action triggerDisplay active object in the interface

React to user activity

Introduction

Awaking Controls allows the user to focus on contextual

controls. They appear when needed, and disappear as soon as

they are no longer needed.

Awaking

Trigger for invisible interactionsChange of status

Revealing new informationShowing extra controls

CODE (and) MOTION

Technology makes what was once impossible possible. Design makes it real.

Technology Vs Motion

MICHAEL GAGLIANO

Why the web?

Code (and) Motion Technology Vs Motion

• Large audience both on desktop & mobile

• New visual effects with good retro-compatibility

• Web technologies enable different things:

from one page apps to games

Development Tools

Code (and) Motion

How we build this examples:

To create the structure of the

layout and define a hierarchy

between objects

To add visual appeal to the

elements, and to define, where

possible, animations and

transitions

To manipulate HTML content

and to compute CSS properties

on-the-fly

General principles

Use JavaScript to extend the events that the

browser exposes, to add new functionalities

and enable the user to interact with your

apps in more natural ways

Code (and) Motion

Always prefer CSS3 over JavaScript animations

Use JavaScript only to compute new

properties of each element, and to

dynamically change CSS

Without JavaScript it will be impossible to create complex animations and transitions.

That said, try to:

The examples

These examples are not real-life cases! it is almost impossible that you'll have to deal

with a pre-defined number of elements in a

page, and that these elements have pre-defined

positions but can be used as inspiration.

Code (and) Motion

Demo code

http://2m3.it/codemotion

Animation patterns

For every category we selected a relevant example to show the

motion characteristic and how to build it in web browser

Extension

Orientation

Feedforward

Feedback

Highlight

Awaking

Half Flip, Folding, Area Split

Fan of stack, Page turn, Saving

Solidarity, Affordance, Hidden features

Availability, Heal, Position accepted

Blur to, Change depth to, Expand to

Mode switch, Swipe to delete, Spring refresh

* Johannes Tonollo, Meaningful Transitions - http://www.ui-transitions.com

Code (and) Motion Design with code

Motion and Gesture interaction in the Digital Age

1. Folding

Code (and) Motion

Extension

A transition to show that only a part of an object is visible.

By interacting with it, it reveals to full size.

Design with code

Revealing selectors Folding a listhttp://capptivate.co/?s=clear

Revealing hidden areahttp://capptivate.co/2013/11/19/taasky/

Code (and) Motion

2. Area splitExtension

A transition to show a direct connection between

the object of interest and further information.

Design with code

Folder area split Message previewhttp://goo.gl/fZABUf

Revealing additional spacehttp://media.idownloadblog.com/wp-content/uploads/2013/05/Stopwatch-for-Velox.gif

Code (and) Motion

Example 3. Flip / Half flipA transition to show a direct connection between

the object of interest and further information.

Design with code

Revealing extra spacehttp://capptivate.co/2013/11/19/445/

Flip screenhttp://capptivate.co/2013/07/15/ifttt/

CSS Card fliphttps://daneden.me/animate/

Extension

Code (and) Motion

1. Fan of stackOrientation

A transition to explain the arrangement of objects,

by showing how they are placed.

Example

Design with code

Revealing menuhttp://capptivate.co/2013/08/14/your-extra-life/

Connected itemshttp://capptivate.co/2013/06/25/lapka/

Item stackhttp://youtu.be/8-3NcnyaehU

Code (and) Motion

2. Page turnOrientation

A Transition to explain the information architecture

of the list in a narrative way.

Example

Flipping pages Android

Design with code

Page turn to reveal menuhttp://capptivate.co/?s=circa

Flipping pageshttp://www.macitynet.it/flipboard-web-magazine/

Code (and) Motion

3. Saving indicatorA transition to tell the user where to find the object

after saving/bookmarking/downloading.

Example

Design with code

Saving eventhttp://capptivate.co/2013/06/26/lift/

Adding a new item

Orientation

Code (and) Motion

1. AffordanceFeedforward

A transition to present a possible interaction.

Card movementhttp://capptivate.co/2013/11/23/potluck/

Design with code

External menu navigationhttp://capptivate.co/2013/11/23/airbnb/

Open/close main navigationhttp://capptivate.co/2013/11/07/breezi/

Code (and) Motion

2. Hidden featuresFeedforward

A transition to explain what is necessary to activate

a hidden function.

Design with code

Show additional informationhttp://capptivate.co/?s=kickstarter

Revealing additional functionhttp://capptivate.co/?s=clear

Code (and) Motion

3. Explaining directionsFeedforward

A transition to explain

a specific directional interaction

Slide indicator for final position

Example

Design with code

Horizontal slide to unlockhttp://youtu.be/3JlOOO14oSU

Code (and) Motion

1. AvailabilityFeedback

A transition to show the progress of a loading behavior

of individual objects.

Example

Design with code

Progressive loadinghttp://capptivate.co/2013/06/25/expereal/

Data/Map content loadinghttp://capptivate.co/2013/06/25/ribbon//

Content loadedhttp://capptivate.co/2013/09/30/phq4-3/

Code (and) Motion

2. HealFeedback

A transition to show that the objects of a list reacts directly on

the user‘s input. The interaction is validated.

Design with code

Content organization & position feedbackhttp://useyourinterface.com/post/55475407955

Code (and) Motion

3. Position acceptedFeedback

A transition to show that a movement into

a new position is accepted.

Design with code

Show additional informationhttp://www.idownloadblog.com/2013/09/26/ios-7-the-ultimate-safari-guide/

Selecting possible tileshttp://capptivate.co/2013/09/18/grid/

Code (and) Motion

1. Change depth toHighlight

A transition to direct the user‘s view by an attention

grabbing animation of the object.

Design with code

Folder zoom iOs7http://goo.gl/wDvwGJ

Panel focus switchhttp://capptivate.co/2013/09/18/memoir/

Panel switchhttp://capptivate.co/2013/08/12/felix-for-app-net/

Code (and) Motion

2. Expand toA transition to show which object is active

on interaction by expanding it.

Keyboard magnify effect

Design with code

Highlight

Zoom on a specific areahttp://capptivate.co/2013/08/12/dark-sky/

Current selectionhttp://capptivate.co/2013/08/20/spendee/

Code (and) Motion

3. Blur toHighlight

A transition to direct the user‘s view

by an attention grabbing animation of the object.

Example

Design with code

Focus on the active areahttp://vimeo.com/66395692

Code (and) Motion

1. Swipe to deleteAwaking

A transition to support an intuitive interaction

of deleting an object in a list.

Design with code

Swipe to delete items (html version)http://goo.gl/IMLjsUf

Swipe left / right to activatehttp://vimeo.com/59504129

Code (and) Motion

2. Spring to refreshAwaking

A transition to use the intuitive interaction of a list. By scrolling a list,

the function appears underneath the related list.

Design with code

Drag to refreshhttp://azumbrunnen.me/wp-content/uploads/Twitter.gif

Load new contentshttp://warezos.com/glide-app-for-ios-and-android/arefresh/

Code (and) Motion

3. Mode switchAwaking

A transition to introduce new controls by sliding in. The moving of the

items‘ labels underlines the importance of the new controls.

Example

Design with code

Task completedhttp://useyourinterface.com/image/63324362515

Show additional controls selecting a rowhttp://goo.gl/KFDps2

Design with code

Code (and) Motion

Coding and testing in real

time a design solution,

to highlight technology

or UX/Interaction related

problems on the go.

it's a good practice when you have:

It is always a good feeling

not to be the last step in

the production/design

process, and this will lead

to better products

a Designer a developer

Design with code

The future is already here.It's just not evenly distributed.

WILLIAM GIBSON

Motion & Gesture Interactions in the digital age

Design

Describe motion using Cornell notes, questions, and interactions

Chapter 1 Interactions and Motion - University of …bulldog2.redlands.edu/fac/eric_hill/Phys231/ch1.pdf2 Chapter 1: Interactions Chapter 1 Interactions and Motion This course deals

Touching Isn't It? Consumer Response to Touch and Gesture Technology for IHS Touch Gesture Motion 2014

Enabling Early Gesture Recognition by Motion Augmentation...early gesture recognition accuracy. When forecasting 25 future frames of a partially observed input gesture sequence of

Gesture-based Interactions in Video Games with the Leap ...jpirker.com/wp-content/uploads/2013/09/hcii2017-gesture-based.pdf · Gesture-based Interactions in Video Games with the

MS-PS2-1 Motion and Stability: Forces and Interactions

Gesture-Based Human Computer Interactions: Signs … · Abstract: Sign and gesture ... Systems are built which run ... Computer vision is a branch of artificial Intelligence and image

Motion Energy & Motion Historyhome.engineering.iastate.edu/.../09_MotionHistory.pdf · Motion Profile Calculation Face and Hands Segmentation Neural Network Hidden Markov Models Gesture

Gesture-Driven Interaction Using the Leap Motion to

Real Time Motion Controlling Using Hand Gesture Recognition

Markerless Hand Gesture Interface Based on LEAP …Markerless Hand Gesture Interface Based on LEAP Motion Controller Danilo Avola, Andrea Petracca Giuseppe Placidi, Matteo Spezialetti

Multi-modal Gesture Recognition Using Skeletal …vigir.missouri.edu/~gdesouza/Research/Conference_CDs/...and motion trail model [16]. The framework consists of gesture spotting module

Hand Gesture Recognition with Leap Motion · Hand Gesture Recognition with Leap Motion Youchen Du, Shenglan Liu, Lin Feng, Menghui Chen, Jie Wu Dalian University of Technology Abstract

Arm Motion Gesture Recognition using Dynamic Movement Primitives …sites.utexas.edu/hcrl/files/2016/01/Jorgensen_HRI_2016... · 2016. 8. 17. · Arm Motion Gesture Recognition using

8 Particles in Motion (Chemical Interactions—Grade 8)

HAND GESTURE AND BRAND COMMUNICATION - Ico-D · development of motion-sensing technology, ... tectural space through the use of a Leap Motion ... HAND GESTURE AND BRAND COMMUNICATION

Gesture Semantics Reconstruction Based on Motion …

7. Designing Motion Gesture Interfaces in Mobile Phones for Blind

Motion event expressions in language and gesture: Evidence ...Motion event expressions in language and gesture: Evidence from Persian Niloofar Akhavan (nakhavan13@ku.edu.tr) Department

The Future of Motion/Gesture Technology