UX in Action: IBM Watson

UX in Action: IBM Watson

#Utwebinar @UserTesting

Carol Smith

Sr Design Manager, IBM Watson

#UTwebinar

UserTesting’s on demand customer insights platform enables companies to create powerful products and exceptional customer experiences.

We Partner With Thousands Of Customers

#UTwebinar

Across All Platforms, At All Stages Of Development

Watchcustomersinteractwith:

DESKTOP SMARTPHONE TABLET

WEBSITES APPS&PROGRAMS

PROTOTYPES&WIREFRAMES

SURVEYS PHYSICALPRODUCTS

Anywhere people interact with your brand:

ATHOME ATASTORE ONTHEGO

On any pla6orm or device:

#UTwebinar

Traditional Testing

2 to 3 Weeks 2 to 3 Weeks 1 Week

Results In Hours, Not Weeks. Save Time. Save Money.

#UTwebinar

Plan & Recruit Run Study Analyze Data & Write Report

1 Day 1 Day

Analyze Data & Iterate

Plan, Recruit & Run Study

Within Sprint 2

Days

4-7 Weeks

Agenda

•  Welcome

•  IBM Design Thinking

•  Watson Design Team

•  Agile

•  Usability Testing

•  Q&A

Adam Rector Webinar Manager, UserTesting @iamadamrector

Carol Smith Sr Design Manager, IBM Watson

#UTwebinar

@carologic

8

Carol Smith Sr Design Manager, IBM Watson

#UTwebinar

Carol Smith conducted her first usability study in 2000 and has been researching to understand user needs and then advocating for improved designs on their behalf since then. She has led projects for the US government, non-profits, and corporations in a wide variety of industries. She currently manages a team of designers at IBM, creating and improving Watson software products. She has presented over 70 talks and workshops around the world and has collaborated with luminaries in the field on a variety of topics.

@carologic

Carol Smith • User Experience

•  Mix of consulting and in-house •  Government, non-profits

and corps in many industries •  Usability testing, research, interaction design •  Web, software, mobile

• MS, Human-Computer Interaction, DePaul University, 2002

•  Sr. Design Manager at IBM Watson, Pittsburgh, PA

• My thoughts – not those of my employer •  Twitter: @carologic

IBM

• 105 year old company “Big Blue” • Continuous reinvention • Patents: ATM, floppy disk, relational

database, SQL, UPC barcode • Over 170 countries • Nearly 380,000 employees • 100’s of Designers

https://en.wikipedia.org/wiki/IBM and https://en.wikipedia.org/wiki/IBM_Selectric_typewriter

IBM Design Thinking Human-centered outcomes at speed and scale

Understand people’s needs.

IBM Design Thinking © 2016 IBM Corporation | IBM Confidential

Form intent. IBM Design Thinking © 2016 IBM Corporation | IBM Confidential

Deliver outcomes at speed and scale.



The Loop The Keys

Principles


The Loop Principles

The Keys

A focus on useroutcomes

Multidisciplinaryteams

Restlessreinvention


Principles

Principles


The Loop The Keys

IBM Design Thinking

The Loop

Observe Reflect Make

© 2016 IBM Corporation | IBM Confidential

The Loop Principles


The Keys

Hills Playbacks Sponsor users


The Keys


The Loop The Keys

Principles

Watson Designers and Guilds

Watson Design Team (~60 Designers)

Explore & Discover (8)

Management & Guild Leads

Director Watson Design Director

Carol

Researchers Ix Designers Viz Designers

Staffing • Balance of skills and experience • Provide mentoring and guidance to less experienced folks • Best designers working on toughest problems • Focus on skill set

•  Enable team to do what they enjoy and practice skill •  Encourage shadowing of others to learn new skills •  Encourage depth and T shape

The Work •  Individuals focus on one primary product/problem at a time

•  Secondary short projects •  10% time is available •  Tuesdays are ‘No Meeting’ day

• Management facing constant change •  Multiple projects •  Customers around the world •  Big challenges

Remote Research •  Interviews • Observations

•  Limited, but helpful • Usability testing

•  Prototypes and working software •  Moderated through screen shares •  Unmoderated with usertesting.com

• Card Sorting and/or co-creation activities (limited)

Ask for artifacts! hHp://www.flickr.com/photos/camknows/viahHp://creaOvecommons.org/licenses/by-nc-sa/2.0/

In Person Research • At customer’s location

•  With sales or product owners (Offering Managers) •  Initial meetings are typically to gain trust

• Return to do additional work •  Interviews •  Observations •  Usability Testing •  Co-Creation activities

• Limitations •  Recording can be very difficult •  Access only to whom they approve

Photo: http://koin.com/2016/07/12/how-to-protect-yourself-from-computer-camera-hackers/

Security

Conferences and Events • World of Watson • Design Studio Sessions

•  Listed as Feedback Sessions •  Use social media and signage

to attract attendees

• Primary Goal: Meet many customers and build relationships for future work.

• Secondary Goal: Learn as much as possible through research activities.

Workshops • Start with Design and product owners (Offering Management)

•  Incubator phase •  Create Concept Cars and determine value •  Review with Development teams

• Collaboration sessions •  Design, Development and Offering Management (QA and Content as well)

Remote Work and Collaboration • Screen sharing

•  SmartCloud, WebEx, GoToMeeting, etc. • Sharing ideas

•  Virtual post-its - Mural.ly •  Real-time co-editing (Google Docs, Box) •  Document cameras (higher quality)

This is only a small sample of available tools. Endorsement is not implied by inclusion here.

Co-located and international teams • Communication

•  Skype, Google Hangout, Appear.in • Chatting

•  Slack •  WhatsApp desktop version

This is only a small sample of available tools. Endorsement is not implied by inclusion here.

IBM Design Thinking and Agile • Emphasize focus on:

•  Team collaboration •  Continuous learning •  Outcomes

• Sprints or Kanban •  3 week cadence •  Demos near end •  Daily squad standups •  Weekly cross-team reviews

• Usability test should take no more than 2 sprints to complete

Agile Manifesto

Full version: http://agilemanifesto.org/

Individuals and Interactions Customer Collaboration

over processes and tools over contract negotiation

Working Software Responding to Change

over comprehensive documentation over following a plan

We have come to value the items in the dark boxes more.

Agile and Prioritizing Work • Things move very quickly • Constant re-prioritization of work

•  Offering Management sets overall priority (fairly stable) •  Teams pick what to work on

• Dependencies •  What we learn through research •  What we learn through dev process •  Time available (if no time to implement, don’t do the research) •  Staff availability (other projects, personal leave, etc.) •  Customer availability

Agile Frustrations via Spotify

Scaling Agile at Spotify via Slideshare of Vlad Mysla http://www.slideshare.net/vmysla/scrum-at-spotify?qid=2345c3ad-7e68-4383-9673-9e715ff47a75&v=default&b=&from_search=14

squad member

Agile UX: The Good • Most important functionality is done first • Working together – not “over the wall” • Keep up with technology and environmental changes • Enables iteration of requirements • Less “design drift” and less wasted design

Agile UX: The Good (continued) •  Issues get fixed •  “Done” includes design • Satisfying to see designs in real use • Learn from actual product use • User data has effect on current release

Be Part of the Team • They’re not going to “stop the train” for you • Make UX processes Agile • Build trust by providing value of work • Attend daily standups/scrums

Design Adapts to Agile • Falling short of end goals is a constant

•  Focus on meeting user’s primary needs •  Constant Improvement is key

• Widen the Design family •  Distribute the work •  Empower entire squad to inform design

• Agile is reactive – no time for predictive work •  Prepare to react

Adapted from Jim Laing’s presentation at UX Pittsburgh, May 2014

Moderated Usability Testing • Everyone is invited to observe

•  Separate viewing room and project to a screen •  Development, offering management, QA, content, etc. •  Box note to track progress within Design team •  Slack channel for larger group discussion and to ask questions

• Discussions start during the study •  Team gets excited – especially those that are new to usability testing •  I have mixed emotions – sometimes wrong emphasis - but at least they

are communicating!

Moderated Usability Testing • Raise and/or discover significant issues through usability testing

•  Hard to argue with what you see •  Have to more carefully vet our participants when testing high visibility

functionality •  Have had situations with intense scrutiny – which we welcome!

• Create highlight reels of most important topics •  Optional editing – frequently we tell them at what time marker to watch

Unmoderated Usability testing • UserTesting.com

•  Clickable prototypes and working software •  IBM Bluemix for hosting

• One participant completed and watched before inviting more •  Find issues with study design •  Determine if getting desired results •  If not, edit and re-start

• Small studies •  Up to 5 participants •  Review and repeat as needed

Remote Tool Considerations • Prototype

•  Must be online (impossible to test if not online) •  Document or clickable site/prototype

• How is recruiting handled? •  How selective can I be? •  Can I provide my own list?

• Limited number of observers? • Limited number of participants?

44

Remote Tool Considerations (cont) • Typically only one facilitator • Logging and video editing needs met?

•  Time on task, highlight video creation •  Ease of use

• Able to provide surveys before or after?

45

Towards Building Teams • Hire great people • Balance skills and experience • Enable to work to their strengths • Encourage to grow, attend webinars, conferences, etc. • Make time for 1:1’s

Questions & Answers

#UTwebinar

Q A

Carol Smith

in/[email protected]/carologic

Shared Documents Aren’t Shared Understanding

CartoonbyLukeBarreH©JeffPaHon,allrightsreserved,www.AgileProductDesign.com

IBM Health Corps • Three weeks, completed entire software project • Partnered with American Cancer Society • Built a tool to help Ministries of Health in low-income countries get

access to more complete and cheaper therapies for cancer.

Blog: https://ibmhealthcorpsblog.wordpress.com/2016/09/29/delivering-high-value-in-a-short-time-colocation-agile-and-design-thinking/ Video: https://www.instagram.com/p/BK9N5K_jEN_/?taken-by=ibmhealthcorps

World Community Grid – unused computing

Web Site: https://www.worldcommunitygrid.org/discover.action

More about Watson • CBS News, 60 Minutes on AI

Video: http://www.cbsnews.com/news/60-minutes-artificial-intelligence-charlie-rose-robot-sofia/

Internet

UX in Action: IBM Watson