Embed Size (px)
DESCRIPTION
Contextual inquiry as a research method has gained its popularity these years among user experience practitioners. As a user researcher, we face excessive user data that are collected from field studies. Most of us review and analyze the field data by looking for trends of users’ responses and behaviors. For example, “Affinity diagram” has been commonly used to group and analyze the field data to identify any trends. However, in many cases, it is not enough to draw our conclusions based on a few “Aha!” moments. We should also consider the rich and “random” data that are not obvious to form trends, and abstract hidden implications from them. How we could accomplish it, however, has remained as a challenge. In this presentation, I will start with a case study from our own work, and demonstrate how we found the hidden implications from our data. Then we will explore and discuss strategies and techniques from different perspectives.
Citation preview
Look Beyond Data Trends –
A Technique to Find Hidden Design Implications From Field Data Analysis
Wei Zhou, Oracle
UPA 2014London
Agenda Topic Introduction (5 min)
Case Study Presentation (20 min)
Summary of Techniques Learned(5 min)
Group Discussion for Cases and Possible Solutions Using the Techniques (25 min)
Wrap Up (5 min)
Topic IntroductionContextual inquiry is a user research method to
collect field data through inquiry and observations in the real environment users live in or work in.
The output from contextual inquiry is not structured data. It does not always show direct links to new user requirements and design issues.
Analyzing and understanding the field data become a discovery process. And it is always a challenge to find design implications from such rich and unstructured data.
Topic Introduction and Challenges“Affinity diagram” has been used as a way to
organize notes and insights from field studies. From “Affinity diagram”, we could cluster the
data into trends. However, in many cases, it is not enough to draw
our conclusions based on a few “Aha!” moments. We should try to abstract hidden design
implications from the data that don’t seem to form a trend as well.
CASE STUDY
Oracle Financial User Experience: Finding the Design Implications From the Differences
Site Visits Study for Designing Credit Management and Collections System
• We visited 3 companies and interviewed 40+ users at their sites.
• User Profiles:• Credit manager• Credit analyst• Collections manager• Collections analyst
Different Data for Credit AnalystsCredit Analyst – Release an order hold
Contact: Wei ZhouFrom Schreiber Foods Inc
Open Crystal Report (Order holds report for the entire team) from email
to retrieve case folder #, credit exposure and customer’s name
(see Screenshot)
Go to EBS Oracle Navigator(see Screenshot)
Go to Oracle Collection IEXH (See Screenshot)
Are credit exposure and order behavior or pattern changed?
Thru “Application” link
Thru SFI Collections user role and Collection
Search Customer Name to find the customer with order holds
Crystal Report was sent periodically in the morning, and then 3pm, 4:30pm…Crystal Report was opened up in Excel
Go to Intranet
Start to retrieve customer information to evaluate before doing anything on the
order hold
Select the customer found in search result
(See Screenshot)
Frustration with Oracle Search, had to put “%” in the beginning and the end of the string to get result, otherwise, may
fail. Google search style is desired
Search Result has scrunched columns every time and couldn’t view the whole
customer names. Hard to move the column splitter.
Go to Customer Detailed View (default
was “transaction”
Go to “General Tab” to check credit, and WAP
(See Screenshot)
Ignores Transaction tab, go to General Tab.
Click “Trend” button to see Trend Information such as
historical WAPs and balances(See Screenshot)
Upon closing the Trend dialog, May also go to “Trend” tab to get
additional information (See Screenshot)
YesGo to Order
Management Apps to check the order
Here, user had to go back to EBS, switch user role to OM view.
Collections view and OM can’t be open at the same time!
No
2
Example: Company 1
Each company has their own work flow and steps!
We also collected a lot of screenshots from each company.
Similarities From the DifferencesWhat is the ultimate goal? Release a held
order!What are the reasons of those different
steps?Review held orders Check customers’ credits, order history and
payment infoUpdate credit recommendations for customers
after obtaining approvals, such asRelease an order holdIncrease/decrease credit limits
If it is a new customer, check public credit information and set up new account
Analysis of DifferencesCredit held orders were reviewed in different
tools. Company 1 users: Crystal Report received by emailCompany 2 users: ExpressOS & BPCS systemCompany 3 users: GMIL
Credit review was conducted in different third-party applicationsCompany 1 users: DNBi, SeaFax and SEC/EdgarCompany 2 users: Global CreditCompany 3 users: DNBi
Held order release was completed in different tools.
Analysis of DifferencesDifferent authorities to release a held order
Company 1 users: Credit analysts have authority to release the orders.
Company 2 users: Credit analysts provide hard-copy credit review and obtain physical signatures of management before releasing any orders, higher $ amount needs further approval.
Company 3 users: Credit analysts have authority to release the orders under certain $ amount. If an order $ amount is over the limit, escalate to manager to release it.
Finding the Design Implications from DifferencesLet’s drill deeper into the consolidated data
Are there any reasons why no trends were seen at all?
Why are they different?
Are these differences actually related to a same topic?
Finding the Design Implications from Differencesthe reason for the differences is because
there were no integrated solutions in the market to help our users!
Users struggled with too many separate applications to complete a single task. It is crucial to provide appropriate seamless integration within a complete task flow such as release a credit hold.
Another Example – Enrollment and Payment for Continuing Education
One University we visited, we found the different schools and departments have its own customized or home grown systems for Enrollment and Payment.
They said they needed to apply different discounts for the students who are government employees and military family, plus ad hoc, random programs etc.
The payments become more complicated due to different discount programs and different ways to apply the discounts.
We observed 4 different departments’ current systems.
Another Example – Enrollment and Payment for Continuing Higher Education
Although they are all different, but yet not so different!
Why are they so different? - We noticed the differences are different design solutions, not the underlying problems to solve!
What are the underlying reasons? - apply various discounts and be able to collect payments.
We believe we can provide a simple streamlined system for Continuing Higher Education.
Summary of the TechniqueDocument and Analyze “Differences”Find “Similarities” from “Differences”Drill deeper beyond the surface of the data,
ask:Are there any reasons why no trends were seen
at all?Why are they different?Is underlying problem the same?Are these differences actually related to a
same topic?Are these differences just different design
solutions?
Group DiscussionWhat kinds of data are difficult to analyze using
affinity diagrams? And why?What kinds of data have been ignored or
discarded during field data analysis? And why?What are the challenges and strategies to
abstract hidden design implications from field data?
How many different perspectives could we analyze consolidated field data? By similarities, by differences and what else?
Is there a method or process that we could come up with to guide us step by step to find hidden design implications?
