Assignment Cover Sheet

MSc in User Experience Design

Student Name: Stephen Norman

Student Number: N00147768

Programme: MSc UX Design

Year of Programme: 2015

Module Name: Fundamentals of UX

Assignment: Literature Review

Assignment Deadline: 24/11/2015

I declare that that this submission is my own work. Where I have read, consulted and used the work of others I have acknowledged this in the text.

Signature: Stephen Norman Date: 24/11/2015

Approaches to conducting a Usability/User Experience Design Project By Stephen Norman

Introduction

This literature review will begin by evaluating different papers on the variety of

usability/UX evaluation methods. It will aim to identify similarities and differences

between the most commonly used evaluation methods. Though a few articles have

touched upon the full breadth of evaluation methods, for the purpose of this review

only a few methods will be discussed. The literature review will then explore some of

the main approaches to conducting a design project. Each approach will be

assessed and compared. The literature review aims to discover if there is a

preferred UX/Usability method which is considered to be the best. The review will

also identify any gaps or areas where future research is necessary.

What are the current evaluation methods?

UX is described by the International Standards Organisation (ISO DIS 9241-210,

2010) as: “a person’s perception and responses that result from the user and/or

anticipated use of a product, system or service”. The current user experience (UX)

evaluation toolkit varies in technique depending on the type of design project. There

is such a variety that Bevan (2009) argues that the ISO standards are not being as

widely use as they could be. One major detractor to their availability is cost. For

example, the minimum cost is around $110 per article, with others reaching double

that. The second major issue is that the U.S. Department of Health and Human

Services have created a free online usability website1 which is far superior, and

much easier to digest than the ISO articles. However, costs aside, it can be hugely

beneficial to argue designs backed by ISO standards (Bevan, 2009).

Another evaluation method; usability, consists of evaluating how user interacts with a

product. This method can be used in isolation; Agile, but Vermeeren et al. (2010)

discuss that usability and UX are directly related and both should be considered

during evaluations. UX can augment the existing usability methods during

evaluations where environmental context applies. Trivedi & Khanum (2012) agree

that a user’s experience may vary depending on the context in which they are

interacting with the device i.e. Field vs Lab studies.

1 http://www.usability.gov/

Advantages & Disadvantages of Field Methods

While conducting their research Vermeeren et al. (2010) found that to their surprise

almost half of all early evaluation methods could be used in the field. Another study

conducted by Monahan et al. (2008) revealed that the greatest advantage to field

methods was the real world context which would lead to rich qualitative data. It was

discovered that new tasks could emerge from the user during these studies. The

importance of context in the field is agreed by Trivedi and Khanum (2012). They

discuss the importance of characteristics of context (the users, tasks an

environment) as being as important as usability on the product itself.

When it came to disadvantages Monahan et al. (2008) learned that the biggest was

the labour intensity required, the long timescale, and availability of the users and

also the difficulty of the analysis, which is hard to replicate. Researchers looking for

more control when conducting their research will opt for lab methods as this allows

the evaluation to take place in a controlled environment.

Advantages & Disadvantages of Lab Methods

Monahan et al. (2008) concluded that the main advantages of lab methods were that

these methods were more focused and there was more control over the user’s tasks.

Other benefits included less cost, not as labour intensive, and the ability to control

the environment.

The main disadvantage is that lab methods do not look at tasks in context, instead,

the tasks are created by the researchers. Field studies would allow the user to

generate their own tasks. Recruitment was also a problem in lab studies.

What are the current User Experience and Usability methods?

Agile UX

Agile, as discussed by Schwartz (2013) is: “a method to quickly provide software that

answers the users’ needs with a certain level of quality”. Agile development is based

on continuous integration and development to radically reduce their cycle times,

(Gothelf, & Seiden, 2013). Schwartz (2013) also found that the emergence of Agile

was instigated by software developers who were frustrated with delayed projects,

project overruns and stressful jobs. Initially Agile did not incorporate a user centred

approach, however they both shared common goals which served as a starting point

for integration. Agile itself is focused on customer satisfaction through usability by

delivering small updates at a time to gauge and or rework client feedback (Jurca,

Hellmann, & Maurer, 2014). However, the Agile camps are divided.

Supporters of Big Upfront Design

Supporters of big upfront design argue that it is absolutely necessary to capture user

needs, usability goals and context of use and other design criteria before

development. Agile methods move at such a pace that usability experts could

potentially not have enough time to ensure consistency across the design. Supports

such as Champerlain, Sharp and Maiden (2006) would agree that; “UCD

practitioners must be given ample time in order to discover the basic needs of their

users before any code gets released into the shared coding environment”.

Opposing Big Upfront Design

Jurca, Hellmann, and Maurer (2014) discuss how Armitage (2004) argues against

big upfront design citing that “it is too risky and time and money- consuming to

design deeply beforehand and it is totally against Agile practices”. It also would

make any significant change further down the development process much harder to

fulfil. However, as some form of design is necessary Armitage (2004) discussed that

to enable designers to work efficiently in an agile environment they will need to

reduce their designs to their simplest form by eliminating pixel perfect prototypes for

more unrefined sketches.

Although there are two strategies when applying Agile UX, it seems the general

agreement of when to use either big upfront design or not will depend greatly on the

project, and the team.

Design Thinking

Design thinking is the adoption of the cognitive thinking process of designers by

firms, management and media agencies to solve difficult problems, (Kimbell, 2011).

Razzouk and Shute (2012) describe design thinking as; “a synthetic process dealing

with the real world content were at its core it is based on and iterative and interactive

flow of how designers visualise solving problems or conceptualising ideas, and how

they draw relations between each”. It takes good design skills, as well as design

thinking processes, can assist in solving complex challenges as well as adjust to

unexpected project changes. It has been said that this style of thinking has been

known to be a chaotic work process. To ensure this does not happen the project life

cycle follows a mutual adjustment between specifications and solutions, (Razzouk &

Shute, 2012).

Design thinking can be considered in two discourses described by Johansson‐

Sköldberg, Woodilla, & Çetinkaya (2013) as “designerly” thinking; “Designerly

thinking links theory and practice from a design perspective, and is accordingly

rooted in the academic field of design.” The other being design thinking; “where

design practice and competence are used beyond the design context (including art

and architecture), for and with people without a scholarly background in design,

particularly in management.”

There are clear similarities in both articles by Johansson‐Sköldberg, Woodilla, &

Çetinkaya (2013) and by Kimbell (2011) with regards to the design thinking process

being used by management as a method to solve problems. This was also

discussed briefly in an article by Gothelf & Seiden (2013), but the discussion is more

based around it’s communication and iteration process which has been incorporated

in to Lean UX.

Lean UX

Lean UX is the combination of design thinking and Agile. It is a highly

communicative process where heavy deliverables give way to techniques of shared

understanding between colleagues. Design thinking focuses more on the problems

and challenges through collaborating with iterations towards a perfect design,

(Gothelf, & Seiden, 2013). This differs from Agile, which focuses on development

goals and software usability requirements. Due to the availability of market

feedback, designs can be assessed for what works and can be adjusted accordingly,

instead of having a set list of features and documents for the project.

Are there any similarities or differences between these methods?

All user experience methods have been designed to boost productivity, encourage

more collaboration between all parties, clients and stakeholders and to cut project

costs. Another similarity between these methods is that each begins with little or no

initial regard to a user’s experience. They were development focused and driven but

a growing need and to stay ahead of competition has forced the development of

designing around the user’s needs. User experience, contextual research were all

elements added later in the process specially to address the use of mobile.

Although the similarities lie with reasons for the development of these methods, they

each are very different of their approach in how they deliver their results.

Gaps and future research?

Trivedi & Khanum (2012) and Vermeeren et al. (2010) both agree that there is

significant research required in relation to social UX. As this is a relatively new

frontier there is no single evaluation toolkit to provide all the answers. There also

seems to be an underlying agreement within all articles that there is a definite need

for continual research and development of a unified system. It seems more so than

not that this relates to mostly field methods. Monahan et al. (2008) made

recommendations exactly for an evaluation and research toolkit for practitioners in

which they can mix and match items to suit specific needs. This would directly

address time and budget concerns which arise.

Conclusion The initial research began looking in to the particular methods of evaluation. This

was quite a granular approach yet successfully yielded significant information and an

increased understanding of the multitude of options available when conducting a UX

project. After this it was decided to take a high level approach to determine what

are the current main UX approaches. Although the UX methods do have a

commonality in which they appear to strive for quicker project turn arounds and cost

savings, each have their own set of principles, which are different from the rest. It

was quite interesting to see these differences. However, with the constant stream of

new developments online, Agile UX seems to be the strongest and most efficient

methodology. It joins developers and designers, and encourages small sets of

updates that are continually rolled out and updated as more research and testing is

carried out. There is still work to be done in Agile, it has limitations in being able to

fully integrate all aspects of UX process in to the project flow.

There are definite areas of evaluation studies that more research is required.

Especially around social integration and field studies. It seems to be an ongoing

trend in tandem with the evolution of portable devices. As devices become more

personal the requirement for field studies will inevitably increase. Therefore a set of

standards should be adopted in for future field research methods.

References Armitage, J. (2004). Are agile methods good for design?. interactions, 11(1), 14-23.

Bevan, N. (2009). International standards for usability should be more widely

used. Journal of usability Studies, 4(3), 106-113.

Gothelf, J., & Seiden, J. (2013). Lean UX: Applying lean principles to improve user

experience. " O'Reilly Media, Inc.".

Johansson‐Sköldberg, U., Woodilla, J., & Çetinkaya, M. (2013). Design thinking:

past, present and possible futures. Creativity and Innovation

Management, 22(2), 121-146.

Jurca, G., Hellmann, T. D., & Maurer, F. (2014, July). Integrating agile and user-

centered design: a systematic mapping and review of evaluation and

validation studies of agile-ux. In Agile Conference (AGILE), 2014 (pp. 24-32).

IEEE.

Kimbell, L. (2011). Rethinking design thinking: Part I. Design and Culture, 3(3), 285-

306.

Monahan, K., Lahteenmaki, M., McDonald, S., & Cockton, G. (2008, September). An

investigation into the use of field methods in the design and evaluation of

interactive systems. In Proceedings of the 22nd British HCI Group Annual

Conference on People and Computers: Culture, Creativity, Interaction-

Volume 1 (pp. 99-108). British Computer Society.

Nngroup.com,. (2015). Which UX Deliverables Are Most Commonly Created and

Shared?. Retrieved 18 November 2015, from

http://www.nngroup.com/articles/common-ux-deliverables/

Razzouk, R., & Shute, V. (2012). What is design thinking and why is it

important?. Review of Educational Research, 0034654312457429.

Schwartz, L. (2013). Agile-User Experience Design: an Agile and User-Centered

Process?.

Trivedi, M. C., & Khanum, M. A. (2012). Role of context in usability evaluations: A

review. arXiv preprint arXiv:1204.2138.

Vermeeren, A. P., Law, E. L. C., Roto, V., Obrist, M., Hoonhout, J., & Väänänen-

Vainio-Mattila, K. 2010, October). User experience evaluation methods:

current state and development needs. In Proceedings of the 6th Nordic

Conference on Human-Computer Interaction: Extending Boundaries (pp. 521-530). ACM.

Bibliography Gray, C. M. (2014, April). Evolution of design competence in UX practice. In

Proceedings of the 32nd annual ACM conference on Human factors in

computing systems (pp. 1645-1654). ACM.

Gruber, M., De Leon, N., George, G., & Thompson, P. (2015). Managing by

design. Academy of Management Journal, 58(1), 1-7.

Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., & Sinnelä, A.

(2011). UX Curve: A method for evaluating long-term user experience.

Interacting with Computers, 23(5), 473-483.