View
215
Download
0
Embed Size (px)
Citation preview
1©Serco Reproduction permitted provided the source is acknowledged
Formalising usability as part of ISO standards
Nigel BevanNigel Bevan
[email protected]@usability.serco.com
www.usability.serco.comwww.usability.serco.comwww.usabilitynet.orgwww.usabilitynet.org
Serco Usability ServicesSerco Usability Services
UKUK
2©Serco Reproduction permitted provided the source is acknowledged
TopicsTopics
What is usability?What is usability?
Definitions of usability? ISO 9126-1 and ISO 9241-11Definitions of usability? ISO 9126-1 and ISO 9241-11
Specifying usability as quality in use: ISO 9126-4, the Specifying usability as quality in use: ISO 9126-4, the Common Industry Format and ISO 25030Common Industry Format and ISO 25030
Usability of everyday products: ISO 20282Usability of everyday products: ISO 20282
User interface usability: ISO 9241User interface usability: ISO 9241
User centred design process: ISO 13407User centred design process: ISO 13407
Usability capability: ISO TR 18529Usability capability: ISO TR 18529
3©Serco Reproduction permitted provided the source is acknowledged
What is usability?What is usability?
what do you think usability is ......?what do you think usability is ......?
4©Serco Reproduction permitted provided the source is acknowledged
IBMIBM
““IBM's focus on ease of use has led to technology IBM's focus on ease of use has led to technology breakthroughs and advances in design research”breakthroughs and advances in design research”
““For developers and manufacturers, the advantages of For developers and manufacturers, the advantages of creating usable products far outweigh the costs”creating usable products far outweigh the costs”
““Every dollar invested in ease of use returns $10 to Every dollar invested in ease of use returns $10 to $100”$100”
http://www.ibm.com/ibm/easy/http://www.ibm.com/ibm/easy/
5©Serco Reproduction permitted provided the source is acknowledged
MicrosoftMicrosoft
““The Usability Group is an integral part of the product The Usability Group is an integral part of the product design process at Microsoft”design process at Microsoft” Over 120 usability engineersOver 120 usability engineers Over 25 usability labsOver 25 usability labs
http://www.microsoft.com/usability/http://www.microsoft.com/usability/
6©Serco Reproduction permitted provided the source is acknowledged
Why is usability important?Why is usability important?
Focus on user and organisational needsFocus on user and organisational needs reduce development timesreduce development times less training and support and documentation is requiredless training and support and documentation is required
Improve productivityImprove productivity simpler interface, fewer user errorssimpler interface, fewer user errors
Improve the competitive edgeImprove the competitive edge increasing expectations for ease of useincreasing expectations for ease of use increasing usability of competitive productsincreasing usability of competitive products high profile of usability in advertisinghigh profile of usability in advertising
Improve the quality of lifeImprove the quality of life less stress, users are more satisfiedless stress, users are more satisfied lower staff turnoverlower staff turnover
Health and safety legislationHealth and safety legislation European Directive on Display Screen EquipmentEuropean Directive on Display Screen Equipment
7©Serco Reproduction permitted provided the source is acknowledged
Case study: Hewlett Packard Case study: Hewlett Packard
User-centred design methods were applied to redesign User-centred design methods were applied to redesign software used for identifying network problems:software used for identifying network problems:
OldOld NewNew
time to finish tasktime to finish task 9.4 min9.4 min 4.1 min4.1 min
problems identifiedproblems identified 16%16% 78%78%
average length of callaverage length of call 30 min30 min 10 min10 min
size of manualsize of manual 25 pages25 pages 4 pages4 pages
people needing the manualpeople needing the manual 53%53% 3%3%
user satisfaction ratinguser satisfaction rating 3.53.5 6.86.8
In addition to the benefits to customers, HP recovered their In addition to the benefits to customers, HP recovered their costs in 18 monthscosts in 18 months
8©Serco Reproduction permitted provided the source is acknowledged
ISO/IEC 9126 Software Product Quality ModelISO/IEC 9126 Software Product Quality Model
functionality reliability
usability efficiency
maintainability
portability
accuracysuitability
interoperabilitysecurity
understandabilitylearnabilityoperability
analysabilitychangeability
stabilitytestability
maturityfault tolerancerecoverability
availability
time behaviourresourceutilisation
adaptabilityinstallabilityco-existencereplaceability
9©Serco Reproduction permitted provided the source is acknowledged
ISO/IEC 9126-1 DefinitionsISO/IEC 9126-1 Definitions FunctionalityFunctionality
The capability of the software product to provide functions which meet stated The capability of the software product to provide functions which meet stated and implied needs when the software is used under specified conditions.and implied needs when the software is used under specified conditions.
ReliabilityReliability The capability of the software product to maintain a specified level of The capability of the software product to maintain a specified level of
performance when used under specified conditions performance when used under specified conditions
UsabilityUsability The capability of the software product to be understood, learned, used and The capability of the software product to be understood, learned, used and
attractive to the user, when used under specified conditions.attractive to the user, when used under specified conditions.
EfficiencyEfficiency The capability of the software product to provide appropriate performance, The capability of the software product to provide appropriate performance,
relative to the amount of resources used, under stated conditions.relative to the amount of resources used, under stated conditions.
MaintainabilityMaintainability The capability of the software product to be modified. Modifications may The capability of the software product to be modified. Modifications may
include corrections, improvements or adaptation of the software to changes in include corrections, improvements or adaptation of the software to changes in environment, and in requirements and functional specifications. environment, and in requirements and functional specifications.
PortabilityPortability The capability of the software product to be transferred from one environment to The capability of the software product to be transferred from one environment to
another.another.
10©Serco Reproduction permitted provided the source is acknowledged
Usability : The extent to which a product canbe used by specified users to achievespecified goals with effectiveness, efficiencyand satisfaction in a specified context ofuse.
Effectiveness : The accuracy andcompleteness with which users achievespecified goals.
Efficiency: The resources expended inrelation to the accuracy and completenesswith which users achieve goals.
Satisfaction : The comfort and acceptabilityof use.
ISO 9241-11 Guidance on UsabilityISO 9241-11 Guidance on Usability
11©Serco Reproduction permitted provided the source is acknowledged
ISO/IEC 9126-1 Software Product Quality ModelISO/IEC 9126-1 Software Product Quality Model
quality in use
functionality reliability
usability efficiency
maintainability portability
accuracysuitability
interoperabilitysecurity
understandabilitylearnabilityoperability
attractiveness
analysabilitychangeability
stabilitytestability
maturityfault tolerancerecoverability
availability
time behaviourresourceutilisation
adaptabilityinstallabilityco-existencereplaceability
12©Serco Reproduction permitted provided the source is acknowledged
Quality in UseQuality in Use
ISO/IEC 9126-1 (2001) Software Engineering - ISO/IEC 9126-1 (2001) Software Engineering - Product quality - Part 1: Quality modelProduct quality - Part 1: Quality model
quality in usequality in use
the capability of the software product to enable the capability of the software product to enable specified users to achieve specified goals with specified users to achieve specified goals with effectiveness, productivity, safety and satisfaction effectiveness, productivity, safety and satisfaction in specified contexts of usein specified contexts of use
Quality in use is the user’s view of the quality of a system Quality in use is the user’s view of the quality of a system containing software, and is measured in terms of the result of containing software, and is measured in terms of the result of using the software, rather than properties of the software using the software, rather than properties of the software itselfitself
Quality in use measures the combined effect of ease of use, Quality in use measures the combined effect of ease of use, functionality, efficiency and reliability.functionality, efficiency and reliability.
13©Serco Reproduction permitted provided the source is acknowledged
Approaches to achieving usabilityApproaches to achieving usability
process quality product quality quality in use
usability in context
development process
product effect of the product
user centred process
interface and interaction
ISO 13407 ISO 9241-11ISO 14598-1
ISO/IEC 9126-1ISO/IEC 9126-4
ISO 20282-2
organisational capability
life cycle processes
usability capability
ISOTR 18529 ISO 9241 parts 10, 12-17ISO/IEC 9126-2/3
14©Serco Reproduction permitted provided the source is acknowledged
Approaches to achieving usabilityApproaches to achieving usability
1 Design for quality in useDesign for quality in use ISO 9241-11, ISO 9126-4: effectiveness, efficiency, (safety), ISO 9241-11, ISO 9126-4: effectiveness, efficiency, (safety),
satisfactionsatisfaction ISO 20282-2 Usability of everyday productsISO 20282-2 Usability of everyday products
2 Design the user interfaceDesign the user interface ISO 9241 parts 12 to 17 Ergonomic principlesISO 9241 parts 12 to 17 Ergonomic principles
3 Use user-centred methods in developmentUse user-centred methods in development ISO 13407 Human centred design processesISO 13407 Human centred design processes
4 Establish organisational processesEstablish organisational processes ISO TR 18529 ISO TR 18529 Human-centred lifecycle process descriptionsHuman-centred lifecycle process descriptions
15©Serco Reproduction permitted provided the source is acknowledged
requirements
User requirements
Functional requirements
Detailed specification
inspection
evaluationproduct
Deliver
Prototype
Code
What can be specified and measured?What can be specified and measured?
Test user function
Test user goal
Internalusability metricsISO/IEC 9126-3
requirements
Externalusability metricsISO/IEC 9126-2
Quality in usemetrics
ISO/IEC 9126-4
16©Serco Reproduction permitted provided the source is acknowledged
1. Design for quality in use1. Design for quality in use
Usability/quality in use measures:Usability/quality in use measures:
Effectiveness and efficiencyEffectiveness and efficiency MUSiC Performance Measurement MethodMUSiC Performance Measurement Method
http://www.usability.serco.com/publs.htmlhttp://www.usability.serco.com/publs.html
SatisfactionSatisfaction SUMI questionnaireSUMI questionnaire
www.ucc.ie/hfrg/questionnaires/sumiwww.ucc.ie/hfrg/questionnaires/sumi
Reporting usabilityReporting usability Common industry format for usability test resultsCommon industry format for usability test results includes intended context of use and actual context of includes intended context of use and actual context of
evaluationevaluation
www.nist.gov/iusrwww.nist.gov/iusr
17©Serco Reproduction permitted provided the source is acknowledged
ISO/IEC CD 25030: Software quality requirements ISO/IEC CD 25030: Software quality requirements and evaluation – Quality requirementsand evaluation – Quality requirements
Requirements can be categorised as: Requirements can be categorised as: functional requirementsfunctional requirements quality requirementsquality requirements other non-functional requirementsother non-functional requirements
Software quality requirements are categorised using the Software quality requirements are categorised using the ISO/IEC 9126-1 software quality model into ISO/IEC 9126-1 software quality model into requirements for: requirements for: quality in use (the effectiveness, productivity, safety and quality in use (the effectiveness, productivity, safety and
satisfaction with the software applied in its context of use)satisfaction with the software applied in its context of use) external quality (the behaviour of the finished software product)external quality (the behaviour of the finished software product) internal quality (static properties during development, such as internal quality (static properties during development, such as
adherence to coding standards, and meeting size and adherence to coding standards, and meeting size and complexity thresholds)complexity thresholds)
18©Serco Reproduction permitted provided the source is acknowledged
Requirements life cycle modelRequirements life cycle model
Definition
Measured quality
Measured quality
Stated requirements
Stated requirements
Stated requirements
Formalisation Evaluation
Measurable requirements
Measurable requirements
Measurable requirements
Internal quality
External quality
Quality in use
Quality requirements
Stake-holderneeds
Measured quality
Developed system
Functional requirements
Other non-functional requirements
19©Serco Reproduction permitted provided the source is acknowledged
Quality requirements specificationQuality requirements specification
Define quality requirementsDefine quality requirements Identify stakeholdersIdentify stakeholders
end users, developers, producers, trainers, maintainers, disposers, acquirer, supplier organisations and regulatory bodies
Elicit requirements from stakeholdersElicit requirements from stakeholders Quality in use, External and Internal QualityQuality in use, External and Internal Quality
Analyse the set of requirementsAnalyse the set of requirements Resolve problemsResolve problems Confirm requirementsConfirm requirements Record requirementsRecord requirements
Formalise quality requirementsFormalise quality requirements Specify target values for measuresSpecify target values for measures Demonstrate traceabilityDemonstrate traceability Maintain requirementsMaintain requirements
20©Serco Reproduction permitted provided the source is acknowledged
Elicit requirements from stakeholdersElicit requirements from stakeholders
Include the needs, wants, desires, expectations and Include the needs, wants, desires, expectations and perceived constraints perceived constraints
Identify requirementsIdentify requirements Quality in useQuality in use ExternalExternal InternalInternal
Identify constraintsIdentify constraints Define scenarios of useDefine scenarios of use
21©Serco Reproduction permitted provided the source is acknowledged
Quality in useQuality in use
Extent to which a product used by specified users meets their needs to achieve specified goals with effectiveness, productivity and satisfaction in specified contexts of use
Effectiveness: accuracy and completenessEffectiveness: accuracy and completeness
Productivity: the resources expended by users in relation Productivity: the resources expended by users in relation to the effectiveness achievedto the effectiveness achieved
22©Serco Reproduction permitted provided the source is acknowledged
Why is quality in use important?Why is quality in use important?
It is about meeting business and user needsIt is about meeting business and user needs Effectiveness: success in achieving goalsEffectiveness: success in achieving goals Productivity: staff timeProductivity: staff time Satisfaction: willingness to use the systemSatisfaction: willingness to use the system
23©Serco Reproduction permitted provided the source is acknowledged
Quality in use: context of UseQuality in use: context of Use
The usability of a product is affected not only by The usability of a product is affected not only by the features of the product itself but also by its the features of the product itself but also by its Context of UseContext of Use
Context is the characteristics of:Context is the characteristics of: the users of the productthe users of the product the tasks they carry out the tasks they carry out the technical, organisational the technical, organisational
and physical environmentand physical environmentin which the product is in which the product is usedused
the date and time when the the date and time when the product is being usedproduct is being used
24©Serco Reproduction permitted provided the source is acknowledged
Example: A bank ATMExample: A bank ATM
Context description:Context description: The usersThe users
The user characteristicsThe user characteristics
The tasks users performThe tasks users perform
The technical environmentThe technical environment(hardware and software to(hardware and software tosupport system)support system)
The physical environmentThe physical environment
The social or organisational environmentThe social or organisational environment
25©Serco Reproduction permitted provided the source is acknowledged
New design features to meet contextual New design features to meet contextual needsneeds
Recess for wheelchair access.Recess for wheelchair access.
Speech output for visuallySpeech output for visuallyimpaired users.impaired users.
Customisation features forCustomisation features forrapid access.rapid access.
Finger print for identification.Finger print for identification.
Visor appears during sunny weather.Visor appears during sunny weather.
Buttons light during darkness.Buttons light during darkness.
Alarm button for security alert. Alarm button for security alert.
26©Serco Reproduction permitted provided the source is acknowledged
Efficiency: mean transaction timesEfficiency: mean transaction times
0
0.5
1
1.5
2
2.5
3
DepositingCash
WithdrawingCash
DepositingCheques
WithdrawingCheques
Minutes Old system
New system
27©Serco Reproduction permitted provided the source is acknowledged
User satisfaction: SUMI questionnaireUser satisfaction: SUMI questionnaire
0
10
20
30
40
50
60
Global Efficiency Affect Helpfulness Control Learnability
Old System
New System
28©Serco Reproduction permitted provided the source is acknowledged
The danger of not setting quality in use The danger of not setting quality in use requirementsrequirements
New software for issuing UK passportsNew software for issuing UK passports Installed in passport issuing officesInstalled in passport issuing offices Took operators twice as longTook operators twice as long Caused delays of up to 3 months in obtaining a passportCaused delays of up to 3 months in obtaining a passport Huge cost of additional clerical staffHuge cost of additional clerical staff
29©Serco Reproduction permitted provided the source is acknowledged
Example of quality in use requirementsExample of quality in use requirements
User performanceUser performance ““all data entry clerks will be able to complete the task with at all data entry clerks will be able to complete the task with at
least 95% accuracy in under 10 minutes”least 95% accuracy in under 10 minutes”
User satisfactionUser satisfaction ““the mean score on the SUMI scale will be greater than 50” More the mean score on the SUMI scale will be greater than 50” More
information on quality in use requirements information on quality in use requirements
30©Serco Reproduction permitted provided the source is acknowledged
Common Industry Format usability testsCommon Industry Format usability tests
NIST initiativeNIST initiative National Institute of Standards and TechnologyNational Institute of Standards and Technology
Objective: raise the profile of usability in procurementObjective: raise the profile of usability in procurement Suppliers provide standard test reports to purchasersSuppliers provide standard test reports to purchasers Suppliers include: Suppliers include:
IBM, Microsoft, HP, Sun, Kodak, Oracle, CompaqIBM, Microsoft, HP, Sun, Kodak, Oracle, Compaq Purchasers include: Purchasers include:
Boeing, Northwest Mutual Life, State Farm Insurance, FidelityBoeing, Northwest Mutual Life, State Farm Insurance, Fidelity Reports provided in confidenceReports provided in confidence Could permit comparisonsCould permit comparisons Agreed as US standard Agreed as US standard ANSI/NCITS 354ANSI/NCITS 354, to be submitted to ISO, to be submitted to ISO
http://www.nist.gov/iusrhttp://www.nist.gov/iusr
31©Serco Reproduction permitted provided the source is acknowledged
Report format - technicalReport format - technical
Test objectivesTest objectives ParticipantsParticipants
Total number and user groupsTotal number and user groups Key characteristics and capabilities expectedKey characteristics and capabilities expected How selected: with the key characteristics and capabilities?How selected: with the key characteristics and capabilities?
Context of evaluationContext of evaluation Task scenarios and task data usedTask scenarios and task data used Any performance criteria usedAny performance criteria used Physical and computer environmentPhysical and computer environment
32©Serco Reproduction permitted provided the source is acknowledged
Report format - resultsReport format - results
EffectivenessEffectiveness % completely and correctly achieving their goals% completely and correctly achieving their goals Separate data where assistance was givenSeparate data where assistance was given
EfficiencyEfficiency Mean time taken to complete the taskMean time taken to complete the task Standard deviation (and standard error)Standard deviation (and standard error)
SatisfactionSatisfaction Optional: own scales or standard questionnaireOptional: own scales or standard questionnaire
33©Serco Reproduction permitted provided the source is acknowledged
ISO 20282: Usability of everyday products ISO 20282: Usability of everyday products Part 1: Universal user profilePart 1: Universal user profile
Design should take account ofDesign should take account of Strength and Biomechanical AbilitiesStrength and Biomechanical Abilities HandednessHandedness Body Dimensions.Body Dimensions. Visual Abilities. Visual Abilities. Auditory AbilitiesAuditory Abilities Cognitive AbilitiesCognitive Abilities Language and literacyLanguage and literacy CultureCulture AgeAge GenderGender
34©Serco Reproduction permitted provided the source is acknowledged
ISO WD 20282-2: Test methodISO WD 20282-2: Test method
A summative test method for measuring usability based onA summative test method for measuring usability based on ISO 9241-11 (Guidance on usability)ISO 9241-11 (Guidance on usability) ISO 9126-4 (Quality in use metrics)ISO 9126-4 (Quality in use metrics) Common Industry Format for usability test reportsCommon Industry Format for usability test reports
Identify intended context of use and user groupsIdentify intended context of use and user groups
MeasuresMeasures Success rate (ease of operation)Success rate (ease of operation) Task time (efficiency)Task time (efficiency) Satisfaction scale (being developed)Satisfaction scale (being developed)
35©Serco Reproduction permitted provided the source is acknowledged
Approaches to achieving usabilityApproaches to achieving usability
process quality product quality quality in use
usability in context
development process
product effect of the product
user centred process
interface and interaction
ISO 13407 ISO 9241-11ISO 14598-1
ISO/IEC 9126-1ISO/IEC 9126-4
ISO 20282-2
organisational capability
life cycle processes
usability capability
ISOTR 18529 ISO 9241 parts 10, 12-17ISO/IEC 9126-2/3
36©Serco Reproduction permitted provided the source is acknowledged
2. Design the user interface2. Design the user interfaceISO 9241: Ergonomic requirements for office work with visual display ISO 9241: Ergonomic requirements for office work with visual display terminalsterminals
Parts 10 to 17 give guidance on softwareParts 10 to 17 give guidance on software
1010 Dialogue principles Dialogue principles
1111 Guidance on usability Guidance on usability
1212 Presentation of information Presentation of information
1313 User guidance User guidance
1414 Menu dialogues Menu dialogues
1515 Command Language dialogues Command Language dialogues
1616 Direct Manipulation dialogues Direct Manipulation dialogues
1717 Form-filling dialogues Form-filling dialogues
37©Serco Reproduction permitted provided the source is acknowledged
Approaches to achieving usabilityApproaches to achieving usability
process quality product quality quality in use
usability in context
development process
product effect of the product
user centred process
interface and interaction
ISO 13407 ISO 9241-11ISO 14598-1
ISO/IEC 9126-1ISO/IEC 9126-4
ISO 20282-2
organisational capability
life cycle processes
usability capability
ISOTR 18529 ISO 9241 parts 10, 12-17ISO/IEC 9126-2/3
38©Serco Reproduction permitted provided the source is acknowledged
3. ISO 13407: Human centred design process for 3. ISO 13407: Human centred design process for interactive systemsinteractive systems
1. Plan the human centred process
1. Plan the human centred process
2. Specify the context of use
2. Specify the context of use
4. Produce design solutions
4. Produce design solutions
3. Specify user and organisational
requirements
3. Specify user and organisational
requirements
5. Evaluate designs against
user requirements
5. Evaluate designs against
user requirements
Meets requirements
39©Serco Reproduction permitted provided the source is acknowledged
ISO 13407 ScopeISO 13407 Scope
Computer-based interactive systemsComputer-based interactive systems The detailed recommendations are applicable to computer The detailed recommendations are applicable to computer
hardware and softwarehardware and software
Can apply the same principles to the design of any Can apply the same principles to the design of any system for use by humanssystem for use by humans A penA pen A chairA chair A meeting roomA meeting room A conferenceA conference
40©Serco Reproduction permitted provided the source is acknowledged
TRUMP: Trial Usability Maturity ProcessTRUMP: Trial Usability Maturity Process
EU-funded trial application of user-centred design EU-funded trial application of user-centred design methods developed in previous research projects methods developed in previous research projects (INUSE and RESPECT)(INUSE and RESPECT)
Serco: apply the methodsSerco: apply the methods Lloyds Register: Usability Maturity AssessmentLloyds Register: Usability Maturity Assessment
Inland Revenue/EDS - IT for 60,000 staffInland Revenue/EDS - IT for 60,000 staff RAD methodologyRAD methodology
Israel Aircraft Industries - aerospace systemsIsrael Aircraft Industries - aerospace systems traditional methodologytraditional methodology
41©Serco Reproduction permitted provided the source is acknowledged
Typical approach to usabilityTypical approach to usability
feasibility requirements design implement release
Usability group user and task analysis
usability test
prototyping
42©Serco Reproduction permitted provided the source is acknowledged
TRUMP approach: “ubiquitous usability”TRUMP approach: “ubiquitous usability”
1. Stake-holder meeting
2. Context of use
3. Scenarios
4. Usability requirements
5. Evaluate existing system
6. Prototyping
7. Style guide
8. Evaluation
9. Usability testing
10. Collect feedback
feasibility requirements design implement release
System lifecycle
PlanProcess
SpecifyContext of
Use
DesignSolution
s
SpecifyRequirements
Evaluate against Requirements
www.usability.serco.com/trump/ucdmethods
43©Serco Reproduction permitted provided the source is acknowledged
requirements
User requirements
Functional requirements
Detailed specification
Function test
evaluationproduct
Deliver
Prototype
Code
Conventional design processConventional design processHow do we incorporate user centred design?How do we incorporate user centred design?
44©Serco Reproduction permitted provided the source is acknowledged
requirements
User requirements
Functional requirements
Detailed specification
evaluationproduct
2. Intended context of use
Context used for evaluation
6.Mock up
Use in context
Prototype
Code
6. Paper prototype
6. Paper specificatio
n
3. Intended scenarios of use
Scenarios used for evaluation
1. Plans
4. Quality in userequirements
4. External usability
requirements
4. Internal usability
requirements
5. Evaluate existing system
Function test
7. Style guide
Inspection7. Style guide
45©Serco Reproduction permitted provided the source is acknowledged
Methods: feasibilityMethods: feasibility
1. Stakeholder meeting 1. Stakeholder meeting A half-day meeting to identify and agree on the role of A half-day meeting to identify and agree on the role of usability, broadly identifying the intended context of use and usability, broadly identifying the intended context of use and usability goals, and how these relate to the business usability goals, and how these relate to the business objectives and success criteria for the system. objectives and success criteria for the system.
2. Context of use2. Context of useA half-day workshop to collect and agree detailed information A half-day workshop to collect and agree detailed information about the intended users, their tasks, and the technical and about the intended users, their tasks, and the technical and environmental constraints.environmental constraints.
3. Scenarios of use3. Scenarios of useA half day workshop to document examples of how users are A half day workshop to document examples of how users are expected carry out key tasks in a specified contexts, to expected carry out key tasks in a specified contexts, to provide an input to design and a basis for subsequent provide an input to design and a basis for subsequent usability testing. usability testing.
46©Serco Reproduction permitted provided the source is acknowledged
Methods: requirementsMethods: requirements
4. Evaluate an existing system4. Evaluate an existing systemEvaluate an earlier version or competitor system to identify Evaluate an earlier version or competitor system to identify usability problems and obtain measures of usability as an usability problems and obtain measures of usability as an input to usability requirements.input to usability requirements.
5. Usability requirements5. Usability requirementsA half-day workshop to establish requirements for A half-day workshop to establish requirements for effectiveness, efficiency and satisfaction for the user groups effectiveness, efficiency and satisfaction for the user groups and tasks identified in the context of use analysis and in the and tasks identified in the context of use analysis and in the scenarios.scenarios.
47©Serco Reproduction permitted provided the source is acknowledged
Methods: designMethods: design
6. Paper prototyping6. Paper prototypingEvaluation by users of quick low fidelity prototypes (using Evaluation by users of quick low fidelity prototypes (using paper or other materials) to clarify requirements and enable paper or other materials) to clarify requirements and enable draft interaction designs and screen designs to be rapidly draft interaction designs and screen designs to be rapidly simulated and tested.simulated and tested.
7. Style guide7. Style guide Identify, document and adhere to industry, corporate or Identify, document and adhere to industry, corporate or project conventions for screen and page design.project conventions for screen and page design.
48©Serco Reproduction permitted provided the source is acknowledged
Methods: evaluationMethods: evaluation
8. Evaluation of machine prototypes8. Evaluation of machine prototypes Informal usability testing with 3-5 representative users Informal usability testing with 3-5 representative users carrying out key tasks to provide rapid feedback on the carrying out key tasks to provide rapid feedback on the usability of prototypes. usability of prototypes.
9. Usability testing9. Usability testing Formal usability testing with 8 representatives of a user group Formal usability testing with 8 representatives of a user group carrying out key tasks to identify any remaining usability carrying out key tasks to identify any remaining usability problems and evaluate whether usability objectives have been problems and evaluate whether usability objectives have been achieved.achieved.
10. Collect feedback from users10. Collect feedback from usersCollect information from sources such as usability surveys, Collect information from sources such as usability surveys, help lines and support services to identify any problems that help lines and support services to identify any problems that should be fixed in future versions. should be fixed in future versions.
49©Serco Reproduction permitted provided the source is acknowledged
Cost benefits at IAICost benefits at IAI
Development cost/benefitsDevelopment cost/benefits A all the methods used (except Style Guide) resulted in savings in A all the methods used (except Style Guide) resulted in savings in
development costs of between $5K and $70K for each method, development costs of between $5K and $70K for each method, with a total saving of $330K. with a total saving of $330K.
The cost of using the methods was only $22K, giving a cost-The cost of using the methods was only $22K, giving a cost-benefit ration of benefit ration of 1:151:15. .
Sales cost/benefitsSales cost/benefits Increased sales because customers would be more likely to buy Increased sales because customers would be more likely to buy
the product were estimated to be $400Kthe product were estimated to be $400K Support cost/benefitsSupport cost/benefits
Reduced costs of training and support were estimated at £50K.Reduced costs of training and support were estimated at £50K. Overall cost benefitsOverall cost benefits
The overall costs of the maturity assessments and use of methods The overall costs of the maturity assessments and use of methods was $27K. The total estimated savings and increased sales is was $27K. The total estimated savings and increased sales is $780K, giving a cost-benefit ration of 1:29.$780K, giving a cost-benefit ration of 1:29.
50©Serco Reproduction permitted provided the source is acknowledged
Trial Application at IAI Bottom LineTrial Application at IAI Bottom Line
Positive Feedback from ParticipantsPositive Feedback from Participants A Definite Improvement in the Development Process A Definite Improvement in the Development Process Very Cost Effective and Low CostVery Cost Effective and Low Cost Mostly Intuitive however tailoring sometimes required Mostly Intuitive however tailoring sometimes required Expert Guidance needed in few techniquesExpert Guidance needed in few techniques LAHAV decided to incorporate TRUMP techniques in it’s LAHAV decided to incorporate TRUMP techniques in it’s
standard development processstandard development process
51©Serco Reproduction permitted provided the source is acknowledged
IAI ConclusionsIAI Conclusions
Low cost simple methods for improving systems usability Low cost simple methods for improving systems usability are hereare here
The methods can be easily integrated into any software The methods can be easily integrated into any software development processdevelopment process
The responsibility for quality in use should be transferred The responsibility for quality in use should be transferred to development organizationto development organization
The Methods and supporting means are available on the The Methods and supporting means are available on the WebWeb
No more Excuses No more Excuses
52©Serco Reproduction permitted provided the source is acknowledged
Approaches to achieving usabilityApproaches to achieving usability
process quality product quality quality in use
usability in context
development process
product effect of the product
user centred process
interface and interaction
ISO 13407 ISO 9241-11ISO 14598-1
ISO/IEC 9126-1ISO/IEC 9126-4
ISO 20282-2
organisational capability
life cycle processes
usability capability
ISOTR 18529 ISO 9241 parts 10, 12-17ISO/IEC 9126-2/3
53©Serco Reproduction permitted provided the source is acknowledged
4. Establish usability capability 4. Establish usability capability ISO TR 18529: Human-centred lifecycle process descriptionsISO TR 18529: Human-centred lifecycle process descriptions
HCD.1 Ensure HCD content in system strategyHCD.1 Ensure HCD content in system strategy
HCD.2 Plan and manage the HCD processHCD.2 Plan and manage the HCD process
HCD.3 Specify the user and organisational requirementsHCD.3 Specify the user and organisational requirements
HCD.4 Understand and specify the context of useHCD.4 Understand and specify the context of use
HCD.5 Produce design solutionsHCD.5 Produce design solutions
HCD.6 Evaluate designs against requirementsHCD.6 Evaluate designs against requirements
HCD.7 Introduce and operate the systemHCD.7 Introduce and operate the system
54©Serco Reproduction permitted provided the source is acknowledged
Process improvement overviewProcess improvement overview
1.1. Identify needs for usability process improvement Identify needs for usability process improvement • Use the usability maturity model in ISO TR 18529 to assess the Use the usability maturity model in ISO TR 18529 to assess the
current capability of each organisationcurrent capability of each organisation
2.2. Make the identified improvements to the software Make the identified improvements to the software development processesdevelopment processes• Introduce simple user-based methods implementing ISO 13407Introduce simple user-based methods implementing ISO 13407
3.3. Reassess the usability capability the organisation Reassess the usability capability the organisation • Assess the extent of the improvementAssess the extent of the improvement
4.4. Identify the cost-benefits of the improvementsIdentify the cost-benefits of the improvements• Integrate the methods into the documented processesIntegrate the methods into the documented processes
55©Serco Reproduction permitted provided the source is acknowledged
Usability Maturity Model Usability Maturity Model HCD.2 Plan and manage the HCD processHCD.2 Plan and manage the HCD process
Consult stakeholders.Consult stakeholders. Identify and plan user involvement.Identify and plan user involvement. Select human-centred methods and techniques.Select human-centred methods and techniques. Ensure a human-centred approach within the project Ensure a human-centred approach within the project
team.team. Plan human-centred design process.Plan human-centred design process. Manage human-centred process.Manage human-centred process. Champion human-centred activities.Champion human-centred activities. Provide support for human-centred designProvide support for human-centred design
56©Serco Reproduction permitted provided the source is acknowledged
Usability Maturity Model Usability Maturity Model HCD.3 Specify the user and organisational HCD.3 Specify the user and organisational requirementsrequirements
Clarify and document the system goals.Clarify and document the system goals. Define stakeholders.Define stakeholders. Assess risk to stakeholders.Assess risk to stakeholders. Define the system.Define the system. Generate the user and organisational requirements.Generate the user and organisational requirements. Set usability objectives.Set usability objectives.
57©Serco Reproduction permitted provided the source is acknowledged
Usability Maturity ModelUsability Maturity Model HCD.4 Understand and specify the context of useHCD.4 Understand and specify the context of use
Identify and documentIdentify and document user’s tasks.user’s tasks. significant user attributes.significant user attributes. organisational environment.organisational environment. technical environment.technical environment. physical environment.physical environment.
58©Serco Reproduction permitted provided the source is acknowledged
Usability Maturity ModelUsability Maturity Model HCD.5 Produce design solutionsHCD.5 Produce design solutions
Allocate functionsAllocate functions Produce composite task modelProduce composite task model Produce system designProduce system design Use existing knowledge to develop design solutionsUse existing knowledge to develop design solutions Specify systemSpecify system Develop prototypesDevelop prototypes Provide user trainingProvide user training Provide user supportProvide user support
59©Serco Reproduction permitted provided the source is acknowledged
Usability Maturity ModelUsability Maturity Model HCD.6 Evaluate designs against HCD.6 Evaluate designs against requirementsrequirements
Specify and validate context of evaluation.Specify and validate context of evaluation. Evaluate early prototypes in order to define the Evaluate early prototypes in order to define the
requirements for the system.requirements for the system. Evaluate prototypes in order to improve the design.Evaluate prototypes in order to improve the design. Evaluate the system in order to check that the system Evaluate the system in order to check that the system
requirements have been met.requirements have been met. Evaluate the system in order to check that the Evaluate the system in order to check that the
required practice has been followed.required practice has been followed. Evaluate the system in use in order to ensure that it Evaluate the system in use in order to ensure that it
continues to meet organisational and user needs.continues to meet organisational and user needs.
60©Serco Reproduction permitted provided the source is acknowledged
United Kingdom’s tax collection departmentUnited Kingdom’s tax collection department
Employ over 60,000 staff in more than 600 local offices Employ over 60,000 staff in more than 600 local offices
Partnership with EDS for software development now Partnership with EDS for software development now running for more than 5 yearsrunning for more than 5 years
JAD: Joint Application DevelopmentJAD: Joint Application Development RAD: Rapid Application DevelopmentRAD: Rapid Application Development
61©Serco Reproduction permitted provided the source is acknowledged
Usability capability assessment Usability capability assessment
Formal CMM/SPICE style assessment by Lloyds RegisterFormal CMM/SPICE style assessment by Lloyds Register Interviewed 13 stakeholders in twelve 3-hour sessions Interviewed 13 stakeholders in twelve 3-hour sessions
over one week. over one week. A conventional software process assessment procedure based A conventional software process assessment procedure based
on SPICE (ISO 15504) was used. on SPICE (ISO 15504) was used. Produced a detailed profile and rich information about Produced a detailed profile and rich information about
where improvements would be beneficial. where improvements would be beneficial. At a feedback meeting on the last day, it provided the At a feedback meeting on the last day, it provided the
basis for an agreed set of improvement activitiesbasis for an agreed set of improvement activities A second assessment a year later showed significant A second assessment a year later showed significant
improvementsimprovements
62©Serco Reproduction permitted provided the source is acknowledged
Inland Revenue/EDS ConclusionsInland Revenue/EDS Conclusions
Significant improvement in all areas of usability maturitySignificant improvement in all areas of usability maturity IR/EDS now strong in the “core” areas of context, IR/EDS now strong in the “core” areas of context,
requirements, design and evaluationrequirements, design and evaluation Evaluation activity rated as outstandingEvaluation activity rated as outstanding The methods are very cost effective and will be The methods are very cost effective and will be
incorporated into the documented development processincorporated into the documented development process ““A step change in the awareness and practice of human A step change in the awareness and practice of human
centred design within IR/EDS”.centred design within IR/EDS”.
63©Serco Reproduction permitted provided the source is acknowledged
IAI LAHAV BackgroundIAI LAHAV Background
Expertise in Military Aircraft “Avionics Upgrade Expertise in Military Aircraft “Avionics Upgrade Programs”Programs”
Customers WorldwideCustomers Worldwide
User needs addressed by a group of PilotsUser needs addressed by a group of Pilots
Trial Application: PC-based Mission Planning systemTrial Application: PC-based Mission Planning system
64©Serco Reproduction permitted provided the source is acknowledged
Usability capability assessment workshopUsability capability assessment workshop
Process improvement workshopProcess improvement workshop One-day workshopOne-day workshop Review each process and activity in ISO TR 18529 in Review each process and activity in ISO TR 18529 in
detaildetail Score each process as: Score each process as:
Not, Partly, Largely or Fully performedNot, Partly, Largely or Fully performed Note whether the process is managed Note whether the process is managed
a documented part of normal proceduresa documented part of normal procedures Establish a target level for each process on the same Establish a target level for each process on the same
scalesscales A second assessment a year later showed significant A second assessment a year later showed significant
improvementsimprovements
65©Serco Reproduction permitted provided the source is acknowledged
Trial Application at IAI Bottom LineTrial Application at IAI Bottom Line
Positive Feedback from ParticipantsPositive Feedback from Participants
A Definite Improvement in the Development Process A Definite Improvement in the Development Process
Very Cost Effective and Low Cost methodsVery Cost Effective and Low Cost methods
LAHAV decided to incorporate TRUMP techniques in it’s LAHAV decided to incorporate TRUMP techniques in it’s standard development processstandard development process
66©Serco Reproduction permitted provided the source is acknowledged
Process improvement conclusionsProcess improvement conclusions
The usability maturity model was a valuable tool for The usability maturity model was a valuable tool for identifying needs for process improvementidentifying needs for process improvement Inland Revenue:summative assessment requiring three person Inland Revenue:summative assessment requiring three person
weeks effortweeks effort IAI: a simpler formative one-day assessmentIAI: a simpler formative one-day assessment
A very promising way to make major process A very promising way to make major process improvementsimprovements
www.usability.serco.com/trumpwww.usability.serco.com/trump
67©Serco Reproduction permitted provided the source is acknowledged
ISO 13407 ConclusionsISO 13407 Conclusions
ISO 13407 and ISO TR 18529 provide an internationally-agreed ISO 13407 and ISO TR 18529 provide an internationally-agreed description of good practice in user-centred designdescription of good practice in user-centred design
The content can be used to specify or evaluate:The content can be used to specify or evaluate: How usability should be incorporated into the development processHow usability should be incorporated into the development process Whether a project has applied user centred designWhether a project has applied user centred design Organisational capability in user-centred designOrganisational capability in user-centred design Basis for process improvementBasis for process improvement Appropriate content for education and training courses in user centred Appropriate content for education and training courses in user centred
designdesign Necessary areas of professional competence: professional accreditationNecessary areas of professional competence: professional accreditation
68©Serco Reproduction permitted provided the source is acknowledged
www.usabilitynet.orgwww.usabilitynet.org