View
218
Download
0
Tags:
Embed Size (px)
Citation preview
Managed Learning Environment Project
User Issues - Formative and Summative Assessment
MarkMark Simpson Simpson
How did we specify the system - specification?How did we decide on content- requirements?How did we verify content?Develop conventions for pages?Usability trade-offs and compromises?
Managed Learning Environment Project
What is UsabilityPerhaps it is:Learnability, Efficiency, MemorabilityAccessibility, Recoverability, Satisfaction
OrIf you use the system enough you will be able to use it
OrIf you are too stupid to work it out it isn’t my problem and why are you here.
Managed Learning Environment Project
Usability how hard is it?Everybody has an opinion
“I hate …… …… …”
“Frames / Flash / PDF are rubbish”
“we know what we want, don’t waste anymore time exploring and understanding, just get on and create it”. Steve Krug, Don’t make me think
Anything new will not satisfy X % of users, until they have used it.
Managed Learning Environment Project
Difference between screen and paper
Sun
Absorbed & Reflected constant light
Electron Gun
Direct & Emittedflickering light
What is this, in your opinion ?
RED
BLUE
What colour is this, in your opinion?
Managed Learning Environment Project
Development cycle
Changing login & password
October 2000
March 2001
September2001
December2001
February 2002
April 2002
June 2002
July 2002
August 2002
Web Site Requirements / Specification
Develop Content List
PreparingContent / Layout
Designing initial prototype
1st user testing card sort
Modifying design menus
2nd user testing4th user testingNavigation &Logic Testing
CorrectingNavigation Issues
3rd user testing
Modifying design
WritingHelp and Search
Project evaluation report
Implement uPortal
Managed Learning Environment Project
SpecificationWhat does the system do?
What are the users likeWhat do they want the system to doWhat can we do from the users want listWhat are the usability issues with current systemsCan the MLE solve these issues
Project team produced a list of what they wanted as indicative student users
Managed Learning Environment Project
User Requirements / Content How do you start
Ask yourself / project team / steering committee Ask users Research other systems or similar sites What is unique to your project – DMU it was QLS Produce a list of content areas Refine list to remove duplication / impossible content Record list with reasons
Managed Learning Environment Project
Sources of Requirements1. DMU PC ownership and use
survey 1997, 98, 99, & 00
2. Collaboration in Research Support by Academic Libraries in Leicestershire survey [CORSALL]
3. ANGEL user analysis results
4. DMU Nursing School, online course survey
5. DMU Library survey of student information needs
6. DMU Research papers on using the Internet for teaching.
7. DMU Keyskills survey of all first year students
8. Student Experience Unit feedback from students leaving the university
9. DMU Student focus group meeting
10. Usage reports from similar systems
11. User interviews with users of similar existing systems
Managed Learning Environment Project
10. User tracking data User Logs for six months from a similar system
Logon Data, route through site, user names
Course being studied, Self assessments completed, discussion involvement
Didn’t match interview or focus group reports
Managed Learning Environment Project
Results of Specification Consistent increase in PC ownership and use of online resources – 34 % in 2001
All users ask for more content than they use
Academic users want the MLE to be integrated in university and not isolated
Navigation done using mouse and back arrow
Content and feature list for student users
Managed Learning Environment Project
Version 1
Managed Learning Environment Project
Card Sorting
menus, content and priority
Managed Learning Environment Project
Dendrogram
Cluster Analysis – grouping of content
Managed Learning Environment Project
Managed Learning Environment Project
MLE Version 2
Managed Learning Environment Project
Further login trials, testing navigation
Login trials - user name and password
Simple content location
Logging in and out
Interface issues
Don’t look at too much each time
Managed Learning Environment Project
Testing login &
photographs
Managed Learning Environment Project
Login trials with HNC, BA, MA students
12 first year students HNC Housing Part-time
7 second year students BA Accounting and Finance Full-time
26 postgraduates taking MA degrees Full-time
6 postgraduates last part of MA Full-time
Between each test correct problems and develop a few new features for testing
Managed Learning Environment Project
How many times do you test it, how many subjects do you need?
As many times as will yield useful data
Between 4 and 10 subjects at a time
What are you testing ?what is the questionwhat will you do with the resultswhat resources are available
Don’t leave it hoping it will be alright
Managed Learning Environment Project
MLE Version 3
Managed Learning Environment Project
Managed Learning Environment Project
Future UsabilityTraining and Help pages
Focus groups – face to face
Online questionnaires
Feedback from modules with teaching resources online
User tracking and what people actually do
Managed Learning Environment Project
Benefits of the methods
Provide a handle on the university administration information issues
Publicise the project – making it visible
Resolve problems as they were discovered
Kept in touch with all types of users
Managed Learning Environment Project
How would we do it differently
Have an off-line non development system
Use external evaluation only in the second half of the project
When carrying out usability testing freeze the design and clearly name versions
A recorded and agreed method of tracking changes and developments in the interface.
Managed Learning Environment Project
What should you do? Set-up a steering group or panel
Think about what the site / page / system is providing and so make your site user centred, not content centred
Do testing of content and navigation regularly
Always have a usability issue to hand
Use conventions and standard layouts – they work
– what did the users tell you
Managed Learning Environment Project
Summary
6 Formative and 3 Summative methods of assessment
Ask users what they want
Test navigation regularly
Implement changes and test again
Don’t expect to please everybody – usability is opinions based on evidence
Managed Learning Environment Project
Where am I now ?