Upload
sanjoy-sanyal
View
567
Download
0
Embed Size (px)
Citation preview
Tech Systems and Testing Processes – Beta Testing of an Online Exam System Dr. Sanjoy Sanyal; MBBS, MS (Surgery), MSc (Royal College of Surgeons of Edinburgh), ADPHA (Public Health)
Associate Dean (Dean I/C) and Clinical Neuroscience Professor
September 2015
Contents• Hitchhiker’s Guide to the Galaxy – Insanely great!• Technological Systems in Medical Settings • Systems Development Processes• Systems Testing Processes • Usability / Usefulness, Usability Testing Instruments• Computer System Usability Questionnaire (CSUQ)
using Easy Test Creator (ETC)• Beta testing, Score grading, Preliminary results of an
Online Examination System• Tailpiece – What was that again?
SMART: A project should be• Simple• Measurable• Achievable• Realistic• Time-bound
Project Tested: An Online Examination System
• Beta testing of an Online Examination System
Systems Defined
Technological Systems• Medical care settings:
Strategic (DSS, EIS); Tactical (MIS); Operational (PAS; Clinical EMR / EHR / EPR, CPOE, ETP, LIS, PACS)
Technological Systems• Telemedicine –
Telematics systems• Subject / Task based
systems (OT, Admission-Discharge systems); Simulation systems (Acute capability model, Pollution Asthma project, Renal services simulation)
• Medical education setting: Many of above; LMS (Moodle); Online lecture systems; Online Exam systems (Questionmark, InQsit)
Systems Development
Systems Development
• Multiple iterations till Alpha / Internal Acceptance Testing
Testing ProcessesAlpha (α) testing: Internal Acceptance Testing
• Simulated / actual operational testing by independent team at developers' site
• It is a form of Internal Acceptance testing, before it goes to Beta (β) testing
Beta (β) testing:• Comes after Alpha (α) testing • β versions released to limited audience outside
programming team, or made available to open public to increase number of feedback
• Ensures product has few faults / ‘bugs’
Beta Testing
Testing Processes(External) Acceptance testing:
• Can be conducted by end-user, customer or client• Whether or not to accept product
Regression testing:• After modifying S/W, re-run previously passing tests• Ensures modifications have not caused regression
of previous functionality– Sanity testing: Quickly checking for bizarre
behaviour– Smoke testing: Testing for basic functionality
Usability – Usefulness – Users –Qs• Usability: How easy is it for me to use the system?
• Usefulness: Does it make my work easier?
• Users: Consultative; Representative; Consensus
Usability Testing Instruments• QUIS (27): Questionnaire for User Interface Satisfaction –
Chin et al (Number in bracket indicates number of questions)• PUEU (12): Perceived Usefulness and Ease of Use – Davis• NAU (5): Nielsen’s Attributes of Usability – Nielsen • NHE (10): Nielsen’s Heuristic Evaluation – Nielsen• CSUQ (19): Lewis (Described here)• ASQ (3): After-scenario Questionnaire – Lewis• PHUE (13): Practical Heuristics for Usability Evaluation –
Perlman• PUTQ (100): Purdue Usability Testing Questionnaire – Lin et
CSUQ – Lewis
CSUQ – Lewis
Score GradingScore 19 - 56
Dis-satisfied
We are concerned that you are not entirely satisfied with the online examination system. We would like you tell us about it in more detail.
57 - 95
Neither +/-
It seems you are still double-minded about the new online exam system. Is there anything we can do to help you make up your mind?
96 - 133
Satisfied We are happy to note that you are satisfied with the new online exam system. We can assure you, your faith in the system is quite justified.
Beta Test Preliminary Results
Beta Test Preliminary Results• Average
question-wise score = 5.33 / 7 (Mildly-Moderately agree)
• Average total score = 101.2 / 133 (Satisfied)
Total Score Frequency Distribution
2 2 2
5
2 2
0
1
2
3
4
5
6
70-79 80-89 90-99 100-109 110-119 120-129
Total Score
Num
ber o
f stu
dent
s
Avg Score Frequency Distribution
012345678
4.1 4.7 5.2 5.8 </=7
Questioniwise Average score
Freq
uenc
y
Beta Test Preliminary Results
• Qs 9: The system gives error messages that clearly tells me how to fix problems (This was an ambiguous question)
Tailpiece
Tailpiece – What was that, again?