Upload
michael-halloran
View
24
Download
0
Embed Size (px)
DESCRIPTION
This research report details a usability study of an online grammar course for teachers of English as a foreign language. The purpose of the study was to investigate if participants’ understanding of course concepts improved after completing self-reflection exercises.Ten teachers divided into two teams acted as test participants. All participants did the first unit of the course. However, the test intervention required one team to complete all embedded self-reflection exercises while the other team ignored them. Afterwards, the researcher tested each participant on grammar concepts in a post-course exam of ten questions. The researcher used these exam results to carry out a t-test of two independent means in order to evaluate the test hypothesis.This report includes an extensive literature review, a methodology section, a discussion section, and three appendices.
Citation preview
SCHOOL OF LANGUAGES, LITERATURE, CULTURE AND COMMUNICATION
ASSIGNMENT SUBMISSION FORM
Student Name: Michael Halloran
Student ID Number: 13035657
Course of Study: Technical Writing (Distance Learning) Graduate Certificate
Year: 2013
Lecturer Name: Yvonne Cleary
Module Code: TW5221
Date of Submission: 29 November 2013
I, Michael Halloran, declare that the attached essay/project is entirely my own work, in my own words, and that all sources used in researching it are fully acknowledged and all quotations properly identified
How Self-Reflection Exercises Influence User Comprehension: A Usability Study Report
Michael Halloran
Student Number: 13035657
Course Name: Graduate Certificate in Technical Communication
Module Code: TW5221
Supervisor: Dr. Yvonne Cleary, University of Limerick
Date of Submission: 29 / 11/ 2013
TW5221 Michael Halloran - 13035657
i
Abstract
This research report details a usability study of an online grammar course for teachers of
English as a foreign language. The purpose of the study was to investigate if participants’
understanding of course concepts improved after completing self-reflection exercises.
Ten teachers divided into two teams acted as test participants. All participants did the
first unit of the course. However, the test intervention required one team to complete all
embedded self-reflection exercises while the other team ignored them. Afterwards, the
researcher tested each participant on grammar concepts in a post-course exam of ten
questions. The researcher used these exam results to carry out a t-test of two independent
means in order to evaluate the test hypothesis.
This report includes an extensive literature review, a methodology section, a
discussion section, and three appendices.
KEY WORDS: Constructivism, e-learning, hypothesis test, online course, self-reflection,
usability test.
Acknowledgements
I want to thank Prof. Philip Rubens for helping me to refine this research project. I also want
to thank Dr. Yvonne Cleary for her advice, encouragement, and mentorship over the course
of TW5221. I want to thank the participants for setting aside time to take part in the test.
Finally, I want to thank Liam Halloran and Bryna Greenlaw for proofreading the final report.
TW5221 Michael Halloran - 13035657
ii
Contents
Abstract ....................................................................................................................................... i
Acknowledgements ..................................................................................................................... i
Section 1: Introduction ............................................................................................................... 1
Section 2: Literature review ....................................................................................................... 2
Section 2.1: Overview ........................................................................................................... 2
Section 2.2: Theories of education and e-learning: behaviourism, cognitivism,
constructivism and connectivism ........................................................................................... 2
Section 2.3: Usability, user experience and e-learning .......................................................... 5
Section 2.4: Data analysis of usability tests (hypothesis testing) .......................................... 8
Section 2.5: Conclusion ......................................................................................................... 8
Section 3: Methodology ............................................................................................................. 9
Section 3.1: Hypothesis ......................................................................................................... 9
Section 3.2: Quantitative usability testing – an empirical research method .......................... 9
Section 3.2.1: Sample selection: choosing participants ..................................................... 9
Section 3.2.2: Control conditions..................................................................................... 10
Section 3.2.3: Ethical considerations ............................................................................... 10
Section 3.2.4: Procedure ................................................................................................. 10
Section 3.2.5: Statistical data analysis ............................................................................. 11
Section 4: Results and discussion ............................................................................................ 12
Section 4.1: Testing the difference between two means ...................................................... 12
Section 4.2: Discussion of research questions ..................................................................... 13
Section 4.3: Possible reasons for test results ....................................................................... 13
Section 4.4: Conclusion and recommendations ................................................................... 14
Section 5: References ............................................................................................................... 15
Appendix 1: Exam sheet .......................................................................................................... 18
Appendix 2: Instruction sheet .................................................................................................. 19
TW5221 Michael Halloran - 13035657
iii
Appendix 3: Research ethics committee consent form ............................................................ 20
List of Figures
Figure 1. The “User Experience Honeycomb” (Morville 2004). ............................................... 5
Figure 2. Post-course exam results. ......................................................................................... 12
TW5221 Michael Halloran - 13035657
1
Section 1: Introduction
This is a report on a usability study of e-learning interfaces and whether self-reflection
exercises should be included in online courses. The usability study was quantitative in that it
gathered numeric data from user testing.
The purpose of the usability study was to test the impact of embedded self-reflection
exercises on users of online courses and whether self-reflection is useful in helping users
understand concepts they come across.
The course that I tested was Grammar for Teachers: Language Awareness. Teachers
at the university where I work did the course as part of their professional development.
However, I tended to ignore the self-reflection exercises when I did the course myself and I
wanted to know why.
Through this study, I wanted to find out why the designer included the self-reflection
exercises in the course, whether the exercises were really necessary, and whether they raised
or lowered student attrition rates. However, these questions proved to be subjective and not
easily tested. Therefore, Professor Philip Rubens helped me refine my ideas into a
quantitative study based on a post-course exam of ten questions (see Appendix 1). This report
details the background, the methodology of data collection, the control conditions, the ethical
considerations, the results of the study, and a discussion of the data. Ten participants
volunteered to do the first unit of the course as well as the exam. The number of participants,
the scope and the length of the study were limited by time, location, and resources.
The layout of this report is as follows: Section 2 is a review of literature dealing with
learning theory, usability testing and hypothesis testing. Section 3 deals with the test
hypothesis and the methodology used to carry out the usability study. Finally, Section 4
presents the results of the study and offers some recommendations.
TW5221 Michael Halloran - 13035657
2
Section 2: Literature review
This review looks at some of the literature that has been written on the theory and usability of
e-learning programs. It also includes a short description of hypothesis testing as outlined by
Hughes and Hayhoe (2008, pp.64-71).
Section 2.1: Overview E-learning is becoming a major organizational use of the internet (Zaharias and
Poylymenakou 2009, p.76; Angelino et al 2007, p.2). Blended learning, already ubiquitous in
universities, is penetrating second-level education; e-learning programs are also being used
for corporate and professional development programs.
Koohang et al (2009, p.91) state that constructivist learning theory “focuses on
knowledge construction based on learners’ previous experience” and is therefore “a good fit
for e-learning because it ensures learning among learners.” Ally (2001, p.31) states that
“learners should be given time and the opportunity to reflect”; embedded questions will
encourage learners “to reflect on and process information in a relevant and meaningful
manner.” Alley suggests getting students to generate a learning journal. Koohang et al (2009,
p.95) agree by stating that reflection activities “will encourage the learner to be responsible
for his or her own learning.” But what are the other theories behind e-learning?
Section 2.2: Theories of education and e-learning: behaviourism, cognitivism, constructivism and connectivism Early online courses were designed based on behaviourism (Ally 2011, p.19). This school of
thought focuses on learners’ observable and measurable behaviours (Ally 2011, p.19). A
behaviourist online course would:
Inform students of course outcomes so they can set expectations.
Have regular tests as an imbedded part of course design.
Introduce learning material in a sequenced way.
Encourage students to provide feedback.
Cognitivism looks at learning from an information processing point of view (Ally 2011,
p.20). A cognitivist online course would:
Place important information in the centre of the screen for reading.
TW5221 Michael Halloran - 13035657
3
Highlight critical information with headings and clear formatting.
Tell learners why they should take the course.
Match the difficulty level to the learner’s cognitive level.
Jakob Nielsen in an interview with elearningpost (Nichani 2001) touches on an aspect of
cognitivism:
You need to keep all the content fresh in learners mind [sic]…For example, response time. Even after a
few seconds you always forget what was the track or sequence you were following…It is important that
your brain keeps the context.
(Nichani 2001 para. 4.)
Ally also says a cognitivist course would encourage students to use their existing knowledge
to help them make sense of the new information (Ally 2011, p.24).
Some other aspects of this approach include:
Chunking information to make it more memorable (Miller 1956).
Varied learning strategies to accommodate different kinds of learners.
Varied modes of information delivery: textual, visual and verbal.
Learner motivation strategies: intrinsic motivation and extrinsic motivation.
Metacognition: make a student aware of their learning capabilities.
Assignments that have real-life application and information.
As mentioned above, constructivism sees the learner as active. Stimuli are received from the
outside but it is the learner who actively creates the knowledge (Ally 2011, p.30). A
constructivist online course would:
Give learners meaningful activities in practical situations.
Provide first-hand information, without the contextual influence of an instructor, so
that students can personalise the information themselves.
Encourage cooperative learning.
Provide guided discovery activities.
Use embedded questions (or a learning journal) to encourage learner reflection.
Provide a high-level of interactivity.
Finally, connectivism is “a theory for the digital age, where individuals learn and work in a
networked environment” (Ally 2011, p.34). Ally sketches some general guidelines based on
this theory. Learners need to:
TW5221 Michael Halloran - 13035657
4
Be autonomous and independent; appropriate use of the internet is encouraged.
Unlearn old information and models in favour of the most up-to-date information and
models; learners need to identify the most important information.
Be active in a network of learning and acquire knowledge on an ongoing basis.
Must be allowed to connect with others around the world in order to share knowledge
and opinions.
Gather information from many resources to reflect the networked world and the
diversity of thinking within it.
Ally states that “further work needs to be done on how this theory can be used by educators
to design learning materials” (Ally 2011, p.38).
Finally, Ally suggests these different theories can be used to deal with different
aspects of a course:
Behavourism to teach facts.
Cognitivism to teach principles and processes.
Constructivism to teach real-life applications of learning.
Anderson (2011) provides a framework of how people learn:
Knowledge-centred learning give access to a vast selection of content and activities
but quality information is highlighted and filtered by the community of users.
Assessment-centred learning is based on “formative and summative assessment by
self, peer and teachers” (Anderson 2011, p.66).
Learner-centred learning changes in response to group and learner models and content
is changed based on student and teacher use.
Community-centred learning uses many formats for collaborative and individual
interaction.
In relation to knowledge-based and community-centred learning, Nielsen discusses usability,
design and aesthetics of good discussion forums for learners:
I actually believe much more in discussion groups than I believe in chat rooms as ways of allowing
students to interact…real-time chat effectively becomes very thin and not nearly as valuable as
discussion groups where people can think a little bit before they post and the instructor can moderate it
which a also good.
(Nichani 2001, para. 10)
TW5221 Michael Halloran - 13035657
5
Section 2.3: Usability, user experience and e-learning What is usability? Nielsen (2012, What – Definition of Usability, para. 1) defines usability as
“a quality attribute that assesses how easy user interfaces are to use” and defines it by five
qualities:
Learnability.
Efficiency.
Memorability.
Errors (the quantity and quality of errors a user makes).
Satisfaction.
Utility (does the interface do what the user needs?).
User experience (UX) is related to usability in that it “focuses on having a deep
understanding of users, what they need, what they value, their abilities, and also their
limitations” (U.S. Department of Health & Human Services 2013a, para. 1). Morville (2004)
uses a honeycomb to illustrate the facets of user experience:
Figure 1. The “User Experience Honeycomb” (Morville 2004).
The most basic way to improve usability is user-testing (Nielsen 2012, How to Improve
Usability, para. 1) and this process is three-fold:
Get representative users to test the interface.
Ask users to do representative tasks.
TW5221 Michael Halloran - 13035657
6
Observe users and take notes on their experience.
The U.S Department of Health & Human Services (2013b) outlines some other evaluation
methods:
Focus groups: moderated discussion involving five to ten participants.
Card sort testing: participants organise topics into categories that make sense to them.
Wireframing: creating a two-dimensional illustration of a page’s interface.
First click testing: examines what a test participant would click on first on the
interface in order to complete their intended task.
Satisfaction surveys.
The U.S Department of Health & Human Services goes on to discuss what the researcher
should do after gathering data from one of the above methods:
Evaluate the usability of the website.
Recommend improvements.
Implement recommendations.
Re-test the site to measure the effectiveness of your changes.
While these methods can help researchers test usability, there are recognised usability
principles for interaction design, often referred to as heuristics (Nielsen 1995a). Heuristic
evaluation is part of an iterative design process and usual involves a team of evaluators.
Nielsen recommends the use of five evaluators, but three at the least (Nielsen 1995b, para. 2).
Jefferies and Desurvire (1992, p.39) found that just one evaluator was the least powerful
evaluating technique when they experimented with different usability tests.
Heuristic evaluation does not provide a systematic way to generate fixes, but rather
aims to solve design issues by reference to established usability principles (Nielsen 1995b,
para. 12). Nielsen (1995a) provides a list of “10 Usability Heuristics for User Interface
Design”:
Visibility of system status.
Match between system and the real world.
User control and freedom.
Consistency and standards.
Error prevention.
TW5221 Michael Halloran - 13035657
7
Recognition rather than recall.
Flexibility and efficiency of use.
Aesthetic and minimalist design.
Help users recognize, diagnose, and recover from errors.
Help and documentation.
Do these principles apply to e-learning? A team at The University of Georgia found that
Nielsen’s list needed to be augmented (Benson et al 2002). They evaluated an e-learning
program designed for the American Red Cross. They created a “protocol for e-learning
heuristic evaluation” and fifteen usability and instructional design heuristics for the
evaluation of e-learning programs. Their augmented evaluation heuristics included Nielsen’s
original ten and five new principles:
Learning Design.
Media Integration.
Instructional Assessment.
Resources.
Feedback.
Zaharias and Poylymenakou (2009) suggest that a usability evaluation method for e-learning
needs to place motivation above functionality. They split e-learning usability attributes in
two: usability and instructional design. Under usability they include, navigation learnability,
accessibility, consistency and visual design. Under instructional design they include,
interactivity/engagement, content and resources, media use, learning strategies design,
feedback, instructional assessment and learner guidance and support (Zaharias and
Poylymenakou 2009, p.80). This ultimately feeds into the most important part of e-learning,
motivation by students: attention, relevance, confidence and satisfaction. Intrinsic motivation
can be characterised as the drive arising within the self to carry out an activity whose reward
is derived from enjoyment of the activity itself (Zaharias and Poylymenakou 2009, p.80) The
learning interface needs to encourage intrinsic learning motivation. Ally says that extrinsic
motivation should also be used, citing Keller’s ARCS model (Attention, Relevance
Confidence and Satisfaction) (Keller 1987; Ally 2011, p.28).
TW5221 Michael Halloran - 13035657
8
Section 2.4: Data analysis of usability tests (hypothesis testing) To test and analyse a usability intervention (the independent variable), a researcher will use
two groups: a test group and a control group (Hughes and Hayhoe 2008, p.64). The results of
the test are known as the dependent variable. The research analyses the average results of the
test group and control group to see if there is a statistically significant difference. Therefore,
it is important to include these factors when stating a test hypothesis.
In order to test the hypothesis Hughes and Hayhoe (2008, p.65; p.71) suggest using
Microsoft Excel to conduct a t-test of two independent means:
State the test hypothesis, including the independent variable, the dependent variable
and the expected direction.
Recast the test hypothesis as a null hypothesis.
Collect the data through the researchers chosen method.
Enter data into a spreadsheet.
Use the function COUNT to tally the sample sizes of the test and control groups.
Use the function AVERAGE to calculate the mean.
Use the function STDEV to calculate the standard deviation for each group. Standard
deviation is “an indicator of the variation of the data in the sample…[it] is helpful for
envisioning how widely the data vary from the average” (Hughes and Hayhoe 2008,
p.63).
Use the function TTEST to calculate the probability (the p value) that the results
could be caused by differences in the samples, rather than the intervention.
Hughes and Hayhoe (2008, p71) state that “typically, you can reject the null if the p
value is less than 0.1.
Finally, if you can reject the null, then accept the test hypothesis.
Section 2.5: Conclusion This literature review has looked at the theories behind e-learning course design and how to
test the usability of such courses.
Online courses will become more central to general education and connectivism, as
mentioned by Ally (2011), might be a very large target for research and potential
applications.
TW5221 Michael Halloran - 13035657
9
Section 3: Methodology
Grammar for Teachers: Language Awareness is a course designed for future, inexperienced
and experienced teachers of English as a foreign language. To successfully complete the
course users must “reflect upon [their] own knowledge of the English language…participate
in discussion forums with other teachers…keep a learner journal” (Cambridge University
Press and UCLES 2013). All of these components suggest a strong constructivist basis for the
course.
Section 3.1: Hypothesis Does self-reflection actually help students process information and become better learners? I
set about proving the following hypothesis:
In online courses, users will better understand course concepts if they complete self-
reflection exercises.
The goal of testing this hypothesis was to establish whether designers should include self-
reflection exercises in online courses. To reach this goal, I proposed these questions:
Do users understand course concepts better after they self-reflect?
Do users misinterpret course concepts if they do not self-reflect?
Is there any noticeable difference between users who self-reflect and those who don’t?
Section 3.2: Quantitative usability testing – an empirical research method In order to generate data suitable for a t-test of two independent means, I chose the following
testing methodology.
Section 3.2.1: Sample selection: choosing participants
The course is aimed at teachers of any level of experience. I chose ten participants, male and
female, between the ages of twenty-five and sixty-five. All participants were teachers at
different stages in their careers; none had attempted the course before. Based on Nielsen’s
(1995b, para. 2) recommendation of using five usability evaluators, I divided the participants
randomly into two groups of five. I will refer to these two groups as Team A (the test group)
and Team B (the control group).
TW5221 Michael Halloran - 13035657
10
Section 3.2.2: Control conditions
The usability study took place in a large, air-conditioned and brightly-lit university
classroom. I observed one participant at a time. I spaced out the participant slots over five
days between 8.30am and 4.30pm depending on the participants’ availability. Cambridge
English Teacher’s suggested completion time of the whole course was five hours which is too
long for most people to put time aside to complete. Instead, I chose “Unit 1: Nouns and
pronouns” to act as a representative example of the whole course. I found that registration,
Unit 1 itself, and the post-course exam took forty-five minutes for me to complete. I took this
timing into account when informing participants about the length of the test. Finally, I
provided all participants with the following items:
A pencil and eraser.
A Lenovo G480 laptop with wireless mouse and internet access.
A pre-course task sheet (see Appendix 2).
A post-course exam sheet (see Appendix 1).
A bottle of water.
Section 3.2.3: Ethical considerations
The FAHSS Ethics Committee at the University of Limerick approved this quantitative
research project. The approval included conducting pre-/post-surveys as part of a usability
study. I answered “No” to all questions on the approval form checklist. Participants signed a
consent form (Appendix 3) but will remain anonymous. I did not make any audio or video
recordings. The anonymous exam sheet is the only record.
Section 3.2.4: Procedure
Participants sat at a desk with the laptop. They signed the consent form and received the task
sheet. After they complete the task sheet, I directed them to complete ‘Unit 1 – Nouns and
Pronouns’ using the onscreen directions only. I emphasised that they must read everything
and watch every video. However, I implemented the following intervention:
Team A members must attempt all self-reflection exercises.
Team B members must ignore all the self-reflection exercises.
I observed all participants to ensure they followed this intervention protocol. I prompted them
when they strayed from the instructions or provided them with assistance when they asked.
TW5221 Michael Halloran - 13035657
11
Registration and unit completion took forty to eighty minutes depending on the participant.
After they complete the unit, I removed the laptop and task sheet and gave the participants a
pencil, eraser and exam sheet. The exam consists of ten questions which tested participants’
comprehension of the course concepts. Participants had to attempt all questions. The exam
took less than five minutes to complete per participant.
Section 3.2.5: Statistical data analysis
The exam sheet consisted of ten questions and was based on concepts covered in the course.
The first five questions required one answer. The second five questions required three
answers. This added up to a potential total score of 20 out of twenty 20. I pilot tested the
exam sheet on two colleagues. This was to see whether the questions were readable and easy
to understand. While these two colleagues did sign the consent form, I did not keep their
results for the record as this was simply a proofreading exercise.
After the ten participants of the actual test had finished, I tallied up their exam scores
and followed Hughes and Hayhoe’s (2008, p.71) hypothesis testing method as I outlined in
the literature review.
TW5221 Michael Halloran - 13035657
12
Section 4: Results and discussion
Below are the results of the analysis. In section 4.2 I restate my initial goal questions and
discuss the answers.
Section 4.1: Testing the difference between two means
Hypothesis: In online courses, users will better understand course concepts if they
complete self-reflection exercises.
Null Hypothesis: In online courses, users will better understand course concepts if
they do not complete self-reflection exercises.
When assigning arguments in the function TTEST (probability) in Microsoft Excel, I chose a
one-tailed (directional) test. Under the type argument, I chose “3” as this was a “nonpaired
test where the variances were not equal (Hughes and Hayhoe 2008, p. 69). See Figure 2
below for details of the data analysis.
Post-Course Exam Results
Team A Team B
8 18
17 12
15 14
16 14
11 15
n 5 5
mean 13.4 14.6
SD 3.78153 2.19089
p 0.28018
n= total sample size
SD = standard deviation
p= probability
Figure 2. Post-course exam results.
By reviewing the means of both groups we can see that the average score for Team A was
13.4 out of 20 while for Team B it was 14.6 out of 20. Already we see that the hypothesis is
not accepted based on the raw averages. If the probability is less than 0.1 than the null
hypothesis can be rejected. However, the p value is 0.28018, meaning the null hypothesis
TW5221 Michael Halloran - 13035657
13
cannot be rejected. Therefore, based on the available data: the test hypothesis is not
accepted.
Section 4.2: Discussion of research questions Based on the data, I will now answer my research questions as outlined in Section 3.1:
Do users understand course concepts better after they self-reflect?
No, Team A participants had a lower total average than Team B.
Do users misinterpret course concepts if they do not self-reflect?
No, Team B had a higher average than Team A and one Team B participant scored
the highest of all the participants, 18 out of 20.
Is there any noticeable difference between users who self-reflect and those who don’t?
No, there is only a 9% difference between the two groups’ averages.
Section 4.3: Possible reasons for test results Does this all mean that self-reflection exercises are redundant and that we should accept the
null hypothesis? Possibly. However, this would require us to reject a huge part of
constructivist theory. Therefore, it is more likely that the test itself was flawed. There are
several possible reasons why these results occurred:
The sample size was not big enough.
Unit 1 is an eighth of the total course. The test may not have been representative
enough of the whole course. A test of two units may have been better.
The exam sheet questions should be re-worded because two participants
misunderstood questions 2 and 6.
Unit 1 may not have been challenging enough. Nouns and pronouns are basic
concepts for experienced teachers.
Some participants may not have used the self-reflection exercises to their full
potential. Some wrote very long passages while others wrote only a couple of lines.
Participants may not have read the questions thoroughly. The participant who scored
18 out of 20 read every part of the online unit aloud. This participant also read the
exam questions aloud twice.
The exam questions may not have accurately reflected the self-reflection exercises.
TW5221 Michael Halloran - 13035657
14
In hindsight, the hypothesis is flawed. A better hypothesis might be worded to include
the dependent variable; in this case, the results of the post-course exam:
Users will have stronger recall of online course concepts in a post-task exam if they
complete embedded self-reflection exercises.
Allowing participants to access the online journal of their self-reflections may have
altered their test results.
Allowing participants to discuss the reflections in the course forum could also have
altered the results as well.
The course self-reflection exercises may have been poorly designed themselves; they
may have discouraged participants from really considering the course concepts.
Section 4.4: Conclusion and recommendations In this report, I have presented a literature review of e-learning theory and usability, the
methodology I used to test my hypothesis, and finally, an analysis of the resulting data. In
Section 3.1, I stated that “the goal of testing this hypothesis was to establish whether
designers should include self-reflection exercises in online courses.” Considering my test
results, the answers to my research questions, as well as the possible faults with the test itself
(as mentioned in Section 4.3), I have to state that the test results are inconclusive.
It is possible that self-reflection exercises do not aid learners. However, self-reflection
is an integral part of constructivist learning theory, and constructivism is central to
contemporary e-learning design theory. Therefore, it is essential that further research is
carried out in this area. I would recommend another study is carried out on Grammar for
Teachers: Language Awareness, using a larger sample size and a larger representative section
of the course. The future researcher should also take into account all the other faults I
mentioned in Section 4.3 and address them before conducting the research.
TW5221 Michael Halloran - 13035657
15
Section 5: References
Ally, M. (2011) ‘Foundation of Educational Theory for Online Learning’, in Anderson, T.,
ed., The Theory and Practice of Online Learning, 2nd
ed., Edmonton: AU Press, 15-44.
Anderson, T. (2011) ‘Towards a Theory of Online Learning’ in Anderson, T., ed., The
Theory and Practice of Online Learning, 2nd
ed., Edmonton: AU Press, 45-74.
Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., Kim, H., Lauber, E., Loh, S. and
Reeves, T.C. (2002) ‘Usability and Instructional Design Heuristics for E-Learning
Evaluation’, in Barker, P. and Rebelsky, S., eds., Proceedings of World Conference on
Educational Multimedia, Hypermedia and Telecommunications 2002, Chesapeake, VA:
AACE, 1615-1621.
Cambridge University Press and UCLES 2013 (2013) ‘Grammar for Teachers: Language
Awareness’, Cambridge English Teacher available:
http://www.cambridgeenglishteacher.org/courses/details/18606 [accessed 2 Nov 2013].
David, A. and Glore, P. (2010) ‘The Impact of Design and Aesthetics on Usability,
Credibility, and Learning in an Online Environment’, Online Journal of Distance Learning
[online], 13(4), available:
http://www.westga.edu/~distance/ojdla/winter134/david_glore134.html [accessed 13 Oct
2013].
Hughes, M. and Hayhoe, G. (2007) A Research Primer for Technical Communication:
Methods, Exemplars, and Analyses, New York: Lawrence Erlbaum Associates.
Jeffries, R. and Desurvire, H. (1992) ‘Usability testing vs. heuristic evaluation: was there a
contest?’, SIGCHI Bulletin, 24 (4), 39-41.
Keller, J. (1987) ‘Development and use of the ARCS model of instructional design’ in
Journal of Instructional Development, 10(3), 2-10.
TW5221 Michael Halloran - 13035657
16
Koohang, A., Riley, L. and Smith, T. (2009) ‘E-learning and Constructivism: From Theory to
Application’ in Interdisciplinary Journal of E-Learning and Learning Objects [online], 5, 91-
109, available: http://ijklo.org/Volume5/IJELLOv5p091-109Koohang655.pdf [accessed 2
Nov 2013].
Miller, G.A. (1956) ‘The magical number seven, plus or minus two: Some limitations on our
capacity for processing information’ in Psychological Review, 63, 81-97.
Morville, P. (2004) ‘User Experience Design’, Semantic Studios [online], 21 June, available:
http://semanticstudios.com/publications/semantics/000029.php [accessed 24 Nov 2013].
Nielsen, J. (1995a) ‘10 Usability Heuristics for User Interface Design’, Nielsen Norman
Group [online], 1 January, available: http://www.nngroup.com/articles/ten-usability-
heuristics/ [accessed 13 Oct 2013].
Nielsen, J. (1995b) ‘How to Conduct a Heuristic Evaluation’, Nielsen Norman Group
[online], 1 January, available: http://www.nngroup.com/articles/how-to-conduct-a-heuristic-
evaluation/ [accessed 13 Oct 2013].
Nielsen, J. (2001) ‘First Rule of Usability? Don't Listen to Users’, Nielsen Norman Group
[online], 5 Aug, available: http://www.nngroup.com/articles/first-rule-of-usability-dont-
listen-to-users/ [accessed 13 Oct 2013].
Nielsen, J. (2012) ‘Usability 101: Introduction to Usability’, Nielsen Norman Group [online]
4 January, available: http://www.nngroup.com/articles/usability-101-introduction-to-
usability/ [accessed 13 Oct 2013].
Nichani, M., ed. (2001) ‘Jakob Nielsen on e-learning’, elearningpost [online], 16 January,
available:
http://www.elearningpost.com/articles/archives/jakob_nielsen_on_e_learning/ [accessed 13
Oct 2013].
TW5221 Michael Halloran - 13035657
17
U.S. Department of Health & Human Services (2013b) ‘Usability Evaluation Basics’
Usability.gov [online], available at: http://www.usability.gov/what-and-why/usability-
evaluation.html [24 Nov 2013].
U.S. Department of Health & Human Services (2013a) ‘User Experience Basics’,
Usability.gov [online], available: http://www.usability.gov/what-and-why/user-
experience.html [accessed 23 Nov 2013].
Zaharias, P. and Poylymenakou, A. (2009) ‘Developing a Usability Evaluation Method for e-
Learning Applications: Beyond Functional Usability’, International Journal of Human-
Computer Interaction, 25(1), 75-98.
TW5221 Michael Halloran - 13035657
18
Appendix 1: Exam sheet EXAM SHEET Questions 1-5 are worth one point each 1. What is an alternative name for
“word class”?
______________________________________
2. Give one example of an abstract,
countable, common noun
______________________________________
3. Complete the sentence: ____________________is a word that is used to
show a sudden expression of emotion.
4. Complete the sentence: ____________________is a word that connects
words, phrases and clauses in a sentence.
5. Complete the sentence:
Concrete nouns can be seen, touched or
____________________.
Questions 6-10 are worth three points each
6. Give three examples of determiners
1. 2. 3.
7. Give three examples of demonstrative pronouns
1. 2. 3.
8. Give three examples of ordinal number quantifiers
1. 2. 3.
9. Give three examples of collective nouns
1. 2. 3.
10. Give three examples of indefinite pronouns
1. 2. 3.
Total Score: /20
TW5221 Michael Halloran - 13035657
19
Appendix 2: Instruction sheet
Instructions
1. Open Google Chrome.
2. In the search bar type “www.cambridgeenglishteacher.org” and press
return.
3. In the top-right corner click “register.”
4. Fill in the registration form with your details (N.B. only use numbers and/or
letters in your password).
5. Click “create account.”
6. Open your email inbox and select the message titled “validate email
address.”
7. Follow the link in the message.
8. Sign into your new account.
9. Click “courses” in the navigation bar.
10. Scroll down to “Grammar for Teachers: Language Awareness” and click
“open.”
11. Select “Unit 1 – Nouns and Pronouns” and begin the course.
TW5221 Michael Halloran - 13035657
20
Appendix 3: Research ethics committee consent form
FACULTY OF ARTS, HUMANITIES AND SOCIAL SCIENCES
RESEARCH ETHICS COMMITTEE
CONSENT FORM
Consent Section:
I, the undersigned, declare that I am willing to take part in research for the project entitled
TW5221 Usability Study.
I declare that I have been fully briefed on the nature of this study and my role in it and
have been given the opportunity to ask questions before agreeing to participate.
The nature of my participation has been explained to me and I have full knowledge of
how the information collected will be used.
I am also aware that my participation in this study may be audio recorded and I agree
to this. However, should I feel uncomfortable at any time I can request that the
recording equipment be switched off. I am entitled to copies of all recordings made
and am fully informed as to what will happen to these recordings once the study is
completed.
I fully understand that there is no obligation on me to participate in this study.
I fully understand that I am free to withdraw my participation at any time without
having to explain or give a reason.
I am also entitled to full confidentiality in terms of my participation and personal
details.
______________________________________ __________________________
Signature of participant Date