Project leader: Helen Correia

Project report: Helen Correia

LTAP project no: P7262

P7262 Final Report Page 1 of 11

(LG) vUWS (Web CT) learning guides: interactive student competency

assessments for plagiarism and referencing

A UWS Learning and Teaching Action Plan (LTAP) 2006-2008 Project

TABLE OF CONTENTS

Project aims...................................................................................................................3

Actual outcomes achieved...........................................................................................3

Comments......................................................................................................................3

Evaluation of the project...............................................................................................3

Scalability of the project...............................................................................................3

Sustainability of project................................................................................................3

Project implementation issues.....................................................................................3

Dissemination of project outcomes.............................................................................4

Other comments............................................................................................................4

SECTION 2: DETAILED REPORT, RELATED DOCUMENTS AND MEDIA................4

WebCT learning guides: interactive student competency assessments for plagiarism and referencing...........................................................................................4

1) Evaluation of the guide.............................................................................................5

2) Modification of the guide..........................................................................................6

Appendix - Plagiarism and Referencing Guide (APA)...............................................8

P7262 Final Report Page 2 of 11

Project aims

The project aims to improve plagiarism and referencing literacy through the development and assessment of relevant knowledge and skills using an interactive, feedback-driven, self paced, online tutorial assessment.

Actual outcomes achieved

Data regarding the use of the guide in addition to feedback from students has been analysed and the guide has been modified to improve clarity of the task, reduce the time commitment to the task, and to improve student understanding of plagiarism and referencing. This has resulted in improved accuracy by students on the task itself. Unit coordinators and markers have also independently reported that referencing in assessments has improved. Additional items for the assessment components of the guide have been created increasing the item pool. The guide has been developed for scalability and a manual is being finalised to allow staff to modify the task to suit their own unit and assessment.

Comments

The decisions regarding the development and refinement of the guide has been based on sound scholarly literature as well as based on evidence from students and the task itself. Publications have not yet been produced, but research is currently underway..

Evaluation of the project

Data regarding the use of the guide and feedback from students has been collected. A HREC application for use of the data for publication purposes has been accepted and this data will form the basis of a publication on the way in which students have engaged in the task and will be submitted for review later in 2008

Scalability of the project

The guide has been adopted in the College of Health and Science (School of Biomedical and Health Sciences – Occupational Therapy) and will continue to be used in that school. The unit coordinator has reported that referencing skills had largely improved. Interest in the guide continues and discussions are being held regarding use in other schools and colleges. A manual regarding use of the guide is being finalised to assist designers in adapting the task to suit their unit.

Sustainability of project:

Discussions are being held with TDU regarding the potential of this guide to be included in a learning object repository. In addition, the guide is being used in other units with the proviso that unit coordinators contribute assessment items to the item pool so that the item pool can continue to expand.

Project implementation issues:

Initial aims to evaluate the generalisation of skills to written assessments by using Turnitin were not possible before the project end date.

P7262 Final Report Page 3 of 11

Dissemination of project outcomes:

The project was presented at a HoP forum in November 2007. The evaluation of student engagement with the task is currently ongoing as part of a research project and will be disseminated via journal publication. The guide is also highlighted as part of a vUWS exemplar project that focuses on the unit in which this guide is used and will be published via the UWS QILT site.

Other comments

The manual for designers is currently being finalised. Data from the task will continue to be analysed and the guide modified for improvement.

SECTION 2: DETAILED REPORT, RELATED DOCUMENTS AND MEDIA

WebCT learning guides: interactive student competency assessments for plagiarism and referencing

The Plagiarism and Referencing Task is an integrated learning guide, feedback tool, and assessment that is delivered via the UWS learning management system vUWS (originally developed in WebCT). Despite being informed about the policy on academic misconduct in unit outlines, lectures, and tutorials, there is no guarantee that students understand what constitutes cheating or plagiarism or how to prevent it (e.g. through appropriate referencing) prior to assignment submission. Student responses to allegations of academic misconduct typically include a defence based on insufficient information and misunderstanding of relevant issues. At present, assessment practices rely on the implicit assumption that students have already attained sufficient literacy in plagiarism and referencing – an assumption that is often untested prior to students submitting assessments. As such, this interactive online guide was developed to ensure that students are both educated and assessed on critical information regarding plagiarism but also how to avoid it through appropriate referencing.

The guide is presented using the quiz tool in the online learning management system and is structured such that students are guided through a series of sections or levels relating to plagiarism and referencing in APA style (the referencing style used in the School of Psychology). It provides the context and content necessary, but also assesses the students understanding in a sequential and structured way, to maximise their mastery of concepts before progressing to the next level. The guide is an integrated component of a written assessment task and is presented as a prerequisite for assignment submission.

Further details relating to the outcomes of the LTAP project regarding evaluation of the guide (section 1), modification of the task (section 2), scalability and sustainability (section 3) and dissemination (section 4) are presented below.

P7262 Final Report Page 4 of 11

1) Evaluation of the guide

Data from the task

Item responses from data collected in Autumn and Spring 2007 were downloaded. The most noteworthy data is from the section of the task that is an overview from the UWS policy on academic misconduct, including providing text from the policy itself to highlight key aspects (some of the aggregate data from items on the task are presented in the attached document). What is particularly important is that not all students are able to accurately answer questions relating to key policy statements, despite having been presented with the exact information within the task itself, usually within the previous two or three items. For example:

Approximately 5% of students do not initially accurately define plagiarism despite the definition being included in the previous item.

Approximately 8% of students do not initially select the following example as plagiarism: “An assignment is submitted, where some of it, or all of it, has been prepared by someone other than the person submitting the assignment”

Inaccurate responses may be due to a number of possibilities, such as clarity of questions, inattention, misunderstanding, or recall difficulties. In any case, whilst the majority of students may be able to answer the question correctly, it cannot be assumed in the first instance that all students have read or understood the policy on academic misconduct. Guides and assessments such as this are the only way to ensure that students have actively engaged with the necessary material.

All students must repeat the questions until they have achieved 100% accuracy, which is the case for all sections of the task. From the Spring 2007 data, in the section of the task focusing on plagiarism, the median number of attempts was 2, with the maximum number of attempts by a student being 14. Subsequent revisions of the task (see section 2 below for more detail) reduced the frequency of repetitions (mean number of attempts reduced from above 2 to below 2 across this section). Improvements in accuracy were similarly achieved across other sections following modifications.

Student Feedback on Unit (SFU) formsSFU forms from Autumn 2007 and Spring 2007 were reviewed for any feedback relevant to the task. There were a mix of comments regarding its usefulness and suggestions for improvement.

Negative comments Time consuming Frustrating Difficult to complete Delay time between being able to repeat a level (included to ensure time to

review answers before repeating the level)

Positive comments Helpful Relevant across subjects Worth completing, even when challenging

The feedback was incorporated into modifications of the task for Autumn 2008 (see section 2 below).

P7262 Final Report Page 5 of 11

Focus group feedbackInformal focus groups were held with six students in Spring 2007 to gain more detailed information regarding completion of the task, particularly in relation to those sections with a high frequency of repetition to achieve mastery. Students reported that the task itself was useful, particularly given the relevance of the task across multiple units. Students did report that some sections required greater clarification or further examples, and indicated that the length of some sections meant that an error on just one item would make it necessary to repeat the entire section. This feedback was incorporated into revisions of the task (see section 2 below).

2) Modification of the guide

Task structureInitially the task was divided into 12 sections, each of which students were required to complete to 100% mastery before progressing to the next level. Following data analysis and feedback from students, the task was further divided into smaller subsections to reduce the frequency of repetitions of large sections based on minor errors. Analysis of the Spring 2008 data revealed that frequency of repetitions had reduced which also reduced the time commitment to the task.

The task includes time delays between resubmissions to ensure that students are provided with time to review the feedback. Based on student feedback, these delay times were reviewed and reduced in some sections where students commonly made minor errors (e.g. a spelling error). Again, this reduced the time commitment to the task.

Clarification of questionsThe task was initially designed with the aim of keeping items to a minimum to reduce the length of the task. However, feedback from students regarding particular items where data analysis revealed low levels of accuracy suggested that some items required division, further examples were needed, further clarification of information would be beneficial, or clarification of the assessment item was required. Data analysis from Spring 2008 indicates that accuracy across a number of these items has improved.

Additional assessment itemsThe task has been designed such that students are assessed on particular concepts within the task itself and these assessment items are randomly drawn from a pool of assessment items. This ensures that different, but equivalent, items are present to different students so that it is highly unlikely that two students receive exactly the same set of questions. In addition, each student is presented with a different set of assessment items if they need to repeat a level so that they must apply the principle to a new item rather than simply learn the answer to an existing item. In order for the task to remain effective, additional items must be deposited in the assessment item pool. More than 50 new items have since been created and included in the pool of items.

Scalability and sustainability Portability

P7262 Final Report Page 6 of 11

The task has been designed so that it can effectively be uploaded and downloaded into any unit with a vUWS site as a package of quizzes or as a pool of items.

Use in other unitsThe task has been adopted in the College of Health and Science (School of Biomedical and Health Sciences – Occupational Therapy) and will continue to be used in that school. Feedback from the unit coordinator of this unit indicates that the task was considerably successful in improving referencing practices as demonstrated in student’s written assignments. Interest from unit coordinators in use of the task in other Schools has also been shown (School of Social Sciences, School of Law) and discussions are underway regarding the use and adaptability of the task to make it applicable within relevant units.

Procedural manualA procedural manual is currently being finalised that will provide unit coordinators with an overview of the task itself, how to import the task into their unit vUWS site, as well as how to modify the task for applicability in a particular unit.

Contribution to the item poolThe item pool of assessments needs to be contributed to over time to ensure the originality of the items being used within the task itself. To maximise the range of items available for use in the long term, unit coordinators who decide to use the task within their units are asked to contribute to the overall item pool by creating assessment items each time the task is used.

Dissemination HoP forumAn overview of the online guide was presented at the Head of Program forum in November 2007 and subsequently generated interest in use of the guide.

PublicationA research project investigating to student’s engagement with the task itself has been approved by the UWS Human Research Ethics Committee (Approval code: H6323) and student responses to the task are currently being collected and analysed for the purpose of publication. Publication through a journal article will make available information about the development of the guide as well as data regarding student responses relating to the process of engaging with about plagiarism and referencing.

QILT siteThis guide appears in the unit 101183 - Psychology: Behavioural Science. This unit is currently being developed as a vUWS exemplar and the guide is a feature of the exemplar. As such, information about the guide will be made available through the Quality in Learning and Teaching (QILT) site - http://tdu.uws.edu.au/qilt/ - once it has been finalised for publication.

P7262 Final Report Page 7 of 11

Appendix - Plagiarism and Referencing Guide (APA)

Screenshots

Task: Guide and embedded assessment (Plagiarism)

P7262 Final Report Page 8 of 11

Task: Feedback

P7262 Final Report Page 9 of 11

Task: Guide and embedded assessment (Referencing)

P7262 Final Report Page 10 of 11

P7262 Final Report Page 11 of 11